Lawn boundary control method and control system of mower and intelligent mower

Abstract

The invention provides a lawn boundary control method and system for a lawn mower and an intelligent lawn mower, and the method comprises the steps: respectively collecting RGB image data and lawn point cloud data corresponding to different moments, and the position information, attitude information and motion information of the lawn mower through a multi-sensor fusion mode based on a visual sensor, a TOF sensor and an inertial measurement unit; according to the acquired data and information, on one hand, boundary identification and boundary optimization are carried out on the RGB image data, the optimal three-dimensional lawn boundary line of the RGB image data is obtained, and the lawn boundary identification accuracy of the intelligent lawn mower is improved; and on the other hand, the position information of the mower at the next moment is predicted, and whether the mower has a border crossing risk is judged, so that when the mower has the border crossing risk, the working track of the mower is changed, the border crossing risk of the intelligent mower is avoided in time, and the safety of the intelligent mower is improved.

Description

Technical Field

[0001] The present application relates to the technical field of lawn mower boundary recognition, and in particular to a lawn mower lawn boundary control method, a control system and an intelligent lawn mower. Background Art

[0002] With the rapid development of smart agriculture and automation technology, smart lawn mowers, as an important tool for modern gardening management, have gradually attracted widespread attention. Smart lawn mowers have become an ideal choice for home and commercial lawn maintenance because they can autonomously complete lawn mowing tasks, significantly improve work efficiency and reduce labor costs. However, in actual applications, the lawn terrain environment is complex and diverse, or there are many uncertain obstacles, etc., which puts higher requirements on the positioning ability, navigation ability, boundary recognition ability and working accuracy of smart lawn mowers.

[0003] Existing smart lawn mowers mainly rely on RTK (real-time dynamic differential positioning) technology for self-positioning. However, when the RTK signal is blocked or interfered by the outside world, its positioning accuracy will drop significantly. In this case, the smart lawn mower may go beyond the lawn boundary, causing damage to the lawn or equipment. In order to prevent the occurrence of cross-border problems when the positioning accuracy is insufficient, many smart lawn mowers mainly adopt the method of pre-burying boundary lines on the periphery of the lawn, but this method not only consumes a lot of manpower, but also increases the initial investment cost, which greatly limits the promotion and application of smart lawn mowers. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a lawn mower lawn boundary control method, control system and intelligent lawn mower, which are used to solve the boundary crossing problem of the existing intelligent lawn mower.

[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides a lawn boundary control method for a lawn mower, the lawn boundary control method for a lawn mower comprising: based on a preset frequency, collecting RGB image data corresponding to different times through a visual sensor, obtaining lawn point cloud data at each time through a TOF sensor, and obtaining position information, posture information and motion information of the lawn mower at each time through an inertial measurement unit; sequentially performing boundary recognition on each RGB image data, and performing boundary detection operation or boundary prediction operation on the current RGB image data according to the boundary line recognition result of the previous RGB image data, respectively, to obtain the corresponding primary two-dimensional lawn boundary line; performing boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and performing boundary correction operation on the optimized two-dimensional lawn boundary line based on the lawn point cloud data at the current time to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; predicting the position information of the lawn mower at the next time according to the position information, posture information and motion information of the lawn mower at the current time, and judging whether the lawn mower has a risk of crossing the boundary based on the optimal three-dimensional lawn boundary line, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

[0006] In some embodiments of the first aspect of the present application, the method of performing a boundary detection operation or a boundary prediction operation on the current RGB image data according to the boundary line recognition result of the previous RGB image data includes: judging whether the previous RGB image data recognizes the lawn boundary line according to the boundary line recognition result of the previous RGB image data; if the previous RGB image data does not recognize the lawn boundary line, performing a boundary detection operation on the current RGB image data to obtain the corresponding primary two-dimensional lawn boundary line; if the previous RGB image data has recognized the lawn boundary line, performing a boundary prediction operation on the current RGB image data to obtain the corresponding primary two-dimensional lawn boundary line.

[0007] In some embodiments of the first aspect of the present application, the method of performing a boundary detection operation on the current RGB image data includes: performing distortion correction on the current RGB image data, and extracting the lower half of the corrected RGB image data as the current lawn boundary RGB image data; removing high-frequency noise in the current lawn boundary RGB image data, and performing smoothing noise processing on the denoised current lawn boundary RGB image data to obtain the corresponding current lawn boundary optimal RGB image data; performing color gamut conversion on the current lawn boundary optimal RGB image data to generate the corresponding current lawn boundary HSV image data, and performing image segmentation on the current lawn boundary HSV image data according to the lawn hue threshold obtained by the test to obtain the corresponding lawn area mask; performing corrosion operations and expansion operations on the lawn area mask, and identifying and marking the connected domains in the lawn area mask, eliminating isolated noise areas, and obtaining the corresponding optimal lawn area mask; extracting multiple lawn boundary points in the optimal lawn area mask, and constructing a lawn boundary line equation based on each lawn boundary point to fit the primary two-dimensional lawn boundary line of the current RGB image data.

[0008] In some embodiments of the first aspect of the present application, the method of performing a boundary prediction operation on the current RGB image data includes: calculating the posture change of the lawn mower at the current moment based on the position information, posture information and motion information of the lawn mower at the previous moment and the current moment; predicting the position of the lawn boundary line in the current RGB image data based on the optimal three-dimensional lawn boundary line of the previous RGB image data and the posture change of the lawn mower at the current moment, and generating a corresponding primary two-dimensional lawn boundary line.

[0009] In some embodiments of the first aspect of the present application, the method of performing a boundary particle filtering operation on the primary two-dimensional lawn boundary line includes: dividing the primary two-dimensional lawn boundary line into multiple continuous sampling areas, and dynamically sampling the boundary line in each sampling area to obtain multiple boundary line particles in each sampling area; calculating the weight value of each boundary line particle, and using a robust optimization algorithm to suppress abnormal boundary line particles; screening the particles with the highest weight value in each sampling area for fitting to obtain the optimized two-dimensional lawn boundary line of the current RGB image data.

[0010] In some embodiments of the first aspect of the present application, based on the lawn point cloud data at the current moment, a method of performing a boundary line correction operation on the optimized two-dimensional lawn boundary line includes: projecting the lawn point cloud data at the current moment onto the current RGB image data according to the coordinate conversion relationship between the RGB image data collected by the visual sensor and the TOF data collected by the TOF sensor to obtain corresponding lawn point cloud projection data; performing geometric consistency correction on multiple lawn boundary points in the optimized two-dimensional lawn boundary line based on the lawn point cloud projection data to obtain the optimal two-dimensional lawn boundary line of the current RGB image data; performing three-dimensional coordinate conversion and three-dimensional boundary line fitting on multiple lawn boundary points in the optimal two-dimensional lawn boundary line based on the lawn point cloud data to obtain the primary three-dimensional lawn boundary line of the current RGB image data; calculating the point cloud distribution density and normal change rate of multiple lawn boundary points in the primary three-dimensional lawn boundary line based on the lawn point cloud data, and selecting multiple optimal lawn boundary points according to the point cloud distribution density and normal change rate of each lawn boundary point; converting each optimal lawn boundary point from the mower coordinate system to the world coordinate system, and fitting the optimal three-dimensional lawn boundary line of the current RGB image data.

[0011] In some embodiments of the first aspect of the present application, the method of converting each optimal lawn boundary point from the lawn mower coordinate system to the world coordinate system includes: obtaining the position information robot_x and robot_y of the lawn mower in the world coordinate system by real-time dynamic positioning technology, and defining a translation matrix The yaw angle information robot_yaw of the lawn mower in the lawn mower coordinate system is obtained through the inertial measurement unit, and the rotation matrix is ​​defined According to the defined translation matrix T and rotation matrix R, the position information x_world, y_world and z_world of each optimal lawn boundary point in the world coordinate system are calculated; wherein the calculation formula is: Among them, x_robot, y_robot and z_robot are the position information of each optimal lawn boundary point in the lawn mower coordinate system.

[0012] In some embodiments of the first aspect of the present application, the method of changing the working trajectory of the lawn mower includes: determining the current position of the lawn mower and the next index position based on the original working trajectory according to the position information of the lawn mower at the current moment, and connecting these two position points to obtain a straight line; wherein the original working trajectory is a serpentine route; calculating the slope of the straight line, and obtaining a perpendicular line of the straight line based on the current position of the lawn mower; obtaining the intersection of the perpendicular line and the remaining section of the original working trajectory after the current position, and using the remaining section after the intersection as the new working trajectory.

[0013] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides a lawn mower lawn boundary control system, the lawn mower lawn boundary control system comprising: a data acquisition module, including: a visual sensor, a TOF sensor and an inertial measurement unit, which are respectively used to collect RGB image data corresponding to different times based on a preset frequency, collect lawn point cloud data at each time, and collect position information, posture information and motion information of the lawn mower at each time; a boundary recognition module, connected to the data acquisition module, for sequentially performing boundary recognition on each RGB image data, and according to the boundary line recognition result of the previous RGB image data, performing boundary detection operation or boundary prediction operation on the current RGB image data, to obtain the corresponding primary two-dimensional lawn. a boundary optimization module connected to the boundary recognition module, used to perform a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and based on the lawn point cloud data at the current moment, perform a boundary correction operation on the optimized two-dimensional lawn boundary line to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; a boundary control module connected to the data acquisition module and the boundary recognition module, used to predict the position information of the lawn mower at the next moment according to the position information, posture information and motion information of the lawn mower at the current moment, and based on the optimal three-dimensional lawn boundary line, determine whether the lawn mower has a risk of crossing the boundary, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

[0014] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides an intelligent lawn mower, the intelligent lawn mower comprising: a lawn mower body and a lawn mower lawn boundary control system communicatively connected to the lawn mower body; wherein the lawn mower lawn boundary control system comprises: a data acquisition module, comprising: a visual sensor, an inertial measurement unit and a TOF sensor arranged on the lawn mower body, respectively used to collect RGB image data corresponding to different times based on a preset frequency, collect lawn point cloud data at each time, and collect position information, posture information and motion information of the lawn mower at each time; a boundary recognition module, connected to the data acquisition module, used to perform boundary recognition on each RGB image data in turn, and perform boundary recognition on the current RGB image data according to the boundary line recognition result of the previous RGB image data. A boundary detection operation or a boundary prediction operation is performed to obtain a corresponding primary two-dimensional lawn boundary line; a boundary optimization module is connected to the boundary recognition module and is used to perform a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and based on the lawn point cloud data at the current moment, a boundary correction operation is performed on the optimized two-dimensional lawn boundary line to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; a boundary control module is connected to the data acquisition module and the boundary recognition module and is used to predict the position information of the lawn mower at the next moment according to the position information, posture information and motion information of the lawn mower at the current moment, and based on the optimal three-dimensional lawn boundary line, determine whether the lawn mower has a risk of crossing the boundary, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

[0015] As described above, the present application has the following beneficial effects: the present application provides a lawn mower lawn boundary control method, a control system and an intelligent lawn mower, which respectively collects RGB image data, lawn point cloud data and position information, posture information and motion information of the lawn mower corresponding to different times through a multi-sensor fusion method based on a visual sensor, a TOF sensor and an inertial measurement unit; and based on the collected various types of data and information, on the one hand, performs boundary recognition and boundary optimization on each RGB image data to obtain the optimal three-dimensional lawn boundary line of each RGB image data, thereby improving the accuracy of the intelligent lawn mower in identifying the lawn boundary; on the other hand, predicts the position information of the lawn mower at the next moment, and determines whether the lawn mower has a risk of crossing the boundary, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary, thereby timely avoiding the risk of crossing the boundary of the intelligent lawn mower and improving the safety of the intelligent lawn mower. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a flow chart of a lawn mower lawn boundary control method according to an embodiment of the present application.

[0017] Figure 2Shown is a schematic diagram of a flow chart of performing a boundary detection operation in an embodiment of the present application.

[0018] Figure 3 Shown is a flowchart of performing a boundary prediction operation in an embodiment of the present application.

[0019] Figure 4 Shown is a schematic diagram of a flow chart of performing a boundary particle filtering operation in an embodiment of the present application.

[0020] Figure 5 Shown is a schematic diagram of a flow chart of performing a boundary line correction operation in one embodiment of the present application.

[0021] Figure 6 Shown is a schematic diagram of a serpentine working trajectory in an embodiment of the present application.

[0022] Figure 7 Shown is a structural schematic diagram of a lawn mower lawn boundary control system in one embodiment of the present application. DETAILED DESCRIPTION

[0023] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0024] In order to solve the problems in the above-mentioned background technology, the present application provides a lawn mower lawn boundary control method, a control system and an intelligent lawn mower, aiming to solve the crossing-boundary problem of existing intelligent lawn mowers. It adopts a multi-sensor fusion method based on visual sensors, inertial measurement units IMU and TOF sensors, fully leveraging the advantages of visual sensors in feature recognition, the real-time advantages of inertial measurement units IMU in motion state monitoring and the precision advantages of TOF sensors in depth measurement. It can provide comprehensive environmental information for the intelligent lawn mower, which not only improves the accuracy of the intelligent lawn mower in identifying lawn boundaries, reduces missed detection and false detection of lawn boundaries, but also enhances the adaptability of the intelligent lawn mower in complex environments, so that the intelligent lawn mower can achieve real-time feedback and autonomous decision-making, optimize its working trajectory and movement speed, and improve overall work efficiency and safety.

[0025] At the same time, in order to make the purpose, technical solutions and advantages of this application more clear, the technical solutions in the embodiments of this application are further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain this application and are not used to limit the invention.

[0026] like Figure 1 FIG. 2 is a flow chart showing a lawn mower lawn boundary control method in an embodiment of the present application. The lawn mower lawn boundary control method in this embodiment mainly includes the following steps.

[0027] Step S1: Based on the preset frequency, the visual sensor is used to collect the RGB image data corresponding to different times, and the TOF sensor is used to obtain the lawn point cloud data at each time, and the inertial measurement unit is used to obtain the position information, posture information and motion information of the lawn mower at each time.

[0028] It should be understood that a visual sensor is a device that can simulate the human eye to observe and analyze images. It captures light in the environment and converts it into digital signals to obtain corresponding RGB image data.

[0029] TOF (Time of Flight) sensor is a device that uses light or electromagnetic waves to measure the distance between the sensor and the object. By calculating the time required for the sensor to transmit light to the lawn and return, the distance between the lawn and the sensor is determined, thereby obtaining lawn point cloud data. It has the advantages of high precision, fast response and adaptability to a variety of environments.

[0030] The inertial measurement unit (IMU) is a device that integrates multiple sensors, mainly including accelerometers and gyroscopes, which are used to measure and detect the motion information of the lawn mower, such as linear acceleration and angular velocity, and calculate the position information and attitude information of the lawn mower. Among them, the position information of the lawn mower includes: the three-dimensional coordinate values ​​of the lawn mower (robot_x, robot_y and robot_z); the attitude information of the lawn mower includes: the yaw angle yaw of the lawn mower around the z-axis, the pitch angle pitch around the y-axis, and the roll angle roll around the x-axis.

[0031] It should be noted that the data collected by each sensor is mainly based on its own coordinate system. In order to ensure the accuracy of the lawn boundary line fitted by multi-sensor fusion technology, it is necessary to obtain the conversion relationship between the coordinate systems of each sensor and spatially align each sensor to ensure the consistency of data from different sensors in space and time.

[0032] Preferably, the preset frequency is 100 Hz, and data is collected 100 times every 1 second, that is, RGB image data, lawn point cloud data, and position information, posture information, and motion information of the lawn mower are collected once every 10 ms. It should be noted that the frequency of data collection is not limited in this application.

[0033] Step S2: performing boundary recognition on each RGB image data in turn, and performing boundary detection operation or boundary prediction operation on the current RGB image data according to the boundary line recognition result of the previous RGB image data, to obtain the corresponding primary two-dimensional lawn boundary line.

[0034] Specifically, step S2 includes: judging whether the previous RGB image data has identified a lawn boundary line according to the boundary line recognition result of the previous RGB image data; if the previous RGB image data has not identified the lawn boundary line, performing a boundary detection operation on the current RGB image data to obtain a corresponding primary two-dimensional lawn boundary line; if the previous RGB image data has identified the lawn boundary line, performing a boundary prediction operation on the current RGB image data to obtain a corresponding primary two-dimensional lawn boundary line.

[0035] For example, the first RGB image data is collected at the 0th second, a boundary detection operation is performed on the first RGB image data, and a boundary line recognition result of the first RGB image data is determined; if the first RGB image data does not recognize the lawn boundary line, a boundary detection operation is performed on the second RGB image data collected at the 10th ms; if the first RGB image data has recognized the lawn boundary line, a boundary prediction operation is performed on the second RGB image data; and then the boundary line recognition result of the second RGB image data is determined to perform a boundary detection operation or a boundary prediction operation on the third RGB image data collected at the 20th ms, and so on.

[0036] In one embodiment, if Figure 2 As shown, when the lawn boundary line is not identified in the previous RGB image data, the boundary detection operation is performed on the current RGB image data. It should be noted that when the lawn mower performs the lawn mowing task, it is carried out step by step, and the position of the lawn boundary line cannot change suddenly when it moves to the lawn boundary line. Based on the preset 100Hz frequency, when the lawn boundary line is not identified in the previous RGB image data, the boundary detection operation performed on the current RGB image data can detect the lawn boundary line position in time, and change the working trajectory in time when there may be a risk of crossing the boundary at the next moment.

[0037] In this embodiment, the specific method of performing the boundary detection operation on the current RGB image data includes the following steps.

[0038] Step S21: performing distortion correction on the current RGB image data, and extracting the lower half area of ​​the corrected RGB image data as the current lawn boundary RGB image data.

[0039] Preferably, the resolution of the current RGB image data is set to 640×360 to reduce the processing delay and computational burden of the image data and keep the details of the boundary detection clear enough. Then, the current RGB image data is calibrated using the camera internal parameters to perform distortion correction to improve the image quality and lay a good foundation for subsequent boundary line fitting.

[0040] The lawn mower walks on the lawn to perform lawn mowing tasks. Since the lawn area is basically located in the lower half of each RGB image data, in order to reduce the impact of noise and the time-consuming algorithm processing, the lower half of the image is extracted as the current lawn boundary RGB image data for the next step of denoising and smoothing noise processing, which helps to improve the accuracy and efficiency of obtaining the primary two-dimensional lawn boundary line.

[0041] Step S22: removing high-frequency noise from the current lawn boundary RGB image data, and performing noise smoothing processing on the denoised current lawn boundary RGB image data to obtain corresponding current lawn boundary optimal RGB image data.

[0042] Specifically, Gaussian filtering technology is used to remove high-frequency noise in the current lawn boundary RGB image data, and the denoised current lawn boundary RGB image data is smoothed by noise processing, while still retaining image edge information, so as to improve the robustness of the obtained current lawn boundary optimal RGB image data.

[0043] It should be understood that Gaussian filtering technology refers to using a Gaussian function as a weight coefficient to convolve a specified image to reduce image noise and details. Gaussian filtering is based on Gaussian distribution, and its mathematical expression is: Wherein, x and y are the coordinates from the center of the Gaussian kernel; σ is the standard deviation, which is used to control the width of the Gaussian kernel. In a preferred embodiment, the Gaussian kernel size of the Gaussian function can be set to 5 and the standard deviation to 1.5 to smooth the noise.

[0044] Step S23: performing color gamut conversion on the optimal RGB image data of the current lawn boundary to generate corresponding HSV image data of the current lawn boundary, and performing image segmentation on the HSV image data of the current lawn boundary according to the lawn hue threshold obtained by the test to obtain a corresponding lawn area mask.

[0045] In a specific embodiment, the method of converting the current lawn boundary optimal RGB image data from the RGB color gamut to the HSV color gamut includes: normalizing the RGB value of each pixel in the current lawn boundary optimal RGB image data, and calculating the HSV value of each pixel, thereby generating corresponding current lawn boundary HSV image data.

[0046] It should be understood that the HSV color gamut refers to a color model that represents colors as points in a three-dimensional space, including hue, saturation, and value.

[0047] Preferably, according to the test, the hue range of the lawn is 35-85, therefore, the lawn hue threshold can be set according to the range, and the current lawn boundary HSV image data is segmented to identify the target lawn area with a specific color, and obtain a lawn area mask. The lawn area mask is usually a binary image or Boolean image of the same size as the original image, in which the selected lawn area is marked as 1 (or True), and the remaining areas are marked as 0 (or False), which can accurately mark the lawn area.

[0048] Step S24: performing an erosion operation and a dilation operation on the lawn region mask, identifying and marking connected domains in the lawn region mask, removing isolated noise regions, and obtaining a corresponding optimal lawn region mask.

[0049] Performing an erosion operation on the lawn area mask will erode the foreground area in the image by removing pixels where the structural element does not completely cover the foreground area. The erosion operation can shrink the lawn boundary in the lawn area mask, remove small protrusions on the lawn boundary, and reduce the irregularity of the lawn boundary.

[0050] Performing a dilation operation on the lawn region mask grows the foreground region in the image by sliding the structure element across the image and increasing the pixel value when the structure element completely covers the foreground region. The dilation operation can cause the lawn boundary in the lawn region mask to expand outward, fill in small holes and cracks near the lawn boundary, and thus smooth the lawn boundary.

[0051] In a specific embodiment, the method of identifying and marking connected domains in the lawn area mask includes: using a connected domain analysis method, marking adjacent pixel regions with the same pixel values ​​in the binary image of the lawn area mask as connected domains, and proposing remaining isolated noise regions with smaller areas, so as to further optimize the quality of the lawn area mask.

[0052] Step S25: extracting a plurality of lawn boundary points in the optimal lawn area optimal mask, and constructing a lawn boundary line equation based on each lawn boundary point to fit a primary two-dimensional lawn boundary line of the current RGB image data.

[0053] In a preferred embodiment, based on the lawn boundary line equation, the primary two-dimensional lawn boundary line is fitted to each extracted lawn boundary point by using a random sampling consensus algorithm (RANSAC).

[0054] The Random Sampling Consensus Algorithm (RANSAC) is an iterative method for estimating the parameters of a mathematical model from a data set containing outliers. The basic idea involves randomly extracting a subset of the data, using this subset to estimate the parameters of the model, and then verifying whether other data points also conform to this model. The above process is repeated many times to finally select the optimal model, thereby reducing the impact of noise points.

[0055] In this embodiment, the model of the random sampling consensus algorithm (RANSAC) is as follows.

[0056]

[0057] Among them, ax i +by i +c is the constructed lawn boundary equation, x i and i is the two-dimensional coordinate value of each lawn boundary point; n is the number of iterations, preferably set to 100; d is the number of internal points, preferably set to 5.

[0058] In one embodiment, step S25 also includes: if multiple lawn boundary points in the optimal mask of the optimal lawn area are extracted, it is determined that the boundary line recognition result of the current RGB image data is that the lawn boundary line has been recognized, and a boundary detection operation should be performed on the next RGB image data; otherwise, it is determined that the boundary line recognition result of the current RGB image data is that the lawn boundary line has not been recognized, and a boundary prediction operation should be performed on the next RGB image data.

[0059] In one embodiment, if Figure 3 As shown, when the lawn boundary line has been identified in the previous RGB image data, the method of performing the boundary prediction operation on the current RGB image data includes the following steps.

[0060] Step S26: Calculate the position and posture change of the lawn mower at the current moment according to the position information, posture information and motion information of the lawn mower at the previous moment and the current moment.

[0061] In one embodiment, the position information, posture information and motion information of the lawn mower at two adjacent moments are dynamically integrated to calculate the posture change of the lawn mower at the current moment based on the previous moment. The posture change includes the rotation change ΔR k and displacement change ΔT k Specifically, ΔR k =ΔR k-1 exp(ω k Δt); Among them, ΔR k is the change in the rotation amount of the lawn mower at the kth moment based on the k-1th moment, ΔR k-1is the change in the rotation of the lawn mower at the k-1th moment based on the k-2th moment; exp() is the exponential mapping of the Lie algebra; ω k is the angular velocity of the lawn mower at the kth moment; ΔT k is the displacement change of the lawn mower at the kth moment based on the k-1th moment, ΔT k-1 is the displacement change of the lawn mower at the k-1th moment based on the k-2th moment; v k-1 is the speed of the lawn mower at the k-1th moment; a k is the acceleration of the lawn mower at the kth moment; Δt is the time interval between the kth moment and the k-1th moment, preferably, Δt=0.01s=10ms is preset.

[0062] Step S27: predict the position of the lawn boundary line in the current RGB image data according to the optimal three-dimensional lawn boundary line of the previous RGB image data and the posture change of the lawn mower at the current moment, and generate the corresponding primary two-dimensional lawn boundary line.

[0063] It should be noted that when collecting RGB image data, lawn point cloud data, and the position information, posture information, and motion information of the lawn mower, the lawn mower is in a dynamic driving process. The lawn boundary line of the existing lawn mower is usually predicted only based on the current position information, so the prediction accuracy is limited and the obtained lawn boundary line position has deviations.

[0064] In this embodiment, the present application performs a boundary prediction operation on the current RGB image data when the lawn boundary line has been identified in the previous RGB image data. The lawn boundary line is jointly fitted based on the lawn boundary line identified in the previous RGB image data, and the current position information and the previous position information of the lawn mower are combined. This can effectively improve the fitting accuracy, ensure the accuracy of the obtained lawn boundary line, and improve the continuity of the detected lawn boundary line.

[0065] In one embodiment, step S27 also includes: if a predicted primary two-dimensional lawn boundary line is obtained, it is determined that the boundary line recognition result of the current RGB image data is that the lawn boundary line has been recognized, and a boundary detection operation should be performed on the next RGB image data; otherwise, it is determined that the boundary line recognition result of the current RGB image data is that the lawn boundary line has not been recognized, and a boundary prediction operation should be performed on the next RGB image data.

[0066] Step S3: performing a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and performing a boundary correction operation on the optimized two-dimensional lawn boundary line based on the lawn point cloud data at the current moment to obtain an optimal three-dimensional lawn boundary line of the current RGB image data.

[0067] In one embodiment, if Figure 4As shown, the method of performing boundary particle filtering operation on the primary two-dimensional lawn boundary line includes the following steps.

[0068] Step S31: Divide the primary two-dimensional lawn boundary line into a plurality of continuous sampling areas, and dynamically sample the boundary line in each sampling area to obtain a plurality of boundary line particles in each sampling area.

[0069] Specifically, when dynamically sampling each sampling area, the sampling range can be adjusted by the density of the boundary points to ensure that the distribution of the boundary line particles covers the potential lawn boundary line area. In a preferred embodiment, 100 boundary line particles are dynamically sampled for each sampling area. It should be noted that the number of sampling areas and the number of sampled boundary line particles in each sampling area are not limited in this application.

[0070] Step S32: Calculate the weight value of each boundary line particle and use a robust optimization algorithm to suppress abnormal boundary line particles.

[0071] In one embodiment, the weight value of each boundary line particle may be calculated using environmental perception factors such as gradient characteristics, color distribution characteristics, and optical flow consistency of image regions on both sides of each boundary line particle.

[0072] Specifically, the calculation formula for the weight value of each boundary line particle is: Among them, exp() is the exponential mapping of Lie algebra, is the gradient magnitude of the image, is the standard deviation of the gradient, d HSV is the HSV color difference of the image, σH is the standard deviation of the hue value in the HSV color, and d flow is the optical flow difference of the image, and σf is the standard deviation of the optical flow.

[0073] At the same time, in this embodiment, a robust optimization algorithm is introduced to suppress abnormal boundary line particles and find the optimal solution in an uncertain environment, thereby ensuring the stability and accuracy of the boundary line particle weight evaluation.

[0074] Step S33: Screen the particles with the highest weight value in each sampling area for fitting to obtain the optimized two-dimensional lawn boundary line of the current RGB image data.

[0075] In one embodiment, if Figure 5 As shown, based on the lawn point cloud data at the current moment, the method of performing a boundary line correction operation on the optimized two-dimensional lawn boundary line includes the following steps.

[0076] Step S34: According to the coordinate conversion relationship between the RGB image data collected by the visual sensor and the TOF data collected by the TOF sensor, the lawn point cloud data at the current moment is projected onto the current RGB image data to obtain corresponding lawn point cloud projection data.

[0077] Step S35: Based on the lawn point cloud projection data, a plurality of lawn boundary points in the optimized two-dimensional lawn boundary line are corrected for geometric consistency to obtain an optimal two-dimensional lawn boundary line of the current RGB image data.

[0078] In one embodiment, the objective function is used Perform geometric consistency correction on each lawn boundary point. i is each lawn boundary point, and l is the optimal two-dimensional lawn boundary line. The objective function is used to minimize the sum of squares of the differences of all lawn boundary points, so that the distance dist(p) from each lawn boundary point to the optimal two-dimensional lawn boundary line is i ,l) and its expected position dist in the lawn point cloud projection data TOF (p i ) is the smallest, thereby ensuring that the optimal two-dimensional lawn boundary line is consistent with the lawn point cloud projection data.

[0079] Step S36: Based on the lawn point cloud data, perform three-dimensional coordinate transformation and three-dimensional boundary line fitting on multiple lawn boundary points in the optimal two-dimensional lawn boundary line to obtain a primary three-dimensional lawn boundary line of the current RGB image data.

[0080] In one embodiment, the method of fitting three-dimensional boundary lines for multiple lawn boundary points after three-dimensional coordinate conversion includes: fitting the three-dimensional boundary lines using an improved random sampling consensus (RANSAC) algorithm based on geometric features, which can use the lawn surface morphology as a constraint condition to further optimize the primary three-dimensional lawn boundary line of the current RGB image data.

[0081] It should be understood that the improved random sampling consensus (RANSAC) algorithm based on geometric features is a method that improves the efficiency of feature matching and outlier removal by utilizing the geometric relationship between points in the image. By measuring the distance and angle between each lawn boundary point in the current RGB image data and its corresponding point in the lawn point cloud data, the matching points can be pre-filtered, thereby increasing the proportion of internal points and ensuring the accuracy of the primary three-dimensional lawn boundary line obtained by fitting. In addition, the algorithm effectively reduces the number of iterations, improves the calculation efficiency, and ensures the speed of fitting the primary three-dimensional lawn boundary line.

[0082] Step S37: Based on the lawn point cloud data, calculate the point cloud distribution density and normal change rate of multiple lawn boundary points in the primary three-dimensional lawn boundary line, and select multiple optimal lawn boundary points according to the point cloud distribution density and normal change rate of each lawn boundary point.

[0083] The point cloud distribution density of lawn boundary points refers to the density of point cloud distribution near the corresponding points of lawn boundary points in the lawn point cloud data, which can accurately reflect the spatial distribution state of lawn boundary points in the lawn point cloud data, and is measured by calculating the number of points within a unit volume.

[0084] The normal change rate is used to describe the degree of change of the normal vector on the surface. It can reflect the local geometric characteristics of the surface, especially in edge detection, such as lawn boundary detection. The normal change rate can be obtained by calculating the difference between the normal vectors of adjacent points.

[0085] Step S38: converting each optimal lawn boundary point from the lawnmower coordinate system to the world coordinate system, and fitting the optimal three-dimensional lawn boundary line of the current RGB image data.

[0086] In a specific embodiment, the method of converting each optimal lawn boundary point from the lawnmower coordinate system to the world coordinate system in step S38 includes the following steps.

[0087] ①Use real-time kinematic positioning technology (RTK) to obtain the position information robot_x and robot_y of the lawn mower in the world coordinate system, and define the translation matrix

[0088] ② Obtain the yaw angle information robot_yaw of the lawn mower in the lawn mower coordinate system through the inertial measurement unit and define the rotation matrix

[0089] ③ According to the defined translation matrix T and rotation matrix R, calculate the position information x_world, y_world and z_world of each optimal lawn boundary point in the world coordinate system; the calculation formula is as follows.

[0090]

[0091] z_world = z_robot;

[0092] Among them, x_robot, y_robot and z_robot are the position information of each optimal lawn boundary point in the lawn mower coordinate system.

[0093] Step S4: predicting the position information of the lawn mower at the next moment based on the position information, posture information and motion information of the lawn mower at the current moment, and judging whether the lawn mower is at risk of crossing the boundary based on the optimal three-dimensional lawn boundary line, so as to change the working trajectory of the lawn mower when the lawn mower is at risk of crossing the boundary.

[0094] In one embodiment, the method for determining whether the lawn mower has a risk of crossing the boundary includes: determining whether a line segment of the lawn mower body intersects with the optimal three-dimensional lawn boundary line; if so, determining that the lawn mower has a risk of crossing the boundary and changing the working trajectory of the lawn mower; if not, determining that the lawn mower has no risk of crossing the boundary and the lawn mower still performs the lawn mowing task along the original working trajectory.

[0095] Specifically, the method of changing the working trajectory of the lawn mower includes the following steps.

[0096] ① According to the position information of the lawn mower at the current moment, determine the current position of the lawn mower and the next index position based on the original working trajectory, and connect these two position points to obtain a straight line L.

[0097] Among them, the original working trajectory is as follows Figure 6 The serpentine route shown. Figure 6 As shown, point A represents the current position of the lawn mower, and point B represents the next index position of the lawn mower based on the original working trajectory.

[0098] ② Calculate the slope of the straight line L and obtain the perpendicular line L′ based on the current position of the lawn mower. That is, the perpendicular line L′ is perpendicular to the straight line L and passes through point A.

[0099] ③ Obtain the intersection of the vertical line L′ and the remaining section of the original working trajectory after the current position, and use the remaining section after the intersection as the new working trajectory. Figure 6 As shown, point C is the intersection of the vertical line L′ and the remaining section of the original working trajectory after the current position, and the new working trajectory is the bold line portion in the figure.

[0100] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0101] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.

[0102] like Figure 7 , which shows a schematic diagram of the structure of a lawn mower lawn boundary control system 700 in an embodiment of the present application. The lawn mower lawn boundary control system 700 includes: a data acquisition module 701, a boundary recognition module 702, a boundary optimization module 703 and a boundary control module 704.

[0103] The data acquisition module 701 includes: a visual sensor, a TOF (Time of Flight) sensor and an inertial measurement unit (IMU), which are used to collect RGB image data corresponding to different times based on a preset frequency, collect lawn point cloud data at each time, and collect position information, posture information and motion information of the lawn mower at each time.

[0104] The boundary recognition module 702 is connected to the data acquisition module 701, and is used to perform boundary recognition on each RGB image data in turn, and according to the boundary line recognition result of the previous RGB image data, perform boundary detection operation or boundary prediction operation on the current RGB image data to obtain the corresponding primary two-dimensional lawn boundary line.

[0105] The boundary optimization module 703 is connected to the boundary recognition module 702, and is used to perform a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and based on the lawn point cloud data at the current moment, perform a boundary correction operation on the optimized two-dimensional lawn boundary line to obtain the optimal three-dimensional lawn boundary line of the current RGB image data.

[0106] The boundary control module 704 is connected to the data acquisition module 701 and the boundary optimization module 703, and is used to predict the position information of the lawn mower at the next moment based on the position information, posture information and motion information of the lawn mower at the current moment, and judge whether the lawn mower has a risk of crossing the boundary based on the optimal three-dimensional lawn boundary line, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

[0107] The present application also provides an intelligent lawn mower. In one embodiment, the intelligent lawn mower includes: a lawn mower body and a lawn mower lawn boundary control system that is communicatively connected to the lawn mower body. The lawn mower lawn boundary control system is a lawn mower lawn boundary control system 700 provided in the above system embodiment, including: a data acquisition module 701, a boundary recognition module 702, a boundary optimization module 703, and a boundary control module 704. The connection relationship and functions of each module are as described in the above embodiment, and for the sake of brevity, they are not repeated here.

[0108] Furthermore, it should be understood that the specific process of each module executing the corresponding steps in the above system embodiment and the intelligent lawn mower embodiment has been described in detail in the above method embodiment, and will not be repeated here for the sake of brevity.

[0109] It should also be understood that the division of modules in the above system embodiments and the intelligent lawn mower embodiments is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated into a processor, or may exist physically separately, or two or more modules may be integrated into one module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0110] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0111] In summary, the present application provides a lawn mower lawn boundary control method, control system and intelligent lawn mower, which respectively collects RGB image data, lawn point cloud data and position information, posture information and motion information of the lawn mower corresponding to different times through a multi-sensor fusion method based on a visual sensor, a TOF sensor and an inertial measurement unit; and according to the various types of data and information collected, on the one hand, the boundary recognition and boundary optimization of each RGB image data are performed to obtain the optimal three-dimensional lawn boundary line of each RGB image data, thereby improving the accuracy of the intelligent lawn mower in identifying the lawn boundary; on the other hand, the position information of the lawn mower at the next moment is predicted to determine whether the lawn mower has a risk of crossing the boundary, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary, thereby timely avoiding the danger of crossing the boundary of the intelligent lawn mower and improving the safety of the intelligent lawn mower. Therefore, the present application effectively overcomes the various shortcomings in the prior art and has a high industrial utilization value.

[0112] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.

Claims

1. A lawn mower lawn boundary control method, characterized in that: include: Based on the preset frequency, the visual sensor collects the corresponding RGB image data at different times, and the TOF sensor obtains the lawn point cloud data at each time, and the inertial measurement unit obtains the position information, posture information and motion information of the lawn mower at each time; Perform boundary recognition on each RGB image data in turn, and according to the boundary line recognition result of the previous RGB image data, perform boundary detection operation or boundary prediction operation on the current RGB image data to obtain the corresponding primary two-dimensional lawn boundary line; Performing a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and performing a boundary correction operation on the optimized two-dimensional lawn boundary line based on the lawn point cloud data at the current moment to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; According to the position information, posture information and motion information of the lawn mower at the current moment, the position information of the lawn mower at the next moment is predicted, and based on the optimal three-dimensional lawn boundary line, it is judged whether the lawn mower has a risk of crossing the boundary, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

2. The lawn mower lawn boundary control method according to claim 1, characterized in that: Methods of performing a boundary detection operation or a boundary prediction operation on the current RGB image data according to the boundary line recognition result of the previous RGB image data include: According to the boundary line recognition result of the previous RGB image data, it is determined whether the previous RGB image data recognizes the lawn boundary line; If the lawn boundary line is not identified in the previous RGB image data, a boundary detection operation is performed on the current RGB image data to obtain a corresponding primary two-dimensional lawn boundary line; If the lawn boundary line has been identified in the previous RGB image data, a boundary prediction operation is performed on the current RGB image data to obtain a corresponding primary two-dimensional lawn boundary line.

3. The lawn mower lawn boundary control method according to claim 2, characterized in that: The methods for performing boundary detection operations on the current RGB image data include: Performing distortion correction on the current RGB image data, and extracting the lower half area of ​​the corrected RGB image data as the current lawn boundary RGB image data; Removing high-frequency noise from the current lawn boundary RGB image data, and performing noise smoothing processing on the denoised current lawn boundary RGB image data to obtain corresponding current lawn boundary optimal RGB image data; The optimal RGB image data of the current lawn boundary is converted into a color gamut to generate corresponding HSV image data of the current lawn boundary, and the HSV image data of the current lawn boundary is segmented according to the lawn hue threshold obtained by the test to obtain a corresponding lawn area mask; Performing erosion and dilation operations on the lawn area mask, identifying and marking connected domains in the lawn area mask, removing isolated noise areas, and obtaining a corresponding optimal lawn area mask; A plurality of lawn boundary points in the optimal lawn area optimal mask are extracted, and a lawn boundary line equation is constructed based on each lawn boundary point to fit a primary two-dimensional lawn boundary line of the current RGB image data.

4. The lawn mower lawn boundary control method according to claim 2, characterized in that: The methods for performing boundary prediction operations on the current RGB image data include: Calculate the position change of the lawn mower at the current moment according to the position information, posture information and motion information of the lawn mower at the previous moment and the current moment; According to the optimal three-dimensional lawn boundary line of the previous RGB image data and the posture change of the lawn mower at the current moment, the position of the lawn boundary line in the current RGB image data is predicted, and the corresponding primary two-dimensional lawn boundary line is generated.

5. The lawn mower lawn boundary control method according to claim 1, characterized in that: The method of performing the boundary particle filtering operation on the primary two-dimensional lawn boundary line includes: The primary two-dimensional lawn boundary line is divided into a plurality of continuous sampling areas, and the boundary line in each sampling area is dynamically sampled to obtain a plurality of boundary line particles in each sampling area; Calculate the weight value of each boundary line particle and use a robust optimization algorithm to suppress abnormal boundary line particles; The particles with the highest weight value in each sampling area are selected for fitting to obtain the optimized two-dimensional lawn boundary line of the current RGB image data.

6. The lawn mower lawn boundary control method according to claim 1, characterized in that: Based on the lawn point cloud data at the current moment, a method of performing a boundary line correction operation on the optimized two-dimensional lawn boundary line includes: According to the coordinate conversion relationship between the RGB image data collected by the visual sensor and the TOF data collected by the TOF sensor, the lawn point cloud data at the current moment is projected onto the current RGB image data to obtain the corresponding lawn point cloud projection data; Based on the lawn point cloud projection data, a plurality of lawn boundary points in the optimized two-dimensional lawn boundary line are corrected for geometric consistency to obtain an optimal two-dimensional lawn boundary line for the current RGB image data; Based on the lawn point cloud data, a plurality of lawn boundary points in the optimal two-dimensional lawn boundary line are subjected to three-dimensional coordinate transformation and three-dimensional boundary line fitting to obtain a primary three-dimensional lawn boundary line of the current RGB image data; Based on the lawn point cloud data, the point cloud distribution density and the normal change rate of multiple lawn boundary points in the primary three-dimensional lawn boundary line are calculated, and multiple optimal lawn boundary points are screened according to the point cloud distribution density and the normal change rate of each lawn boundary point; Each optimal lawn boundary point is converted from the lawnmower coordinate system to the world coordinate system, and the optimal three-dimensional lawn boundary line of the current RGB image data is fitted.

7. The lawn mower lawn boundary control method according to claim 6, wherein the method of converting each optimal lawn boundary point from the lawn mower coordinate system to the world coordinate system comprises: The position information robot_x and robot_y of the lawn mower in the world coordinate system is obtained through real-time dynamic positioning technology, and the translation matrix is ​​defined The yaw angle information robot_yaw of the lawn mower in the lawn mower coordinate system is obtained through the inertial measurement unit, and the rotation matrix is ​​defined According to the defined translation matrix T and rotation matrix R, the position information x_world, y_world and z_world of each optimal lawn boundary point in the world coordinate system are calculated; wherein the calculation formula is: z_world = z_robot; Among them, x_robot, y_robot and z_robot are the position information of each optimal lawn boundary point in the lawn mower coordinate system.

8. The lawn mower lawn boundary control method according to claim 1, wherein: Ways to change the working trajectory of the lawn mower include: According to the position information of the lawn mower at the current moment, determine the current position of the lawn mower and the next index position based on the original working trajectory, and connect these two position points to obtain a straight line; wherein the original working trajectory is a serpentine route; Calculate the slope of the straight line and obtain the perpendicular line of the straight line based on the current position of the lawn mower; The intersection of the vertical line and the remaining section of the original working trajectory after the current position is obtained, and the remaining section after the intersection is used as the new working trajectory.

9. A lawn mower lawn boundary control system, characterized in that: include: The data acquisition module includes: a visual sensor, a TOF sensor and an inertial measurement unit, which are used to collect RGB image data corresponding to different times based on a preset frequency, collect lawn point cloud data at each time, and collect position information, posture information and motion information of the lawn mower at each time; A boundary recognition module, connected to the data acquisition module, is used to perform boundary recognition on each RGB image data in turn, and according to the boundary line recognition result of the previous RGB image data, perform boundary detection operation or boundary prediction operation on the current RGB image data respectively to obtain the corresponding primary two-dimensional lawn boundary line; A boundary optimization module, connected to the boundary recognition module, is used to perform a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and based on the lawn point cloud data at the current moment, perform a boundary correction operation on the optimized two-dimensional lawn boundary line to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; The boundary control module is connected to the data acquisition module and the boundary optimization module, and is used to predict the position information of the lawn mower at the next moment according to the position information, posture information and motion information of the lawn mower at the current moment, and to judge whether the lawn mower has a risk of crossing the boundary based on the optimal three-dimensional lawn boundary line, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

10. An intelligent lawn mower, characterized in that: include: A lawn mower body and a lawn mower lawn boundary control system communicatively connected to the lawn mower body; Wherein, the lawn mower lawn boundary control system comprises: The data acquisition module includes: a visual sensor, an inertial measurement unit and a TOF sensor arranged on the lawn mower body, which are respectively used to collect RGB image data corresponding to different times based on a preset frequency, collect lawn point cloud data at each time, and collect position information, posture information and motion information of the lawn mower at each time; A boundary recognition module, connected to the data acquisition module, is used to perform boundary recognition on each RGB image data in turn, and according to the boundary line recognition result of the previous RGB image data, perform boundary detection operation or boundary prediction operation on the current RGB image data respectively to obtain the corresponding primary two-dimensional lawn boundary line; A boundary optimization module, connected to the boundary recognition module, is used to perform a boundary particle filtering operation on the primary two-dimensional lawn boundary line to obtain an optimized two-dimensional lawn boundary line of the current RGB image data, and based on the lawn point cloud data at the current moment, perform a boundary correction operation on the optimized two-dimensional lawn boundary line to obtain an optimal three-dimensional lawn boundary line of the current RGB image data; The boundary control module is connected to the data acquisition module and the boundary optimization module, and is used to predict the position information of the lawn mower at the next moment according to the position information, posture information and motion information of the lawn mower at the current moment, and to judge whether the lawn mower has a risk of crossing the boundary based on the optimal three-dimensional lawn boundary line, so as to change the working trajectory of the lawn mower when the lawn mower has a risk of crossing the boundary.

Images