Intelligent mower operation control method, collaborative operation control method and medium

Abstract

The invention discloses an intelligent mower operation control method, a collaborative operation control method and a medium, the intelligent mower operation control method comprises the following steps: after an intelligent mower enters an operation area from a boundary, extracting a boundary contour track point of the operation area in a field of view range according to a collected image; performing curve fitting to obtain a reference boundary; shifting the reference boundary by an operation radius to serve as an expected path in the current vision field range; tracking the expected path to obtain a map of the current vision field range; when the intelligent mower enters the next vision field range, the maps are spliced, and a round of mowing operation is completed; and the intelligent mower is controlled to travel towards the center of the operation area by one operation radius distance, and a round of mowing operation is executed again until the mowing task of the whole operation area is completed. The method has the advantages of being simple in implementation method, low in cost, high in control efficiency and precision, high in anti-interference performance and flexibility, wide in application range and the like.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent lawn mowers, and in particular to an intelligent lawn mower operation control method, a collaborative operation control method and a medium. Background Art

[0002] The traditional way of mowing grass relies on manual operation, which is not only inefficient, but also not suitable for operation in bad weather conditions or uneven terrain. Manual mowing grass often cannot guarantee the uniformity and efficiency of mowing grass. With the continuous advancement of sensor technology, path planning algorithms and wireless communication technology, the development of smart lawn mowers has also achieved remarkable results. By integrating a variety of sensors, smart lawn mowers can perceive the environmental information of the lawn, such as terrain, obstacles, etc. in real time. At the same time, using advanced path planning algorithms, smart lawn mowers can autonomously plan mowing routes to achieve efficient and uniform mowing operations.

[0003] However, most of the smart lawn mowers in the prior art need to be equipped with expensive RTK (Real-Time Kinematic) modules to achieve precise positioning and complete path planning based on RTK positioning data, which will not only greatly increase the equipment cost and maintenance cost, but also be unfavorable for the promotion and application of smart lawn mowers. In addition, since RTK equipment relies on satellite signals, its applicable environment is limited and easily affected by environmental factors. For example, satellite signals may be blocked in areas with dense trees, dense buildings or mountainous areas, affecting the accuracy and reliability of RTK positioning, making it impossible for smart lawn mowers to accurately and reliably complete mowing tasks. Summary of the invention

[0004] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention provides an intelligent lawn mower operation control method, a collaborative operation control method and a medium, which have a simple implementation method, low cost, high control efficiency and precision, strong anti-interference and flexibility, and a wide range of applications.

[0005] In order to solve the above technical problems, the technical solution proposed by the present invention is:

[0006] A method for controlling operation of an intelligent lawn mower, wherein the intelligent lawn mower is equipped with a binocular camera, and the steps include:

[0007] After the intelligent lawn mower enters the working area from the boundary, it extracts the boundary contour trajectory points of the working area within the field of view based on the images collected by the binocular camera;

[0008] The fitted line obtained by curve fitting the boundary contour trajectory points within the current field of view is used as the reference boundary;

[0009] The reference boundary is offset toward the center of the working area by an operating radius as the desired path within the current field of view;

[0010] The desired path is tracked while the smart lawn mower is traveling along the desired path, and the error information of the smart lawn mower deviating from the desired trajectory is calculated in real time based on the images collected by the binocular camera. The driving path of the smart lawn mower is controlled and adjusted based on the error information until the mowing task within the current field of view is completed, thereby obtaining a map of the current field of view.

[0011] When the smart lawn mower enters the next field of view, the map of the previous field of view is spliced ​​with the map of the current field of view until the smart lawn mower completes a round of mowing operation after circling the boundary of the entire operation area, and obtains a map of the entire operation area;

[0012] The intelligent lawn mower is controlled to travel an operating radius distance toward the center of the operating area and re-execute a round of mowing operation. In each round of mowing operation, the next round of mowing operation path is obtained by offsetting the previous round of mowing operation path toward the center of the operating area by an operating radius until the mowing task of the entire operating area is completed.

[0013] Furthermore, the step of extracting the boundary contour trajectory points of the working area within a field of view according to the image collected by the binocular camera includes:

[0014] Obtain two images captured by the binocular camera for stereo matching to obtain matching point pairs in the two images;

[0015] Calculate the disparity of each pixel point according to the matching point pairs obtained by stereo matching to obtain a disparity map;

[0016] Using the disparity map, the three-dimensional information of the working area within the current field of view is restored to obtain the three-dimensional point cloud data of the working area within the front field of view;

[0017] Boundary contour trajectory points are extracted from the three-dimensional point cloud data of the working area within the current field of view.

[0018] Furthermore, performing curve fitting on the acquired boundary contour trajectory points within the current field of view includes:

[0019] Perform edge detection on the currently acquired image to obtain a boundary curve;

[0020] The boundary curve is sampled according to a fixed step size d to decompose into a number of points to form a boundary curve sequence point;

[0021] A starting point is randomly selected from the boundary curve sequence points, and a fixed-size window n is selected along the search direction of the boundary curve, where n is the length of the window;

[0022] Starting from the current starting point of the boundary curve according to the current window, a linear fit is performed for each point in the current window using the least square method and the fitting error is calculated;

[0023] Comparing the calculated fitting error with a preset error threshold v;

[0024] If the minimum fitting error is less than the preset error threshold v, the current window size is enlarged to n+1, and then the fitting error is recalculated and compared with the preset error threshold v;

[0025] If the fitting error is greater than or equal to the preset error threshold v, the current point is selected as the segment end point, and a straight line L is obtained from the current starting point to the current segment end point to complete the generation of a segmented straight line. The current segment end point is used as the starting point of the next segmented straight line, and a window of a fixed size of n is selected along the search direction of the boundary curve. Then, the fitting error is calculated again and compared with the preset error threshold v, until all points on the edge curve sequence point are traversed.

[0026] Furthermore, the step of splicing the map of the previous field of view with the map of the current field of view includes:

[0027] Respectively obtain first point cloud data from a current view range map and second point cloud data from a previous view range map;

[0028] Extracting feature points from the first point cloud data and the second point cloud data respectively and generating feature descriptors;

[0029] Matching the feature descriptors corresponding to the first point cloud data and the second point cloud data to obtain a matched feature point pair;

[0030] Registering the first point cloud data with the second point cloud data according to the matched feature points;

[0031] The registered first point cloud data and the second point cloud data are spliced ​​to generate spliced ​​map point cloud data.

[0032] Further, the registering the first point cloud data with the second point cloud data according to the matched feature points includes:

[0033] Performing preliminary registration of the first point cloud data and the second point cloud data using a coarse registration algorithm according to the matched feature point pairs;

[0034] Iteratively optimizing the point cloud data after preliminary registration to adjust the position process between the first point cloud data and the second point cloud data to achieve accurate registration;

[0035] The precise registration results are globally optimized to ensure that the stitching results are globally consistent.

[0036] A method for controlling cooperative operation of multiple intelligent lawn mowers, comprising the following steps:

[0037] Get map information of the operation area;

[0038] If the work area is of regular shape, divide the work area into multiple sub-areas;

[0039] Control multiple intelligent lawn mowers to enter the working area from the boundaries of each sub-area;

[0040] According to the above method, each intelligent lawn mower is controlled to complete the mowing operation in each sub-area.

[0041] Furthermore, if the area is an irregular shape, the improved bio-inspired neural network algorithm is used to plan the driving path of each intelligent lawn mower, and the steps include:

[0042] Rasterize the map and mark the obstacles in the grid;

[0043] The activity value of the obstacle point is the first activity value, and the activity value in the remaining area to be operated is the second activity value, wherein each intelligent lawn mower is regarded as an obstacle to another;

[0044] Initialize the activity values ​​of all neurons to 0, and update the activity values ​​of neurons according to the preset frequency map and the following formula:

[0045]

[0046] Among them, x i is the activity value of the i-th neuron, A, B, and D represent the decay rate of the activity value and the upper and lower values ​​of the neuron activity, respectively. Represents the stimulus input of the neuron, and the calculation expression is:

[0047]

[0048] Among them, w ij is a monotonically decreasing function, It is an inhibitory signal of neurons;

[0049] When the smart lawn mower is at a grid position, it selects the optimal path based on the activity values ​​of the surrounding neurons and the penalty function until the operation in the current area is completed.

[0050] Furthermore, the smart lawn mower selects the next path point at the next moment according to the activity values ​​of the surrounding neurons and the penalty function as follows:

[0051] qn ←x qn =max(G j )

[0052] in:

[0053] G j =ax j +cy j

[0054] y j =1-Δθ j / π

[0055]

[0056] In the formula, a and c are the activity coefficient and turning coefficient respectively, G j is the evaluation function, (x qj ,y qj )(x qc ,y qc )(x qp ,y qp ) are the coordinate positions of the next moment, the current moment, and the previous moment, respectively, Δθ j is the corner, x qn represents the path point at the current moment, q n represents the path point at the next moment, q n ←x qn Represents the path point x from the current moment qn Select the path point q at the next moment n .

[0057] Furthermore, when there are more than two intelligent lawn mowers that select the same point as the next path point, the priority value K of each lawn mower is calculated according to the following formula: i :

[0058] K i =αE i +βT 2i +γP i

[0059] Where E i is the stored energy value of the i-th smart lawn mower, T pi is the amount of work completed by the i-th smart lawn mower, P i is the priority coefficient of the i-th smart lawn mower;

[0060] The smart lawn mower with the largest K value is selected to perform the task at the conflict point, and the remaining smart lawn mowers continue to select the next path point according to the updated active field.

[0061] A computer-readable storage medium storing a computer program, wherein the computer program implements the above method when executed by a processor.

[0062] Compared with the prior art, the advantages of the present invention are:

[0063] 1. The present invention controls the intelligent lawn mower to enter the working area from a certain boundary, obtains the outline at the boundary through visual information, fits the outline in the line of sight into segmented lines to obtain a reference boundary, obtains the expected path based on the reference boundary, and controls the intelligent lawn mower to perform mowing operations. After completing a round of mowing operations, the driving path of the next round of mowing operations is planned according to the driving trajectory of the previous round. No RTK positioning device is required, and the full coverage mowing task can be achieved based on the binocular camera, thereby greatly reducing equipment costs, improving mowing efficiency, and reducing the control complexity of the overall mowing operation.

[0064] 2. The present invention further solves the problems of tortuous and incomplete paths caused by the sudden drop in the activity values ​​of neurons near boundaries and obstacles in the traditional algorithm by combining and improving the traditional bio-inspired neural network algorithm. By utilizing the collaborative operation of multiple machines, each intelligent lawn mower regards each other as an obstacle, and then combines the use of their own improved bio-inspired neural network algorithms, it can efficiently and reliably perform full-coverage mowing tasks, greatly improving the efficiency and coverage of mowing tasks in irregular large areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a schematic diagram of the implementation flow of the intelligent lawn mower operation control method of this embodiment.

[0066] Figure 2 It is a schematic diagram of the principle of generating a reference boundary in a specific application embodiment.

[0067] Figure 3 It is a schematic diagram of the principle of the N*R operation trajectory planned in a specific application embodiment.

[0068] Figure 4 It is a schematic diagram of the principle of achieving obstacle avoidance in a specific application embodiment. DETAILED DESCRIPTION

[0069] The present invention is further described below in conjunction with the accompanying drawings and specific preferred embodiments, but the protection scope of the present invention is not limited thereby.

[0070] As shown in the disclosure of the present invention, unless the context clearly indicates an exception, the words "a", "an", "a kind" and / or "the" do not specifically refer to the singular, but may also include the plural. The words "first", "second" and similar words used in the disclosure of the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. Similarly, the words "include" or "comprise" and the like mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0071] Embodiment 1:

[0072] The present invention does not need to be equipped with RTK positioning modules, laser radar modules and other equipment on the smart lawn mower. It only needs to be equipped with a binocular vision camera on the smart lawn mower. Combined with the image data collected by the binocular vision camera, it can efficiently realize real-time online path planning and complete the full coverage mowing task. Figure 1 As shown, the steps of the intelligent lawn mower operation control method of this embodiment include:

[0073] Step S101: After the intelligent lawn mower enters the working area from the boundary, the boundary contour trajectory points of the working area within the field of view are extracted according to the image collected by the binocular camera.

[0074] In this embodiment, a binocular camera with a binocular camera is pre-configured on the smart lawn mower, and the depth information within the field of view can be obtained by calculation. After the smart lawn mower is started, it enters the working area from a certain boundary, and the outline of the boundary can be obtained through the visual information collected by the binocular camera on the smart lawn mower. Specifically, an image within the field of view of the binocular camera can be obtained at each interval, and then the boundary contour trajectory points of the working area within the field of view can be extracted from the image. For example, an image within a certain field of view is obtained from the starting moment of entering the working area, and the boundary contour trajectory points within the field of view are extracted. After the smart lawn mower has traveled for a specified time, the boundary contour trajectory points within the new field of view are re-extracted.

[0075] In a specific application embodiment, before the binocular camera works, it is necessary to calibrate the binocular camera to determine the internal parameters (such as focal length, principal point, etc.) and external parameters (such as the relative position and direction between cameras) of each camera. After calibration, the distortion of the camera image can be corrected so that the image of the binocular camera remains geometrically consistent.

[0076] In this embodiment, the step of extracting the boundary contour trajectory points of the working area within a field of view according to the image collected by the binocular camera includes:

[0077] Step S111. Stereo matching: obtain two images taken by the binocular camera for stereo matching to obtain matching point pairs in the two images.

[0078] First, stereo matching is performed on the two images taken by the binocular camera. The goal of this process is to find the corresponding two pixel points in the two images to form matching point pairs. These matching point pairs can reflect the positional relationship of the same target object under the perspectives of the two cameras.

[0079] Step S112. Calculate the disparity map: Calculate the disparity of each pixel point based on the matching point pairs obtained by stereo matching to obtain a disparity map.

[0080] Using the matching point pairs obtained through stereo matching, the disparity of each pixel can be calculated, and then a disparity map can be obtained. A disparity map is a two-dimensional image in which the grayscale value of each pixel represents the disparity of the point in the left and right images. The disparity is inversely proportional to the distance from the object to the camera, that is, the larger the disparity, the closer the object is to the camera.

[0081] Step S113. 3D reconstruction: Use the disparity map to restore the 3D information of the working area within the current field of view, and obtain the 3D point cloud data of the working area within the front field of view.

[0082] By using the disparity map and camera calibration parameters, the three-dimensional information of the objects in the scene can be restored through the principle of triangulation. In this process, the three-dimensional point cloud data of the object can be obtained, which contains the shape and position information of the object. In this embodiment, the three-dimensional point cloud data of the working area within the front field of view can be restored by using the disparity map.

[0083] After the 3D point cloud data is restored, the data can be further preprocessed, such as removing noise, outliers, or unnecessary points.

[0084] Step S114. Boundary contour extraction: extract boundary contour trajectory points from the three-dimensional point cloud data of the working area within the current field of view.

[0085] After obtaining the three-dimensional point cloud data, the boundary contour of the object can be extracted through a point cloud processing algorithm (such as a curvature-based method, a region growing-based method, etc.). The boundary contour is expressed as a series of points or line segments in three-dimensional space, that is, the boundary contour trajectory points are obtained.

[0086] by Figure 2 As an example, the dotted line in the figure represents the point cloud trajectory of the extracted boundary contour. When the smart lawn mower starts working, the point cloud information of the boundary contour within the field of view can be extracted from the image collected by the binocular camera at the initial time t0, such as Figure 2 As shown by the dotted line in (a), at time t1 and t2, the images collected by the binocular camera are respectively obtained to extract the point cloud information of the boundary contour within the corresponding field of view, and the point cloud information of the boundary contour within the corresponding field of view is respectively obtained. Figure 2 The dotted lines in (b) and (c).

[0087] Step S102: Perform curve fitting on the acquired boundary contour trajectory points within the current field of view to obtain a fitting line as a reference boundary.

[0088] By curve fitting the boundary contour trajectory points extracted in step S101, a fitting line can be obtained, and the fitting line is used as the reference boundary of the current field of view. During the driving process of the intelligent lawn mower, the reference boundaries of different field of view can be obtained by continuously extracting boundary contour trajectory points for curve fitting.

[0089] For the extracted boundary contours, their geometric characteristics can be further analyzed. In this embodiment, the obtained boundary contour trajectory point cloud information within the current field of view can be specifically subjected to curve fitting by a segmented fitting method, and the steps include: dividing the extracted boundary contour trajectory points into multiple subsets according to the local characteristics of the curve, each subset corresponding to a fitting to form a straight line segment, and splicing the straight line segments corresponding to each subset to obtain the final fitting line.

[0090] Optionally, a sliding window method can be used to obtain a segmented subset, and a least square method can be used to obtain a fitting line. The specific steps are as follows:

[0091] Step S121. Perform edge detection on the currently acquired image using an edge detection algorithm (such as Canny edge detection or Sobel operator, etc.) to obtain a boundary curve;

[0092] Step S122. Sampling the boundary curve according to a fixed step length d to decompose it into a number of points, forming a boundary curve sequence point {P1, P2, ..., PN}, where N is the number of points on the boundary curve;

[0093] Step S123. Randomly select a starting point on the boundary curve sequence {P1, P2, ..., PN}, and select a window of a fixed size n along the search direction of the boundary curve;

[0094] Step S124. Starting from the current starting point of the boundary curve in the current window, for each point (x1, y1), (x2, y2), ..., (xn, yn) in the current window, use the least squares formula to perform linear fitting and calculate the fitting error;

[0095] Specifically, the mean square error (MSE) can be calculated by using the following formula:

[0096]

[0097] In order to minimize the fitting error, the above formula is differentiated to determine the optimal parameters a and b values ​​of the fitting line. The fitting error is calculated using the obtained optimal parameters a and b values, that is, the fitting error can be obtained.

[0098] It is understandable that other methods may be used to calculate the fitting error according to actual needs.

[0099] Step S125. Compare the calculated fitting error with a preset error threshold v;

[0100] Step S126. If the minimum fitting error is less than the preset error threshold v, the current window size is enlarged to n+1, that is, one more point is added, and the process returns to step S114 to recalculate the fitting error and compare it with the preset error threshold v; otherwise, the process proceeds to step S117;

[0101] Step S127. Select the current point as the end point of the segment, obtain a straight line L from the current starting point to the current end point of the segment, complete the generation of a segmented straight line, and use the current end point of the segment as the starting point of the next segmented straight line. Select a window of a fixed size of n along the search direction of the boundary curve, return to execute step S114, and end until all points on the edge curve sequence point have been traversed.

[0102] Assume that the working area is an irregular curve. Figure 2 For example, the above method is used to perform segmented fitting on the extracted boundary contour trajectory points (shown by the dotted line) to obtain an approximate fitting line (shown by the solid line). The approximate fitting line is the obtained reference boundary, which is formed by connecting several segmented lines.

[0103] Step S103: offset the reference boundary toward the center of the working area by an operating radius as the expected path within the current field of view.

[0104] After the smart lawn mower enters the working area from the boundary, it starts to perform the first round of operations around the boundary of the working area. During the first round of operations, the fitted reference boundary is offset by an operating radius toward the center of the working area as the expected path, so that a round of operations can be completed around the boundary of the working area with the help of visual information.

[0105] Specifically, assuming that the operating radius of the intelligent lawn mower is R, the initial point is the starting point of the lawn mower, and the direction of the straight line containing the initial point (starting point) in the fitting curve obtained by the above curve fitting is the heading of the lawn mower, that is, a straight line containing the starting point is taken, so that the heading of the lawn mower is parallel to the straight line and the distance is R, and then the reference boundary obtained by the curve fitting is offset by a distance of R in the direction of the device, and the expected path and expected heading angle of the lawn mower in the field of view can be obtained. The edge curve (reference boundary) can be divided into several straight lines through the curve fitting of step S102, and a straight line containing the starting point is selected so that the heading of the lawn mower is parallel to the straight line and the distance is R, that is, the initial position of the lawn mower is determined, and then the lawn mower is driven parallel to the straight line direction and maintains a distance of R until the next divided straight line in the reference boundary.

[0106] For example, assuming that a boundary curve is composed of points P1 to Pn, in the straight line fitting stage, assuming that points P1-P10 are divided into straight line L1, P11-P30 are divided into straight line L2, and P31-Pn are divided into straight line L3, then the directions of the three straight lines can be determined, that is, the slope; if the current starting point is included in one of the straight lines, for example, the starting point is point P3 (on straight line L1), then the heading of the lawn mower can be set according to the slope of straight line L1 (parallel to the slope).

[0107] Step S104. Track the desired path while the smart lawn mower is traveling along the desired path, calculate the error information of the smart lawn mower deviating from the desired trajectory in real time based on the images collected by the binocular camera, and control and adjust the driving path of the smart lawn mower based on the error information until the mowing task within the current field of view is completed, thereby obtaining a map of the current field of view.

[0108] Since there is no RTK module, the location information of the device cannot be obtained, and there may be idling and slipping during the movement. In this embodiment, the binocular camera can be used to obtain the error information of the lawn mower deviating from the desired trajectory during actual operation. The control based on error feedback can realize the tracking of the desired trajectory, and the smart lawn mower can be accurately controlled to complete the mowing task along the reference boundary.

[0109] Step S105. When the smart lawn mower enters the next field of view, the map of the previous field of view is spliced ​​with the map of the current field of view until the smart lawn mower completes a round of mowing operation after going around the boundary of the entire working area, thereby obtaining a map of the entire working area.

[0110] Within each field of view, a reference boundary can be obtained by fitting. After a desired path is determined using the reference boundary, the smart lawn mower can obtain a map of the field of view after completing the operation along the desired path. Then, the map of the previous field of view is continuously spliced ​​with the map of the current field of view. After the smart lawn mower completes a circle of mowing operations around the boundary, the map of the entire operating area can be spliced.

[0111] In this embodiment, the step of splicing the map of the previous field of view with the map of the current field of view specifically includes:

[0112] Step S151. Data preparation and preprocessing: respectively obtain the first point cloud data from the current field of view map and the second point cloud data from the previous field of view map.

[0113] Collect point cloud data from different maps. Preprocess the point cloud data, such as removing noise, filtering and smoothing, and ground removal, to improve the quality of the point cloud and reduce interference in the matching process, ensuring that the format and quality of the data meet the splicing requirements.

[0114] Step S152. Feature extraction and description: extract feature points from the first point cloud data and the second point cloud data respectively and generate feature descriptors.

[0115] Representative feature points are extracted from each point cloud data. The feature points can be based on geometric shapes or extracted based on other algorithms (such as SIFT, SURF, etc.). The extracted feature points are then described to generate feature descriptors for subsequent feature matching.

[0116] Step S153: Match the feature descriptors corresponding to the first point cloud data and the second point cloud data to obtain matching feature point pairs.

[0117] Step S154. Registering the first point cloud data with the second point cloud data according to the matched feature points;

[0118] Step S155: splicing the registered first point cloud data and the second point cloud data to generate spliced ​​map point cloud data.

[0119] In a specific application embodiment, the above step S154 may divide the registration process into two stages: coarse registration and precise registration to ensure the registration accuracy. The steps may include:

[0120] Step S1541. Perform preliminary registration of the first point cloud data and the second point cloud data using a coarse registration algorithm according to the matched feature point pairs;

[0121] Step S1542: iteratively optimizing the point cloud data after preliminary registration to adjust the position process between the first point cloud data and the second point cloud data to achieve accurate registration;

[0122] Step S1543: Globally optimize the result after precise registration so that the stitching result is consistent in the global range.

[0123] For example, you can first use a feature matching algorithm to match feature descriptors from different point clouds to generate feature point pairs; based on the matched feature point pairs, use a coarse registration algorithm (such as the RANSAC algorithm combined with the ICP algorithm) to perform preliminary registration to roughly align different point clouds. On the basis of coarse registration, use a precise registration algorithm (such as the ICP algorithm) for iterative optimization to further adjust the positional relationship between point clouds to make them match more accurately. In the precise registration process, by minimizing the distance between point pairs or other metrics, the optimal transformation matrix is ​​found so that the corresponding points between different point clouds overlap as much as possible. Taking into account the contextual information of the entire scene, global optimization is performed to ensure the consistency of the stitching results on a global scale. Finally, the precisely registered point cloud data is stitched to generate complete multi-map point cloud data.

[0124] To further ensure the stitching accuracy, the stitched point cloud data can also be post-processed, such as smoothing and filtering, to further improve the data quality. The quality of the stitching results can also be evaluated, such as checking the continuity, smoothness and color consistency of the stitching area, to further improve the stitching quality.

[0125] Step S106. Control the intelligent lawn mower to travel an operating radius distance toward the center of the operating area and re-execute a round of mowing operation. In each round of mowing operation, the next round of mowing operation path is obtained by offsetting the previous round of mowing operation path toward the center of the operating area by an operating radius until the mowing task of the entire operating area is completed.

[0126] After completing the first round of operation along the boundary of the operation area according to steps S101 to S105, the intelligent lawn mower is controlled to travel an operation radius distance toward the center of the operation area, and then the mowing operation path of the first round of operation (that is, the reference boundary is offset by an operation radius) is offset by an operation radius toward the center of the operation area as the expected path for the second round of mowing operation. The intelligent lawn mower completes the second round of mowing operation according to the expected path, and after completion, it travels another operation radius distance toward the center of the operation area and restarts the third round of operation. The above steps are repeated until the mowing task of the entire operation area is completed.

[0127] In a specific application embodiment, the path trajectory of the lawn mower is a cattle-plowing trajectory. Through the boundary information fed back by the camera and combined with its own operating radius R, an operating trajectory parallel to the boundary and N*R away from the boundary is designed, where N is the number of round trips parallel to the boundary. In the first round of mowing, the lawn mower continuously fits and generates a reference boundary and a desired path in the new horizon to obtain a map within the current horizon, until the next map horizon, the map of the current horizon is spliced ​​with the map of the previous horizon, and the above steps are repeated. When the lawn mower goes around the operating area, the map information of the entire area can be spliced, such as Figure 2 As shown in (d), the first round of operation tasks are completed, and then the mowing equipment moves inward a distance of 2R, which is twice the distance of the operation radius, and restarts a new round of operations until the mowing task of the area is completed. Figure 3 shown.

[0128] Furthermore, when there is an obstacle on the path, save the current azimuth and working line distance, mark the heading angle at this time, and use the bug2 algorithm to perform obstacle avoidance operations until the heading angle is consistent with the original heading angle and leaves the obstacle area. Figure 4 As shown in the figure, the obstacle avoidance trajectory planning of the bug2 algorithm is similar to the path planning method around the map boundary, both of which are bypassing along the fitted straight line of the point cloud trajectory. After bypassing the obstacle, it returns to the previous mode of azimuth and working line spacing.

[0129] The present invention controls the intelligent lawn mower to enter the working area from a certain boundary, obtains the outline at the boundary through visual information, fits the outline in the line of sight into segmented segments to obtain a number of connected straight line segments, uses the segment as the reference boundary of the direction, and combines the depth information obtained in the camera to obtain the coordinate information under the reference boundary. The lawn mower travels to the next reference boundary with a fixed row spacing in the direction, then switches the reference reference coordinates, and re-executes the working task in the direction of the segment. During the travel process, the control input of the lawn mower is adjusted in real time using the error feedback signal to avoid the lawn mower from slipping and idling. No RTK positioning device is required, and only the binocular camera carried by the lawn mower can be used to complete the acquisition of the boundary features of the mowing area, the planning of the path, the avoidance of obstacles, etc., so as to achieve full coverage of mowing tasks, thereby greatly reducing the cost of the equipment, reducing the complexity of the overall operation, and improving the mowing efficiency.

[0130] Embodiment 2:

[0131] For large lawn operation environments, the operating efficiency of a single device is low, the operating efficiency of the mower will be limited by its own performance, and a single mower is difficult to cope with complex and changeable lawn environments. The collaborative operation mode can achieve higher operating efficiency and better operating results through communication, collaborative planning, and obstacle avoidance between multiple mowers. The use of multi-machine collaboration combined with efficient path planning can greatly improve operating efficiency and achieve efficient operation in large-area operating spaces.

[0132] The steps of the multi-intelligent lawn mower collaborative operation control method of this embodiment include:

[0133] Get map information of the operation area;

[0134] If the work area is of regular shape, divide the work area into multiple sub-areas;

[0135] Control multiple intelligent lawn mowers to enter the working area from the boundaries of each sub-area;

[0136] According to the method of Example 1, each intelligent lawn mower is controlled to complete the mowing operation in each sub-area.

[0137] The above-mentioned regular shape refers to a regular shape that can be divided into a square, a circle, etc., such as a large square or a large circle, or a combination of a square and a circle.

[0138] The present embodiment can cover a larger area through the collaborative operation of multiple lawn mowers, significantly improve work efficiency, and reduce the time required to complete the entire lawn mowing. The collaborative operation of multiple lawn mowers can ensure that the lawn is mowed more evenly. Different lawn mowers can cooperate with each other during operation to avoid uneven mowing caused by the operation of a single lawn mower, making the lawn look more beautiful. The collaborative operation of multiple machines can also reduce the load of a single lawn mower and reduce energy consumption. At the same time, by reasonably arranging the number of lawn mowers and the operating area, it can also save labor costs and reduce maintenance costs, thereby maximizing cost-effectiveness. The collaborative operation of multiple machines can also cope with complex terrain, such as slopes, hills, etc. Different lawn mowers can be flexibly adjusted according to the characteristics of the terrain to ensure mowing quality and efficiency.

[0139] Since Example 1 completes the mowing operation by identifying the boundary and then following the N*R operation trajectory, it is suitable for planning the regular shape area of ​​the N*R operation trajectory. For irregular areas, there will be areas that cannot be completely covered in this way. For this type of irregular area, if the area is an irregular shape, this embodiment further adopts an improved biologically inspired neural network algorithm to plan the driving path of each intelligent lawn mower, combined with the improved biological neural network algorithm to plan the paths of multiple intelligent lawn mowers, each grid map is regarded as a neuron with an activity value that is updated at any time, and the steps include:

[0140] Step S201. Rasterize the map and mark obstacles in the grid;

[0141] Step S202: The activity value of the obstacle point is the first activity value, and the activity value in the remaining area to be operated is the second activity value, wherein each intelligent lawn mower regards each other as an obstacle;

[0142] Step S203. Initialize the activity values ​​of all neurons to 0, and update the activity values ​​of neurons according to the preset frequency map and the following formula:

[0143]

[0144] Among them, x iis the activity value of the i-th neuron, A, B, and D represent the decay rate of the activity value and the upper and lower values ​​of the neuron activity, respectively. Represents the stimulus input of the neuron, and the calculation expression is:

[0145]

[0146] Among them, w ij is a monotonically decreasing function, It is an inhibitory signal for neurons.

[0147] The above formula (2) calculates the stimulus input of the neuron by adopting the method of averaging the signal in the neighborhood. Compared with the traditional direct summation method, it can avoid the problem that the narrow channel is not regarded as a passable area.

[0148] Step S204: When the intelligent lawn mower is at a grid position, the next path point is determined according to the activity values ​​and turning coefficients of the surrounding neighborhood points until the operation in the current area is completed.

[0149] This embodiment improves the traditional bio-inspired neural network algorithm to solve the problems of tortuous and incomplete paths caused by the sudden drop in the activity values ​​of neurons near boundaries and obstacles in the traditional algorithm. By utilizing multi-machine collaborative operation, each intelligent lawn mower regards each other as an obstacle, and then combines the use of their own improved bio-inspired neural network algorithms to efficiently and reliably perform full-coverage mowing tasks, greatly improving the efficiency and coverage of mowing tasks in irregular large areas.

[0150] The target point in the map transmits the activity value to the node in the topological space, forming a different activity field for each node. The robot takes a node in the neighborhood as the next waypoint according to a certain rule, and moves to the target position. The changes in the surrounding environment are directly reflected in the activity field. In the initial stage, the activity values ​​of all neurons are 0. The map is refreshed at a certain frequency to update the activity values ​​of neurons. In this embodiment, it is assumed that each lawn mower is randomly placed within the operating range. To ensure operating efficiency, it is preferred that each lawn mower is evenly dispersed in the border area of ​​the map, and then the improved bio-inspired neural network algorithm is used for path planning and mowing operations.

[0151] In this embodiment, each intelligent lawn mower selects the optimal path according to the activity values ​​of the surrounding neurons and the penalty function. Specifically, the next path point at the next moment can be selected according to the following formula:

[0152] q n ←x qn =max(G j ) (3)

[0153] in:

[0154] Gj =ax j +cy j (4)

[0155] y j =1-Δθ j / π (5)

[0156]

[0157] In the formula, a and c are the activity coefficient and turning coefficient respectively, G j is the evaluation function, (x qj ,y qj )(x qc ,y qc )(x qp ,y qp ) are the coordinate positions of the next moment, the current moment, and the previous moment, respectively, Δθ j is the corner, x qn represents the path point at the current moment, q n represents the path point at the next moment, q n ←x qn Represents the path point x from the current moment qn Select the path point q at the next moment n .

[0158] This embodiment takes into account factors such as steering and trajectory smoothness, and adopts the above-mentioned formulas (3), (4), and (5), so as to more quickly and accurately determine the next path point to be selected at the next moment.

[0159] In the collaborative full coverage path planning of multiple intelligent lawn mowers in this embodiment, the activity value of the obstacle point is first marked as a large negative number (-E), and the activity value in the remaining area to be operated is a positive number E. In order to ensure that there is no collision between the machines, the machines are also regarded as obstacles in the working space. When the robot is in the grid position, eight surrounding points are selected as neighborhood points, and the evaluation function is calculated according to formula (6). The activity value of each neuron is updated according to formula (1), and each machine selects the next path point according to formula (3); when the area is patrolled, its activity value is set to 0, which can achieve avoidance of dynamic obstacles and real-time online path planning.

[0160] Furthermore, when there are more than two intelligent lawn mowers that select the same point as the next path point, the priority value K of each lawn mower is calculated according to the following formula: i :

[0161] K i =αE i +βT 2i +γP i (7)

[0162] Where E i is the stored energy value of the i-th smart lawn mower, T pi is the amount of work completed by the i-th smart lawn mower, P i is the priority coefficient of the i-th smart lawn mower.

[0163] The smart lawn mower with the largest K value is selected to perform the task at the conflict point, and the remaining smart lawn mowers continue to select the next path point according to the updated active field.

[0164] When the same area is selected by multiple machines as the working area at the next moment, this embodiment calculates the evaluation function according to formula (7) and selects the lawn mower with the largest function value as the working equipment in the area at the next moment. This method can average the working tasks, achieve a balance between the equipment priority and the previous workload, and realize the task assignment of the lawn mower.

[0165] This embodiment aims at mowing tasks in irregular large areas. An RTK module can be installed in the lawn mower, and map information of the working area can be obtained first. The map is rasterized and given different and real-time updated neuron activity values. By using the above-mentioned improved bio-inspired neural network algorithm, each machine equipment completes the covering task of the entire area by attracting the activity value. At the same time, combined with the value function, it can handle the competition of multiple devices for the working tasks in the same area.

[0166] It is understandable that the above method of this embodiment can be executed by a single device, such as a computer or server, etc., and can also be applied to a distributed scenario and completed by multiple devices in cooperation with each other. In the case of a distributed scenario, one of the multiple devices can only execute one or more steps in the above method of this embodiment, and multiple devices interact to complete the above method. The processor can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, etc., for executing related programs to implement the above method of this embodiment. The memory can be implemented in the form of a read-only memory ROM, a random access memory RAM, a static storage device, and a dynamic storage device. The memory can store an operating system and other applications. When the above method of this embodiment is implemented by software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0167] This embodiment further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above method is implemented.

[0168] Those skilled in the art will appreciate that the above-mentioned embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0169] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment, it is not intended to limit the present invention. Therefore, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A method for controlling the operation of an intelligent lawn mower, wherein the intelligent lawn mower is equipped with a binocular camera, wherein the steps include: After the intelligent lawn mower enters the working area from the boundary, it extracts the boundary contour trajectory points of the working area within the field of view based on the images collected by the binocular camera; The fitted line obtained by curve fitting the boundary contour trajectory points within the current field of view is used as the reference boundary; The reference boundary is offset toward the center of the working area by an operating radius as the desired path within the current field of view; The desired path is tracked while the smart lawn mower is traveling along the desired path, and the error information of the smart lawn mower deviating from the desired trajectory is calculated in real time based on the images collected by the binocular camera. The driving path of the smart lawn mower is controlled and adjusted based on the error information until the mowing task within the current field of view is completed, thereby obtaining a map of the current field of view. When the smart lawn mower enters the next field of view, the map of the previous field of view is spliced ​​with the map of the current field of view until the smart lawn mower completes a round of mowing operation after circling the boundary of the entire operation area, and obtains a map of the entire operation area; The intelligent lawn mower is controlled to travel an operating radius distance toward the center of the operating area and re-execute a round of mowing operation. In each round of mowing operation, the next round of mowing operation path is obtained by offsetting the previous round of mowing operation path toward the center of the operating area by an operating radius until the mowing task of the entire operating area is completed.

2. The intelligent lawn mower operation control method according to claim 1, characterized in that: The step of extracting the boundary contour trajectory points of the working area within a field of view according to the image collected by the binocular camera comprises: Obtain two images captured by the binocular camera for stereo matching to obtain matching point pairs in the two images; Calculate the disparity of each pixel point according to the matching point pairs obtained by stereo matching to obtain a disparity map; Using the disparity map, the three-dimensional information of the working area within the current field of view is restored to obtain the three-dimensional point cloud data of the working area within the front field of view; Boundary contour trajectory points are extracted from the three-dimensional point cloud data of the working area within the current field of view.

3. The intelligent lawn mower operation control method according to claim 1, characterized in that: The curve fitting of the acquired boundary contour trajectory points within the current field of view includes: Perform edge detection on the currently acquired image to obtain a boundary curve; The boundary curve is sampled according to a fixed step size d to decompose into a number of points to form a boundary curve sequence point; A starting point is randomly selected from the boundary curve sequence points, and a fixed-size window n is selected along the search direction of the boundary curve, where n is the length of the window; Starting from the current starting point of the boundary curve according to the current window, a linear fit is performed for each point in the current window using the least square method and the fitting error is calculated; Comparing the calculated fitting error with a preset error threshold v; If the minimum fitting error is less than the preset error threshold v, the current window size is enlarged to n+1, and then the fitting error is recalculated and compared with the preset error threshold v; If the fitting error is greater than or equal to the preset error threshold v, the current point is selected as the segment end point, and a straight line L is obtained from the current starting point to the current segment end point to complete the generation of a segmented straight line. The current segment end point is used as the starting point of the next segmented straight line, and a window of a fixed size of n is selected along the search direction of the boundary curve. Then, the fitting error is calculated again and compared with the preset error threshold v, until all points on the edge curve sequence point are traversed.

4. The intelligent lawn mower operation control method according to claim 1, 2 or 3, characterized in that: The step of splicing the map of the previous field of view with the map of the current field of view includes: Respectively obtain first point cloud data from a current view range map and second point cloud data from a previous view range map; Extracting feature points from the first point cloud data and the second point cloud data respectively and generating feature descriptors; Matching the feature descriptors corresponding to the first point cloud data and the second point cloud data to obtain a matched feature point pair; Registering the first point cloud data with the second point cloud data according to the matched feature points; The registered first point cloud data and the second point cloud data are spliced ​​to generate spliced ​​map point cloud data.

5. The intelligent lawn mower operation control method according to claim 4, characterized in that: The registering the first point cloud data with the second point cloud data according to the matched feature points comprises: Performing preliminary registration of the first point cloud data and the second point cloud data using a coarse registration algorithm according to the matched feature point pairs; Iteratively optimizing the point cloud data after preliminary registration to adjust the position process between the first point cloud data and the second point cloud data to achieve accurate registration; The precise registration results are globally optimized to ensure that the stitching results are globally consistent.

6. A method for controlling the coordinated operation of multiple intelligent lawn mowers, characterized in that the steps include: Get map information of the operation area; If the work area is of regular shape, divide the work area into multiple sub-areas; Control multiple intelligent lawn mowers to enter the working area from the boundaries of each sub-area; According to the method described in any one of claims 1 to 5, each intelligent lawn mower is controlled to complete the mowing operation in each sub-area.

7. The method for controlling the coordinated operation of multiple intelligent lawn mowers according to claim 6, characterized in that: If the area is irregular in shape, the improved bio-inspired neural network algorithm is used to plan the driving path of each intelligent lawn mower, and the steps include: Rasterize the map and mark the obstacles in the grid; The activity value of the obstacle point is the first activity value, and the activity value in the remaining area to be operated is the second activity value, wherein each intelligent lawn mower is regarded as an obstacle to another; Initialize the activity values ​​of all neurons to 0, and update the activity values ​​of neurons according to the preset frequency map and the following formula: Among them, x i is the activity value of the i-th neuron, A, B, and D represent the decay rate of the activity value and the upper and lower values ​​of the neuron activity, respectively. Represents the stimulus input of the neuron, and the calculation expression is: Among them, w ij is a monotonically decreasing function, It is an inhibitory signal of neurons; When the smart lawn mower is at a grid position, it selects the optimal path based on the activity values ​​of the surrounding neurons and the penalty function until the operation in the current area is completed.

8. The method for controlling the coordinated operation of multiple intelligent lawn mowers according to claim 7, characterized in that: The smart lawn mower selects the next path point at the next moment according to the activity values ​​of the surrounding neurons and the penalty function as follows: q n ←x qn =max(G j ) in: G j =ax j +cy j y j =1-Δθ j / p In the formula, a and c are the activity coefficient and turning coefficient respectively, G j is the evaluation function, (x qj ,y qj )(x qc ,y qc )(x qp ,y qp ) are the coordinate positions of the next moment, the current moment, and the previous moment, respectively, Δθ j is the corner, x qn represents the path point at the current moment, q n represents the path point at the next moment, q n ←x qn Represents the path point x from the current moment qn Select the path point q at the next moment n .

9. The method for controlling cooperative operation of multiple intelligent lawn mowers according to claim 7 or 8, characterized in that: When there are more than two intelligent lawn mowers that select the same point as the next path point, the priority value K of each lawn mower is calculated according to the following formula: i : K i =αE i +βT 2i +γP i Where E i is the stored energy value of the i-th smart lawn mower, T pi is the amount of work completed by the i-th smart lawn mower, P i is the priority coefficient of the i-th smart lawn mower; The smart lawn mower with the largest K value is selected to perform the task at the conflict point, and the remaining smart lawn mowers continue to select the next path point according to the updated active field.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images