Smart mower operation control method, collaborative operation control method, and medium

Abstract

The application discloses a kind of intelligent mower operation control method, collaborative operation control method and medium, the intelligent mower operation control method steps include: after intelligent mower enters work area from boundary, the boundary profile trajectory point of work area in a view range is extracted according to the image collected, reference boundary is obtained by curve fitting;The expected path in the current view range is taken as the reference boundary offset a work radius;Desired path is tracked, and the map of current view range is obtained;When intelligent mower enters next view range, map is spliced, and a round of mowing operation is completed;Control intelligent mower towards the center of work area travels a work radius distance, and re-executes a round of mowing operation, until the mowing task of entire work area is completed.The application has the advantages of simple implementation method, low cost, high control efficiency and precision, strong anti-interference and flexibility, wide application range and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent lawnmower technology, and in particular to an intelligent lawnmower operation control method, a collaborative operation control method, and a medium. Background Technology

[0002] Traditional lawn mowing relies on manual operation, which is not only inefficient but also unsuitable for inclement weather or uneven terrain. Manual mowing also often struggles to guarantee evenness and efficiency. However, with continuous advancements in sensor technology, path planning algorithms, and wireless communication, the development of intelligent lawnmowers has achieved remarkable results. Intelligent lawnmowers integrate multiple sensors to perceive real-time environmental information about the lawn, such as terrain and obstacles. Simultaneously, utilizing advanced path planning algorithms, intelligent lawnmowers can autonomously plan mowing routes, achieving efficient and even mowing operations.

[0003] However, most existing smart lawnmowers require expensive RTK (Real-Time Kinematic) modules to achieve precise positioning and path planning based on RTK positioning data. This not only significantly increases equipment and maintenance costs, hindering the widespread application of smart lawnmowers, but also limits their applicability to certain environments due to their reliance on satellite signals. They are easily affected by environmental factors, such as densely wooded areas, densely built-up areas, or mountainous regions where satellite signals may be blocked, affecting the accuracy and reliability of RTK positioning and preventing smart lawnmowers from accurately and reliably completing their mowing tasks. Summary of the Invention

[0004] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides an intelligent lawnmower operation control method, collaborative operation control method and medium that is simple to implement, low in cost, highly efficient and precise in control, strong in anti-interference and flexibility, and widely applicable.

[0005] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0006] A method for controlling the operation of an intelligent lawnmower, wherein the intelligent lawnmower is equipped with a binocular camera, the steps of which include:

[0007] After the intelligent lawnmower enters the work area from the boundary, it extracts the boundary contour trajectory points of the work area within the field of view based on the images captured by the binocular cameras.

[0008] The fitted line obtained by curve fitting of the boundary contour trajectory points within the current field of view is used as the reference boundary.

[0009] offsetting the reference boundary towards a center region of the work area by a work radius as a desired path within a current field of view;

[0010] tracking the desired path during the intelligent mower driving along the desired path, calculating error information of the intelligent mower deviating from the desired path according to images collected by the binocular camera in real time, and controlling and adjusting the driving path of the intelligent mower according to the error information until the mowing task within the current field of view is completed, and a map of the current field of view is obtained;

[0011] when the intelligent mower enters the next field of view, splicing the map of the previous field of view with the map of the current field of view, until the intelligent mower completes a round of mowing operation after driving around the boundary of the entire work area, and a map of the entire work area is obtained;

[0012] controlling the intelligent mower to drive towards the center of the work area by a work radius distance, and re-executing a round of mowing operation, and the path of the next round of mowing operation in each round of mowing operation is obtained by offsetting the path of the previous round of mowing operation towards the center region of the work area by a work radius, until the mowing task of the entire work area is completed.

[0013] Further, the step of extracting the boundary contour trajectory points of the work area within a field of view according to images collected by the binocular camera comprises:

[0014] stereo matching two images photographed by the binocular camera to obtain a matching point pair in the two images;

[0015] calculating the disparity of each pixel point according to the matching point pair obtained by stereo matching to obtain a disparity map;

[0016] restoring three-dimensional information of the work area within the current field of view using the disparity map to obtain three-dimensional point cloud data of the work area within the current field of view;

[0017] extracting the boundary contour trajectory points from the three-dimensional point cloud data of the work area within the current field of view.

[0018] Further, the curve fitting of the obtained boundary contour trajectory points within the current field of view comprises:

[0019] performing edge detection on the currently collected image to obtain a boundary curve;

[0020] sampling the boundary curve according to a fixed step length d to decompose it into a plurality of points to form a boundary curve sequence point;

[0021] randomly selecting a starting point in the boundary curve sequence point, and selecting a fixed size window n along the search direction of the boundary curve, n being the length of the window;

[0022] According to the current window, a linear fitting is performed on each point in the current window from the current starting point of the boundary curve using a least square method, and a fitting error is calculated;

[0023] A comparison is made between the calculated fitting error and a preset error threshold v;

[0024] If the minimum fitting error is less than the preset error threshold v, the current window size is expanded to n+1, and the fitting error is recalculated and compared with the preset error threshold v again;

[0025] If the fitting error is greater than or equal to the preset error threshold v, the current point is selected as a segment endpoint, a straight line L is obtained from the current starting point to the current segment endpoint, the generation of a segment straight line is completed, the current segment endpoint is taken as a starting point of a next segment straight line, a window with a fixed size n is selected along the search direction of the boundary curve, and the fitting error is recalculated and compared with the preset error threshold v again until all points on the edge curve sequence are traversed.

[0026] Further, the splicing of the map of the previous view range and the map of the current view range comprises:

[0027] First point cloud data from the map of the current view range and second point cloud data from the map of the previous view range are respectively acquired;

[0028] Feature points are extracted from the first point cloud data and the second point cloud data respectively, and feature descriptors are generated;

[0029] The feature descriptors corresponding to the first point cloud data and the second point cloud data are matched, and matched feature point pairs are obtained;

[0030] The first point cloud data and the second point cloud data are registered according to the matched feature point pairs;

[0031] The first point cloud data and the second point cloud data after registration are spliced to generate spliced map point cloud data.

[0032] Further, the registration of the first point cloud data and the second point cloud data according to the matched feature point pairs comprises:

[0033] The first point cloud data and the second point cloud data are preliminarily registered using a coarse registration algorithm according to the matched feature point pairs;

[0034] The point cloud data after preliminary registration is iteratively optimized to adjust the position process between the first point cloud data and the second point cloud data, so as to realize accurate registration;

[0035] The post-precise registration result is globally optimized to make the splicing result consistent in the global range.

[0036] A multi-intelligent mower cooperative operation control method, comprising the steps of:

[0037] Obtaining map information of the operation area;

[0038] If the operation area is a regular shape, the operation area is divided into multiple sub-areas;

[0039] Control multiple intelligent mowers to enter the operation area from the boundary of each sub-area;

[0040] According to the above method, each intelligent mower completes the mowing operation of each sub-area.

[0041] Further, if the operation area is an irregular shape, an improved biological excitation neural network algorithm is used to plan the driving path of each intelligent mower, comprising the steps of:

[0042] Grid processing is performed on the map, and markers are placed in the obstacle grid;

[0043] The activity value of the obstacle point is marked as the first activity value, and the activity value of the remaining to-be-operated area is marked as the second activity value, wherein each intelligent mower is considered as an obstacle between each other;

[0044] Initialize the activity value of all neurons to 0, and update the activity value of the neurons according to the preset frequency map and according to the following formula:

[0045]

[0046] wherein x i is the activity value of the i-th neuron, A, B, and D represent the decay rate of the activity value and the upper and lower values of the neuron activity, respectively, represents the stimulus input of the neuron, and the calculation expression is:

[0047]

[0048] wherein w ij is a monotonic decreasing function, is the inhibition signal of the neuron;

[0049] When the intelligent mower is at the grid position, the intelligent mower selects the optimal path according to the activity value of the surrounding neurons and the penalty function until the operation of the current area is completed.

[0050] Further, the intelligent mower selects the next path point at the next time according to the activity value of the surrounding neurons and the penalty function according to the following formula:

[0051] qn ← x qn = max(G j )

[0052] wherein:

[0053] G j = ax j + cy j

[0054] y j = 1 - Δθ j / π

[0055]

[0056] In the formula, a and c are respectively an activity value coefficient and a turning coefficient, G j is an evaluation function, (x qj , y qj )(x qc , y qc )(x qp , y qp ) are respectively coordinate positions at a next time, a current time and a previous time, Δθ j is a corner, x qn represents a path point at the current time, q n represents a path point at the next time, and q n ← x qn represents that the path point q qn at the next time is selected from the path point x n at the current time.

[0057] Further, when there are two or more intelligent mowing machines selecting the same point as the next path point, the priority value K i of each intelligent mowing machine is calculated according to the following formula:

[0058] K i = αE i + βT 2i + γP i

[0059] In the formula, E i is a storage energy value of the i-th intelligent mowing machine, T pi is a work amount already completed by the i-th intelligent mowing machine, and P i is a priority coefficient of the i-th intelligent mowing machine.

[0060] The intelligent mowing machine with the largest K value is selected to perform the work task of the conflict point, and the remaining intelligent mowing machines continue to select the next path point according to the updated activity field.

[0061] A computer readable storage medium storing a computer program, the computer program being executed by a processor to implement the method described above.

[0062] Compared with the prior art, the present application has the advantages that:

[0063] 1、The present application controls the intelligent mower to enter the working area from a certain boundary, obtains the profile at the boundary through visual information, fits the segmented line segment of the profile in the line of sight to obtain the reference boundary, controls the intelligent mower to perform the mowing work based on the reference boundary, plans the driving path of the mowing work of the next round according to the driving track of the last round after completing a round of mowing work, does not need an RTK positioning device, and can realize the full coverage mowing task according to the binocular camera, so that the equipment cost can be greatly reduced, the mowing efficiency can be improved, and the control complexity of the whole mowing work can be reduced.

[0064] 2、The present application further solves the problems of path zigzagging and incompleteness caused by the sudden drop of the activity value of neurons near the boundary and obstacles in the traditional algorithm by combining and improving the traditional biological excitation neural network algorithm, utilizes the multi-machine collaborative work, regards each intelligent mower as an obstacle, and combines the improved biological excitation neural network algorithm to efficiently and reliably perform the full coverage mowing task, and greatly improves the efficiency and coverage rate of the mowing task in an irregular large area. BRIEF DESCRIPTION OF DRAWINGS

[0065] Figure 1 It is a schematic diagram of the implementation process of the intelligent mower working control method of the present embodiment.

[0066] Figure 2 It is a schematic diagram of the principle of generating the reference boundary in the specific application embodiment.

[0067] Figure 3 It is a schematic diagram of the principle of planning the N*R working track in the specific application embodiment.

[0068] Figure 4 It is a schematic diagram of the principle of realizing obstacle avoidance in the specific application embodiment. DETAILED DESCRIPTION

[0069] The present application is further described below in combination with the drawings and specific preferred embodiments of the present application, but the protection scope of the present application is not limited thereto.

[0070] As shown in the present disclosure, unless the context clearly indicates otherwise, the words "one", "an", "a", and / or "the" do not specify a singular number, but can also include a plural number. The words "first", "second", and similar words used in the present disclosure do not indicate any order, quantity, or importance, but are only used to distinguish different components. Similarly, the words "include" or "contain" and similar words mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, and do not exclude other elements or objects.

[0071] Embodiment 1:

[0072] The present application does not need to be equipped with RTK positioning module, laser radar module and other devices on the intelligent mower, only needs to configure binocular vision camera on the intelligent mower, and can efficiently realize real-time online planning path combining with the image data collected by the binocular vision camera, and complete the full coverage mowing task. As shown in Figure 1 The steps of the intelligent mower operation control method of the present embodiment include:

[0073] Step S101. After the intelligent mower enters the operation area from the boundary, the boundary contour trajectory points of the operation area within a field of view are extracted according to the images collected by the binocular camera.

[0074] In the present embodiment, a binocular camera with a binocular camera is configured on the intelligent mower, and the depth information within the field of view can be obtained by calculation. After the intelligent mower starts, it enters the operation area from a certain boundary, and the contour at the boundary can be obtained through the visual information collected by the binocular camera mounted on the intelligent mower. Specifically, the images within the field of view of the binocular camera can be obtained at every interval time, and then the boundary contour trajectory points of the operation area within the field of view are extracted from the images. For example, the images within a certain field of view are obtained from the starting time of entering the operation area, the boundary contour trajectory points within the field of view are extracted, and then the new boundary contour trajectory points within the field of view are re-extracted after the intelligent mower travels for a specified time.

[0075] In a specific application embodiment, before the binocular camera works, the binocular camera needs to be calibrated to determine the internal parameters (such as focal length, principal point, etc.) and external parameters (such as the relative position and direction between the cameras) of each camera. After calibration, the distortion of the camera image can be corrected, so that the images of the binocular camera are consistent in geometry.

[0076] In the present embodiment, the step of extracting the boundary contour trajectory points of the operation area within a field of view according to the images collected by the binocular camera includes:

[0077] Step S111. Stereo matching: stereo matching is performed on the two images taken by the binocular camera to obtain the matching point pairs in the two images.

[0078] First, stereo matching is performed on the two images captured by the binocular camera. The goal of this process is to find corresponding pixel pairs in the two images, forming matching point pairs, which can reflect the positional relationship of the same target object under the perspective of the two cameras.

[0079] Step S112. Disparity map calculation: Calculate the disparity of each pixel point according to the matching point pairs obtained by stereo matching, and obtain the disparity map.

[0080] Using the matching point pairs obtained by stereo matching, the disparity of each pixel point can be calculated, and then the disparity map is obtained. The disparity map is a two-dimensional image, where the gray value of each pixel represents the disparity size of the point in the left and right images. Disparity is inversely proportional to the distance of the object to the camera, that is, the larger the disparity, the closer the object to the camera.

[0081] Step S113. Three-dimensional reconstruction: Use the disparity map to recover the three-dimensional information of the working area within the current field of view, and obtain the three-dimensional point cloud data of the working area within the front field of view.

[0082] Using the disparity map and camera calibration parameters, the three-dimensional information of the object in the scene can be recovered by the principle of triangulation. In this process, the three-dimensional point cloud data of the object can be obtained, which contains the shape and position information of the object. In this embodiment, the three-dimensional point cloud data of the working area within the front field of view can be recovered by using the disparity map.

[0083] After recovering the three-dimensional point cloud data, further preprocessing can be performed on the data, such as removing noise, outliers or unnecessary points, etc.

[0084] Step S114. Boundary contour extraction: Extract the boundary contour trajectory points from the three-dimensional point cloud data of the working area within the current field of view.

[0085] After obtaining the three-dimensional point cloud data, the boundary contour of the object can be extracted by point cloud processing algorithms (such as curvature-based method, region growing-based method, etc.), which is a series of points or line segments in three-dimensional space, that is, the boundary contour trajectory points are obtained.

[0086] For example, the dashed line in the figure represents the point cloud trajectory of the extracted boundary contour. When the intelligent mower starts working, the point cloud information of the boundary contour within the field of view can be extracted from the image captured by the binocular camera at the initial t0 time, as shown by the dashed line in (a) in Figure 2 At t1 and t2, the point cloud information of the boundary contour within the corresponding field of view can be extracted from the images captured by the binocular camera, respectively obtaining the dashed lines in (b) and (c) in Figure 2 . Figure 2 .

[0087] Step S102. The fitting line obtained by curve fitting of the boundary contour trajectory points in the current field of view range is taken as the reference boundary.

[0088] The boundary contour trajectory points extracted in step S101 are fitted by curve fitting to obtain a fitting line, which is taken as the reference boundary of the current field of view range. In the process of driving the intelligent mower, the reference boundaries of different field of view ranges can be obtained by continuously extracting boundary contour trajectory points and performing curve fitting.

[0089] For the extracted boundary contour, their geometric characteristics can be further analyzed. In this embodiment, the acquired boundary contour trajectory point cloud information in the current field of view range can be fitted by curve fitting in a segmented fitting manner, which includes: dividing the extracted boundary contour trajectory points into multiple subsets according to the local characteristics of the curve, each subset corresponds to a straight line segment, and the straight line segments corresponding to each subset are spliced to obtain the final fitting line.

[0090] Optionally, the sliding window method can be used to obtain the segmented subsets, and the least square method is used to obtain the fitting straight line, and the specific steps are as follows:

[0091] Step S121. The edge detection algorithm (such as Canny edge detection or Sobel operator) is used to perform edge detection on the current collected image to obtain a boundary curve;

[0092] Step S122. The boundary curve is sampled according to a fixed step size d to decompose into a plurality of points to form a boundary curve sequence of points {P1, P2, …, PN}, wherein N is the number of points of the boundary curve;

[0093] Step S123. A starting point is randomly selected on the boundary curve sequence {P1, P2, …, PN}, and a window with a fixed size n is selected along the search direction of the boundary curve;

[0094] Step S124. According to the current window, starting from the current starting point of the boundary curve, for each point (x1, y1), (x2, y2), …, (xn, yn) in the current window, the least square method formula is used for linear fitting, and the fitting error is calculated.

[0095] Specifically, the mean square error (MSE) can be calculated by using the following formula:

[0096]

[0097] In order to minimize the fitting error, the optimal parameters a and b values of the fitting straight line can be determined by taking the derivative of the above formula, and the fitting error can be calculated using the obtained optimal parameters a and b values, that is, the fitting error can be obtained.

[0098] It can be understood that other ways can also be used to calculate the fitting error according to actual needs.

[0099] Step S125. Compare the size between the calculated fitting error and the preset error threshold v.

[0100] Step S126. If the minimum fitting error is less than the preset error threshold v, the current window size is enlarged to n+1, that is, one point is added, and the step S114 is returned to recalculate the fitting error and compare it with the preset error threshold v, otherwise, go to step S117.

[0101] Step S127. Select the current point as the end point of the segment, obtain a straight line L from the current start point to the current end point of the segment, complete the generation of a segmented straight line, and take the current end point of the segment as the start point of the next segmented straight line, select a window with a fixed size n in the search direction of the boundary curve, and return to execute step S114 until all points on the edge curve sequence are traversed, and then end.

[0102] Suppose the working area is an irregular curve, for example, the above method is used to segment and fit the extracted boundary contour trajectory points (shown by a dashed line) to obtain an approximate fitting line (shown by a solid line), which is the obtained reference boundary, which is connected by several segmented lines. Figure 2

[0103] Step S103. Offset the reference boundary towards the center area of the working area by a working radius as the expected path in the current field of view.

[0104] After the intelligent mower enters the working area from the boundary, it starts to perform the first round of work around the boundary of the working area. During the first round of work, the reference boundary obtained by fitting is offset towards the center area of the working area by a working radius as the expected path, so that a round of work around the boundary of the working area can be completed with the help of visual information.

[0105] Specifically, suppose the working radius of the intelligent mower is R, and the initial point is the starting point of the mower. The direction of the straight line containing the initial point (starting point) obtained by the above curve fitting is the heading direction of the mower, that is, a straight line containing the starting point is taken, so that the heading direction of the mower is parallel to the straight line and the distance is R. Then, by offsetting the reference boundary obtained by curve fitting by a distance R in the direction of the device, the expected path and expected heading angle of the mower in the field of view can be obtained. The edge curve (reference boundary) can be divided into several straight lines by the curve fitting in step S102. A straight line containing the starting point is selected, so that the heading direction of the mower is parallel to the straight line and the distance is R, that is, the initial position of the mower is determined, and then the mower travels parallel to the straight line direction and maintains a distance of R until the next segmented straight line in the reference boundary.​

[0106] For example, assuming that a boundary curve is composed of P1~Pn points, in the straight line fitting stage, it is assumed that points P1-P10 are divided into straight line L1, P11-P30 are divided into straight line L2, and P31-Pn are divided into straight line L3, the direction of the three straight lines can be determined, that is, the slope; if the starting point of the current start is contained in a certain straight line, such as the starting point P3 point (on the L1 straight line), the heading of the mower can be set according to the slope of the L1 straight line (parallel to the slope).

[0107] Step S104. Tracking the expected path during the driving of the intelligent mower along the expected path, calculating the error information of the intelligent mower deviating from the expected trajectory in real time according to the images collected by the binocular camera, and controlling and adjusting the driving path of the intelligent mower according to the error information until the current field of view range is completed, and the map of the current field of view range is obtained.

[0108] Since there is no RTK module, the position information of the device cannot be obtained, and there may be phenomena such as idling and slipping during the driving process. The embodiment can obtain the error information of the mower deviating from the expected trajectory in the actual running process by using the binocular camera, and realize the tracking of the expected trajectory based on the error feedback control, so as to accurately control the intelligent mower to complete the mowing task along the reference boundary.

[0109] Step S105. When the intelligent mower enters the next field of view range, the map of the previous field of view range is spliced with the map of the current field of view range, until the intelligent mower completes a round of mowing operation after one round around the boundary of the entire working area, and the map of the entire working area is obtained.

[0110] In each field of view range, a reference boundary can be obtained by fitting, and after determining an expected path using the reference boundary, the intelligent mower can obtain the map of the field of view range after completing the operation along the expected path, and then continuously splicing the map of the previous field of view range with the map of the current field of view range. After the intelligent mower completes a round of mowing operation around the boundary, the map of the entire working area can be spliced.

[0111] In the embodiment, the step of splicing the map of the previous field of view range with the map of the current field of view range specifically includes:

[0112] Step S151. Data preparation and preprocessing: obtaining first point cloud data from the map of the current field of view range and second point cloud data from the map of the previous field of view range.

[0113] The point cloud data from different maps can be pre-processed, such as removing noise, filtering smoothing, ground removal, etc., to improve the quality of the point cloud and reduce interference in the matching process, and to ensure that the format and quality of the data meet the splicing requirements.

[0114] Step S152. Feature extraction and description: extracting feature points from the first point cloud data and the second point cloud data respectively and generating feature descriptors.

[0115] Representative feature points are extracted from each point cloud data, which can be based on geometric shapes or extracted based on other algorithms (such as SIFT, SURF, etc.), and then the extracted feature points are described to generate feature descriptors for subsequent feature matching.

[0116] Step S153. Matching the feature descriptors corresponding to the first point cloud data and the second point cloud data to obtain matched feature point pairs.

[0117] Step S154. Registering the first point cloud data and the second point cloud data according to the matched feature point pairs;

[0118] Step S155. Splicing the registered first point cloud data and second point cloud data to generate spliced map point cloud data.

[0119] In specific application embodiments, the above step S154 can divide the registration process into two stages of coarse registration and accurate registration to ensure registration accuracy, and the steps can include:

[0120] Step S1541. Using a coarse registration algorithm to preliminarily register the first point cloud data and the second point cloud data according to the matched feature point pairs;

[0121] Step S1542. Iteratively optimizing the preliminarily registered point cloud data to adjust the position process between the first point cloud data and the second point cloud data, and realizing accurate registration;

[0122] Step S1543. Globally optimizing the result after accurate registration to make the splicing result consistent in the global range.

[0123] For example, feature matching algorithms can be used to match feature descriptors from different point clouds first, generating feature point pairs; based on the matched feature point pairs, a coarse registration algorithm (such as the RANSAC algorithm combined with the ICP algorithm) is used for preliminary registration, and different point clouds are roughly aligned. On the basis of coarse registration, an accurate registration algorithm (such as the ICP algorithm) is used for iterative optimization, and the positional relationship between the point clouds is further adjusted to make them match more accurately. In the process of accurate registration, the best transformation matrix is found by minimizing the distance or other measurement indicators between the point pairs, so that the corresponding points between different point clouds coincide as much as possible. Considering the context information of the entire scene, global optimization is performed again to ensure the consistency of the splicing result in the global range, and finally the point cloud data after accurate registration is spliced to generate complete multi-map point cloud data.

[0124] To further ensure the splicing accuracy, post-processing such as smoothing and filtering can also be performed on the spliced point cloud data to further improve the data quality. The quality of the splicing result can also be evaluated, such as checking the continuity, smoothness and color consistency of the splicing area, to further improve the splicing quality.

[0125] Step S106. Control the intelligent mower to travel towards the center of the working area by a working radius distance, and re-perform a round of mowing work. The mowing work path of the next round of mowing work is obtained by offsetting the mowing work path of the previous round of mowing work towards the center of the working area by a working radius, until the mowing task of the entire working area is completed.

[0126] After completing the first round of work along the boundary of the working area according to steps S101-S105, the intelligent mower is controlled to travel towards the center of the working area by a working radius distance, and then the mowing work path of the first round of work (i.e., offsetting the reference boundary by a working radius) is used as the expected path for the second round of mowing work by offsetting the center of the working area by a working radius. The intelligent mower completes the second round of mowing work according to the expected path, and then travels towards the center of the working area by a working radius distance to start the third round of work, and the above steps are repeated until the mowing task of the entire working area is completed.

[0127] In a specific application embodiment, the path trajectory of the mower is a cow-cultivation trajectory. Based on the boundary information fed back by the camera and the working radius R of the mower, a working trajectory parallel to the boundary and N*R away from the boundary is designed, where N is the number of round trips in the direction parallel to the boundary. In the first round of mowing work, the mower continuously generates a reference boundary and an expected path in the new field of view to obtain a map within the current field of view, until the next map field of view. The map of the current field of view is spliced with the map of the previous field of view, and the above steps are repeated. When the mower completes one round around the working area, the map information of the entire area can be spliced, as shown in the following figure:Figure 2 As shown in (d), the first round of mowing tasks is completed, and then the mower device walks inward by a distance of 2R, i.e., a distance of twice the working radius, to start a new round of work until the mowing task in the area is completed, as shown in (e). Figure 3

[0128] Further, when there is an obstacle on the travel path, the current azimuth angle and the working row distance are saved, the heading angle at this time is marked, and the bug2 algorithm is used to perform obstacle avoidance until the heading angle is consistent with the original heading angle and the obstacle area is exited, as shown in (f). Figure 4 The obstacle avoidance trajectory planning of the bug2 algorithm is similar to the path planning of the map boundary planning, which is to follow the fitted straight line of the point cloud trajectory. After avoiding the obstacle, the previous azimuth angle and working row distance mode are returned to.

[0129] The intelligent mower enters the working area from a certain boundary, obtains the profile at the boundary through visual information, fits the segmented line segments of the profile in the line of sight to obtain a plurality of connected straight line segments, takes the line segment as the reference boundary in the direction, obtains the coordinate information under the reference boundary combined with the depth information obtained by the camera, drives the mower to the next reference boundary at a fixed row distance in the direction, then switches the reference coordinate, re-executes the working task in the line segment direction, and adjusts the control input of the mower in real time through the error feedback signal during the travel to avoid slipping and idling of the mower. Without the RTK positioning device, the mower can complete the acquisition of the boundary features of the mowing area, the planning of the path, and the avoidance of obstacles, etc. by relying only on the binocular camera carried by itself, and can realize full coverage of the mowing task, thereby greatly reducing the cost of the device and reducing the complexity of the overall operation, and improving the mowing efficiency.

[0130] Embodiment 2:

[0131] For large lawn working environments, the working efficiency of a single device is low, and the working efficiency of the mower is limited by its own performance, and a single mower is difficult to cope with complex and variable lawn environments. The cooperative working mode can realize higher working efficiency and better working effect through communication, cooperative planning, and obstacle avoidance between multiple mowers. The use of multi-machine cooperation combined with efficient path planning can greatly improve the working efficiency and realize efficient work in a large working space.

[0132] The steps of the multi-intelligent mower cooperative working control method of the embodiment include:

[0133] Obtain map information of the working area;

[0134] If the working area is a regular shape, divide the working area into a plurality of sub-areas;

[0135] ​Controlling multiple intelligent mowers to enter the working area from the boundary of each sub-area;

[0136] According to the method of embodiment 1, each intelligent mower completes the mowing work of each sub-area.

[0137] The above-mentioned regular shape refers to a shape that can be divided into a square, a circle, etc., such as a large square or a large circle, or a combination of a square and a circle.

[0138] The embodiment can cover a larger area through the cooperative work of multiple mowers, significantly improve work efficiency, and reduce the time required to complete the entire lawn mowing. The cooperative work of multiple mowers can ensure that the lawn is more evenly mowed, and different mowers can cooperate with each other during work to avoid uneven mowing caused by single mower work, making the lawn look more beautiful. Multi-machine cooperative work can also reduce the load of a single mower and reduce energy consumption. At the same time, by reasonably arranging the number of mowers and the working area, labor costs and maintenance costs can be saved, thereby maximizing cost efficiency. Using multi-machine cooperative work can also deal with complex terrains such as slopes and hills. Different mowers can be flexibly adjusted according to the terrain characteristics to ensure mowing quality and efficiency.

[0139] Since embodiment 1 is to complete mowing work by identifying the boundary and then following the N*R working track, it is suitable for regular shape areas with N*R working tracks. For irregular areas, this method cannot completely cover the area. For this type of irregular area, if the area is irregular in shape, the embodiment further uses an improved biological excitation neural network algorithm to plan the driving path of each intelligent mower. The improved biological neural network algorithm is used to plan the path of multiple intelligent mowers. Each grid map is regarded as a neuron with an updated activity value at any time. The steps include:

[0140] Step S201. Perform grid processing on the map and mark obstacles in the grid.

[0141] Step S202. Mark the activity value of the obstacle point as a first activity value, and the activity value in the remaining working area as a second activity value, wherein each intelligent mower is regarded as an obstacle between each other.

[0142] Step S203. Initialize the activity value of all neurons to 0, and update the activity value of the neurons according to the preset frequency map according to the following formula:

[0143]

[0144] wherein, x iis the activity value of the ith neuron, A, B, D represent the decay rate of the activity value and the upper and lower values of the neuron activity respectively, represents the stimulation input of the neuron, and the calculation expression is:

[0145]

[0146] wherein w ij is a monotonic decreasing function, is the inhibition signal of the neuron.

[0147] The above formula (2) is to calculate the stimulation input of the neuron by using the method of average signal in the neighborhood, which can avoid the problem that the narrow channel is not regarded as a passable area compared with the traditional direct summation method.

[0148] Step S204. When the intelligent mower is at the grid position, the next path point is determined according to the activity value of the surrounding neighborhood point and the turning coefficient until the work in the current area is completed.

[0149] The embodiment improves the traditional biological excitation neural network algorithm, solves the problems of path meandering and incompleteness caused by the sudden drop of the activity value of the neurons near the boundary and obstacles in the traditional algorithm, uses multi-machine cooperative work, and each intelligent mower regards each other as an obstacle. In combination with the improved biological excitation neural network algorithm of each intelligent mower, the full coverage mowing task can be efficiently and reliably executed, and the efficiency and coverage rate of the mowing task in the irregular large area are greatly improved.

[0150] The target point in the map transmits the activity value to the node in the topological space to form an activity field with different activity values of each node. The robot moves to the target position according to a certain rule of a certain node in the neighborhood as the next path point. The change of the surrounding environment is directly reflected on the activity field. The activity value of all neurons in the initial stage is 0. The map is refreshed at a certain frequency, so as to update the activity value of the neurons. In the embodiment, the intelligent mowers are randomly placed in the working range. In order to ensure the working efficiency, the intelligent mowers are preferably uniformly arranged in the boundary area of the map, and then the improved biological excitation neural network algorithm is used for path planning and mowing work.

[0151] In the embodiment, each intelligent mower selects the optimal path according to the activity value of the surrounding neurons and the penalty function. Specifically, the next path point at the next moment can be selected according to the following formula:

[0152] q n ←x qn =max(G j ) (3)

[0153] wherein:

[0154] Gj = ax j + cy j (4)

[0155] y j = 1 - Δθ j / π (5)

[0156]

[0157] wherein a, c are respectively activity value coefficient and turning coefficient, G j is evaluation function, (x qj , y qj )(x qc , y qc )(x qp , y qp ) are respectively coordinate position of next time, current time and previous time, Δθ j is corner, x qn represents path point of current time, q n represents path point of next time, q n ← x qn represents selection of path point q qn of next time from path point x n of current time.

[0158] The embodiment can more quickly and accurately determine the next path point of next time by considering factors such as turning and trajectory smoothing, and using the above formula (3), (4), (5).

[0159] In the multi-intelligent mower cooperative full-coverage path planning, the activity value of the obstacle point is marked as a larger negative number (-E), and the activity value in the remaining to-be-worked region is a positive number E. In order to ensure that the machines do not collide with each other, the machines are also regarded as obstacles in the working space. When the robot is in a grid position, eight points around the robot are selected as neighborhood points, the evaluation function is calculated according to formula (6), the activity value of each neuron is updated according to formula (1), and each machine selects the next path point according to formula (3). When the region is patrolled and worked, the activity value is set to 0, so as to realize avoidance of dynamic obstacles and real-time online path planning.

[0160] Further, when two or more intelligent mowers select the same point as the next path point, the priority value K i of each mower is calculated according to the following formula:

[0161] K i = αE i + βT 2i + γP i (7)

[0162] E = (1 - e) * E + e * Ei i Ei is the storage energy value of the i-th intelligent mower, T pi Ei is the storage energy value of the i-th intelligent mower, T i Ei is the storage energy value of the i-th intelligent mower, T

[0163] The intelligent mower with the maximum K value is selected to perform the work task at the conflict point, and the remaining intelligent mowers continue to select the next path point according to the updated activity field.

[0164] When the same region is selected as the work region at the next moment by multiple machines, the evaluation function is calculated according to formula (7) in this embodiment, and the mower with the maximum function value is selected as the work equipment at the next moment in the region. Through this method, the work task can be averaged, the equipment priority and the previous work amount can be balanced, and the task assignment of the mower can be realized.

[0165] For the mowing task of an irregular large area, the RTK module can be installed in the mower in this embodiment, and the map information of the work region can be obtained first, and the map is rasterized and given different and real-time updated neuron activity values. Using the improved biological excitation neural network algorithm, each machine equipment completes the coverage work task of the entire region through the attraction of the activity value, and the price function can be combined to process the competition of multiple devices for the same region work task.

[0166] It can be understood that the above-mentioned method of this embodiment can be executed by a single device, such as a computer or a server, etc., and can also be applied to a distributed scenario to be completed by multiple devices in cooperation. In the case of a distributed scenario, one of the multiple devices can only execute one or more steps in the above-mentioned method of this embodiment, and the multiple devices interact to complete the above-mentioned method. The processor can be implemented in the form of a general-purpose CPU, a microprocessor, an application-specific integrated circuit, or one or more integrated circuits, etc., for executing related programs to implement the above-mentioned method of this embodiment. The memory can be implemented in the form of a read-only memory ROM, a random access memory RAM, a static storage device, and a dynamic storage device, etc. The memory can store an operating system and other application programs, and when the above-mentioned method of this embodiment is implemented by software or firmware, the related program code is saved in the memory and executed by the processor.

[0167] This embodiment further provides a computer readable storage medium storing a computer program, which is executed by a processor to implement the above-mentioned method.

[0168] Those skilled in the art will appreciate that the embodiments of the present application described above can be provided as a method, system, or computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage devices, etc.) embodying computer readable program code. The present application is described in terms of flowcharts and / or block diagrams in accordance with embodiments of the application. It will be understood that each block of the flowchart and / or block diagrams, and combinations of blocks in the flowchart and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart and / or block diagram block or blocks. These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart and / or block diagram block or blocks. Figure 1 one or more flowcharts and / or block diagrams. Figure 1 one or more flowcharts and / or block diagrams. Figure 1 one or more flowcharts and / or block diagrams. Figure 1 one or more flowcharts and / or block diagrams. Figure 1 one or more flowcharts and / or block diagrams. Figure 1 one or more flowcharts and / or block diagrams.

[0169] The foregoing is merely illustrative of the principles of the application, and various modifications can be made by those skilled in the art without departing from the scope of the application. The above description is embodied in the best mode presently contemplated by the inventors. The application, however, is capable of other embodiments and of being practiced and carried out in various ways. Accordingly, the application is not intended to be limited to the embodiments described above but is to be accorded the widest scope consistent with the principles and features disclosed herein.

Claims

1. A smart mower operation control method, a binocular camera is mounted on the smart mower, characterized in that the steps of The method comprises the following steps: After the intelligent mower enters the working area from the boundary, the boundary contour track points of the working area within a view range are extracted according to the images collected by the binocular camera; The fitting line obtained by curve fitting of the boundary contour track points within the current view range is taken as the reference boundary, including: dividing the extracted boundary contour track points into multiple subsets, each subset corresponding to a straight line segment, and splicing the straight line segments corresponding to each subset to obtain the final fitting line; The reference boundary is offset towards the center area of the working area by a working radius as the expected path within the current view range, and the heading of the intelligent mower is set according to the slope of the straight line segment containing the starting point of the fitting reference boundary; During the driving of the intelligent mower according to the expected path, the expected path is tracked, the error information of the deviation of the intelligent mower from the expected path is calculated in real time according to the images collected by the binocular camera, and the driving path of the intelligent mower is controlled and adjusted according to the error information until the mowing task within the current view range is completed, and the map of the current view range is obtained; When the intelligent mower enters the next view range, the map of the previous view range is spliced with the map of the current view range, and after the intelligent mower completes a round of mowing operation around the entire boundary of the working area, the map of the entire working area is obtained; The intelligent mower is controlled to drive towards the center of the working area by a working radius distance, and a round of mowing operation is re-executed, the driving path of the next round of mowing operation is planned according to the driving track of the previous round of mowing operation in each round of mowing operation, the path of the next round of mowing operation is obtained by offsetting the path of the previous round of mowing operation towards the center area of the working area by a working radius, and the mowing task of the entire working area is completed.

2. The intelligent lawnmower work control method according to claim 1, characterized in that, The step of extracting the boundary contour track points of the working area within a view range according to the images collected by the binocular camera comprises: Stereo matching is performed on two images captured by the binocular camera to obtain a matching point pair in the two images; The disparity of each pixel point is calculated according to the matching point pair obtained by stereo matching to obtain a disparity map; The three-dimensional information of the working area within the current view range is recovered using the disparity map to obtain three-dimensional point cloud data of the working area within the current view range; The boundary contour track points are extracted from the three-dimensional point cloud data of the working area within the current view range.

3. The intelligent lawnmower work control method according to claim 1, characterized in that, The curve fitting of the boundary contour track points within the current view range comprises: Edge detection is performed on the currently collected image to obtain a boundary curve; The boundary curve is sampled at a fixed step d to decompose into a plurality of points to form a boundary curve sequence point; A starting point is randomly selected in the boundary curve sequence point, and a fixed-size window n is selected along the search direction of the boundary curve, n being the length of the window; Linear fitting is performed on each point in the current window using the least square method from the current starting point of the boundary curve according to the current window, and the fitting error is calculated; The size between the calculated fitting error and the preset error threshold v is compared. If the minimum fitting error is less than the preset error threshold v, the current window size is enlarged to n+1, and the fitting error is recalculated and compared with the preset error threshold v again; If the fitting error is greater than or equal to the preset error threshold v, the current point is selected as a segment end point, a straight line L is obtained from the current start point to the current segment end point, the generation of a segment straight line is completed, the current segment end point is taken as a start point of a next segment straight line, a window with a fixed size n is selected along a search direction of the boundary curve, and the fitting error is recalculated and compared with the preset error threshold v again until all points on the edge curve sequence point are traversed.

4. The intelligent lawnmower work control method according to claim 1 or 2 or 3, characterized in that, The stitching of the map of the previous view range and the map of the current view range comprises: Respectively acquiring first point cloud data from the map of the current view range and second point cloud data from the map of the previous view range; Respectively extracting feature points from the first point cloud data and the second point cloud data and generating feature descriptors; Matching the feature descriptors corresponding to the first point cloud data and the second point cloud data to obtain matched feature point pairs; Registering the first point cloud data and the second point cloud data according to the matched feature point pairs; Stitching the registered first point cloud data and second point cloud data to generate stitched map point cloud data.

5. The intelligent lawnmower work control method according to claim 4, characterized in that, The registration of the first point cloud data and the second point cloud data according to the matched feature point pairs comprises: Preliminarily registering the first point cloud data and the second point cloud data using a coarse registration algorithm according to the matched feature point pairs; Iteratively optimizing the preliminarily registered point cloud data to adjust the position relationship between the first point cloud data and the second point cloud data, so as to realize accurate registration; Performing global optimization on the result of accurate registration, so that the stitching result has consistency in a global range.

6. A multi-intelligent mower cooperative work control method, characterized by the steps of Comprise: Obtain the map information of the working area; If the working area is a regular shape, divide the working area into a plurality of sub-areas; Control a plurality of intelligent mowers to enter the working area from the boundaries of the sub-areas; Control each intelligent mower to complete the mowing work of each sub-area according to the method of any one of claims 1-5.

7. The multi-intelligent mower cooperative work control method according to claim 6, characterized in that, If the working area is an irregular shape, use an improved biological excitation neural network algorithm to plan the driving path of each intelligent mower, and the steps comprise: Grid processing the map and marking in the obstacle grid; Mark the activity value of the obstacle point as a first activity value, and the activity value of the remaining to-be-worked area as a second activity value, wherein each intelligent mower is regarded as an obstacle between each other; Initialize the activity value of all neurons to 0, and update the activity value of the neurons according to the preset frequency map and the following formula: wherein x i is the activity value of the i-th neuron, A , B , D respectively represent the decay rate of the activity value and the upper and lower values of the neuron activity, represents the stimulus input of the neuron, the calculation expression is: wherein, is a monotonically decreasing function, is an inhibitory signal of the neuron; When the intelligent mower is at the grid position, the intelligent mower selects the optimal path according to the activity value of the surrounding neurons and the penalty function until the work of the current area is completed.

8. The multi-intelligent mower cooperative work control method according to claim 7, characterized in that, The intelligent mower selects the next path point at the next moment according to the activity value of the surrounding neurons and the penalty function according to the following formula: Wherein: wherein a, c are respectively the activity value coefficient and the turning coefficient, is an evaluation function, (x qj , y qj ) is the coordinate position of the next time, (x qc , y qc ) is the coordinate position of the current time, and (x qp , y qp ) is the coordinate position of the previous time, is a corner, represents the path point of the current time, represents the path point of the next time, represents the path point of the next time selects the path point of the next time .

9. The multi-intelligent mower cooperative work control method according to claim 7 or 8, characterized in that, When two or more intelligent mowers exist and select the same point as the next path point, the priority value of each mower is calculated according to the following formula : In the formula, E i is the storage energy value of the i-th intelligent mower, T pi is the work amount already completed by the i-th intelligent mower, P i is the priority coefficient of the i-th intelligent mower; The intelligent mower with the maximum K value is selected to perform the task of the conflict point, and the rest of the intelligent mowers continue to select the next path point according to the updated activity field.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by a processor to implement the method of any one of claims 1-9.

Images