Point cloud data analysis-based mowing robot autonomous obstacle avoidance method and device, medium and terminal
Through the method based on point cloud data analysis, point cloud data is collected and preprocessed, obstacle layers are constructed and multi-frame data sets are fused, which solves the shortcomings of mowing robots in route planning and obstacle avoidance, and achieves more reasonable route planning and more accurate obstacle avoidance behavior.
Patent Information
- Application Number
- CN202311376844.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-23
- Publication Date
- 2025-04-25
AI Technical Summary
The existing mowing robots are unreasonable in route planning, unable to effectively divide grasslands and obstacles, are susceptible to strong sunlight, and lack information memory of historical obstacles, resulting in the inability to perform obstacle avoidance behaviors in time at border bends or in blind spots in sight.
Using a method based on point cloud data analysis, we use point cloud data to collect and preprocess point cloud data, build obstacle layers and fusion of multi-frame data sets, and formulate obstacle avoidance strategies to optimize route obstacle avoidance. Specific steps include grass threshold segmentation, downsampling, coordinate system conversion, and fusion update and optimization of obstacle layers.
During the work process, the grass mowing robot constructs a reasonable route plan based on the obstacle information during the time period, effectively identifying high grass areas, improving the work accuracy in the strong light state, and giving the robot the ability to remember historical obstacle information, so that it can avoid obstacles in a timely manner at the bends of the boundary and in blind spots of sight.
Smart Images
Figure CN119882711A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent robots, and in particular to an autonomous obstacle avoidance method, device, medium and terminal for a lawn mowing robot based on point cloud data analysis. Background Art
[0002] With the increasing degree of urban greening in China, the scale of lawn greening in residential areas, villa areas and other places has gradually increased, and parks, golf courses and other entertainment and leisure places have been trimming and maintaining lawns to improve the beauty of the environment. In foreign countries, the culture of planting family lawns is also very popular, and the law also has relevant regulations on lawn trimming. Therefore, lawn trimming has become repetitive and cumbersome, requiring a lot of material and manpower. The use of lawn mowing robots can effectively reduce the labor cost of lawn maintenance operations. At present, there are two main types of lawn mowing robots on the market, namely non-intelligent lawn mowing robots and intelligent lawn mowing robots. Non-intelligent lawn mowing robots require manual operation or follow preset paths to mow, and cannot achieve planned mowing. The random walking method has a high repetition rate, low efficiency, high energy consumption, and great damage to the lawn; intelligent lawn mowing robots benefit from the support of various sensors and have the functions of autonomous navigation and autonomous obstacle avoidance. Different sensors have slightly different obstacle avoidance strategies. However, when facing dynamic obstacles on the grass and complex grass environment, the autonomous obstacle avoidance ability of intelligent lawn mowing robots seems to need to be improved. They often tend to regard grass information as obstacles and mistakenly trigger obstacle avoidance behavior.
[0003] TOF (Time of Flight) is a three-dimensional distance measurement technology that calculates the distance between an object and a sensor by measuring the time it takes for light to be emitted from a transmitter to be received by a receiver. It uses the speed of light and the time difference to calculate the distance, and can achieve three-dimensional spatial perception of an object. When point cloud data is combined with corresponding sensors for obstacle avoidance, it has the characteristics of fast response and strong real-time performance, and can provide timely obstacle distance information, which helps the robot to achieve fast and accurate obstacle avoidance. However, the existing point cloud technology has problems such as the inability of lawn mowing robots to effectively separate grass from obstacles, and they are easily disturbed by strong sunlight. At the same time, they lack memory of historical obstacle information, which makes it impossible to make timely obstacle avoidance actions at boundary turns or blind spots. Summary of the invention
[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present application is to provide a method for solving the problems that the existing lawn mowing robots have unreasonable route planning, cannot effectively separate the grass and obstacles, are easily disturbed by strong sunlight, and lack memory of historical obstacle information, resulting in the inability to perform obstacle avoidance actions in time at boundary turns or blind spots.
[0005] To achieve the above-mentioned purpose and other related purposes, the first aspect of the present application provides an autonomous obstacle avoidance method for a lawn mowing robot based on point cloud data analysis, comprising: collecting a point cloud dataset of the lawn mowing robot during navigation, and preprocessing the point cloud dataset; constructing an obstacle layer, embedding the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion; formulating an obstacle avoidance strategy based on the obstacle layer fused with the point cloud dataset, so as to perform route obstacle avoidance according to the obstacle avoidance strategy.
[0006] In some embodiments of the first aspect of the present application, the obstacle layer is constructed, and the point cloud dataset is embedded in the constructed obstacle layer to perform multi-frame dataset fusion, and the process includes: constructing the obstacle layer and initializing it; mapping the point cloud dataset in the world coordinate system to the map coordinate system; in the map coordinate system, saving the multi-frame dataset and fusing and updating the multi-frame dataset with the initialized obstacle layer; wherein the multi-frame dataset includes multiple frame datasets; and each frame dataset is composed of all point cloud data in the same frame.
[0007] In some embodiments of the first aspect of the present application, the method includes pre-processing the point cloud data by performing grass threshold segmentation on the point cloud dataset; and / or, by downsampling the point cloud dataset; and / or, by performing coordinate system conversion on the point cloud dataset.
[0008] In some embodiments of the first aspect of the present application, the process of performing high grass filtering processing on the point cloud dataset includes: performing cluster analysis on the point cloud dataset according to a grass height threshold to identify and filter the point cloud dataset in which the tall grass area is located.
[0009] In some embodiments of the first aspect of the present application, the process of performing high grass filtering processing on the point cloud dataset includes: performing cluster analysis on the point cloud dataset according to a grass height threshold to identify and filter the point cloud dataset in which the tall grass area is located.
[0010] In some embodiments of the first aspect of the present application, the process of performing coordinate system conversion on the point cloud dataset includes: converting the point cloud dataset from a camera coordinate system to a robot coordinate system, and then converting from the robot coordinate system to a world coordinate system.
[0011] In some embodiments of the first aspect of the present application, the method further includes: performing an optimization operation on the obstacle layer after the obstacle layer is fused and updated; the optimization operation includes: in the obstacle layer after the fusion and update operation, dilating the obstacle edge in the obstacle layer.
[0012] In some embodiments of the first aspect of the present application, an obstacle avoidance strategy is formulated based on an obstacle layer fused with a point cloud data set, so as to perform route obstacle avoidance according to the obstacle avoidance strategy, including: based on the current position information of the mowing robot and the obstacle layer fused with the point cloud data set, in response to the initial trajectory of the mowing robot coinciding with an obstacle in the obstacle layer, traversing multiple free area points in the initial trajectory, determining the free area point closest to the obstacle among the multiple free area points, and setting the free area point as the trajectory end point. An obstacle avoidance curve is planned based on the current position of the mowing robot and the trajectory end point, and the planned obstacle avoidance curve is optimized using a smoothing algorithm; based on the optimized planned obstacle avoidance curve, the mowing robot is controlled to perform obstacle avoidance.
[0013] To achieve the above-mentioned purpose and other related purposes, the second aspect of the present application provides an autonomous obstacle avoidance device for a lawn mowing robot based on point cloud data analysis. Data acquisition and preprocessing module: used to acquire the point cloud data set of the lawn mowing robot during navigation and preprocess the point cloud data set; obstacle layer construction module: used to construct an obstacle layer, embed the point cloud data set into the constructed obstacle layer to perform multi-frame data set fusion; obstacle avoidance strategy planning module: used to formulate an obstacle avoidance strategy based on the obstacle layer fused with the point cloud data set, so as to perform route obstacle avoidance according to the obstacle avoidance strategy.
[0014] To achieve the above-mentioned purpose and other related purposes, the third aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the autonomous obstacle avoidance method of the lawn mowing robot based on point cloud data analysis.
[0015] To achieve the above-mentioned purpose and other related purposes, the fourth aspect of the present application provides an electronic terminal, including: a processor and a memory; the memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes the autonomous obstacle avoidance method of the lawn mowing robot based on point cloud data analysis.
[0016] As described above, the autonomous obstacle avoidance method, device, medium and terminal of the lawn mower robot based on point cloud data analysis in the field of intelligent robots of the present application have the following beneficial effects: the method proposed in the present invention can enable the lawn mower robot to construct a reasonable route planning according to the obstacle information within the time period during work, and can effectively identify the high grass area to avoid area omission. The present invention also improves the working accuracy of the lawn mower robot under strong light conditions by preprocessing the point cloud data. Because the present invention enables the lawn mower robot to have the ability to memorize historical obstacle information, the lawn mower robot can make timely obstacle avoidance actions at boundary turns and blind spots. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 Shown is a schematic flow chart of an autonomous obstacle avoidance method for a lawn mowing robot based on point cloud data analysis in one embodiment of the present application.
[0018] Figure 2 Shown is a schematic diagram of the process of obstacle avoidance of a lawn mowing robot based on point cloud data in one embodiment of the present application.
[0019] Figure 3 Shown is a schematic diagram of point cloud data preprocessing in one embodiment of the present application.
[0020] Figure 4a Shown is a schematic diagram of point cloud data before downsampling in one embodiment of the present application.
[0021] Figure 4b Shown is a schematic diagram of point cloud data downsampling in one embodiment of the present application.
[0022] Figure 5 Shown is a schematic diagram of an obstacle avoidance strategy in an embodiment of the present application.
[0023] Figure 6 Shown is a schematic diagram of an autonomous obstacle avoidance device for a lawn mowing robot based on point cloud data analysis in one embodiment of the present application.
[0024] Figure 7 Shown is a schematic diagram of the structure of an electronic terminal in one embodiment of the present application. DETAILED DESCRIPTION
[0025] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0026] It should be noted that in the following description, with reference to the accompanying drawings, several embodiments of the present application are described in the accompanying drawings. It should be understood that other embodiments may also be used, and mechanical composition, structure, electrical and operational changes may be made without departing from the spirit and scope of the present application. The following detailed description should not be considered restrictive, and the scope of the embodiments of the present application is limited only by the claims of the published patents. The terms used here are only for describing specific embodiments and are not intended to limit the present application. Spatially related terms, such as "upper", "lower", "left", "right", "below", "below", "lower", "above", "upper", etc., may be used in the text to facilitate the description of the relationship between an element or feature shown in the figure and another element or feature.
[0027] In this application, unless otherwise clearly specified and limited, the terms "install", "connect", "connect", "fix", "hold" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.
[0028] Furthermore, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless there is an indication to the contrary in the context. It should be further understood that the terms "comprise", "include" indicate the presence of the described features, operations, elements, components, items, kinds, and / or groups, but do not exclude the presence, occurrence or addition of one or more other features, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Therefore, "A, B or C" or "A, B and / or C" means "any of the following: A; B; C; A and B; A and C; B and C; A, B and C". Exceptions to this definition will only occur when the combination of elements, functions or operations is inherently mutually exclusive in some way.
[0029] In order to solve the problems in the above-mentioned background technology, the present invention provides a method, device, medium and terminal for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis, which aims to solve the problem of filtering shallow grass / tall grass based on point cloud data of the lawn mower robot, thereby realizing the segmentation of grass and obstacles. At the same time, it can also solve the problem that the point cloud data obtained by the lawn mower camera will be disturbed by strong sunlight during the day and generate fragmented point cloud points, thereby avoiding accidental touches during obstacle avoidance behavior; at the same time, it solves the problem that the obstacle point cloud data is too dense, dilutes the obstacle point cloud data, and perfectly retains the basic morphological characteristics of the obstacle. In addition, it solves the problem that the lawn mower lacks memory of historical obstacle information, which leads to the problem that the lawn mower cannot make obstacle avoidance strategies in time under certain circumstances (for example: at the boundary turning point, there is a blind spot during the turning process).
[0030] At the same time, in order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution in the embodiment of the present invention is further described in detail through the following embodiments and in combination with the accompanying drawings. It should be understood that the specific embodiments described here are only used to explain the present invention and are not used to limit the invention.
[0031] Before further describing the present invention in detail, the nouns and terms involved in the embodiments of the present invention are explained. The nouns and terms involved in the embodiments of the present invention are applicable to the following interpretations:
[0032] <1> Point cloud data: Point cloud data is a collection of discrete points in three-dimensional space acquired by lidar, depth cameras or other sensors. Each point contains its position information in space and possible other attributes, such as color, normal, etc. Point cloud data is often used in applications such as 3D modeling, map construction, and target detection.
[0033] <2> TOF point cloud data: TOF (Time of Flight) point cloud data is point cloud data obtained by a TOF camera (Time of Flight Camera). The TOF camera calculates the distance of the object surface by measuring the round-trip time of light. Each point in the TOF point cloud data contains three-dimensional coordinates and distance values, which can provide more accurate distance information.
[0034] <3> Grass filtering: Grass filtering is a point cloud data processing algorithm that is mainly used to remove debris such as grass and low vegetation in point cloud data in order to better identify and analyze targets such as buildings, roads, and vehicles on the ground. The grass filtering algorithm analyzes the height information of each point in the point cloud data, marks points with a height below a certain threshold as debris, and deletes them from the point cloud data.
[0035] <4> Obstacle layer: An obstacle layer is a data structure used to represent the location and shape of obstacles in the environment. In applications such as autonomous driving and robot navigation, obstacle layers are often used for tasks such as path planning and obstacle avoidance. Obstacle layers can contain information such as point cloud data, object bounding boxes, and grid representations.
[0036] <5> Cost map: A cost map is a data structure used to represent the cost or weight of different areas in an environment. Cost maps are often used for tasks such as path planning and obstacle avoidance. Different cost values can be assigned to each area according to the characteristics of the environment to indicate the feasibility or priority of the area.
[0037] <6> Dilation: Dilation is an image processing or geometry processing algorithm that is used to expand or dilate an image or geometric shape. In obstacle detection and avoidance, dilation is often used to expand the obstacle area by a certain distance to better consider the obstacle boundary and obstacle avoidance path.
[0038] <7> Bezier curve: A Bezier curve is a mathematical curve defined by a series of control points. Bezier curves have the characteristics of smoothness and local control and are often used in computer graphics, animation, and curve design.
[0039] <8> PID regulation control: PID regulation control is a commonly used feedback control algorithm used to adjust the output of a system to achieve a desired value. The PID controller calculates the control signal based on the system's error, error change rate, and error integral to achieve system stability and responsiveness.
[0040] <9> RTK sensor: RTK (Real-Time Kinematic) sensor is a sensor used to measure position and attitude, which can obtain high-precision position and attitude information in real time.
[0041] <10> IMU Inertial Measurement Unit: IMU is the abbreviation of Inertial Measurement Unit, which is a device that integrates multiple inertial sensors, including accelerometers, gyrometers, and magnetometers, etc. It is used to measure and record the acceleration and angular velocity of an object.
[0042] The embodiments of the present invention provide a method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis, a system for the method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis, and a storage medium storing an executable program for implementing the method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis. As for the implementation of the method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis, the embodiments of the present invention will describe an exemplary implementation scenario of the method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis.
[0043] like Figure 1 FIG. 2 is a flow chart showing a method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis in an embodiment of the present invention. The method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis in this embodiment mainly includes the following steps:
[0044] Step S11: collecting a point cloud data set of the lawn mowing robot during navigation, and preprocessing the point cloud data set.
[0045] In one embodiment of the present invention, the method includes pre-processing the point cloud data by performing grass threshold segmentation processing on the point cloud data set; and / or, downsampling processing on the point cloud data set; and / or, pre-processing the point cloud data by performing coordinate system conversion on the point cloud data set.
[0046] Furthermore, the technical solution included in the present invention can be the grassland threshold segmentation processing among the above three processing methods, or the grassland threshold segmentation processing and downsampling processing can be used simultaneously. A more preferred solution is to use the grassland threshold segmentation processing, downsampling processing and coordinate system conversion methods for preprocessing. When the three processing methods are used simultaneously, the point cloud data set can be preprocessed by using the grassland threshold segmentation processing, downsampling processing and coordinate system conversion methods in sequence.
[0047] It is worth noting that, since the amount of point cloud data collected is too dense and huge, directly calculating the collected point cloud data can also well describe the morphological characteristics of obstacles, but it will greatly increase the time complexity of data processing, thereby affecting the real-time obstacle avoidance performance of the mowing robot during navigation. However, the technical problem to be solved by the present invention is to isolate and efficiently identify obstacles and plan obstacle avoidance routes for the mowing robot, while solving the problem of difficulty in effectively distinguishing tall grass areas from obstacle areas. Therefore, the present invention deliberately sets a tall grass area filtering algorithm and a point cloud data preprocessing algorithm, so that while reducing the amount of point cloud data, it can also maintain the main morphological characteristics of the obstacle point cloud data as much as possible, so as to achieve the effect of efficient navigation and obstacle avoidance.
[0048] In one embodiment of the present invention, the process of performing high grass filtering processing on the point cloud dataset includes: performing cluster analysis on the point cloud dataset according to a grass height threshold to identify and filter the point cloud dataset in the high grass area.
[0049] In one embodiment of the present invention, the point cloud data used is a TOF point cloud data set. Figure 2The flowchart in this embodiment is shown. The specific process of collecting the point cloud data set of the lawn mower robot during navigation includes installing the TOF camera on the lawn mower robot and keeping a preset height distance from the ground to filter the shallow grass area in the image collected by the TOF camera. For example, the distance between the TOF camera and the ground is set to 0.18 cm. According to the test results of this embodiment, by setting the height threshold to -0.13, the shallow grass area can be completely filtered.
[0050] Furthermore, for the tall grass area, we first perform cluster analysis on the point cloud data set after the shallow grass filtering within a preset range to further filter the tall grass area in the point cloud data set. Exemplarily, the height range of the point cloud data for cluster analysis is set between -0.13 and 0.0, and the K-means algorithm is used to analyze the point cloud data within the preset range. If the clustered point cloud data occupies 60% of the horizontal x range corresponding to the point cloud data value of each y dimension in front of the camera, it is considered to be a tall grass area and is filtered out; otherwise, the point cloud data is retained and regarded as an obstacle, so that the local small area of tall grass is identified as an obstacle.
[0051] In one embodiment of the present invention, the process of downsampling the point cloud data set includes: constructing a three-dimensional cube based on the point cloud data set, and dividing the three-dimensional cube into multiple sub-cubes; determining the amount of point cloud data contained in each sub-cube, identifying and filtering the point cloud data corresponding to the sub-cubes whose data amount is lower than a preset threshold; and obtaining the characteristic value used to characterize the sub-cube for the point cloud data set in each sub-cube.
[0052] Furthermore, based on the point cloud data set, the process of constructing a three-dimensional cube includes: creating a three-dimensional cube according to the point cloud data range collected by the camera set on the lawn mowing robot, wherein the number of sub-cubes in each dimension is the range of the dimension divided by the set voxel value, and then storing the obstacle point cloud data collected by the camera in turn into its corresponding three-dimensional sub-cube. Exemplarily, the selected point cloud data range is (x: 0-1, y: -1.5-1.5, z: -0.18-0.5), and the voxel value is set to 0.03.
[0053] In order to further improve the effect of obstacle recognition of the lawn mowing robot, the fragmented point cloud data in the sub-cube is removed. The specific process includes: identifying and judging the amount of point cloud data for each of the above sub-cubes, and removing the fragmented point cloud data through a preset threshold. For example, because the point cloud data in the obstacle area is densely distributed, taking the preset threshold as 5 can reasonably remove the fragmented points while retaining the obstacle characteristics. That is, if the amount of point cloud data contained in a single sub-cube is less than 5, the point cloud data in the sub-cube is deleted, otherwise it is retained.
[0054] like Figure 3 The schematic diagram of an embodiment of the present invention shows that the original collected point cloud data set is pre-processed by means of grassland threshold segmentation, downsampling and coordinate system conversion. Figure 3 It can be seen that the amount of point cloud data can be greatly reduced through grass filtering, and the noise points in the point cloud data are greatly reduced after voxel filtering. Figure 4a , Figure 4b The schematic diagram before and after downsampling of the point cloud data are shown. It can be seen that the downsampling operation can significantly reduce the amount of point cloud data while preserving the features of the point cloud data.
[0055] Furthermore, the process of obtaining the characteristic value for characterizing the sub-cube from the point cloud data set in each sub-cube includes: obtaining the characteristic value of the horizontal coordinate value (x), the vertical coordinate value (y) and the height coordinate value (z) of the point cloud data in each sub-cube. For example, Figure 3 As shown, a schematic diagram of downsampling of point cloud data in an embodiment of the present invention is shown. In this embodiment, the eigenvalue is extracted by summing and averaging the point cloud data in each sub-cube, and the single eigenvalue point is used to replace all other points in its sub-cube, so as to efficiently achieve the effect of reducing the dimension of the point cloud data. The dimensionality reduction method proposed by the present invention can effectively reduce the amount of point cloud data to about 1 / 15 of the original amount, greatly reducing the amount of data while also ensuring the original form of the point cloud, which can ensure the obstacle avoidance effect.
[0056] It is worth noting that the present invention does not limit the method for obtaining the eigenvalue, and the above-mentioned method of obtaining the average value by summing is only used to explain the present invention and is not a limitation. The methods that can be used to obtain the eigenvalue in the present invention include but are not limited to: obtaining the average value, median, median and weighted average of the point cloud data in each sub-cube.
[0057] In one embodiment of the present invention, the process of converting the coordinate system of the point cloud data set includes: converting the point cloud data set from the camera coordinate system to the robot coordinate system, and then converting it from the robot coordinate system to the world coordinate system. Among them, the cloud data set in the camera coordinate system refers to a coordinate system constructed with the camera installed on the lawn mower robot as the center. The robot coordinate system refers to a coordinate system constructed with the lawn mower as the center. It is worth noting that the reason for converting from the camera coordinate system to the robot coordinate system is that the corresponding collected images are different depending on the installation position of the camera on the lawn mower robot. According to the position information of the camera relative to the lawn mower robot, the point cloud data set is converted from the camera coordinate system to the lawn mower coordinate system, so as to generate an obstacle avoidance route centered on the lawn mower robot in the subsequent steps.
[0058] Furthermore, the process of converting the point cloud data set from the camera coordinate system to the robot coordinate system includes calculating an offset value based on the installation position of the camera on the mowing robot, converting the point cloud data set x in the camera coordinate system to camera ,y camera 、z camera Add the offset value to obtain the point cloud coordinate information in the robot coordinate system (as shown in Formula 1).
[0059]
[0060] Among them, x robot ,y robot 、z robot Represents the point cloud coordinate information in the robot coordinate system, x dis ,y dis 、z dis Indicates the offset value between the camera coordinate system and the robot coordinate system in the x, y, and z directions.
[0061] Furthermore, the process of transferring the point cloud data from the robot coordinate system to the world coordinate system includes: obtaining the position information of the mowing robot in the world coordinate system through the RTK sensor, obtaining the radian value of the orientation angle of the mowing robot through the IMU inertial measurement unit robot yaw , and then use the rotation matrix plus the translation matrix to get the point cloud data in the world coordinate system x world ,y world 、z world As shown in Formula 2, it shows the process of transforming the obstacle point cloud data from the camera coordinate system to the robot coordinate system and finally to the world coordinate system.
[0062]
[0063] Among them, cos(robot yaw ),sin(robot yaw) represents the radian value of the robot's heading angle robot yaw The sin and cos values of robot x and robot y Represents the position information of the lawn mower robot, that is, the x and y coordinates of the lawn mower robot in the world coordinate system.
[0064] Step S12: construct an obstacle layer, and embed the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion.
[0065] In one embodiment of the present invention, the obstacle layer is constructed, and the point cloud dataset is embedded in the constructed obstacle layer to fuse multiple frame datasets, and the process includes: constructing the obstacle layer and initializing it; mapping the point cloud dataset in the world coordinate system to the map coordinate system; in the map coordinate system, saving multiple frame datasets and fusing and updating the multiple frame datasets with the initialized obstacle layer; wherein the multiple frame dataset includes multiple frame datasets; each frame dataset is composed of all point cloud data in the same frame. The obstacle layer is used to represent the cost map of obstacles identified from the point cloud data.
[0066] Furthermore, the process of constructing and initializing the obstacle layer includes: constructing an initial mowing area map based on the mapping mode of the mowing robot, and constructing an obstacle layer according to the size, origin and resolution of the initial mowing area map to ensure that the obstacle layer can be fused with the point cloud data.
[0067] Specifically, the point cloud data set in the world coordinate system is mapped to the map coordinate system and the process includes: if the point cloud data mapping after coordinate system conversion is located in a grid area in the map coordinate system, the obstacle information is represented in the cost map of the obstacle map by setting the corresponding grid value to 255.
[0068] It is worth noting that in order to improve the memory function of the lawn mower robot for obstacle information and improve the ability to identify obstacles. The present invention not only relies on the latest single-frame obstacle point cloud data to represent the obstacle information in the environment, but also combines multiple frames of obstacle point cloud data sets within a preset time for fusion and update. Exemplarily, when the frame rate of the camera carried by the lawn mower robot is 13 frames, the point cloud data within 5 seconds is saved, that is, 65 frames of point cloud data are fused and updated, so that the sweeping robot can keep memory of obstacles within the previous 5 seconds.
[0069] In one embodiment of the present invention, the process of saving a multi-frame data set and fusing and updating the multi-frame data set with the initialized obstacle layer includes: setting the value of the grid information corresponding to the front queue in the multi-frame data set queue and the obstacle layer in the map coordinate system to 0 to represent the free area. Then, the grid information of the obstacle map is updated according to the newly generated point cloud data, and the index value of the point cloud data of the newly generated point in the map coordinate system is pushed into the data queue, so that the mowing robot can have a memory function for the obstacle point cloud information within a certain time range, and better realize the maintenance of the obstacle layer.
[0070] In one embodiment of the present invention, after the obstacle layer is fused and updated, an optimization operation is also performed on the obstacle layer; the optimization operation includes: in the obstacle layer after the fusion and update operation, an expansion process is performed on the obstacle edge in the obstacle layer. Exemplarily, the obstacle is expanded outward from the center according to the radius of 0.5m of the circumscribed circle of the lawn mowing robot.
[0071] Step S13: formulating an obstacle avoidance strategy according to the obstacle layer fused with the point cloud data set, so as to avoid obstacles along the route according to the obstacle avoidance strategy.
[0072] In one embodiment of the present invention, an obstacle avoidance strategy is formulated based on an obstacle layer fused with a point cloud data set, so as to perform route obstacle avoidance according to the obstacle avoidance strategy, including: based on the current position information of the mowing robot and the obstacle layer fused with the point cloud data set, in response to the initial trajectory of the mowing robot coinciding with an obstacle in the obstacle layer, traversing multiple free area points in the initial trajectory, determining the free area point closest to the obstacle among the multiple free area points, and setting the free area point as the trajectory end point. An obstacle avoidance curve is planned based on the current position of the mowing robot and the trajectory end point, and the planned obstacle avoidance curve is optimized using a smoothing algorithm; based on the optimized planned obstacle avoidance curve, the mowing robot is controlled to perform obstacle avoidance.
[0073] Specifically, Figure 5 As shown in the figure, a schematic diagram of the mowing robot's route planning is shown. The current position information of the mowing robot is obtained through the RTK sensor, and the ranging range and map resolution of the camera mounted on the mowing robot are used to determine whether the point in the existing trajectory of the mowing robot coincides with the obstacle in the obstacle layer. If so, the subsequent points of the obstacle point on the trajectory are traversed in sequence until the value of the next point on the obstacle layer is a free area point, the current position of the mowing robot is set as the starting point, and the above-mentioned free area point is set as the end point.
[0074] Furthermore, after setting the starting point and the end point of the lawn mower that do not overlap with the obstacles, a path search algorithm is used to plan the obstacle avoidance curve, and a curve optimization algorithm is used to optimize the obstacle avoidance curve. Exemplarily, the obstacle avoidance curve planning can be performed using the A* algorithm, the Dijkstra algorithm, the BFS algorithm, the DFS algorithm, the Bellman-Ford algorithm, and the Floyd-Warshall algorithm. At the same time, the Bezier curve algorithm, the Catmull-Rom curve algorithm, the natural spline curve algorithm, and the Gaussian filter algorithm make the lines of the generated obstacle avoidance curve smoother. Based on the optimized obstacle avoidance curve, the lawn mower robot is controlled by PID regulation to achieve the corresponding obstacle avoidance behavior according to the generated obstacle avoidance trajectory.
[0075] like Figure 6 , which shows a schematic diagram of the structure of an autonomous obstacle avoidance device for a lawn mower robot based on point cloud data analysis in an embodiment of the present invention. In this embodiment, the autonomous obstacle avoidance device 600 for a lawn mower robot based on point cloud data analysis includes:
[0076] The data collection and preprocessing module 601 is used to collect the point cloud data set of the lawn mowing robot during navigation and preprocess the point cloud data set.
[0077] The obstacle layer construction module 602 is used to construct an obstacle layer, and embed the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion.
[0078] The obstacle avoidance strategy planning module 603 is used to formulate an obstacle avoidance strategy according to the obstacle layer fused with the point cloud data set, so as to avoid obstacles along the route according to the obstacle avoidance strategy.
[0079] In one embodiment of the present invention, the data acquisition and preprocessing module 601 preprocesses the point cloud data by performing grass threshold segmentation processing on the point cloud data set; and / or, by performing downsampling processing on the point cloud data set; and / or, by performing coordinate system conversion on the point cloud data set. The process of performing high grass filtering processing on the point cloud data set includes: performing cluster analysis on the point cloud data set according to the grass height threshold to identify and filter the point cloud data set in the high grass area. The process of performing downsampling processing on the point cloud data set includes: constructing a three-dimensional cube based on the point cloud data set, and dividing the three-dimensional cube into a plurality of sub-cubes; determining the amount of point cloud data contained in each sub-cube, identifying and filtering the point cloud data corresponding to the sub-cube whose amount of data is lower than a preset threshold; and obtaining the feature value for characterizing the sub-cube for the point cloud data set in each sub-cube. The process of performing coordinate system conversion on the point cloud data set includes: converting the point cloud data set from the camera coordinate system to the robot coordinate system, and then converting from the robot coordinate system to the world coordinate system.
[0080] In one embodiment of the present invention, the obstacle layer construction module 602 constructs an obstacle layer, embeds the point cloud data set into the constructed obstacle layer to fuse multiple frame data sets, and the process includes: constructing and initializing the obstacle layer; mapping the point cloud data set in the world coordinate system to the map coordinate system; in the map coordinate system, saving multiple frame data sets and fusing and updating the multiple frame data sets with the initialized obstacle layer; wherein the multiple frame data sets include multiple frame data sets; each frame data set is composed of all point cloud data in the same frame. And after fusing and updating the obstacle layer, the obstacle layer is also optimized; the optimization operation includes: in the obstacle layer after the fusion and update operation, the obstacle edge in the obstacle layer is expanded.
[0081] In one embodiment of the present invention, the obstacle avoidance strategy planning module 603 formulates an obstacle avoidance strategy based on the obstacle layer fused with the point cloud data set, so as to perform route obstacle avoidance according to the obstacle avoidance strategy, including: based on the current position information of the mowing robot and the obstacle layer fused with the point cloud data set, in response to the initial trajectory of the mowing robot coinciding with the obstacle in the obstacle layer, traversing multiple free area points in the initial trajectory, determining the free area point closest to the obstacle among the multiple free area points, and setting the free area point as the trajectory end point. Based on the current position of the mowing robot and the trajectory end point, an obstacle avoidance curve is planned, and the planned obstacle avoidance curve is optimized by a smoothing algorithm; based on the optimized planned obstacle avoidance curve, the mowing robot is controlled to perform obstacle avoidance.
[0082] It should be noted that: the autonomous obstacle avoidance device for a lawn mower robot based on point cloud data analysis provided in the above embodiment is only illustrated by the division of the above program modules when performing autonomous obstacle avoidance for a lawn mower robot based on point cloud data analysis. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the autonomous obstacle avoidance device for a lawn mower robot based on point cloud data analysis provided in the above embodiment and the autonomous obstacle avoidance method for a lawn mower robot based on point cloud data analysis belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0083] The method for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis provided in the embodiment of the present invention can be implemented on the terminal side or the server side. As for the hardware structure of the terminal for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis, please refer to Figure 7, is an optional hardware structure diagram of a lawn mowing robot autonomous obstacle avoidance terminal 700 based on point cloud data analysis provided in an embodiment of the present invention. The terminal 700 may be a mobile phone, a computer device, a tablet device, a personal digital processing device, a factory background processing device, etc. The lawn mowing robot autonomous obstacle avoidance terminal 700 based on point cloud data analysis includes: at least one processor 701, a memory 702, at least one network interface 704 and a user interface 706. The various components in the device are coupled together through a bus system 705. It can be understood that the bus system 705 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 705 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, in Figure 7 In the specification, various buses are labeled as bus systems.
[0084] The user interface 706 may include a display, a keyboard, a mouse, a trackball, a click gun, keys, buttons, a touch pad or a touch screen.
[0085] It is understood that the memory 702 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM). The memory described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable categories of memory.
[0086] The memory 702 in the embodiment of the present invention is used to store various categories of data to support the operation of the lawn mowing robot autonomous obstacle avoidance terminal 700 based on point cloud data analysis. Examples of these data include: any executable program used to operate on the lawn mowing robot autonomous obstacle avoidance terminal 700 based on point cloud data analysis, such as an operating system 7021 and an application 7022; the operating system 7021 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application 7022 can include various applications, such as a media player (Media Player), a browser (Browser), etc., for implementing various application services. The method for implementing the lawn mowing robot autonomous obstacle avoidance method based on point cloud data analysis provided in the embodiment of the present invention can be included in the application 7022.
[0087] The method disclosed in the above embodiment of the present invention can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 701 or the instruction in the form of software. The above processor 701 may be a general processor, a digital signal processor (DSP, Digital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 701 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general processor 701 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.
[0088] In an exemplary embodiment, the lawn mower robot autonomous obstacle avoidance terminal 700 based on point cloud data analysis can be used by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD) to execute the aforementioned lawn mower robot autonomous obstacle avoidance method based on point cloud data analysis.
[0089] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk and other media that can store program codes.
[0090] In the embodiments provided in the present application, the computer readable and writable storage medium may include a read-only memory, a random access memory, an EEPROM, a CD-ROM or other optical disk storage device, a disk storage device or other magnetic storage device, a flash memory, a USB flash drive, a mobile hard disk, or any other medium that can be used to store a desired program code in the form of an instruction or data structure and can be accessed by a computer. In addition, any connection can be appropriately referred to as a computer-readable medium. For example, if the instruction is sent from a website, a server or other remote source using a coaxial cable, an optical fiber cable, a twisted pair, a digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, the coaxial cable, optical fiber cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of the medium. However, it should be understood that computer readable and writable storage media and data storage media do not include connections, carriers, signals, or other temporary media, but are intended to be non-temporary, tangible storage media. Disk and disc, as used in this application, includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers.
[0091] In summary, the present application provides a method, device, terminal and medium for autonomous obstacle avoidance of a lawn mower robot based on point cloud data analysis. The method proposed by the present invention can enable the lawn mower robot to construct a reasonable route plan according to the obstacle information within a time period during operation, and can effectively identify high grass areas to avoid area omissions. The present invention also improves the working accuracy of the lawn mower robot under strong light conditions by preprocessing the point cloud data. Because the present invention enables the lawn mower robot to have the ability to memorize historical obstacle information, the lawn mower robot can perform timely obstacle avoidance actions at boundary turns and blind spots. Therefore, the present application effectively overcomes the various shortcomings of the prior art and has a high industrial utilization value.
[0092] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. An autonomous obstacle avoidance method for a lawn mowing robot based on point cloud data analysis, characterized in that: include: Collecting a point cloud data set of the lawn mowing robot during navigation, and preprocessing the point cloud data set; Constructing an obstacle layer, and embedding the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion; An obstacle avoidance strategy is formulated according to the obstacle layer fused with the point cloud dataset, so as to avoid obstacles along the route according to the obstacle avoidance strategy.
2. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 1, characterized in that: The step of constructing an obstacle layer and embedding the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion includes: Build and initialize the obstacle layer; Mapping the point cloud data set in the world coordinate system to the map coordinate system; In the map coordinate system, a multi-frame data set is saved and the multi-frame data set is merged and updated with the initialized obstacle layer; wherein the multi-frame data set includes a plurality of frame data sets; each of the frame data sets is composed of all point cloud data in the same frame.
3. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 1, characterized in that: The method comprises performing a grass threshold segmentation process on the point cloud data set; And / or, by downsampling the point cloud data set; And / or, the point cloud data is preprocessed by performing coordinate system conversion on the point cloud data set.
4. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 3, characterized in that: The process of performing high grass filtering processing on the point cloud dataset includes: performing cluster analysis on the point cloud dataset according to a grass height threshold to identify and filter the point cloud dataset in the high grass area.
5. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 3, characterized in that: The process of downsampling the point cloud data set includes: Based on the point cloud data set, construct a three-dimensional cube, and divide the three-dimensional cube into a plurality of sub-cubes; Determine the amount of point cloud data contained in each sub-cube, identify and filter the point cloud data corresponding to the sub-cubes whose data amount is lower than a preset threshold; The feature value used to characterize the sub-cube is obtained for the point cloud data set in each sub-cube.
6. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 3, characterized in that: The process of performing coordinate system conversion on the point cloud data set includes: The point cloud data set is transformed from the camera coordinate system to the robot coordinate system, and then from the robot coordinate system to the world coordinate system.
7. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 1, characterized in that: The method further includes: performing an optimization operation on the obstacle layer after fusing and updating the obstacle layer; the optimization operation includes: in the obstacle layer after the fusing and updating operation, performing an expansion process on the obstacle edge in the obstacle layer.
8. The method for autonomous obstacle avoidance of a lawn mowing robot based on point cloud data analysis according to claim 1, characterized in that: An obstacle avoidance strategy is formulated based on the obstacle layer fused with the point cloud dataset to avoid obstacles along the route according to the obstacle avoidance strategy, including: Based on the current position information of the lawn mower robot and the obstacle layer integrated with the point cloud data set, in response to the initial trajectory of the lawn mower robot overlapping with the obstacle in the obstacle layer, traverse a plurality of free area points in the initial trajectory, determine a free area point closest to the obstacle among the plurality of free area points, and set the free area point as the trajectory end point. Planning an obstacle avoidance curve based on the current position of the mowing robot and the end point of the trajectory, and optimizing the planned obstacle avoidance curve using a smoothing algorithm; Based on the optimized planned obstacle avoidance curve, the lawn mowing robot is controlled to perform obstacle avoidance work.
9. An autonomous obstacle avoidance device for a lawn mowing robot based on point cloud data analysis, characterized in that: include: Data acquisition and preprocessing module: used to collect point cloud data sets of the mowing robot during navigation and preprocess the point cloud data sets; Obstacle layer construction module: used to construct an obstacle layer, embed the point cloud dataset into the constructed obstacle layer to perform multi-frame dataset fusion; Obstacle avoidance strategy planning module: used to formulate an obstacle avoidance strategy based on the obstacle layer fused with the point cloud data set, so as to avoid obstacles along the route according to the obstacle avoidance strategy.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the autonomous obstacle avoidance method for a lawn mowing robot based on point cloud data analysis as described in any one of claims 1 to 8 is implemented.
11. An electronic terminal, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory so that the terminal executes the autonomous obstacle avoidance method for a lawn mowing robot based on point cloud data analysis as described in any one of claims 1 to 8.