Obstacle recognition method and automatic cleaning device

Abstract

This application discloses an obstacle recognition method and an automatic cleaning device. After acquiring multiple initial point cloud data frames from a ranging sensor, this application filters each initial point cloud data frame to obtain a target point cloud data frame corresponding to the obstacle. Then, based on the point cloud features of the point cloud dataset within a preset distance range in each target point cloud data frame, the obstacle category of each point cloud dataset is determined. Based on the obstacle categories of all point cloud datasets, the target obstacle category corresponding to the area where each point cloud dataset is located is determined. Finally, the automatic cleaning device is controlled to avoid obstacles based on the target obstacle category. The automatic cleaning device of this application can utilize the laser emitted by the ranging sensor to acquire environmental information over a large area to identify obstacles, eliminating the need for multiple retraction actions to scan complete obstacle information, thus improving the efficiency of obstacle recognition and overall work efficiency.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to an obstacle recognition method and an automatic cleaning device. Background Technology

[0002] In existing technologies, most robots use dual-line / single-line structured light sensors combined with RGB cameras for obstacle avoidance. Structured light obstacle avoidance technology emits light of a specific pattern (such as linear or planar light) onto an object and receives the reflected light signals to calculate the distance between the device and the obstacle. Meanwhile, RGB cameras can capture color images of the surrounding environment and identify obstacles through image processing algorithms.

[0003] However, since the single-frame point cloud that dual-line / single-line structured light sensors can collect is a point cloud of one or two line segments, it is difficult to obtain the complete point cloud information of the obstacle from the single-frame point cloud. The robot needs to perform multiple retraction actions before it can collect the complete point cloud information of the obstacle. The collection process is complicated and cumbersome, and the obstacle recognition efficiency is low. Summary of the Invention

[0004] This invention provides an obstacle recognition method and an automatic cleaning device, which solves the technical problem of low obstacle recognition efficiency of robots in the prior art.

[0005] In a first aspect, embodiments of the present invention provide an obstacle recognition method applicable to an automatic cleaning device. The automatic cleaning device has a ranging sensor positioned in front of it. The ranging sensor includes a laser emitter and a laser receiver. The laser emitter emits one or more combinations of dot matrix lasers and area lasers. The laser receiver receives light signals reflected from environmental objects by the laser. The ranging sensor generates environmental information based on the light signals. The method includes:

[0006] The ranging sensor collects environmental information, which includes multiple frames of initial point cloud data.

[0007] Each initial point cloud data frame is filtered to obtain the target point cloud data frame corresponding to the obstacle.

[0008] Based on the point cloud features of the point cloud dataset within a preset distance range of each target point cloud data frame, determine the obstacle category corresponding to each point cloud dataset.

[0009] Based on the obstacle categories of each point cloud dataset in all the target point cloud data frames, determine the target obstacle category of the region where each point cloud dataset is located;

[0010] The automatic cleaning equipment is controlled to avoid obstacles based on the type of the target obstacle.

[0011] The step of filtering each initial point cloud data frame to obtain a target point cloud data frame corresponding to the obstacle includes:

[0012] Project the point cloud data of each initial point cloud data frame onto a horizontal plane to obtain the point cloud projection point set in the horizontal plane.

[0013] Cluster the point cloud projection point set corresponding to each frame of the initial point cloud data frame to obtain at least one planar cluster corresponding to each frame of the initial point cloud data frame.

[0014] For each initial point cloud data frame, based on the height of the point cloud data of each corresponding planar cluster, the point cloud data corresponding to the obstacle is filtered out in each initial point cloud data frame to obtain the target point cloud data frame.

[0015] Specifically, for each initial point cloud data frame, based on the height of the point cloud data of each corresponding planar cluster, point cloud data corresponding to the obstacle is filtered out from each initial point cloud data frame to obtain a target point cloud data frame, including:

[0016] For each initial point cloud data frame, the maximum height, minimum height, and height variance are determined based on the height of the point cloud data corresponding to each planar cluster.

[0017] Based on the number of point clouds, maximum height, minimum height, and height variance corresponding to each planar cluster, determine whether the object type corresponding to each planar cluster is ground;

[0018] Point cloud data corresponding to planar clusters of the object type "ground" are removed from the initial point cloud data frame to obtain the target point cloud data frame corresponding to the obstacle.

[0019] Before removing point cloud data corresponding to planar clusters of the object type "ground" from the initial point cloud data frame, the process further includes:

[0020] Plane fitting is performed on the point cloud data corresponding to the planar clusters of objects of the ground type to obtain the plane fitting equation;

[0021] Determine the transformation matrix between the plane fitting equation and the standard Z-axis zero plane;

[0022] After obtaining the target point cloud data frame corresponding to the obstacle, the method further includes:

[0023] The point cloud data in the target point cloud data frame is subjected to coordinate transformation according to the transformation matrix to obtain a target point cloud data frame with the standard Z-axis zero plane as the reference horizontal plane.

[0024] The step of determining the obstacle category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each frame of the target point cloud data includes:

[0025] The point cloud data of each target point cloud data frame is segmented to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame.

[0026] In each point cloud data segmentation set of each target point cloud data frame, the obstacle category of each point cloud dataset is determined based on the point cloud features of point cloud datasets within different preset distance ranges from the distance to the automatic cleaning device.

[0027] The step of segmenting the point cloud data of each target point cloud data frame to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame includes:

[0028] Based on the scanning area corresponding to the preset field of view of the ranging sensor, the point cloud data of each frame of the target point cloud data is segmented to obtain multiple point cloud data segmentation sets corresponding to each frame of the target point cloud data.

[0029] The step of segmenting the point cloud data of each frame of the target point cloud data frame according to the scanning area corresponding to the preset field of view of the ranging sensor, to obtain multiple point cloud data segmentation sets corresponding to each frame of the target point cloud data frame, includes:

[0030] Each frame of the target point cloud data is converted into a depth map, and the number of columns of pixels in the depth map and the field of view of the ranging sensor are determined.

[0031] Based on the field of view and the number of columns, determine the target number of pixels corresponding to the preset field of view, and segment multiple columns of pixels in each depth map according to the target number of columns to obtain multiple image segmentation regions corresponding to each depth map;

[0032] In each frame of the target point cloud data, the point cloud data corresponding to each image segmentation region in the corresponding depth map is determined, and the point cloud data corresponding to each image segmentation region is classified into a point cloud data segmentation set.

[0033] Specifically, in each point cloud data segmentation set of each target point cloud data frame, determining the obstacle category of each point cloud dataset based on the point cloud features of point cloud datasets within different preset distance ranges relative to the automatic cleaning device includes:

[0034] For each frame of the target point cloud data, calculate the distance between the point cloud data in each point cloud data segmentation set and the automatic cleaning device;

[0035] The point cloud data in each point cloud data segmentation set is sorted according to the distance to obtain a sorting queue corresponding to each point cloud data segmentation set.

[0036] Each sorting queue is divided according to a preset distance range to obtain multiple point cloud datasets corresponding to each point cloud data segmentation set;

[0037] Determine the point cloud features for each point cloud dataset, and determine the obstacle category corresponding to each point cloud dataset based on the point cloud features.

[0038] The step of determining the target obstacle category of the region where each point cloud dataset is located based on the obstacle category of each point cloud dataset in all the target point cloud data frames includes:

[0039] Based on the obstacle category of each point cloud dataset in all the target point cloud data frames and the region corresponding to each point cloud dataset, determine the obstacle category with the highest recognition probability in each region;

[0040] Based on the obstacle category with the highest identification probability in each region, determine the target obstacle category corresponding to each region.

[0041] Secondly, embodiments of the present invention provide an automatic cleaning device, which includes a processor and a memory. A ranging sensor is disposed at the front of the automatic cleaning device. The ranging sensor includes a laser emitter and a laser receiver. The laser emitter is used to emit one or more combinations of dot matrix laser and area array laser. The laser receiver is used to receive the light signal reflected by the laser from objects in the environment. The ranging sensor is used to generate environmental information based on the light signal. The automatic cleaning device also includes a laser sensor disposed at the rear. The laser sensor includes a dot matrix laser emitter and a dot matrix laser receiver. The dot matrix laser emitter is used to emit dot matrix laser. The dot matrix laser receiver is used to receive the dot matrix light signal reflected by the dot matrix laser on obstacles. The laser sensor is used to generate map information based on the dot matrix light signal to construct a map.

[0042] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0043] The processor is configured to execute an obstacle recognition method as described in the first aspect, according to instructions in the computer program.

[0044] As described above, this invention provides an obstacle recognition method and an automatic cleaning device. After acquiring multiple initial point cloud data frames from a ranging sensor, each initial point cloud data frame is filtered to obtain a target point cloud data frame corresponding to an obstacle. Then, based on the point cloud features of the point cloud dataset within a preset distance range in each target point cloud data frame, the obstacle category of each point cloud dataset is determined. Based on the obstacle category of each point cloud dataset, the target obstacle category corresponding to the area where each point cloud dataset is located is determined. Finally, the automatic cleaning device is controlled to avoid obstacles based on the target obstacle category. This automatic cleaning device can utilize the laser emitted by the ranging sensor to acquire environmental information over a large area to identify obstacles. During the environmental information acquisition process, the automatic cleaning device does not need to perform multiple retraction actions to scan complete obstacle information, improving the efficiency of obstacle recognition and work efficiency. Furthermore, by filtering the initial point cloud data frames in this embodiment, point cloud data corresponding to the ground and distant objects can be removed, reducing the amount of data and minimizing invalid point cloud information. Secondly, by determining the point cloud features of the point cloud dataset within a preset distance range for each target point cloud data frame, and then determining the target obstacle category corresponding to the region based on the obstacle categories determined in different target point cloud data frames for the same region, this embodiment of the invention can avoid the situation of identifying obstacle categories based on only one frame of data. This reduces noise and the impact of ground undulations on the automatic cleaning equipment when it is driving on the ground. It is also unaffected by background and ambient light, and can stably identify obstacles, thus improving the accuracy of identifying different obstacle categories. Attached Figure Description

[0045] Figure 1 This is a front view of an automatic cleaning device provided in an embodiment of the present invention.

[0046] Figure 2 This is a flowchart of an obstacle recognition method provided in an embodiment of the present invention.

[0047] Figure 3 This is a flowchart illustrating another obstacle recognition method provided in an embodiment of the present invention.

[0048] Figure 4 This is a schematic diagram of an initial point cloud data frame provided in an embodiment of the present invention.

[0049] Figure 5 This is a schematic diagram of a target point cloud data frame provided in an embodiment of the present invention.

[0050] Figure 6 This is a schematic diagram illustrating how a depth map is divided into multiple image segmentation regions, as provided in an embodiment of the present invention.

[0051] Figure 7 This is a schematic diagram illustrating how point cloud data within different preset distance ranges are divided into point cloud datasets, as provided in an embodiment of the present invention.

[0052] Figure 8 This is a schematic diagram of a process for determining the obstacle category of each point cloud dataset in a target point cloud data frame, as provided in an embodiment of the present invention.

[0053] Figure 9 This is a circuit diagram of an automatic cleaning device provided in an embodiment of the present invention. Detailed Implementation

[0054] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of embodiments of this application includes the entire scope of the claims and all available equivalents of the claims. In this document, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is disclosed. Relational terms such as "first" and "second" are used herein only to distinguish one entity or operation from another, without requiring or implying any actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed. The various embodiments in this document are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the structures, products, etc., disclosed in the embodiments, since they correspond to the disclosed parts, the descriptions are relatively simple; relevant details can be found in the method section.

[0055] In existing technologies, most robots use a combination of dual-line / single-line structured light sensors and vision sensors for obstacle avoidance. Structured light obstacle avoidance technology uses structured light sensors to emit light of a specific pattern (such as linear or planar light) onto objects and receives the reflected light signals to calculate the distance between the device and the obstacle. Meanwhile, vision sensors can capture color images of the surrounding environment and identify obstacles through image processing algorithms.

[0056] However, when identifying obstacles using structured light obstacle avoidance technology, the single-frame point cloud data collected by dual-line / single-line structured light sensors consists of one or two line segments. Therefore, it is difficult to obtain complete obstacle point cloud information from a single frame, requiring the robot to perform multiple retraction actions to scan the obstacle from different angles. Through multiple scans, the obstacle point cloud data can be gradually accumulated and eventually pieced together to form a complete obstacle model, thus determining the obstacle category corresponding to the model. This method of obstacle identification is complex and cumbersome, resulting in low obstacle recognition efficiency. Furthermore, when detecting obstacles based on data collected by structured light sensors, a judgment based on absolute height is generally used. This means determining the presence of an obstacle by checking if there are objects with a height greater than a preset height threshold in the single-frame data collected by the structured light sensor. Once an obstacle category is identified at a certain location, all objects within the area are automatically considered to belong to the same obstacle category. However, when detecting obstacles using structured light sensors, the preset height threshold is generally set above 20mm. But when the robot is moving on the ground, the undulations of the ground cause the robot's posture to pitch slightly, which causes the height of the laser emitted by the structured light sensor to fluctuate. If the obstacle is identified based on the single frame data collected when the laser height fluctuates, misidentification is likely to occur.

[0057] Furthermore, when using visual sensors for obstacle recognition, the large amount of data required for training and the long training timeframe result in a limited number of obstacle categories that can be identified. Visual sensors are also susceptible to ambient light, failing to accurately identify obstacles when the background is extremely similar to the obstacle, in dark environments, or in bright light.

[0058] Based on this, to address the technical problem of low efficiency and accuracy in obstacle recognition by robots in existing technologies, this invention provides an obstacle recognition method applicable to automated cleaning equipment, such as robotic vacuum cleaners or floor scrubbers, which possess automatic cleaning functions and can move autonomously. This automated cleaning equipment is equipped with a ranging sensor, which includes a laser emitter and a laser receiver. The laser emitter emits one or more combinations of dot-matrix lasers and area-matrix lasers, and the laser receiver receives the light signals reflected from objects in the environment. The ranging sensor generates environmental information based on the light signals, including information about surrounding objects and their distances. For example, the ranging sensor can be a 3D-TOF sensor, which calculates distance by utilizing the time difference between light waves traveling through air to a target and being reflected back. After the laser emitter of the 3D-TOF sensor emits an array of laser beams, the laser propagates through the air and is reflected by an object in the environment. The laser receiver receives the reflected light signal from the object. Based on the time difference between the emitted and received light signals, and the speed of light, the distance to the object can be determined, thus achieving the function of locating and detecting the target object. In one embodiment, such as... Figure 1 As shown, Figure 1 This is a front view of an automatic cleaning device provided in an embodiment of the present invention. A ranging sensor 20 is disposed at the front of the automatic cleaning device 10. The ranging sensor 20 includes two laser emitters 21 and one laser receiver 22. The two laser emitters 21 are a dot matrix laser emitter and a surface array laser emitter, respectively. The surface array laser emitter can provide more comprehensive environmental information, while the dot matrix laser emitter can achieve precise tracking and measurement of specific targets. The combination of the two can significantly improve the system's environmental perception capability. Furthermore, the dot matrix laser emitter and the surface array laser emitter can work simultaneously to achieve rapid measurement and positioning of objects, improving measurement efficiency. Additionally, the laser emitter may include only a dot matrix laser emitter or a surface array laser emitter; the specific choice can be made according to actual needs, and this embodiment does not impose a specific limitation.

[0059] The automatic cleaning equipment also includes cleaning modules such as a water tank and cleaning cloths, motion modules such as wheels, and a positioning module. The electronically controlled components and ranging sensors in these modules are controlled by the automatic cleaning equipment's processor. Correspondingly, the environmental information collected by the ranging sensors is sent to the processor for processing. The processor can identify obstacles and generate maps based on the received environmental information, and then generate control commands based on the obstacles and maps. These control commands are sent to the cleaning and motion modules to achieve operational control under specific environmental conditions. Additionally, there are charging and communication modules. It is understood that the positioning, cleaning, and motion modules can all be implemented using methods from relevant automatic cleaning equipment technology fields. Specific installation methods and basic working principles will not be elaborated here, nor will their corresponding operational content be described in detail. For example, the process of returning to the base station and charging via the charging module after cleaning is not described in detail.

[0060] like Figure 2 As shown, Figure 2 This is a flowchart illustrating an obstacle recognition method provided by an embodiment of the present invention. The obstacle recognition method provided by the embodiment of the present invention includes the following steps:

[0061] Step 101: Collect environmental information using a ranging sensor. The environmental information includes multiple initial point cloud data frames.

[0062] In this embodiment, when the automatic cleaning equipment is in operation, it needs to periodically acquire environmental information collected by the ranging sensor within the duration of each cycle, and determine multiple initial point cloud data frames based on the environmental information. Each initial point cloud data frame includes multiple raw point cloud data. For example, when the ranging sensor is a 3D-TOF sensor, the 3D-TOF sensor can directly output raw point cloud data as environmental information, or it can output a depth map as environmental information. When the environmental information is raw point cloud data, the automatic cleaning equipment can directly determine the initial point cloud data frame based on the raw point cloud data. However, when the environmental information is a depth map, it is necessary to further convert the depth map into point cloud data to obtain the initial point cloud data frame. Specifically, the depth map can be converted into point cloud data using the following method:

[0063] Assuming (u,v) are the pixel coordinates in the pixel coordinate system of the depth map, f x ,c x ,f y ,c y Let R1 and T1 be the camera intrinsic parameters, and T1 be the camera extrinsic parameters (R1 is the rotation matrix with 3 degrees of freedom, and T1 is the translation matrix; together they form the extrinsic parameters). First, the depth values ​​in the pixel coordinate system are converted to point cloud data in the camera coordinate system based on the camera intrinsic parameters, as shown in the following formula:

[0064]

[0065] Where d is the depth value in the depth map, and (x,y,z) are the point cloud coordinates in the camera coordinate system.

[0066] Next, the point cloud data in the camera coordinate system is transformed into point cloud data in the robot coordinate system based on the camera extrinsic parameters, thereby obtaining the initial point cloud data frame. The specific formula is as follows:

[0067]

[0068] in (x′, y′, z′) are the point cloud coordinates in the robot coordinate system.

[0069] Step 102: Filter each initial point cloud data frame to obtain the target point cloud data frame corresponding to the obstacle.

[0070] After obtaining multiple initial point cloud data frames, each initial point cloud data frame needs to be further filtered. The reason for filtering is that the initial point cloud data frames generated by the 3D-TOF sensor have a large data volume, and due to the multipath and flying point noise inherent in the 3D-TOF sensor itself, it is necessary to filter and denoise the initial point cloud data frames to reduce the data volume and invalid point cloud information. In this embodiment, during the filtering process of the initial point cloud data frames, point cloud data corresponding to the ground and distant objects can be removed, retaining only the point cloud data corresponding to obstacles, thereby obtaining the target point cloud data frame. For example, the point cloud data in the initial point cloud data frame can first be divided into regions, and then the point cloud data in each region can be clustered. Then, based on the number and height of the point cloud data in each cluster, it is determined whether to retain or remove the point cloud data corresponding to that cluster. Finally, the retained point cloud data constitutes the target point cloud data frame.

[0071] Step 103: Determine the obstacle category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each target point cloud data frame.

[0072] After obtaining multiple frames of target point cloud data, these frames need to be processed. Specifically, for each frame, the point cloud dataset within a preset distance range needs to be determined. For example, in this embodiment, for each frame, the distance between each point cloud data point and the automatic cleaning device can be determined first, and then the point cloud data within the preset distance range can be identified as the point cloud dataset. The preset distance can be pre-set according to actual needs. After obtaining the point cloud dataset, further processing is required to extract point cloud features. These features characterize the height, density, and shape of the points in the dataset. Examples of point cloud features include maximum and minimum height values, average height values, and the number of points.

[0073] After determining the point cloud features corresponding to each point cloud dataset in each target point cloud data frame, it is necessary to determine the obstacle category corresponding to each point cloud dataset based on the point cloud features. The obstacle category is used to distinguish different types of obstacles, such as low obstacles, large obstacles, and walls. Since different obstacle categories have different characteristics in the number of points within different height ranges, this embodiment can identify different obstacle categories based on the point cloud features of the point cloud datasets, thereby obtaining the obstacle category corresponding to each point cloud dataset. For example, if the average height of the point cloud features in the point cloud dataset falls within the height range of low obstacles, and the number of points is greater than the obstacle number threshold, then the obstacle category corresponding to this point cloud dataset can be considered as low obstacles. Furthermore, it is understood that when there are no obstacles in the target point cloud data frame, it is not necessary to determine the corresponding obstacle category.

[0074] Step 104: Determine the target obstacle category of the region where each point cloud dataset is located based on the obstacle category of each point cloud dataset in all target point cloud data frames.

[0075] After determining the obstacle category corresponding to the point cloud dataset in each target point cloud data frame, it is necessary to determine the target obstacle category corresponding to the region where each point cloud dataset is located, based on the obstacle categories of each point cloud dataset in all target point cloud data frames. Specifically, based on the obstacle categories of each point cloud dataset in all target point cloud data frames and the regions corresponding to each point cloud dataset, the obstacle category with the highest recognition probability in each region can be determined as the target obstacle category.

[0076] Step 105: Control the automatic cleaning equipment to avoid obstacles according to the type of target obstacle.

[0077] After determining the obstacle identification results for each area, the automatic cleaning equipment can be controlled to move according to the target obstacle category corresponding to each area and the current distance to each area, thereby achieving obstacle avoidance. For example, when the target obstacle category is a low obstacle (such as data cable, power cord, building block, pen, sock, or shoelace), the automatic cleaning equipment is controlled to detour around the low obstacle based on its location, avoiding collisions or entanglement between the automatic cleaning equipment and the low obstacle, preventing the moving modules such as the wheels from getting stuck. When the target obstacle category is a threshold or step, the automatic cleaning equipment can be controlled to overcome the obstacle. In another embodiment, the map can be updated based on the obstacle identification results for each area, and the movement path of the automatic cleaning equipment can be planned based on the updated map. The automatic cleaning equipment is then controlled to move according to the movement path to achieve obstacle avoidance.

[0078] The above-described embodiments of the present invention provide an obstacle recognition method. After acquiring multiple initial point cloud data frames from a ranging sensor, each initial point cloud data frame is filtered to obtain a target point cloud data frame corresponding to the obstacle. Then, based on the point cloud features of the point cloud dataset within a preset distance range in each target point cloud data frame, the obstacle category of each point cloud dataset is determined. Based on the obstacle category of each point cloud dataset, the target obstacle category corresponding to the area where each point cloud dataset is located is determined. Finally, the automatic cleaning device is controlled to avoid obstacles based on the target obstacle category. The automatic cleaning device of this application can use the laser emitted by the ranging sensor to acquire environmental information over a large area to identify obstacles. During the environmental information acquisition process, the automatic cleaning device does not need to perform multiple retraction actions to scan complete obstacle information, improving the efficiency of obstacle recognition and work efficiency. Furthermore, by filtering the initial point cloud data frames in this embodiment, point cloud data corresponding to the ground and distant objects can be removed, reducing the amount of data and minimizing invalid point cloud information. Secondly, by determining the point cloud features of the point cloud dataset within a preset distance range for each target point cloud data frame, and then determining the target obstacle category corresponding to each region based on the obstacle categories determined in different target point cloud data frames for the same region, this embodiment of the invention can avoid the situation of identifying obstacle categories based on only one frame of data. This reduces noise and the impact of ground undulations on the automatic cleaning equipment when it is driving on the ground. It is also unaffected by background and ambient light, and can stably identify obstacles, thus improving the accuracy of identifying different obstacle categories.

[0079] This invention also provides another obstacle recognition method, such as... Figure 3 As shown, Figure 3 This is a flowchart illustrating another obstacle recognition method provided in an embodiment of the present invention. Figure 3 The obstacle recognition method shown is a specific embodiment of the above-mentioned obstacle recognition method. The obstacle recognition method provided in this embodiment of the invention includes the following steps:

[0080] Step 201: Collect environmental information using a ranging sensor. The environmental information includes multiple initial point cloud data frames.

[0081] Step 202: Project the point cloud data of each initial point cloud data frame onto the horizontal plane to obtain the point cloud projection point set in the horizontal plane.

[0082] In this embodiment, the point cloud data of each initial point cloud data frame first needs to be planarized. Specifically, the point cloud data of each initial point cloud data frame is projected onto a horizontal plane, i.e., the XY plane, to obtain point cloud projection points in the horizontal plane. The set of multiple point cloud projection points constitutes the point cloud projection point set.

[0083] Step 203: Cluster the point cloud projection point set corresponding to each initial point cloud data frame to obtain at least one planar cluster corresponding to each initial point cloud data frame.

[0084] After projecting the point cloud data onto the horizontal plane, it is necessary to further perform region clustering on the point cloud projection point set corresponding to each initial point cloud data frame to obtain at least one planar cluster on the horizontal plane corresponding to each initial point cloud data frame.

[0085] Step 204: For each initial point cloud data frame, based on the height of the point cloud data of each corresponding planar cluster, filter out the point cloud data corresponding to the obstacle in each initial point cloud data frame to obtain the target point cloud data frame.

[0086] After obtaining the planar clusters corresponding to each initial point cloud data frame, the height of the point cloud data corresponding to each planar cluster needs to be determined. Based on the height of the point cloud data corresponding to the planar clusters, point cloud data corresponding to obstacles is filtered out from the initial point cloud data frames to obtain the target point cloud data frames. It is understandable that the distribution of point cloud data corresponding to different types of objects in different height regions and the difference in maximum and minimum heights will satisfy certain conditions. Therefore, based on the height of the point cloud data corresponding to the planar clusters, the object type of the point cloud data corresponding to the planar clusters can be roughly determined.

[0087] Based on the above embodiments, in step 204, for each initial point cloud data frame, according to the height of the point cloud data of each corresponding planar cluster, the point cloud data corresponding to the obstacle is filtered out from each initial point cloud data frame to obtain the target point cloud data frame, including:

[0088] Step 2041: For each initial point cloud data frame, determine the maximum height, minimum height, and height variance based on the height of the point cloud data corresponding to each planar cluster.

[0089] In one embodiment, for each initial point cloud data frame, the maximum height, minimum height, and height variance corresponding to each planar cluster can be determined based on the height of the point cloud data corresponding to each planar cluster in the initial point cloud data frame. The maximum height refers to the maximum height of the point cloud data, the minimum height refers to the minimum height of the point cloud data, and the height variance refers to the variance of the height of the point cloud data corresponding to the planar cluster.

[0090] Step 2042: Based on the number of point clouds, maximum height, minimum height, and height variance corresponding to each planar cluster, determine whether the object type corresponding to each planar cluster is ground.

[0091] Then, based on the number of point clouds, maximum height, minimum height, and height variance corresponding to each planar cluster, it can be determined whether the object type corresponding to each planar cluster is ground. Specifically, the difference between the maximum and minimum heights corresponding to each planar cluster can be determined. Next, it is determined whether three conditions are simultaneously met: the cluster variance is less than a preset variance, the difference between the maximum and minimum heights is less than a preset difference, and the number of point clouds is greater than a preset number. If all three conditions are met, the object type corresponding to the planar cluster is determined to be ground. Otherwise, the object type corresponding to the planar cluster is determined not to be ground.

[0092] Step 2043: Remove the point cloud data corresponding to the planar clusters with the object type being ground from the initial point cloud data frame to obtain the target point cloud data frame corresponding to the obstacle.

[0093] After identifying planar clusters of objects of type ground, the point cloud data corresponding to these planar clusters can be determined and removed from the initial point cloud data frame. Simultaneously, point clouds containing suspected noise are also removed. The remaining point cloud data is then identified as the point cloud data corresponding to the obstacles, and the point cloud data corresponding to the obstacles constitutes the target point cloud data frame. In one embodiment, as follows... Figure 4 As shown, Figure 4 This is a schematic diagram of an initial point cloud data frame provided in an embodiment of the present invention. Figure 5 This is a schematic diagram of a target point cloud data frame provided in an embodiment of the present invention. Figure 4 After filtering, the initial point cloud data frame shown retains... Figure 5 The point cloud data corresponding to the obstacle in the target point cloud data frame shown.

[0094] Based on the above embodiments, before removing point cloud data corresponding to planar clusters with ground-type objects from the initial point cloud data frame, the method further includes:

[0095] Plane fitting is performed on the point cloud data corresponding to the planar clusters of objects of type ground to obtain the plane fitting equation.

[0096] Determine the transformation matrix between the plane fitting equation and the standard Z-axis zero plane.

[0097] In one embodiment, after identifying the planar clusters of ground-type objects in each initial point cloud data frame, it is necessary to perform plane fitting on the point cloud data corresponding to these ground-type clusters to determine the plane fitting equation. Specifically, for each initial point cloud data frame, three non-collinear point cloud data points are randomly selected from the point cloud data corresponding to the ground-type planar clusters. These three point cloud data points are used to define a candidate plane. The normal vector and intercept of the plane are calculated using these three point cloud data points to obtain a plane equation, which is typically expressed as Ax + By + Cz + D = 0, where (A, B, C) are the plane's normal vector and D is the intercept. In another embodiment, the RANSAC algorithm can also be used for plane fitting; the specific process can be found in existing technologies and will not be elaborated upon in this embodiment.

[0098] After determining the plane fitting equation, it is also necessary to determine the transformation matrix between the plane fitting equation and the standard Z-axis zero plane (i.e., the XY plane, Z=0). The specific formula for the transformation matrix is ​​as follows:

[0099]

[0100] Where R2 is the rotation matrix, T2 is the translation matrix, and (x″, y″, z″) are the coordinates of the transformed point cloud data.

[0101] After obtaining the target point cloud data frame corresponding to the obstacle, the following is also included:

[0102] The point cloud data in the target point cloud data frame is transformed by the transformation matrix to obtain the target point cloud data frame with the standard Z-axis zero plane as the reference horizontal plane.

[0103] Furthermore, after removing point cloud data corresponding to planar clusters of ground-type objects from the initial point cloud data frame to obtain the target point cloud data frame corresponding to the obstacle, it is necessary to further transform the point cloud data in the target point cloud data frame to the standard Z-axis zero plane using a transformation matrix, resulting in a target point cloud data frame with the standard Z-axis zero plane as the reference horizontal plane. It is understandable that during the movement of the automatic cleaning equipment, due to the undulation of the ground, the point cloud data may be tilted relative to the horizontal plane (i.e., the standard Z-axis zero plane). This tilt causes a change in the orientation of the point cloud data in three-dimensional space, thus introducing angular offset. Using a transformation matrix, the obstacle point cloud can be rotated around an axis (such as the X-axis or Y-axis) until its normal vector (or principal direction) is parallel to the standard Z-axis zero plane, thus transforming the point cloud data to the standard Z-axis zero plane and eliminating the angular offset caused by ground undulations.

[0104] Step 205: Segment the point cloud data of each target point cloud data frame to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame.

[0105] After obtaining multiple frames of target point cloud data, it is necessary to further segment the point cloud data of each frame to obtain multiple point cloud data segmentation sets corresponding to each frame. In one embodiment, for each frame, the point cloud data can be segmented according to a preset area range to obtain multiple point cloud data segmentation sets corresponding to each frame. In another embodiment, the point cloud data of each frame can also be segmented according to the scanning area corresponding to the preset field of view of the ranging sensor to obtain multiple point cloud data segmentation sets corresponding to each frame. The preset field of view can be set according to actual needs. For example, the preset field of view can be set to 1°, and the point cloud data of each frame can be segmented according to the area range scanned by the 1° field of view to obtain multiple point cloud data segmentation sets corresponding to each frame.

[0106] Based on the above embodiments, according to the scanning area corresponding to the preset field of view of the ranging sensor, the point cloud data of each target point cloud data frame is segmented to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame, including:

[0107] Step 2051: Convert each frame of target point cloud data into a depth map, and determine the number of columns of pixels in the depth map and the field of view of the ranging sensor.

[0108] In one embodiment, each frame of target point cloud data first needs to be converted into a depth map. The specific process of converting point cloud data into a depth map can be found in existing technologies and will not be described in detail in this embodiment. Then, the number of columns of pixels in the depth map and the field of view of the ranging sensor are determined.

[0109] Step 2052: Based on the field of view and the number of columns, determine the target number of pixels corresponding to the preset field of view, and segment the multiple columns of pixels in each depth map according to the target number of columns to obtain multiple image segmentation regions corresponding to each depth map.

[0110] After converting to a depth map, it is necessary to determine the number of columns of pixels in the depth map and the field of view of the ranging sensor. Based on the field of view and the number of columns, the target number of columns of pixels corresponding to a preset field of view is then determined. Specifically, in this embodiment, the sub-field of view corresponding to each column of pixels in the depth map is first determined based on the field of view and the number of columns. For example, assuming the field of view of the 3D-TOF sensor is 90° and the number of columns of pixels in the depth map is 240, then the sub-field of view corresponding to each column of pixels is 90° ÷ 240 = 0.375°. Then, based on the sub-field of view, the target number of columns of pixels corresponding to the preset field of view is determined. For example, in this embodiment, the preset field of view can be set to 1°, then 1° ÷ 0.375° = 2.6. Rounding 2.6 further to 3, the target number of columns of pixels corresponding to 1° is 3. Then, the depth map is segmented according to the target number of columns to obtain multiple image segmentation regions. Specifically, in the depth map, the region corresponding to every 3 columns of pixels is divided into an image segmentation region. Since the depth map has 240 columns of pixels, after segmentation, 80 image segmentation regions can be obtained. In one embodiment, such as... Figure 6 As shown, Figure 6 This is a schematic diagram illustrating how a depth map is divided into multiple image segmentation regions, as provided in an embodiment of the present invention. Figure 6 In the image, the depth map 40 is divided into multiple image segmentation regions 50.

[0111] Step 2053: In each frame of target point cloud data, determine the point cloud data corresponding to each image segmentation region in the corresponding depth map, and classify the point cloud data corresponding to each image segmentation region into a point cloud data segmentation set.

[0112] After segmentation, the point cloud data corresponding to each image segmentation region in the corresponding depth map needs to be identified in each target point cloud data frame. Then, in each target point cloud data frame, the point cloud data corresponding to each image segmentation region is classified into a point cloud data segmentation set, thus obtaining multiple point cloud data segmentation sets corresponding to each target point cloud data frame.

[0113] Step 206: In each point cloud data segmentation set of each target point cloud data frame, determine the obstacle category of each point cloud dataset based on the point cloud features of the point cloud dataset within different preset distance ranges of the distance to the automatic cleaning device.

[0114] After obtaining multiple point cloud data segments corresponding to each target point cloud data frame, obstacle category identification is required. Specifically, for the point cloud data in each point cloud data segment of each target point cloud data frame, the distance between each point cloud data and the automatic cleaning device can be determined, and point cloud data within different preset distance ranges can be divided into a point cloud dataset. For example, as shown... Figure 7 As shown, Figure 7 This is a schematic diagram illustrating the division of point cloud data within different preset distance ranges into point cloud datasets, as provided in an embodiment of the present invention. Point cloud data within a distance range of 0–3 cm, 3–6 cm, 6–9 cm, and 9–12 cm are each divided into a separate point cloud dataset. After obtaining multiple point cloud datasets, each dataset is further processed to extract point cloud features. Based on these features, the obstacle category corresponding to each dataset is determined. In one embodiment, as shown... Figure 8 As shown, Figure 8 This is a schematic diagram of a process for determining the obstacle category of each point cloud dataset in a target point cloud data frame, as provided in an embodiment of the present invention.

[0115] Based on the above embodiments, in step 206, in each point cloud data segmentation set of each target point cloud data frame, the obstacle category of each point cloud dataset is determined according to the point cloud features of the point cloud dataset within different preset distance ranges from the distance to the automatic cleaning device, including:

[0116] Step 2061: For each target point cloud data frame, calculate the distance between the point cloud data in each point cloud data segmentation set and the automatic cleaning device.

[0117] In one embodiment, for each frame of target point cloud data, it is necessary to calculate the distance between the point cloud data in each point cloud data segmentation set and the automatic cleaning device. This embodiment uses the Manhattan distance as an example. Assume the automatic cleaning device is currently located at coordinates (Xc, Yc) in the robot coordinate system, and the point cloud data in the point cloud data segmentation set in the robot coordinate system are (x1', y1'), (x2', y2'), ..., (xn', yn'). Then, the Manhattan distance D from each point cloud data to the current position of the automatic cleaning device is:

[0118] D1 = |Xc-x1′| + |Yc-y1′|

[0119] D2=|Xc-x2′|+|Yc-y2′|

[0120]

[0121] Dn=|Xc-xn′|+|Yc-yn′|

[0122] Step 2062: Sort the point cloud data in each point cloud data segmentation set according to the distance to obtain the sorting queue corresponding to each point cloud data segmentation set.

[0123] For each target point cloud data frame, after determining the distance between the point cloud data in each point cloud data segment set and the automatic cleaning device, it is necessary to further sort the point cloud data in each point cloud data segment set from smallest to largest according to the distance and store them in a queue, thereby obtaining a sorting queue corresponding to each point cloud data segment set.

[0124] Step 2063: Divide each sorting queue according to the preset distance range to obtain multiple point cloud datasets corresponding to each point cloud data segmentation set.

[0125] Finally, for each frame of target point cloud data, the sorting queue can be segmented according to a preset distance range, thereby obtaining multiple point cloud datasets corresponding to each point cloud data segment. The preset distance range needs to be set in advance. For example, the preset distance range can be set to 0-10cm, 10-20cm, and 20-30cm, etc. That is, point cloud data within the distance of 0-10cm from the automatic cleaning device in the point cloud data segment is divided into one point cloud dataset, point cloud data within the distance of 10-20cm from the automatic cleaning device is divided into another point cloud dataset, and so on.

[0126] Step 2064: Determine the point cloud features of each point cloud dataset, and determine the obstacle category corresponding to each point cloud dataset based on the point cloud features.

[0127] In each frame of target point cloud data, after dividing each point cloud data segment into multiple point cloud datasets, further processing is needed on the point cloud data in each point cloud dataset to extract point cloud features from each dataset and identify the corresponding obstacle category based on these features. Specifically, in this embodiment, the point cloud data in each dataset can be clustered to obtain clusters. Then, point cloud features are extracted from each cluster. These features include maximum height, minimum height, average height, height variance, direction vector, and the number of points. Based on the maximum height, minimum height, average height, number of points, and preset obstacle identification rules, the obstacle category corresponding to each cluster is determined. It is understood that in this embodiment, different obstacle identification rules can be preset for different obstacle categories. The automatic cleaning device determines whether the extracted point cloud features meet the corresponding obstacle identification rules to determine the corresponding obstacle category. For example, when the obstacle category is a wall, the obstacle category can be determined as a wall if the average height is greater than or equal to a wall height threshold, the number of point clouds is greater than or equal to a wall quantity threshold, there are a large number of direction vectors in the same direction, and the cluster area is greater than or equal to a wall area threshold. When the obstacle category is a plane, the obstacle category can be determined as a plane if the average height is within the plane height range, the number of point clouds is greater than or equal to a plane quantity threshold, there are a large number of direction vectors in the same direction, the cluster area is greater than or equal to a plane area threshold, and the height variance conforms to the plane characteristics. When the obstacle category is a low obstacle, the obstacle category can be determined as a low obstacle if the average height is within the low obstacle height range, the number of point clouds is greater than or equal to a low obstacle quantity threshold, and the direction vectors diverge in direction.

[0128] Step 207: Based on the obstacle category of each point cloud dataset in all target point cloud data frames and the region corresponding to each point cloud dataset, determine the obstacle category with the highest recognition probability in each region.

[0129] After determining the obstacle categories corresponding to the clusters in each point cloud dataset within each frame of target point cloud data, the obstacle recognition result for each region can be determined based on the obstacle category corresponding to each cluster and the region where each cluster is located. Specifically, for the region where each cluster is located in a frame of target point cloud data, the obstacle category with the highest recognition probability can be determined based on the obstacle categories determined by the corresponding clusters in different point cloud data frames. For example, if there are three target point cloud data frames, for the same region in the three target point cloud data frames (referring to the same region in the robot coordinate system), if the obstacle category corresponding to the cluster in the first target point cloud data frame is low obstacle, the obstacle category corresponding to the cluster in the second target point cloud data frame is ramp, and the obstacle category corresponding to the cluster in the third target point cloud data frame is low obstacle, that is, in the three recognition results, the recognition probability of low obstacle is 66%, while the recognition probability of ramp is 33%, and the obstacle category with the highest recognition probability in this region is low obstacle.

[0130] Step 208: Determine the target obstacle category corresponding to each region based on the obstacle category with the highest identification probability in each region.

[0131] Once the obstacle category with the highest recognition probability for each region is determined, it can be used as the target obstacle category corresponding to each region.

[0132] Step 209: Control the automatic cleaning equipment to avoid obstacles according to the type of target obstacle.

[0133] As described above, this embodiment filters the initial point cloud data frame to remove ground point cloud data, reducing both the amount of data and invalid point cloud information. Furthermore, this embodiment determines the obstacle categories in each target point cloud data frame by identifying the obstacle categories within different preset distance ranges from the automatic cleaning device. Then, it determines the target obstacle category for the same area based on the obstacle categories identified in different target point cloud data frames. This avoids identifying obstacle categories based on only one frame of data, reducing noise and the impact of ground undulations on the automatic cleaning device's movement. It is also unaffected by background and ambient light, enabling stable obstacle identification and improving the accuracy of identifying different obstacle categories.

[0134] This embodiment also provides an automatic cleaning device 10, such as Figure 9 As shown, Figure 9This invention provides a circuit diagram of an automatic cleaning device 10. The automatic cleaning device 10 includes a processor 300 and a memory 301. A ranging sensor 20 is disposed at the front of the automatic cleaning device 10. The ranging sensor 20 includes a laser emitter 21 and a laser receiver 22. The laser emitter 21 is used to emit one or more combinations of dot matrix laser and area matrix laser. The laser receiver 22 is used to receive the light signal reflected by the laser from objects in the environment. The ranging sensor 10 is used to generate environmental information based on the light signal. The automatic cleaning device also includes a laser sensor 60 disposed at the rear. The laser sensor 60 includes a dot matrix laser emitter 61 and a dot matrix laser receiver 62. The dot matrix laser emitter 61 is used to emit dot matrix laser. The dot matrix laser receiver 62 is used to receive the dot matrix light signal reflected by the dot matrix laser on obstacles. The laser sensor 60 is used to generate map information based on the dot matrix light signal to construct a map. The laser sensor 60 can be disposed on the left, right, or center of the rear of the automatic cleaning device. The specific placement position can be set according to actual needs, and is not specifically limited in this embodiment.

[0135] The memory 301 is used to store the computer program 302 and to transfer the computer program 302 to the processor 300;

[0136] The processor 300 is used to execute the steps in one of the above-described obstacle recognition method embodiments according to the instructions in the computer program 302.

[0137] For example, computer program 302 may be divided into one or more modules / units, one or more of which are stored in memory 301 and executed by processor 300 to complete this application. One or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 302 in automatic cleaning device 10.

[0138] The automatic cleaning device 10 may include, but is not limited to, a processor 300 and a memory 301. Those skilled in the art will understand that... Figure 9 This is merely an example of the automatic cleaning device 10 and does not constitute a limitation on the automatic cleaning device 10. It may include more or fewer components than shown, or combine certain components, or different components. For example, the automatic cleaning device 10 may also include input / output devices, network access devices, buses, etc.

[0139] The processor 300 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0140] The memory 301 can be an internal storage unit of the automatic cleaning device 10, such as a hard drive or RAM of the automatic cleaning device 10. The memory 301 can also be an external storage device of the automatic cleaning device 10, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the automatic cleaning device 10. Furthermore, the memory 301 can include both internal and external storage units of the automatic cleaning device 10. The memory 301 is used to store computer programs and other programs and data required by the automatic cleaning device 10. The memory 301 can also be used to temporarily store data that has been output or will be output.

[0141] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0142] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0143] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0145] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0146] This invention also provides a storage medium containing computer-executable instructions. When executed by a computer processor, these instructions are used to perform an obstacle recognition method. The obstacle recognition method is applicable to an automatic cleaning device. A ranging sensor is positioned in front of the automatic cleaning device. The ranging sensor includes a laser emitter and a laser receiver. The laser emitter emits one or more combinations of dot matrix lasers and area lasers. The laser receiver receives the light signal reflected from objects in the environment. The ranging sensor generates environmental information based on the light signal. The obstacle recognition method includes:

[0147] Based on the environmental information collected by the ranging sensor, multiple initial point cloud data frames are determined.

[0148] Each initial point cloud data frame is filtered to obtain the target point cloud data frame corresponding to the obstacle.

[0149] Based on the point cloud features of the point cloud dataset within a preset distance range for each target point cloud data frame, the obstacle category corresponding to the point cloud dataset is determined.

[0150] Based on the obstacle categories in the point cloud datasets of each target point cloud data frame, determine the target obstacle category corresponding to the region where each point cloud dataset is located.

[0151] The automatic cleaning equipment is controlled to avoid obstacles based on the type of the target obstacle.

[0152] Note that the above are merely preferred embodiments and the technical principles applied in this invention. Those skilled in the art will understand that the embodiments of this invention are not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the protection scope of this invention. Therefore, although the embodiments of this invention have been described in detail above, the embodiments of this invention are not limited to the above embodiments. More other equivalent embodiments may be included without departing from the concept of the embodiments of this invention, and the scope of the embodiments of this invention is determined by the scope of the appended claims.

Claims

1. An obstacle recognition method, characterized in that, The method is applicable to automatic cleaning equipment, wherein a ranging sensor is disposed in front of the automatic cleaning equipment. The ranging sensor includes a laser emitter and a laser receiver. The laser emitter is used to emit one or more combinations of dot matrix lasers and area lasers. The laser receiver is used to receive the light signal reflected by the laser from environmental objects. The ranging sensor is used to generate environmental information based on the light signal. The method includes: The ranging sensor collects environmental information, which includes multiple frames of initial point cloud data. Each initial point cloud data frame is filtered to obtain the target point cloud data frame corresponding to the obstacle. Based on the point cloud features of the point cloud dataset within a preset distance range of each target point cloud data frame, determine the obstacle category corresponding to each point cloud dataset. Based on the obstacle categories of each point cloud dataset in all the target point cloud data frames, determine the target obstacle category of the region where each point cloud dataset is located; The automatic cleaning equipment is controlled to avoid obstacles based on the type of the target obstacle.

2. The obstacle recognition method according to claim 1, characterized in that, The step of filtering each initial point cloud data frame to obtain a target point cloud data frame corresponding to the obstacle includes: Project the point cloud data of each initial point cloud data frame onto a horizontal plane to obtain the point cloud projection point set in the horizontal plane. Cluster the point cloud projection point set corresponding to each frame of the initial point cloud data frame to obtain at least one planar cluster corresponding to each frame of the initial point cloud data frame. For each initial point cloud data frame, based on the height of the point cloud data of each corresponding planar cluster, the point cloud data corresponding to the obstacle is filtered out in each initial point cloud data frame to obtain the target point cloud data frame.

3. The obstacle recognition method according to claim 2, characterized in that, For each initial point cloud data frame, based on the height of the point cloud data of each corresponding planar cluster, point cloud data corresponding to the obstacle is filtered out from each initial point cloud data frame to obtain a target point cloud data frame, including: For each initial point cloud data frame, the maximum height, minimum height, and height variance are determined based on the height of the point cloud data corresponding to each planar cluster. Based on the number of point clouds, maximum height, minimum height, and height variance corresponding to each planar cluster, determine whether the object type corresponding to each planar cluster is ground; Point cloud data corresponding to planar clusters of the object type "ground" are removed from the initial point cloud data frame to obtain the target point cloud data frame corresponding to the obstacle.

4. The obstacle recognition method according to claim 3, characterized in that, Before removing point cloud data corresponding to planar clusters of the object type "ground" from the initial point cloud data frame, the method further includes: Plane fitting is performed on the point cloud data corresponding to the planar clusters of objects of the ground type to obtain the plane fitting equation; Determine the transformation matrix between the plane fitting equation and the standard Z-axis zero plane; After obtaining the target point cloud data frame corresponding to the obstacle, the method further includes: The point cloud data in the target point cloud data frame is subjected to coordinate transformation according to the transformation matrix to obtain a target point cloud data frame with the standard Z-axis zero plane as the reference horizontal plane.

5. The obstacle recognition method according to claim 1, characterized in that, The step of determining the obstacle category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each target point cloud data frame includes: The point cloud data of each target point cloud data frame is segmented to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame. In each point cloud data segmentation set of each target point cloud data frame, the obstacle category of each point cloud dataset is determined based on the point cloud features of point cloud datasets within different preset distance ranges from the distance to the automatic cleaning device.

6. The obstacle recognition method according to claim 5, characterized in that, The step of segmenting the point cloud data of each target point cloud data frame to obtain multiple point cloud data segmentation sets corresponding to each target point cloud data frame includes: Based on the scanning area corresponding to the preset field of view of the ranging sensor, the point cloud data of each frame of the target point cloud data is segmented to obtain multiple point cloud data segmentation sets corresponding to each frame of the target point cloud data.

7. The obstacle recognition method according to claim 6, characterized in that, The step involves segmenting the point cloud data of each frame of the target point cloud data frame according to the scanning area corresponding to the preset field of view of the ranging sensor, to obtain multiple point cloud data segmentation sets corresponding to each frame of the target point cloud data frame, including: Each frame of the target point cloud data is converted into a depth map, and the number of columns of pixels in the depth map and the field of view of the ranging sensor are determined. Based on the field of view and the number of columns, determine the target number of pixels corresponding to the preset field of view, and segment multiple columns of pixels in each depth map according to the target number of columns to obtain multiple image segmentation regions corresponding to each depth map; In each frame of the target point cloud data, the point cloud data corresponding to each image segmentation region in the corresponding depth map is determined, and the point cloud data corresponding to each image segmentation region is classified into a point cloud data segmentation set.

8. The obstacle recognition method according to claim 5, characterized in that, In each point cloud data segmentation set of each target point cloud data frame, based on the point cloud features of point cloud datasets within different preset distance ranges from the automatic cleaning device, the obstacle category of each point cloud dataset is determined, including: For each frame of the target point cloud data, calculate the distance between the point cloud data in each point cloud data segmentation set and the automatic cleaning device; The point cloud data in each point cloud data segmentation set is sorted according to the distance to obtain a sorting queue corresponding to each point cloud data segmentation set. Each sorting queue is divided according to a preset distance range to obtain multiple point cloud datasets corresponding to each point cloud data segmentation set; Determine the point cloud features for each point cloud dataset, and determine the obstacle category corresponding to each point cloud dataset based on the point cloud features.

9. The obstacle recognition method according to claim 1, characterized in that, The step of determining the target obstacle category of the region where each point cloud dataset is located based on the obstacle category of each point cloud dataset in all the target point cloud data frames includes: Based on the obstacle category of each point cloud dataset in all the target point cloud data frames and the region corresponding to each point cloud dataset, determine the obstacle category with the highest recognition probability in each region; Based on the obstacle category with the highest identification probability in each region, determine the target obstacle category corresponding to each region.

10. An automatic cleaning device, characterized in that, The automatic cleaning device includes a processor and a memory. A ranging sensor is disposed at the front of the automatic cleaning device. The ranging sensor includes a laser emitter and a laser receiver. The laser emitter is used to emit one or more combinations of dot matrix laser and area laser. The laser receiver is used to receive the light signal reflected by the laser from objects in the environment. The ranging sensor is used to generate environmental information based on the light signal. The automatic cleaning device also includes a laser sensor disposed at the rear. The laser sensor includes a dot matrix laser emitter and a dot matrix laser receiver. The dot matrix laser emitter is used to emit dot matrix laser. The dot matrix laser receiver is used to receive the dot matrix light signal reflected by the dot matrix laser on obstacles. The laser sensor is used to generate map information based on the dot matrix light signal to construct a map. The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is configured to execute an obstacle recognition method as described in any one of claims 1 to 9 according to instructions in the computer program.

Images