A robot control method, an automatic cleaning device, and a storage medium

Abstract

This invention discloses a robot control method, an automatic cleaning device, and a storage medium. In this embodiment, after acquiring multiple frames of point cloud data collected by a line laser sensor, the automatic cleaning device first determines the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range in each frame. Then, based on the object category of each point cloud dataset in each frame, it determines the target object category corresponding to each environmental object. Finally, it controls the automatic cleaning device to avoid obstacles based on the target object category. This invention, by determining the object category of the point cloud dataset within a preset distance range in each frame and determining the target object category based on the object category of each point cloud dataset in all frames, avoids the situation of identifying obstacles based solely on a single frame, thus improving the accuracy of obstacle identification.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a robot control method, an automatic cleaning device, and a storage medium. Background Technology

[0002] In existing technologies, robots typically detect obstacles using structured light sensors or vision sensors. When detecting obstacles based on data collected by structured light sensors, the presence of an obstacle is usually determined by checking if an object with a height greater than a preset height threshold exists in a single frame of data collected by the sensor. However, due to ground undulations and the robot's pitch during movement, the height of the laser emitted by the structured light sensor fluctuates. If obstacle identification is based on single-frame data collected during these laser height fluctuations, misidentification is likely. Furthermore, when detecting obstacles using single-frame data collected by vision sensors, the accuracy of obstacle identification is also relatively low due to interference from factors such as obstacle background and ambient light intensity.

[0003] In summary, existing technologies for robot obstacle detection suffer from low accuracy. Summary of the Invention

[0004] This invention provides a robot control method, an automatic cleaning device, and a storage medium, which solves the technical problem in the prior art where robots based on visual sensors cannot accurately identify low obstacles when avoiding obstacles.

[0005] In a first aspect, embodiments of the present invention provide a robot control method applicable to an automated cleaning device. The automated cleaning device includes a line laser sensor, which comprises at least one laser emitter and one laser receiver. Each laser emitter emits a line laser beam into the environment, and the laser receiver receives an environmental image reflected from an environmental object by the line laser beam. The line laser sensor obtains point cloud data based on the environmental image. The method includes:

[0006] Acquire multiple frames of point cloud data collected by the line laser sensor;

[0007] Based on the point cloud features of the point cloud dataset within a preset distance range of each point cloud data frame, determine the object category corresponding to each point cloud dataset;

[0008] Based on the object category of each point cloud dataset in all the point cloud data frames, determine the target object category corresponding to each environmental object in the environment;

[0009] The automatic cleaning equipment is controlled to avoid obstacles based on the target object category.

[0010] The step of determining the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each frame of the point cloud data includes:

[0011] In each point cloud data frame, the object category of each point cloud dataset is determined based on the point cloud features of the point cloud dataset within different preset distance ranges relative to the automatic cleaning device.

[0012] Each point cloud data frame includes multiple point cloud data sets. Within each point cloud data frame, based on the point cloud features of point cloud datasets within different preset distance ranges from the automatic cleaning device, the object category of each point cloud dataset is determined, including:

[0013] Each point cloud data frame is filtered to obtain the corresponding target point cloud data frame.

[0014] Based on the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device, and different preset distance ranges, each frame of the target point cloud data is divided into multiple point cloud datasets.

[0015] In each frame of the target point cloud data, the point cloud features of each point cloud dataset are determined, and the object category corresponding to each point cloud dataset is determined based on the point cloud features.

[0016] The step of dividing each frame of the target point cloud data into multiple point cloud datasets based on the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device, as well as different preset distance ranges, includes:

[0017] Calculate the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device;

[0018] The point cloud data in each frame of the target point cloud data is sorted according to the distance to obtain a sorting queue corresponding to each frame of the target point cloud data.

[0019] Each sorting queue is divided according to different preset distance ranges to obtain multiple point cloud datasets corresponding to each frame of the target point cloud data frame.

[0020] The point cloud features of the point cloud dataset include maximum height, minimum height, average height, and number of points. The object categories include low obstacles, walls, and ground. Determining the object category corresponding to each point cloud dataset based on the point cloud features includes:

[0021] If the minimum height of the point cloud dataset is within the ground height range and the number of point clouds is greater than the effective ground quantity threshold, then the object category corresponding to each point cloud dataset is determined to be the ground.

[0022] If the average height of the point cloud dataset is within the height range of low obstacles and the number of point clouds is greater than the effective number threshold of low obstacles, then the object category corresponding to each point cloud dataset is determined to be the low obstacle.

[0023] If the maximum height of the point cloud dataset is within the wall height range and the number of point clouds is greater than the effective number threshold of the wall, then the object category corresponding to each point cloud dataset is determined to be the wall.

[0024] The point cloud features also include point cloud bounding boxes, and when the number of object categories corresponding to the point cloud dataset is greater than one, they also include:

[0025] Based on the number of points and the bounding box of the points corresponding to the point cloud dataset, determine the probability that the point cloud dataset belongs to different object categories;

[0026] Based on the probability and the object category corresponding to the point cloud dataset, a unique object category corresponding to the point cloud dataset is determined.

[0027] The step of determining the target object category corresponding to each environmental object in the environment based on the object category of each point cloud dataset in all the point cloud data frames includes:

[0028] Cluster all point cloud data in all the point cloud data frames to obtain point cloud clusters corresponding to each of the environmental objects;

[0029] Based on the object category confirmed in the point cloud dataset corresponding to the point cloud data in each point cloud cluster, determine the proportion of point cloud data corresponding to each object category in each point cloud cluster;

[0030] The object category with the highest proportion in each point cloud cluster is determined as the target object category corresponding to the point cloud cluster.

[0031] The line laser sensor is a dual-line sensor, comprising two laser emitters. Acquiring multiple frames of point cloud data collected by the line laser sensor includes:

[0032] The original point cloud data frames collected by the dual-line laser sensor are acquired, and the original point cloud data frames are divided to obtain the original point cloud data frames corresponding to the two laser emitters.

[0033] Secondly, embodiments of the present invention also provide an automatic cleaning device, the automatic cleaning device including a line laser sensor, a processor and a memory, the line laser sensor including at least one laser emitter and a laser receiver, each laser emitter being used to emit a line laser into the environment, the laser receiver being used to receive an environmental image reflected by the line laser on an environmental object, and the line laser sensor being used to obtain point cloud data based on the environmental image;

[0034] The memory is used to store computer programs and to transfer the computer programs to the processor;

[0035] The processor is configured to execute a robot control method as described in the first aspect according to instructions in the computer program.

[0036] Thirdly, embodiments of the present invention also provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform a robot control method as described in the first aspect.

[0037] As described above, this invention provides a robot control method, an automatic cleaning device, and a storage medium. In this embodiment, after acquiring multiple frames of point cloud data collected by a line laser sensor, the automatic cleaning device first determines the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each frame. Then, based on the object category of each point cloud dataset in each frame, it determines the target object category corresponding to each environmental object. Finally, it controls the automatic cleaning device to avoid obstacles based on the target object category. This embodiment, by determining the object category of the point cloud dataset within a preset distance range in each frame and determining the target object category based on the object category of each point cloud dataset in all frames, avoids the situation where obstacles are identified solely based on a single frame, thereby reducing noise and the impact of ground undulations on the automatic cleaning device's movement on the ground, improving the accuracy of obstacle identification, and solving the technical problem of low identification accuracy in existing obstacle detection methods. Furthermore, the method of identifying object categories based on a line laser sensor in this embodiment is unaffected by background and ambient light, and can stably identify obstacles. Attached Figure Description

[0038] Figure 1 A flowchart illustrating a robot control method provided in an embodiment of this application.

[0039] Figure 2 This is a schematic diagram of an automatic cleaning device provided in an embodiment of the present invention.

[0040] Figure 3 A schematic diagram of another automatic cleaning device provided in an embodiment of the present invention.

[0041] Figure 4 This is a flowchart illustrating another robot control method provided in an embodiment of the present invention.

[0042] Figure 5 This is a schematic diagram of the XY axis of a robot coordinate system provided in an embodiment of the present invention.

[0043] Figure 6 This is a schematic diagram of the partitioned point cloud dataset provided in an embodiment of the present invention.

[0044] Figure 7 This is a schematic diagram illustrating how to identify object categories in different point cloud datasets, as provided in an embodiment of the present invention.

[0045] Figure 8 This is a schematic diagram of a point cloud bounding box provided in an embodiment of the present invention.

[0046] Figure 9 This is a schematic diagram of an automatic cleaning device provided in an embodiment of the present invention. Detailed Implementation

[0047] The following description and accompanying drawings fully illustrate specific embodiments of this application to enable those skilled in the art to practice them. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. The scope of embodiments of this application includes the entire scope of the claims and all available equivalents of the claims. In this document, each embodiment may be referred to individually or collectively by the term "invention," which is merely for convenience and is not intended to automatically limit the scope of the application to any single invention or inventive concept if more than one invention is disclosed. Relational terms such as "first" and "second" are used herein only to distinguish one entity or operation from another, without requiring or implying any actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed. The various embodiments in this document are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the structures, products, etc., disclosed in the embodiments, since they correspond to the disclosed parts, the descriptions are relatively simple; relevant details can be found in the method section.

[0048] In existing technologies, robots typically detect obstacles using structured light sensors or vision sensors. Structured light sensors include a laser emitter and an IR camera. The laser emitter projects a laser beam onto the target surface, and the IR camera captures the image of the laser beam projected onto the target surface. The depth and shape of the target surface are then calculated using phase decoding and triangulation principles. When detecting obstacles based on data collected by structured light sensors, robots generally use absolute height-based judgment. This means determining the presence of obstacles by checking if there are objects with a height greater than a preset height threshold in a single frame of data collected by the structured light sensor. Once an object category is identified at a certain location, all objects in the area are considered to belong to the same object category. However, since the preset height threshold for obstacle detection using structured light sensors is generally set above 20mm, the robot's posture can fluctuate slightly due to ground undulations, causing fluctuations in the height of the laser emitted by the structured light sensor. If obstacle identification is based on single frames of data collected during these height fluctuations, misidentification is likely to occur.

[0049] When identifying obstacles using single-frame images captured by visual sensors, the sensors are greatly affected by ambient light. When the background and obstacle are very similar, the obstacle cannot be accurately identified. Furthermore, the accuracy of visual sensors decreases significantly in dark or bright light environments. Secondly, visual sensors require large datasets for training over a long period, resulting in high costs.

[0050] Based on this, in order to solve the above-mentioned technical problems, embodiments of the present invention provide a robot control method, such as... Figure 1 As shown, Figure 1This is a flowchart illustrating a robot control method provided in an embodiment of this application. The robot control method provided in this embodiment can be executed by an automatic cleaning device, such as a sweeping robot or a floor scrubber, which possesses automatic cleaning functions and can move autonomously. The automatic cleaning device is equipped with a line laser sensor, which includes at least one laser emitter and one laser receiver. Each laser emitter emits a line laser beam into the environment, and the laser receiver receives an environmental image reflected from an environmental object. The line laser sensor obtains point cloud data based on the environmental image, where the environmental object refers to an object located in the environment. Specifically, after the laser emitter emits a line laser beam into the environment, the laser receiver receives the environmental image formed by the light signal reflected back from the environmental object and calculates the three-dimensional contour information of the measured object based on the environmental image to generate point cloud data. In this embodiment, the line laser sensor can be a single-line laser sensor or a dual-line laser sensor. When the line laser sensor is a single-line laser sensor, it includes one laser emitter and one laser receiver. For example, as shown... Figure 2 As shown, Figure 2 This is a schematic diagram of an automatic cleaning device provided in an embodiment of the present invention. Figure 2 In the automatic cleaning device 10, a single-line laser sensor 20 is installed. The single-line laser sensor 20 emits a linear laser beam tilted downwards. After the linear laser beam is projected onto the surface of an object, it forms a linear pattern parallel to the ground and is reflected. The single-line laser sensor 20 calculates the point cloud data of the object by receiving the reflected environmental image. Alternatively, when the linear laser sensor is a dual-line laser sensor, it includes two laser emitters and one laser receiver. Figure 3 As shown, Figure 3 This is a schematic diagram of another automatic cleaning device provided in an embodiment of the present invention. Figure 3 In this automatic cleaning device 10, a dual-line laser sensor is installed. The dual-line laser sensor includes a laser emitter 21, a laser emitter 22, and a laser receiver 23. The laser emitter 21 and laser emitter 22 are respectively located on opposite sides of the front of the automatic cleaning device 10, and the laser receiver 23 is located at the center of the front of the automatic cleaning device 10. The laser emitter 21 and laser emitter 22 emit linear laser beams perpendicular to the ground. After the linear laser beams are projected onto the surface of the object, they form a linear pattern perpendicular to the ground and are reflected. The laser receiver 23 calculates the point cloud data of the object by receiving the reflected light signals.

[0051] In addition, the automatic cleaning equipment is equipped with cleaning modules such as water tanks and cleaning cloths, motion modules such as wheels, and positioning modules. The electronically controlled components and environmental data acquisition modules within these modules are all controlled by the automatic cleaning equipment's processor. Correspondingly, point cloud data collected by the line laser sensor is sent to the processor for processing. The processor can identify obstacles and generate maps based on the received point cloud data, and then generate control commands based on the obstacles and maps. These control commands are sent to the cleaning and motion modules to achieve operational control under specific environmental conditions. There are also charging and communication modules. It is understood that the positioning, cleaning, and motion modules can all be implemented using methods from relevant automatic cleaning equipment technology fields. Specific installation methods and basic working principles will not be elaborated here, nor will the corresponding operational content be described in detail. For example, the process of returning to the base station and charging via the charging module after cleaning is not described in detail.

[0052] The robot control method provided in this embodiment of the invention includes the following steps:

[0053] Step 101: Acquire multiple point cloud data frames collected by the line laser sensor.

[0054] In this embodiment, the automatic cleaning device activates a line laser sensor during its movement, causing the sensor to emit laser light to scan the surrounding environment and generate multiple point cloud data frames. Each point cloud data frame includes multiple point cloud data points. After generating the multiple point cloud data frames, the automatic cleaning device acquires them. It should be noted that the automatic cleaning device can periodically acquire the multiple point cloud data frames collected by the line laser sensor within the duration of the period. The specific period can be set according to actual needs, and this embodiment does not impose a specific limitation.

[0055] Step 102: Determine the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each point cloud data frame.

[0056] After obtaining multiple frames of point cloud data, these frames need to be processed. Specifically, for each frame, the point cloud dataset within a preset distance range needs to be determined first. For example, in this embodiment, for each frame, the distance between each point cloud data point and the automatic cleaning device can be determined first. Then, the corresponding point cloud data within the preset distance range is identified, and this preset distance range is used as the point cloud dataset. The preset distance can be pre-set according to actual needs. After obtaining the point cloud dataset, further processing is required to extract point cloud features. These features characterize the height, density, and shape of the points in the dataset. Examples of point cloud features include maximum and minimum height values, average height values, and the number of points.

[0057] After determining the point cloud features corresponding to each point cloud dataset in each frame of point cloud data, it is necessary to determine the object category corresponding to each point cloud dataset based on the point cloud features. The object category is used to distinguish different types of environmental objects; for example, object categories include ground, low obstacles, large obstacles, and walls. Since different object categories have different point cloud quantity characteristics within different height ranges, this embodiment can identify different object categories based on the point cloud features of each point cloud dataset, thereby obtaining the object category corresponding to each point cloud dataset. For example, if the average height of the point cloud features of a certain point cloud dataset falls within the low obstacle height range, and the number of points is greater than the obstacle quantity threshold, then the object category corresponding to that point cloud dataset can be considered as low obstacles.

[0058] Step 103: Determine the target object category corresponding to each environmental object in the environment based on the object category of each point cloud dataset in all point cloud data frames.

[0059] After determining the object category of each point cloud dataset in each point cloud data frame, the automatic cleaning device needs to further determine the final target object category based on the object categories of each point cloud dataset in all point cloud data frames. For example, region clustering can be performed on all point cloud data in all point cloud data frames, grouping similar points or points with the same characteristics into one category to obtain a point cloud cluster corresponding to each environmental object. The region clustering method can employ K-means clustering, DBSCAN clustering, hierarchical clustering, etc., and the specific principles can be found in existing technologies, which will not be elaborated upon in this embodiment. After obtaining the point cloud clusters, the target object category corresponding to each point cloud cluster is further determined based on the object categories identified in the point cloud datasets corresponding to the point cloud data in the clusters. It is understood that each point cloud cluster includes multiple point cloud datasets, and the object category corresponding to each point cloud dataset can be determined based on the point cloud dataset where the previous point cloud data was located. Then, based on the object category corresponding to each point cloud data in each point cloud cluster, the target object category corresponding to the point cloud cluster can be determined, thereby determining the target object category corresponding to the environment object. For example, the object category with the largest proportion among all object categories corresponding to all point cloud data in each point cloud cluster can be taken as the target object category corresponding to the point cloud cluster, thus obtaining the target object category of the environment object corresponding to the point cloud cluster.

[0060] Step 104: Control the automatic cleaning equipment to avoid obstacles according to the target object category.

[0061] Once the target object category is determined, the automatic cleaning equipment can be controlled to avoid obstacles based on the location of the corresponding obstacles. For example, when the target object is a low obstacle (such as a data cable, power cord, building block, pen, sock, or shoelace), the automatic cleaning equipment will be controlled to avoid collisions with low obstacles or entanglement and getting stuck when running over them. Conversely, when the target object is a threshold or step, the automatic cleaning equipment can be controlled to overcome the obstacle.

[0062] The above-described embodiments of the present invention provide a robot control method. In this embodiment, after the automatic cleaning device acquires multiple frames of point cloud data collected by a line laser sensor, it first determines the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range in each frame. Then, based on the object category of each point cloud dataset in each frame, it determines the target object category corresponding to each environmental object. Finally, it controls the automatic cleaning device to avoid obstacles based on the target object category. This embodiment, by determining the object category of the point cloud dataset within a preset distance range in each frame and determining the target object category based on the object category of each point cloud dataset in all frames, avoids the situation where obstacles are identified solely based on a single frame, thereby reducing noise and the impact of ground undulations on the automatic cleaning device's movement on the ground, improving the accuracy of obstacle identification, and solving the technical problem of low identification accuracy in existing obstacle detection methods. Furthermore, the method of identifying object categories based on a line laser sensor in this embodiment is unaffected by background and ambient light, and can stably identify obstacles.

[0063] This invention also provides another robot control method, such as... Figure 4 As shown, Figure 4 This is a flowchart illustrating another robot control method provided in an embodiment of the present invention. Figure 4 The robot control method shown is a specific embodiment of the above-described robot control method. In this embodiment, a dual-line laser sensor is used as an example for illustration. The dual-line sensor includes two laser emitters. The robot control method provided in this embodiment includes the following steps:

[0064] Step 201: Obtain multiple frames of raw point cloud data collected by the dual-line laser sensor, divide the multiple frames of raw point cloud data into multiple frames of point cloud data corresponding to the two laser emitters.

[0065] After acquiring multiple frames of raw point cloud data from a dual-line laser sensor, the automated cleaning equipment needs to divide the raw point cloud data frames into multiple point cloud data frames corresponding to the two laser emitters, based on the laser source of the point cloud data in the raw point cloud data frames. For example, the automated cleaning equipment can determine the laser source of the point cloud data in the multiple raw point cloud data frames based on the coordinates of the point cloud data in the multiple raw point cloud data frames and the scanning area of ​​the laser emitters, and then divide the raw point cloud data into two point cloud datasets based on the laser source.

[0066] Specifically, assuming the two laser emitters of the dual-line laser sensor are symmetrically arranged on both sides directly in front of the automatic cleaning device, this embodiment will be illustrated by processing each frame of raw point cloud data to obtain a frame of point cloud data corresponding to each of the two laser emitters. After obtaining the raw point cloud data frame, since the raw point cloud data in the raw point cloud data frame is acquired by the dual-line laser sensor, the raw point cloud data is located in the pixel coordinate system. In this embodiment, it is necessary to further transform the raw point cloud data in the pixel coordinate system to the camera coordinate system to obtain the first point cloud data in the camera coordinate system. For example, assuming (u,v) is the raw point cloud data in the pixel coordinate system, z is the point cloud height of the raw point cloud data, (x,y) is the first point cloud data in the camera coordinate system, and m_fx, m_cx, m_fy, m_cy are camera intrinsic parameters, the specific formula for transforming the raw point cloud data in the pixel coordinate system to the first point cloud data in the camera coordinate system based on the camera intrinsic parameters is as follows:

[0067] x = (u - m_cx) × z / m_fx

[0068] y = (v - m_cy) × z / m_fy

[0069] After obtaining the first point cloud data in the camera coordinate system, it is necessary to further transform the first point cloud data into the second point cloud data in the robot coordinate system. Specifically, assuming (x', y') is the point cloud in the robot coordinate system, and matrices R and T are the camera extrinsic parameter matrices (R is the rotation matrix with 3 degrees of freedom, and T is the translation matrix, together forming the extrinsic parameter matrix), the specific formula for transforming the first point cloud data into the second point cloud data in the robot coordinate system is as follows:

[0070]

[0071] in,

[0072] After obtaining the second point cloud data in the robot coordinate system, the second point cloud data can be divided according to its coordinates in the robot coordinate system. Since the two sets of laser emitters of the dual-line laser sensor are symmetrically arranged on the left and right sides of the automatic cleaning device, this embodiment can distinguish the second point cloud data originating from different lasers based on the sign of the X-axis coordinate, thereby obtaining the point cloud data corresponding to the two laser emitters in each frame of original point cloud data. For example, as shown... Figure 5 As shown, Figure 5 This is a schematic diagram of the XY axis of a robot coordinate system provided in an embodiment of the present invention. Figure 5 The dashed lines in the image represent lasers emitted by two laser emitters, used to scan the point cloud data on both sides of the automated cleaning device. Figure 5The process involves dividing the second point cloud data (where x' is greater than or equal to 0 on the X-axis) into point cloud data corresponding to one laser emitter, and dividing the second point cloud data (where x' is less than 0 on the X-axis) into point cloud data corresponding to another laser emitter. Then, based on the point cloud data of each laser emitter, the point cloud data frame corresponding to each laser emitter can be determined. After processing all the original point cloud data frames, multiple point cloud data frames corresponding to the two laser emitters are obtained. It should be noted that subsequent processing requires processing each multi-frame point cloud data corresponding to one laser emitter individually, rather than processing the multi-frame point cloud data corresponding to both laser emitters together.

[0073] Step 202: In each point cloud data frame, determine the object category of each point cloud dataset based on the point cloud features of the point cloud dataset within different preset distance ranges relative to the automatic cleaning device.

[0074] In this embodiment, for each point cloud data frame, the distance between each point cloud data in each frame and the automatic cleaning device can be determined first. Then, the point cloud data corresponding to a preset distance range can be determined, and the point cloud data within the preset distance range can be used as the point cloud dataset. For example, as shown... Figure 6 As shown, Figure 6 This is a schematic diagram illustrating the segmentation of point cloud datasets according to an embodiment of the present invention. Point cloud data within a distance of 0–5 cm, 5–10 cm, 10–15 cm, and 15–20 cm are each segmented into a separate point cloud dataset, resulting in multiple point cloud datasets. Then, for each point cloud dataset, the corresponding point cloud features need to be determined, and based on these features, the corresponding object category is determined.

[0075] Based on the above embodiments, in step 202, in each point cloud data frame, according to the point cloud features of the point cloud datasets within different preset distance ranges from the distance to the automatic cleaning device, the object category of each point cloud dataset is determined, including:

[0076] Step 2021: Filter each point cloud data frame to obtain the corresponding target point cloud data frame.

[0077] In this embodiment, each point cloud data frame first needs to be filtered to remove abnormal values ​​caused by malfunctions or interference from the dual-line laser sensor. Furthermore, during the filtering process, only point cloud data within a certain distance range needs to be retained, while point cloud data from farther distances is removed to reduce the amount of data to be processed, thereby reducing the computational load. After filtering, the point cloud data retained in each frame constitutes the target point cloud data frame.

[0078] Step 2022: Based on the distance between the point cloud data in each target point cloud data frame and the automatic cleaning device, as well as different preset distance ranges, divide each target point cloud data frame into multiple point cloud datasets.

[0079] After obtaining multiple frames of target point cloud data, it is necessary to further determine the distance between the point cloud data and the automatic cleaning equipment in these frames. This distance can be Manhattan distance or Euclidean distance, etc. Then, based on different preset distance ranges, each frame of target point cloud data is further divided into multiple point cloud datasets.

[0080] Based on the above embodiments, in step 2022, each target point cloud data frame is divided into multiple point cloud datasets according to the distance between the point cloud data in each target point cloud data frame and the automatic cleaning device, as well as different preset distance ranges;

[0081] Step 20221: Calculate the distance between the point cloud data in each target point cloud data frame and the automatic cleaning device.

[0082] In one embodiment, the distance between the point cloud data in each target point cloud data frame and the automatic cleaning device can be calculated. Taking the Manhattan distance as an example, assuming the current coordinates of the automatic cleaning device in the robot coordinate system are (Xc, Yc), and the point cloud data in the target point cloud data frame in the robot coordinate system are (x1', y1'), (x2', y2')...(xn', yn'), then the Manhattan distance D from each point cloud data to the current position of the automatic cleaning device is:

[0083]

[0084] Step 20222: Sort the point cloud data in each target point cloud data frame according to the distance to obtain the sorting queue corresponding to each target point cloud data frame.

[0085] After determining the Manhattan distance between the point cloud data in each target point cloud data frame and the automatic cleaning device, it is necessary to further sort the point cloud data in each target point cloud data frame from smallest to largest according to the Manhattan distance and store them in a queue, thereby obtaining a sorted queue corresponding to each target point cloud data frame.

[0086] Step 20223: Divide each sorting queue according to different preset distance ranges to obtain multiple point cloud datasets corresponding to each target point cloud data frame.

[0087] Finally, each sorting queue can be segmented according to a preset distance range to obtain multiple point cloud datasets corresponding to each target point cloud data frame. The preset distance range needs to be set in advance. For example, in one embodiment, point cloud data with a Manhattan distance of 0-10cm from the automatic cleaning device can be divided into one point cloud dataset, and point cloud data with a Manhattan distance of 10-20cm from the automatic cleaning device can be divided into another point cloud dataset, and so on.

[0088] The above describes the specific process of dividing each frame of target point cloud data based on the distance to the automatic cleaning equipment.

[0089] Step 203: In each target point cloud data frame, determine the point cloud features of each point cloud dataset, and determine the object category corresponding to the point cloud dataset based on the point cloud features.

[0090] After dividing each frame of target point cloud data into multiple point cloud datasets, further processing is needed on the point cloud data in each dataset to extract point cloud features and identify the corresponding object category based on these features. In one embodiment, the point cloud features of the dataset include maximum height, minimum height, average height, and number of points. The maximum height refers to the maximum height of the point cloud data in the dataset, the minimum height refers to the minimum height, the average height refers to the average height, and the number of points is the total number of points in the dataset. When determining the object category corresponding to the point cloud dataset based on the point cloud features, the object category can be determined according to the maximum height, minimum height, average height, number of points, and preset object recognition rules. It is understood that in this embodiment, different object recognition rules can be preset for different object categories, and the automatic cleaning device determines whether the extracted point cloud features meet the corresponding object recognition rules to determine the corresponding object category.

[0091] Specifically, in one embodiment, object categories include Floor, Low obstacles, and Wall. The specific process of determining the object category corresponding to a point cloud dataset based on point cloud features includes: if the minimum height min_z of a point cloud dataset is within the ground height range and the number of points size is greater than the effective ground number threshold floor_threshold, then the object category corresponding to the point cloud dataset is considered to be Floor; if the average height ave_z is within the low obstacle height range and the number of points size is greater than the effective low obstacle number threshold Lowobstacles_threshold, then the object category corresponding to the point cloud dataset is considered to be Low obstacles; if the maximum height max_z is within the wall height range and the number of points size is greater than the effective wall number threshold wall_threshold, then the object category corresponding to the point cloud dataset is considered to be Wall. The specific judgment rules are as follows:

[0092]

[0093]

[0094] For example, Figure 7 This is a schematic diagram illustrating the identification of object categories in different point cloud datasets provided in an embodiment of the present invention. For example, Figure 6 The point cloud datasets are divided, and the object categories corresponding to each point cloud dataset are as follows: Figure 7 As shown. It is understandable that in complex usage scenarios, objects will be categorized into more types, such as large obstacles, straight walls, sloping walls, ramps, and lower limit areas. The object recognition rules corresponding to these object categories will also increase, and each object recognition rule can be pre-set. In this embodiment, the specific content of the object recognition rules can be set according to actual needs and is not specifically limited.

[0095] Furthermore, in practical use, it has been found that for the same point cloud dataset, more than one object category may be identified. For example, the same point cloud dataset may simultaneously satisfy the object recognition rules for ground and the object recognition rules for low obstacles. To improve the accuracy of obstacle identification, the automated cleaning device can extract more point cloud features from the point cloud dataset to determine the final object category. In one embodiment, the point cloud features also include a point cloud bounding box, which aims to find a geometry that can contain all the point cloud data. This geometry is typically a rectangular box (a cuboid in three-dimensional space) that describes the outer boundary of the point cloud data. For example, such as... Figure 8 As shown, Figure 8 This is a schematic diagram of a point cloud bounding box provided in an embodiment of the present invention. Figure 8 The object in the diagram is a wire, and the dashed bounding box surrounding the wire is the point cloud bounding box.

[0096] When the number of object categories corresponding to the point cloud dataset is greater than one, it also includes:

[0097] Based on the number of points corresponding to the point cloud dataset and the bounding box of the point cloud, determine the probability that the point cloud dataset belongs to different object categories.

[0098] Based on the probability and the object category corresponding to the point cloud dataset, a unique object category corresponding to the point cloud dataset is determined.

[0099] In one embodiment, when a point cloud dataset identifies multiple object categories, the automated cleaning device can further determine the probability that the point cloud dataset belongs to different object categories based on the number of points and the bounding boxes of the points. For example, a corresponding object category prediction model can be pre-determined using machine learning algorithms (such as support vector machines, random forests, etc.) or deep learning algorithms (such as convolutional neural networks, point cloud neural networks, etc.), and the object category prediction model can be trained using point cloud data with known object categories. Subsequently, when making predictions, the number of points and the bounding boxes of the points can be input into the pre-trained object category prediction model, and the object category prediction model outputs the probability corresponding to each object category. After determining the probability that the point cloud dataset belongs to different object categories, the object category with the highest probability can be determined as the final and unique object category corresponding to the point cloud dataset.

[0100] The above describes the specific process for determining the object categories corresponding to the point cloud dataset.

[0101] Step 204: Cluster all point cloud data in all point cloud data frames to obtain point cloud clusters corresponding to each environmental object.

[0102] After determining the object category corresponding to each point cloud dataset, it is necessary to further perform region clustering on all point cloud data in the multi-frame point cloud data to obtain point cloud clusters corresponding to different environmental objects.

[0103] Step 205: Based on the object categories confirmed in the point cloud dataset corresponding to the point cloud data in each point cloud cluster, determine the proportion of point cloud data corresponding to each object category in each point cloud cluster.

[0104] For each clustered point cloud, it is necessary to further determine the object category corresponding to each point cloud data in each cluster. The object category is determined based on the object category corresponding to the point cloud dataset to which the point cloud data belongs. It should be noted that if the same point cloud data corresponds to different object categories in different point cloud datasets, the object category with the highest probability among the corresponding object categories will be used as the object category corresponding to that point cloud data. Next, it is necessary to determine the proportion of point cloud data corresponding to each object category in each point cloud cluster.

[0105] Step 206: Determine the target object category corresponding to the point cloud cluster by identifying the object category with the highest proportion in each point cloud cluster.

[0106] After determining the proportion of point cloud data corresponding to each object category in each point cloud cluster, the object category with the highest proportion in each point cloud cluster is determined as the target object category corresponding to that point cloud cluster. For example, for a point cloud cluster, the proportion of point cloud data for the object category of low obstacles is 76%, the proportion of point cloud data for the object category of ground is 15%, the proportion of point cloud data for the object category of wall is 5%, and the proportion of point cloud data for other object categories is 4%. Then the target object category corresponding to this point cloud cluster is low obstacles.

[0107] Step 207: Control the automatic cleaning equipment to avoid obstacles according to the target object category.

[0108] Finally, based on the target object category determined by each point cloud cluster, the automatic cleaning equipment can be controlled to move and avoid obstacles.

[0109] The above-described embodiments of the present invention provide a robot control method. In this embodiment, an automatic cleaning device sorts point cloud data according to the distance between itself and the device in each frame of point cloud data, dividing the point cloud data within different distance ranges into point cloud datasets. Then, based on the point cloud features of each dataset, the corresponding object category is determined. Finally, all point cloud data in multiple frames are clustered to obtain multiple point cloud clusters. The object category with the highest proportion in each cluster is determined as the final target object category, and the automatic cleaning device is controlled to avoid obstacles based on the target object category. This embodiment, by determining the object category within different distance ranges in each frame of point cloud data and determining the target object category based on the proportion of different object categories in the clustering results of all point cloud data frames, avoids the situation where obstacles are identified solely based on a single frame, thereby reducing noise and the impact of ground undulations on the automatic cleaning device's movement on the ground, improving the accuracy of obstacle identification, and solving the technical problem of low accuracy in existing obstacle detection methods. In addition, the method of identifying object categories based on a line laser sensor in this embodiment is not affected by background and ambient light, and can stably identify obstacles.

[0110] This embodiment also provides an automatic cleaning device, such as... Figure 9 As shown, Figure 9 This is a schematic diagram of the structure of an automatic cleaning device provided in an embodiment of the present invention. The automatic cleaning device 10 includes a line laser sensor 20, a processor 400, and a memory 401. The line laser sensor 20 includes at least one laser emitter and a laser receiver. Each laser emitter is used to emit a line laser into the environment, and the laser receiver is used to receive an environmental image reflected from the line laser on an environmental object. The line laser sensor 20 is used to obtain point cloud data based on the environmental image.

[0111] Memory 401 is used to store computer program 402 and transfer computer program 402 to processor 400;

[0112] The processor 400 is used to execute the steps in one embodiment of the robot control method described above, according to the instructions in the computer program 402.

[0113] For example, computer program 402 may be divided into one or more modules / units, one or more of which are stored in memory 401 and executed by processor 400 to complete this application. One or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 402 in automatic cleaning device 10.

[0114] The automatic cleaning device 10 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will understand that... Figure 9 This is merely an example of the automatic cleaning device 10 and does not constitute a limitation on the automatic cleaning device 10. It may include more or fewer components than shown, or combine certain components, or different components. For example, the automatic cleaning device 10 may also include input / output devices, network access devices, buses, etc.

[0115] The processor 400 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0116] The memory 401 can be an internal storage unit of the automatic cleaning device 10, such as a hard disk or RAM of the automatic cleaning device 10. The memory 401 can also be an external storage device of the automatic cleaning device 10, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the automatic cleaning device 10. Furthermore, the memory 401 can include both internal and external storage units of the automatic cleaning device 10. The memory 401 is used to store computer programs and other programs and data required by the automatic cleaning device 10. The memory 401 can also be used to temporarily store data that has been output or will be output.

[0117] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing computer programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0122] This invention also provides a storage medium containing computer-executable instructions. When executed by a computer processor, these instructions are used to perform a robot control method. The robot control method is applicable to an automated cleaning device. The automated cleaning device includes a line laser sensor, which includes at least one laser emitter and one laser receiver. Each laser emitter emits a line laser beam into the environment, and the laser receiver receives an environmental image of the line laser beam reflected from an object in the environment. The line laser sensor obtains point cloud data based on the environmental image. The robot control method includes:

[0123] Acquire multiple frames of point cloud data collected by the line laser sensor;

[0124] Based on the point cloud features of the point cloud dataset within a preset distance range for each point cloud data frame, determine the object category corresponding to each point cloud dataset;

[0125] Based on the object categories of each point cloud dataset in all point cloud data frames, determine the target object category corresponding to each environmental object in the environment;

[0126] The automatic cleaning equipment is controlled to avoid obstacles based on the category of the target object.

[0127] Note that the above are merely preferred embodiments and the technical principles applied in this invention. Those skilled in the art will understand that the embodiments of this invention are not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the protection scope of this invention. Therefore, although the embodiments of this invention have been described in detail above, the embodiments of this invention are not limited to the above embodiments. More other equivalent embodiments may be included without departing from the concept of the embodiments of this invention, and the scope of the embodiments of this invention is determined by the scope of the appended claims.

Claims

1. A robot control method, characterized in that, The method is applicable to automated cleaning equipment, which includes a line laser sensor. The line laser sensor includes at least one laser emitter and one laser receiver. Each laser emitter emits a line laser beam into the environment, and the laser receiver receives an environmental image of the line laser beam reflected from an environmental object. The line laser sensor obtains point cloud data based on the environmental image. The method includes: Acquire multiple frames of point cloud data collected by the line laser sensor; Based on the point cloud features of the point cloud dataset within a preset distance range of each point cloud data frame, determine the object category corresponding to each point cloud dataset; Based on the object category of each point cloud dataset in all the point cloud data frames, determine the target object category corresponding to each environmental object in the environment; The automatic cleaning equipment is controlled to avoid obstacles based on the target object category.

2. The robot control method according to claim 1, characterized in that, The step of determining the object category corresponding to each point cloud dataset based on the point cloud features of the point cloud dataset within a preset distance range for each frame of the point cloud data includes: In each point cloud data frame, the object category of each point cloud dataset is determined based on the point cloud features of the point cloud dataset within different preset distance ranges relative to the automatic cleaning device.

3. The robot control method according to claim 2, characterized in that, Each point cloud data frame includes multiple point cloud data sets. Within each point cloud data frame, based on the point cloud features of point cloud datasets within different preset distance ranges from the automatic cleaning device, the object category of each point cloud dataset is determined, including: Each point cloud data frame is filtered to obtain the corresponding target point cloud data frame. Based on the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device, and different preset distance ranges, each frame of the target point cloud data is divided into multiple point cloud datasets. In each frame of the target point cloud data, the point cloud features of each point cloud dataset are determined, and the object category corresponding to each point cloud dataset is determined based on the point cloud features.

4. The robot control method according to claim 3, characterized in that, The method involves dividing each frame of the target point cloud data into multiple point cloud datasets based on the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device, as well as different preset distance ranges; Calculate the distance between the point cloud data in each frame of the target point cloud data and the automatic cleaning device; The point cloud data in each frame of the target point cloud data is sorted according to the distance to obtain a sorting queue corresponding to each frame of the target point cloud data. Each sorting queue is divided according to different preset distance ranges to obtain multiple point cloud datasets corresponding to each frame of the target point cloud data frame.

5. The robot control method according to claim 3, characterized in that, The point cloud features of the point cloud dataset include maximum height, minimum height, average height, and number of points. The object categories include low obstacles, walls, and ground. Determining the object category corresponding to each point cloud dataset based on the point cloud features includes: If the minimum height of the point cloud dataset is within the ground height range and the number of point clouds is greater than the effective ground quantity threshold, then the object category corresponding to each point cloud dataset is determined to be the ground. If the average height of the point cloud dataset is within the height range of low obstacles and the number of point clouds is greater than the effective number threshold of low obstacles, then the object category corresponding to each point cloud dataset is determined to be the low obstacle. If the maximum height of the point cloud dataset is within the wall height range and the number of point clouds is greater than the effective number threshold of the wall, then the object category corresponding to each point cloud dataset is determined to be the wall.

6. The robot control method according to claim 5, characterized in that, The point cloud features also include point cloud bounding boxes, and when the number of object categories corresponding to the point cloud dataset is greater than one, they also include: Based on the number of points and the bounding box of the points corresponding to the point cloud dataset, determine the probability that the point cloud dataset belongs to different object categories; Based on the probability and the object category corresponding to the point cloud dataset, a unique object category corresponding to the point cloud dataset is determined.

7. The robot control method according to claim 1, characterized in that, The step of determining the target object category corresponding to each environmental object in the environment based on the object category of each point cloud dataset in all the point cloud data frames includes: Cluster all point cloud data in all the point cloud data frames to obtain point cloud clusters corresponding to each of the environmental objects; Based on the object category confirmed in the point cloud dataset corresponding to the point cloud data in each point cloud cluster, determine the proportion of point cloud data corresponding to each object category in each point cloud cluster; The object category with the highest proportion in each point cloud cluster is determined as the target object category corresponding to the point cloud cluster.

8. The robot control method according to any one of claims 1 to 7, characterized in that, The line laser sensor is a dual-line sensor, which includes two laser emitters. Acquiring multiple frames of point cloud data collected by the line laser sensor includes: The original point cloud data frames collected by the dual-line laser sensor are acquired, and the original point cloud data frames are divided to obtain the original point cloud data frames corresponding to the two laser emitters.

9. An automatic cleaning device, characterized in that, The automatic cleaning device includes a line laser sensor, a processor, and a memory. The line laser sensor includes at least one laser emitter and a laser receiver. Each laser emitter is used to emit a line laser into the environment, and the laser receiver is used to receive an environmental image reflected by the line laser on an environmental object. The line laser sensor is used to obtain point cloud data based on the environmental image. The memory is used to store computer programs and to transfer the computer programs to the processor; The processor is used to execute a robot control method as described in any one of claims 1 to 8 according to instructions in the computer program.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform a robot control method as described in any one of claims 1 to 8.

Images