Method, apparatus, robot, and storage medium for robot motion control

Abstract

The present application relates to a method, device, robot, and storage medium for robot motion control. The method includes: obtaining a laser point cloud frame obtained by scanning an operating environment; filtering data points in the laser point cloud frame according to reflection features corresponding to the respective data points in the laser point cloud frame to obtain a filtered laser point cloud frame; clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters; and if it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, performing motion control on the robot. Using this method can ensure the safe operation of the robot.

Description

Technical Field

[0001] The present application relates to the technical field of mobile robots, and in particular, to a method, device, robot, storage medium, and computer program product for robot motion control. Background Art

[0002] With the development of mobile robot technology, the application environment of mobile robots is becoming more and more complex. During the movement of the robot, it may enter a dangerous area or a prohibited area. For example, the robot may have a risk of falling due to entering a dangerous area. In traditional technologies, infrared detection can be used to control the movement of the robot. However, due to the short sensing distance of infrared detection, when the robot moves at a high speed, it is impossible to prevent the robot from falling or the robot may tip over due to emergency braking. In the method of controlling the robot based on a magnetic stripe, the induction distance of the magnetic stripe is short and it is easy to demagnetize. The accuracy of the anti-fall method based on a depth camera is poor and false detection is likely to occur. Therefore, how to ensure the safe operation of the robot has become an urgent problem to be solved. Summary of the Invention

[0003] Based on this, in view of the above technical problems, it is necessary to provide a method, device, robot, computer-readable storage medium, and computer program product for robot motion control that can ensure the safe operation of the robot.

[0004] In a first aspect, the present application provides a method for robot motion control. The method includes:

[0005] Obtaining a laser point cloud frame obtained by scanning the operating environment;

[0006] Filtering the data points in the laser point cloud frame according to the reflection characteristics corresponding to the respective data points in the laser point cloud frame to obtain a filtered laser point cloud frame;

[0007] Clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters;

[0008] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, performing motion control on the robot.

[0009] In a second aspect, the present application further provides a device for robot motion control. The device includes:

[0010] An obtaining module, configured to obtain a laser point cloud frame obtained by scanning the operating environment;

[0011] A filtering module, configured to filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to the respective data points in the laser point cloud frame to obtain a filtered laser point cloud frame;

[0012] A clustering module, configured to cluster data points in the filtered laser point cloud frame to obtain point cloud clusters;

[0013] A control module, configured to perform motion control on the robot if it is determined based on the point cloud clusters that the laser point cloud frame contains a marker.

[0014] In one embodiment, the control module is further configured to:

[0015] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, determine the distance between the robot and the marker;

[0016] Perform motion control on the robot according to the distance.

[0017] In one embodiment, the laser point cloud frame includes at least two frames; the control module is further configured to:

[0018] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, respectively determine the position coordinates corresponding to each marker;

[0019] If the position coordinates corresponding to the markers in at least two laser point cloud frames are the same, perform motion control on the robot.

[0020] In one embodiment, the marker includes a reflective marker, and the reflection feature includes a reflection intensity; the filtering module is further configured to:

[0021] Determine the average reflection intensity and the maximum reflection intensity of the data points according to the reflection intensities corresponding to the data points in the laser point cloud frame;

[0022] Determine an intensity threshold according to the average reflection intensity and the maximum reflection intensity;

[0023] Filter the data points in the laser point cloud frame based on the intensity threshold to obtain a filtered laser point cloud frame.

[0024] In one embodiment, the marker is a graphic combination composed of at least two graphic elements; the clustering module is further configured to:

[0025] Determine a first clustering threshold and a second clustering threshold according to the distances between the graphic elements in the graphic combination; the first clustering threshold is less than the second clustering threshold;

[0026] Cluster the data points in the filtered laser point cloud frame according to the first clustering threshold, and select a first cluster that satisfies a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element;

[0027] Cluster the data points in the first type of cluster according to the second clustering threshold, and select a second type of cluster that meets the second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination;

[0028] Filter the second type of cluster to obtain a point cloud cluster.

[0029] In one embodiment, the clustering module is further configured to:

[0030] Cluster each of the second type of clusters into a plurality of sub-clusters according to the first clustering threshold, and determine the number of sub-clusters in each of the second type of clusters;

[0031] Filter the second type of cluster based on the number of sub-clusters in the second type of cluster to obtain a point cloud cluster.

[0032] In one embodiment, the control module is further configured to:

[0033] Select at least two target sub-clusters in the sub-clusters of each of the point cloud clusters;

[0034] For each of the point cloud clusters, determine the ratio between the number of data points of the target sub-clusters;

[0035] If there is a target ratio that meets the ratio condition among the ratios, determine that the laser point cloud frame contains a marker, and perform motion control on the robot.

[0036] In a third aspect, the present application further provides a robot. The robot includes a memory and a processor. The memory stores a computer program. A lidar is mounted on the robot. A marker is set in the operating environment of the robot. When the processor executes the computer program, the following steps are implemented:

[0037] Obtain a laser point cloud frame obtained by scanning the operating environment;

[0038] Filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to the data points in the laser point cloud frame to obtain a filtered laser point cloud frame;

[0039] Cluster the data points in the filtered laser point cloud frame to obtain a point cloud cluster;

[0040] If it is determined that the laser point cloud frame contains a marker based on the point cloud cluster, perform motion control on the robot.

[0041] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, the following steps are implemented:

[0042] Obtain a laser point cloud frame obtained by scanning the operating environment;

[0043] Filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame;

[0044] Cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters;

[0045] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, perform motion control on the robot.

[0046] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0047] Obtain a laser point cloud frame obtained by scanning the operating environment;

[0048] Filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame;

[0049] Cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters;

[0050] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, perform motion control on the robot.

[0051] For the method, device, robot, storage medium, and computer program product for robot motion control described above, obtain a laser point cloud frame obtained by scanning the operating environment; since the reflection characteristics of the marker to the laser are different from those of other objects in the operating environment, filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame, so that the data points with reflection characteristics different from those of other objects can be filtered out to obtain a filtered laser point cloud frame. Cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters, so that it can be determined whether the laser point cloud frame contains a marker according to the shape characteristics of the marker. If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, perform motion control on the robot. By using the different characteristics of the marker's reflection to the laser and the shape characteristics to identify the marker in the laser point cloud frame, the recognition accuracy is higher, effectively avoiding the robot from entering dangerous areas and prohibited areas. And the sensing distance of the laser is relatively far, and the marker can be identified at a relatively far distance, enabling the robot to have sufficient time to brake, effectively avoiding the fast-moving robot from falling or toppling due to emergency braking, and ensuring the safe operation of the robot. Description of the Drawings

[0052] Figure 1 It is an application environment diagram of the method for robot motion control in an embodiment;

[0053] Figure 2 It is a schematic flowchart of the method for robot motion control in an embodiment;

[0054] Figure 3 It is a schematic diagram of a retroreflective marker in an embodiment;

[0055] Figure 4 It is a schematic flowchart of the method for motion control of a robot according to distance in an embodiment;

[0056] Figure 5a It is a schematic diagram of the area determined according to the distance from the marker in an embodiment;

[0057] Figure 5b It is a schematic flowchart of the filtering step for a laser point cloud frame in an embodiment;

[0058] Figure 6 It is a schematic flowchart of the method for clustering to obtain point cloud clusters in an embodiment;

[0059] Figure 7 It is a schematic flowchart of the method for robot motion control in another embodiment;

[0060] Figure 8 It is a structural block diagram of the device for robot motion control in an embodiment;

[0061] Figure 9 It is an internal structure diagram of a robot in an embodiment. Detailed implementation manners

[0062] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] The method for robot motion control provided by the embodiments of the present application can be applied to, for example Figure 1In the application environment shown. Among them, the robot 102 obtains the laser point cloud frame obtained by scanning the operating environment; filters the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain the filtered laser point cloud frame; clusters the data points in the filtered laser point cloud frame to obtain point cloud clusters; if it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, motion control is performed on the robot 102. Among them, the robot 102 can be but is not limited to various delivery robots, operation robots, service robots, sorting robots, or cleaning robots, etc. The user sets a marker at the edge of the dangerous area or prohibited area in the operating environment of the robot 102, and the marker has at least one of recognizable structural characteristics, material characteristics, or pattern characteristics. For example, the user can paste markers on both sides of the stairs, such as reflective markers or structured markers. The reflective marker has a specific shape and a high reflectivity to laser pulses; while the structured marker has a specific structure and presents specific structural characteristics in the laser point cloud frame formed after reflecting the laser pulse. A lidar is installed on the robot 102, and the operating environment is scanned through the lidar to obtain the laser point cloud frame. Thus, the reflection intensity or structural characteristics are extracted as the reflection characteristics to detect whether there is a marker in the operating environment, and then motion control can be performed on the robot.

[0064] In one embodiment, as Figure 2 shown, a method for robot motion control is provided. Taking the robot in Figure 1 as an example for illustration, it includes the following steps:

[0065] S202, obtain the laser point cloud frame obtained by scanning the operating environment.

[0066] Among them, the operating environment is the environment in which the robot operates, which can be an indoor environment or an outdoor environment. For example, the operating environment can be the road environment for delivering goods. Another example is that the operating environment can be the hotel environment for providing services. Another example is that the operating environment can be the operating environment in a factory.

[0067] A marker is set in this operating environment, and the marker includes a reflective marker and a structured marker. Among them, the reflective marker has a high reflection intensity to laser pulses. According to the reflection intensity corresponding to each data point, the data points with a high reflection intensity can be filtered out to identify the reflective marker from the filtered data points. The structured marker has a specific structure and presents specific structural characteristics in the laser point cloud frame formed after reflecting the laser pulse. Therefore, filtering can be performed according to the structural characteristics corresponding to each data point to identify the structured marker from the filtered data points.

[0068] The robot is equipped with a lidar, so it can scan through the lidar to obtain a lidar point cloud frame. The lidar point cloud frame consists of multiple data points, and each data point has a corresponding lidar intensity. The lidar is an optical sensor that can emit laser pulses into the operating environment and receive the laser pulses reflected by various objects in the operating environment, and generate a lidar point cloud frame based on the reflected laser pulses.

[0069] S204. Filter the data points in the lidar point cloud frame according to the reflection characteristics corresponding to each data point in the lidar point cloud frame to obtain a filtered lidar point cloud frame.

[0070] Among them, the data point is a point in the lidar point cloud frame, which is used to describe the points in the three-dimensional space scanned by the lidar. Each data point includes various attributes such as position coordinates, reflection characteristics, and scanning angles. The reflection characteristic can be the reflection intensity or the reflected structural characteristic. The reflection intensity is used to represent the pulse echo intensity of the lidar. When the reflectivity of the point in the three-dimensional space to the laser pulse is higher, the reflection intensity of the data point corresponding to the point in the three-dimensional space is higher. The structural characteristic is the structured information presented by the pulse echo of the lidar, which can reflect the specific structures of various objects in the operating environment.

[0071] In one embodiment, when the reflection characteristic is the reflection intensity, S204 specifically includes: determining an intensity threshold according to the reflection intensity corresponding to each data point in the lidar point cloud frame; filtering out the data points with a reflection intensity greater than the intensity threshold from the data points in the lidar point cloud frame to obtain a filtered lidar point cloud frame.

[0072] S206. Cluster the data points in the filtered lidar point cloud frame to obtain point cloud clusters.

[0073] Among them, the point cloud cluster is a cluster composed of similar data points. For example, the point cloud cluster can be a cluster composed of the data points corresponding to the same object scanned by the lidar. Clustering is an unsupervised learning technique that measures the similarity of the objects to be clustered and clusters the similar objects into one category.

[0074] In one embodiment, the robot can use the K-Means (K-means) clustering algorithm, the BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies, comprehensive hierarchical clustering) algorithm, or the Gaussian mixture clustering algorithm to cluster the data points in the filtered lidar point cloud frame to obtain point cloud clusters.

[0075] S208. If it is determined based on the point cloud cluster that the lidar point cloud frame contains a marker, perform motion control on the robot.

[0076] In one embodiment, when the reflection feature is the reflection intensity, the robot can determine whether the laser point cloud frame contains a reflective marker based on the point cloud cluster. If it is determined based on the point cloud cluster that the laser point cloud frame contains a reflective marker, motion control is performed on the robot.

[0077] Among them, the reflective marker is a marker composed of a reflective material, pasted on the edge of a dangerous area or a prohibited area, and can be composed of graphic elements of various shapes. For example, the reflective marker can be composed of one or more rectangles, circles, ellipses, or triangles, etc. The multiple graphic elements that make up the reflective marker can be the same or different. For example, the reflective marker can be composed of two or more rectangles, or the reflective marker can also be composed of a rectangle and a circle, or the reflective marker can also be composed of a circle and a triangle. In one embodiment, as Figure 3 shown, the reflective marker is composed of two rectangles arranged side by side. The size of the rectangle and the spacing between the rectangles can be adjusted according to actual needs. For example, the size of the rectangle can be 50 mm × 100 mm, and the spacing between the two rectangles can be 50 mm.

[0078] The motion control is to control the motion mode of the robot, including controlling the moving speed, moving direction, or moving path of the robot, etc. For example, controlling the robot to stop moving, or controlling the robot to bypass a dangerous area, or controlling the robot to reduce the moving speed, etc. If it is determined based on the point cloud cluster that the laser point cloud frame contains a reflective marker, it means that the robot is about to run into a dangerous area or a prohibited area, and motion control needs to be performed on the robot. For example, the user can paste reflective markers on both sides of the stairs or elevators. When the reflective marker is included in the laser point cloud frame collected by the robot's lidar, it means that the robot is about to run near the stairs or elevators and there is a risk of falling, and motion control is performed on the robot. Another example is that the user can paste reflective markers on the edge of a no-go area such as a water area. When the reflective marker is included in the laser point cloud frame collected by the robot's lidar, it means that the robot is about to run into the no-go area, and motion control is performed on the robot. Another example is that the user can also paste reflective markers on the edge of a fragile obstacle such as glass. When the reflective marker is included in the laser point cloud frame collected by the robot's lidar, it means that the robot may collide with the fragile obstacle, and motion control is performed on the robot.

[0079] In another embodiment, when the reflection feature is the structural feature, the robot can determine whether the laser point cloud frame contains a structured marker based on the point cloud cluster. If it is determined based on the point cloud cluster that the laser point cloud frame contains a structured marker, motion control is performed on the robot.

[0080] In the above embodiments, a laser point cloud frame obtained by scanning the operating environment is acquired; since the reflection characteristics of the marker to the laser are different from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered according to the reflection characteristics corresponding to each data point in the laser point cloud frame, so that the data points with reflection characteristics different from those of other objects can be filtered out to obtain a filtered laser point cloud frame. The data points in the filtered laser point cloud frame are clustered to obtain point cloud clusters, so that it can be determined whether the laser point cloud frame contains a marker according to the shape characteristics of the marker. If it is determined that the laser point cloud frame contains a marker based on the point cloud clusters, motion control is performed on the robot. By using the characteristics of different laser reflections of the marker and the shape characteristics to identify the marker in the laser point cloud frame, the recognition accuracy is higher, effectively avoiding the robot from entering dangerous areas and prohibited areas. And the sensing distance of the laser is relatively far, and the marker can be recognized at a relatively long distance, enabling the robot to have sufficient time to brake, effectively avoiding the falling of the fast-moving robot or toppling due to emergency braking, and ensuring the safe operation of the robot.

[0081] In one embodiment, as Figure 4 shown, S208 specifically includes the following steps:

[0082] S402, if it is determined that the laser point cloud frame contains a marker based on the point cloud clusters, determine the distance between the robot and the marker.

[0083] Wherein, the distance is the distance between the robot and the marker in three-dimensional space. The robot can obtain the time interval from the emission of the laser pulse by the lidar to the reception of the reflected echo, and the distance between the robot and the marker can be determined according to the time interval.

[0084] S404, perform motion control on the robot according to the distance.

[0085] Since the marker is pasted on the edge of the dangerous area, when the robot is far from the marker, the urgency of danger is weaker, and when the robot is close to the marker, the urgency of danger is stronger. Therefore, motion control is performed on the robot according to the distance between the robot and the marker.

[0086] In one embodiment, S404 specifically includes: determining the area where the robot is currently located according to the distance, and performing motion control on the robot according to the area where the robot is currently located. Specifically, as Figure 5aAs shown, the black rectangular frame is a reflective marker. Area A is the braking area, Area B is the avoidance area, and Area C is the deceleration area. When the robot moves to the deceleration area, control the robot to reduce its moving speed. For example, reduce the moving speed of the robot to 0.6 m / s. When the robot is in the avoidance area, control the robot to stop moving when it encounters an obstacle. When the robot is in the braking area, control the robot to stop moving. Among them, the deceleration area is the farthest from the reflective marker, the avoidance area is in the middle of the braking area and the deceleration area, and the braking area is the closest to the reflective marker. The distances between each area and the reflective marker can be adjusted. For example, the braking area can be the area where the distance from the reflective marker is less than 1.2 m, the avoidance area can be the area where the distance from the reflective marker is greater than or equal to 1.2 m and less than 1.5 m, and the deceleration area can be the area where the distance from the reflective marker is greater than or equal to 1.5 m and less than 2 m.

[0087] In one embodiment, S404 specifically includes: controlling the moving path of the robot according to the distance. For example, when the distance is less than a preset value, control the robot to stop moving and return. Or when the distance is less than a preset value, control the robot to adjust its moving direction.

[0088] In the above embodiment, if it is determined that the laser point cloud frame contains a marker based on the point cloud cluster, the distance between the robot and the marker is determined, and the movement of the robot is controlled according to the distance. Thus, the movement mode of the robot can be adjusted according to the distance, improving the flexibility of the robot's movement.

[0089] In one embodiment, S208 specifically includes: if it is determined that the laser point cloud frame contains a marker based on the point cloud cluster, the position coordinates corresponding to each marker are determined respectively; if the position coordinates corresponding to the markers in at least two laser point cloud frames are the same, the movement of the robot is controlled.

[0090] Among them, the position coordinates are the coordinates of the marker in the world coordinate system. Each data point in the laser point cloud frame includes the position coordinates corresponding to the data point. The robot can determine the position coordinates of the marker according to the position coordinates corresponding to each data point. To avoid misdetection by the robot, when the robot recognizes a marker in a certain laser point cloud frame, make the robot continue to collect laser point cloud frames through the lidar. If the position coordinates corresponding to the markers in multiple laser point cloud frames continuously collected by the lidar are the same, it means that these markers all correspond to the markers at the same position, then it is determined that the robot scans the marker, and the movement of the robot is controlled.

[0091] In the above embodiments, if it is determined that the laser point cloud frame contains markers based on the point cloud clusters, the position coordinates of each marker are determined respectively; if the position coordinates of the markers in at least two laser point cloud frames are the same, motion control is performed on the robot. Thus, misdetection of the robot can be effectively avoided, and the accuracy of motion control of the robot is improved.

[0092] In one embodiment, the marker includes a reflective marker, and the reflection feature includes the reflection intensity; as Figure 5b shown, S204 specifically includes the following steps:

[0093] S502, determine the average reflection intensity and the maximum reflection intensity of the data points according to the reflection intensity corresponding to each data point in the laser point cloud frame.

[0094] Among them, the average reflection intensity is the average value of the reflection intensities corresponding to each data point in the laser point cloud frame. The maximum reflection intensity is the maximum value of the reflection intensities corresponding to each data point in the laser point cloud frame. When the robot receives the laser point cloud frame, it statistically calculates the reflection intensity corresponding to each data point in the laser point cloud frame, calculates the average reflection intensity, and searches for the maximum reflection intensity among the reflection intensities corresponding to each data point. The robot can search for the maximum reflection intensity through various search methods such as sequential search method, binary search method or binary tree search method.

[0095] S504, determine the intensity threshold according to the average reflection intensity and the maximum reflection intensity.

[0096] When the marker is a reflective marker, since the reflective marker has a high reflectivity to the laser pulse, the reflection intensities of the data points corresponding to the reflective marker in the laser point cloud frame are relatively high. Therefore, the robot can determine the intensity threshold according to the average reflection intensity and the maximum reflection intensity, and then filter the data points in the laser point cloud frame according to the intensity threshold.

[0097] In one embodiment, S504 specifically includes: performing weighted summation on the average reflection intensity and the maximum reflection intensity, and using the obtained sum value as the intensity threshold. Among them, the weight values corresponding to the average reflection intensity and the maximum reflection intensity can be the same or different. For example, the weight value corresponding to the average reflection intensity is 0.3, and the weight value corresponding to the maximum reflection intensity is 0.7. Developers can configure the weight values corresponding to the average reflection intensity and the maximum reflection intensity when the robot leaves the factory, or users can also set them in the setting interface.

[0098] In one embodiment, S504 specifically includes: the robot determines the average value of the average reflection intensity and the maximum reflection intensity, and uses the average value as the intensity threshold.

[0099] S506. Filter the data points in the laser point cloud frame based on the intensity threshold to obtain a filtered laser point cloud frame.

[0100] The robot filters the data points in the laser point cloud frame based on the intensity threshold, deletes the data points with a reflection intensity lower than the intensity threshold, and retains the data points with a reflection intensity higher than the intensity threshold.

[0101] In the above embodiment, according to the reflection intensity corresponding to each data point in the laser point cloud frame, the average reflection intensity and the maximum reflection intensity of the data points are determined; the intensity threshold is determined according to the average reflection intensity and the maximum reflection intensity; based on the intensity threshold, the data points in the laser point cloud frame are filtered to obtain a filtered laser point cloud frame. Thus, the data points conforming to the reflection characteristics of the reflective marker can be filtered out from the laser point cloud frame, and the characteristics of the high reflection intensity of the laser pulse by the reflective marker are used to identify the dangerous area, improving the accuracy of the robot motion control.

[0102] In one embodiment, the marker is a graphic combination composed of at least two graphic elements; as Figure 6 shown, S206 specifically includes the following steps:

[0103] S602. Determine a first clustering threshold and a second clustering threshold according to the distances between the graphic elements in the graphic combination; the first clustering threshold is less than the second clustering threshold.

[0104] Among them, the graphic elements can be graphics of various shapes, including rectangles, circles, or triangles, etc. The robot can identify the graphic combination in the laser point cloud frame according to the shape characteristics of the graphic combination. In one embodiment, the first clustering threshold determined by the robot is less than the distance between the graphic elements, and the second clustering threshold is greater than the distance between the graphic elements, so that the data points corresponding to the graphic elements can be clustered into clusters by the first clustering threshold, and the data points corresponding to the graphic combination can be clustered into clusters by the second clustering threshold. In one embodiment, assuming that the distance between the graphic elements is R, the robot can determine the first clustering threshold as 0.5R and the second clustering threshold as 1.5R.

[0105] S604. Cluster the data points in the filtered laser point cloud frame according to the first clustering threshold, and select a first cluster that satisfies the first shape condition from the clusters obtained by the clustering; the first shape condition is determined based on the shape of the graphic element.

[0106] Among them, the first shape condition is a filtering condition determined based on the shape of the graphic element, and is used to filter out clusters that conform to the shape characteristics of the graphic element. In one embodiment, the first shape condition may be that the aspect ratio of the minimum circumscribed rectangle of the cluster is within a preset range. The preset range corresponding to the aspect ratio may be determined according to the aspect ratio of the graphic element. For example, if the graphic element is a rectangle with an aspect ratio equal to 2, the preset range may be a numerical interval from 1.8 to 2.2. In another embodiment, the first shape condition may be that the radius of the minimum circumscribed circle of the cluster is within a preset range. The preset range corresponding to the radius may be determined according to the radius of the graphic element. For example, if the graphic element is a circle with a radius equal to 3, the preset range may be a numerical interval from 2.5 to 3.5. The robot clusters the data points in the filtered laser point cloud frame according to the first clustering threshold, and clusters the data points with a distance less than the first clustering threshold from each other into a cluster. Then, the clusters obtained by clustering are filtered according to the first shape condition, and the clusters that are too large, too small, or have dissimilar shapes compared with the graphic element are discarded, and the obtained first clusters conform to the geometric characteristics of a single graphic element.

[0107] S606. Cluster the data points in the first clusters according to the second clustering threshold, and select the second clusters that meet the second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination.

[0108] Among them, the second shape condition is a filtering condition determined based on the shape of the graphic combination, and is used to filter out clusters that conform to the shape characteristics of the graphic combination. In one embodiment, the second shape condition may be that the aspect ratio of the minimum circumscribed rectangle of the cluster is within a preset range. The preset range of the aspect ratio may be determined according to the overall aspect ratio of the graphic combination. For example, if the aspect ratio of the graphic combination is equal to 1.5, the preset range may be a numerical interval from 1 to 2. In another embodiment, the second shape condition may be that the radius of the minimum circumscribed circle of the cluster is within a preset range. The preset range of the radius may be determined according to the overall radius of the graphic combination.

[0109] The robot clusters the data points in the first clusters according to the second clustering threshold, and clusters the data points with a distance less than the second clustering threshold from each other into a cluster. Then, the clusters obtained by clustering are filtered according to the second shape condition, and the clusters that are too large, too small, or have dissimilar shapes compared with the graphic combination are discarded, and the second clusters that conform to the overall geometric characteristics of the graphic combination are obtained.

[0110] S608. Filter the second clusters to obtain point cloud clusters.

[0111] In order to make the number of clusters included in the finally obtained point cloud clusters consistent with the number of graphic elements included in the graphic combination, the robot filters the second clusters to obtain point cloud clusters.

[0112] In one embodiment, S608 specifically includes: clustering each second - type cluster into multiple sub - clusters according to a first clustering threshold, and determining the number of sub - clusters in each second - type cluster; filtering the second - type clusters based on the number of sub - clusters in the second - type clusters to obtain point - cloud clusters.

[0113] The robot clusters each second - type cluster according to the first clustering threshold, clustering each second - type cluster into multiple sub - clusters. The shape and size of the sub - clusters are similar to the graphic elements. The robot determines the number of sub - clusters in each second - type cluster, compares the number of sub - clusters in the second - type cluster with the number of graphic elements in the graphic combination, discards the second - type clusters with too many or too few sub - clusters, and takes the remaining second - type clusters as point - cloud clusters. For example, if the image combination includes two graphic elements, then discard the second - type clusters with the number of sub - clusters less than 2 or greater than 3, so that the remaining point - cloud clusters contain 2 - 3 sub - clusters. The robot filters the second - type clusters according to the number of sub - clusters in the second - type cluster, so that the filtered point - cloud clusters not only conform to the geometric characteristics of the graphic combination, but also the number of sub - clusters in the second - type clusters contained is consistent with the number of graphic elements in the graphic combination, thereby enabling more accurate identification of markers and avoiding misdetection.

[0114] In the above - mentioned embodiment, the first clustering threshold and the second clustering threshold are determined according to the distances between the graphic elements in the graphic combination. Then, according to the first clustering threshold, the data points in the filtered laser - point - cloud frame are clustered, and the first - type clusters that meet the first shape condition are selected from the clusters obtained by clustering. According to the second clustering threshold, the data points in the first - type clusters are clustered, and the second - type clusters that meet the second shape condition are selected from the clusters obtained by clustering. The second - type clusters are filtered to obtain point - cloud clusters. Thus, point - cloud clusters that conform to the geometric characteristics of the image combination can be clustered, and markers can be identified according to the shape of the image combination, improving the accuracy of marker identification.

[0115] In one embodiment, S208 specifically includes: selecting at least two target sub - clusters from the sub - clusters of each point - cloud cluster; for each point - cloud cluster, determining the ratio between the number of data points of the target sub - clusters; if there is a target ratio that meets the ratio condition among the ratios, it is determined that the laser - point - cloud frame contains a marker, and motion control is performed on the robot.

[0116] Among them, the target sub - cluster is a sub - cluster that meets the selection condition among all sub - clusters. For example, the selection condition can be that the number of data points is greater than a preset value. Another example is that the selection condition can be that the ranking of the number of data points among all sub - clusters is within a preset ranking, and the preset ranking can be 2, for example.

[0117] The ratio condition is the condition for determining whether a point cloud cluster is a marker based on the ratio. In one embodiment, the ratio condition may be that the ratio is less than a preset value. For example, the preset value may be 0.5, 0.6, etc. In another embodiment, the ratio condition may be that the ratio is within a preset ratio range. The robot can determine the preset ratio range according to the sizes of the graphic elements in the graphic combination. For example, if the graphic combination includes two graphic elements with the same size and the ratio between the sizes of the two graphic elements is 1, then when the ratio between the numbers of data points of the target sub-clusters is much greater than 1 or much less than 1, it indicates that the sizes of the target sub-clusters vary greatly, which does not conform to the size characteristics of the graphic elements in the graphic combination and is not the point cloud cluster corresponding to the marker. Therefore, the robot can determine the preset ratio range as [0.5, 1.5]. If there is a target ratio that satisfies the ratio condition in the ratio, it indicates that the target sub-clusters in the point cloud cluster conform to the size characteristics of the graphic elements in the graphic combination, and thus it is determined that the point cloud cluster is a marker.

[0118] In the above embodiment, at least two target sub-clusters are selected from the sub-clusters of each point cloud cluster; for each point cloud cluster, the ratio between the numbers of data points of the target sub-clusters is determined; if there is a target ratio that satisfies the ratio condition in the ratio, it is determined that the laser point cloud frame contains a marker, and the motion of the robot is controlled. According to whether the sizes of the target sub-clusters in the point cloud cluster conform to the size characteristics of the graphic elements in the graphic combination, it is judged whether the point cloud cluster is a marker, which further improves the accuracy of identifying the marker.

[0119] In one embodiment, as Figure 7 shown, the method for controlling the motion of the robot includes the following steps:

[0120] S702, Obtain the laser point cloud frame obtained by scanning the operating environment.

[0121] Among them, each data point in the laser point cloud frame corresponds to a reflection feature, and the reflection feature may be a reflection intensity or a structural feature. When the reflection feature is the reflection intensity, S704 is executed.

[0122] S704, Determine the average reflection intensity and the maximum reflection intensity of the data points according to the reflection intensities corresponding to the data points in the laser point cloud frame.

[0123] S706, Determine the intensity threshold according to the average reflection intensity and the maximum reflection intensity, and filter the data points in the laser point cloud frame based on the intensity threshold to obtain the filtered laser point cloud frame.

[0124] S708, When the marker is a graphic combination composed of at least two graphic elements, determine the first clustering threshold and the second clustering threshold according to the distances between the graphic elements in the graphic combination; the first clustering threshold is less than the second clustering threshold.

[0125] S710, cluster the data points in the filtered laser point cloud frame according to a first clustering threshold, and select a first cluster that meets a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of a graphic element.

[0126] S712, cluster the data points in the first cluster according to a second clustering threshold, and select a second cluster that meets a second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of a graphic combination.

[0127] S714, cluster each second cluster into multiple sub-clusters according to the first clustering threshold, and determine the number of sub-clusters in each second cluster.

[0128] S716, filter the second clusters based on the number of sub-clusters in the second clusters to obtain point cloud clusters, and select at least two target sub-clusters from the sub-clusters of each point cloud cluster.

[0129] S718, for each point cloud cluster, determine the ratio between the number of data points of the target sub-clusters.

[0130] S720, if there is a target ratio that meets the ratio condition among the ratios, determine that the laser point cloud frame contains a marker, and determine the distance between the robot and the marker; perform motion control on the robot according to the distance.

[0131] The specific content of the above S702 to S720 can refer to the specific implementation process described above.

[0132] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0133] Based on the same inventive concept, the embodiments of the present application also provide a device for robot motion control corresponding to the method for implementing the above-mentioned robot motion control. The implementation solutions provided by this device to solve problems are similar to the implementation solutions described in the above method. Therefore, the specific limitations in one or more embodiments of the device for robot motion control provided below can refer to the limitations on the method for robot motion control in the above text, and will not be repeated here.

[0134] In one embodiment, as Figure 8 shown, a device for robot motion control is provided, including: an acquisition module 802, a filtering module 804, a clustering module 806, and a control module 808, where:

[0135] The acquisition module 802 is configured to acquire a laser point cloud frame obtained by scanning the operating environment;

[0136] The filtering module 804 is configured to filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame, so as to obtain a filtered laser point cloud frame;

[0137] The clustering module 806 is configured to cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters;

[0138] The control module 808 is configured to perform motion control on the robot if it is determined based on the point cloud clusters that the laser point cloud frame contains a marker.

[0139] In the above embodiment, a laser point cloud frame obtained by scanning the operating environment is acquired; since the reflection characteristics of the marker to the laser are different from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered according to the reflection characteristics corresponding to each data point in the laser point cloud frame, so that the data points with reflection characteristics different from those of other objects can be filtered out to obtain a filtered laser point cloud frame. The data points in the filtered laser point cloud frame are clustered to obtain point cloud clusters, so that it can be determined whether the laser point cloud frame contains a marker according to the shape characteristics of the marker. If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, motion control is performed on the robot. By using the different characteristics of the marker's reflection to the laser and the shape characteristics to identify the marker in the laser point cloud frame, the recognition accuracy is higher, effectively avoiding the robot from entering dangerous areas and prohibited areas. And the sensing distance of the laser is relatively far, and the marker can be recognized at a relatively long distance, enabling the robot to have sufficient time to brake, effectively avoiding the fast-moving robot from falling or toppling due to emergency braking, and ensuring the safe operation of the robot.

[0140] In one embodiment, the control module 808 is further configured to:

[0141] If it is determined based on the point cloud clusters that the laser point cloud frame contains a marker, determine the distance between the robot and the corresponding marker;

[0142] Perform motion control on the robot according to the distance.

[0143] In one embodiment, the laser point cloud frame includes at least two frames; the control module 808 is further configured to:

[0144] If it is determined that the laser point cloud frame contains a marker based on the point cloud cluster, the position coordinates corresponding to each marker are determined respectively;

[0145] If the position coordinates corresponding to the markers in at least two laser point cloud frames are the same, motion control is performed on the robot.

[0146] In one embodiment, the filtering module 804 is further configured to:

[0147] Determine the average reflection intensity and the maximum reflection intensity of the data points according to the reflection intensity corresponding to each data point in the laser point cloud frame;

[0148] Determine the intensity threshold according to the average reflection intensity and the maximum reflection intensity;

[0149] Based on the intensity threshold, filter the data points in the laser point cloud frame to obtain a filtered laser point cloud frame.

[0150] In one embodiment, the marker is a graphic combination composed of at least two graphic elements; the clustering module 806 is further configured to:

[0151] Determine a first clustering threshold and a second clustering threshold according to the distances between the graphic elements in the graphic combination; the first clustering threshold is less than the second clustering threshold;

[0152] Cluster the data points in the filtered laser point cloud frame according to the first clustering threshold, and select a first cluster that satisfies the first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic elements;

[0153] Cluster the data points in the first cluster according to the second clustering threshold, and select a second cluster that satisfies the second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination;

[0154] Filter the second cluster to obtain a point cloud cluster.

[0155] In one embodiment, the clustering module 806 is further configured to:

[0156] Cluster each second cluster into multiple sub-clusters according to the first clustering threshold, and determine the number of sub-clusters in each second cluster;

[0157] Filter the second cluster based on the number of sub-clusters to obtain a point cloud cluster.

[0158] In one embodiment, the control module 808 is further configured to:

[0159] Select at least two target sub-clusters from the sub-clusters of each point cloud cluster;

[0160] For each point cloud cluster, determine the ratio of the number of data points of the target sub-cluster;

[0161] If there is a target ratio in the ratios that meets the ratio condition, it is determined that the laser point cloud frame contains a marker, and motion control is performed on the robot.

[0162] Each module in the above device for robot motion control can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each of the above modules.

[0163] In one embodiment, a robot is provided, and its internal structure diagram can be as Figure 9 shown. The robot includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the robot is used to provide computing and control capabilities. The memory of the robot includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the robot is used for the processor to exchange information with external devices. The communication interface of the robot is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a method for robot motion control. The display unit of the robot is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the robot can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the robot housing, or an external keyboard, touchpad, or mouse, etc.

[0164] Those skilled in the art can understand that Figure 9 the structure shown in

[0165] In one embodiment, a robot is provided, which includes a memory and a processor. A computer program is stored in the memory. A lidar is mounted on the robot, and a marker is set in the operating environment of the robot. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0166] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0167] In one embodiment, a computer program product is provided, which includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0168] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. The collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0169] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0170] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0171] The above-described embodiments merely represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for robot motion control, characterized in that, A lidar is mounted on the robot, and markers are set in the operating environment of the robot. The method includes: Obtaining a lidar point cloud frame obtained by scanning the operating environment; Filtering the data points in the lidar point cloud frame according to the reflection characteristics corresponding to each data point in the lidar point cloud frame to obtain a filtered lidar point cloud frame; Clustering the data points in the filtered lidar point cloud frame to obtain point cloud clusters; If it is determined based on the point cloud clusters that the lidar point cloud frame contains a marker, then perform motion control on the robot, including: selecting at least two target sub-clusters in the sub-clusters of each point cloud cluster; for each point cloud cluster, determining the ratio between the number of data points of the target sub-clusters; if there is a target ratio that satisfies the ratio condition among the ratios, then determine that the lidar point cloud frame contains a marker and perform motion control on the robot.

2. The method according to claim 1, characterized in that, The step of performing motion control on the robot if it is determined based on the point cloud clusters that the lidar point cloud frame contains a marker includes: If it is determined based on the point cloud clusters that the lidar point cloud frame contains a marker, then determining the distance between the robot and the marker; Performing motion control on the robot according to the distance.

3. The method according to claim 1, characterized in that, The lidar point cloud frame includes at least two frames. The step of performing motion control on the robot if it is determined based on the point cloud clusters that the lidar point cloud frame contains a marker includes: If it is determined based on the point cloud clusters that the lidar point cloud frame contains a marker, then respectively determining the position coordinates corresponding to each marker; If the position coordinates corresponding to the markers in at least two lidar point cloud frames are the same, then perform motion control on the robot.

4. The method according to claim 1, characterized in that, The marker includes a reflective marker, and the reflection characteristic includes reflection intensity. The step of filtering the data points in the lidar point cloud frame according to the reflection characteristics corresponding to each data point in the lidar point cloud frame to obtain a filtered lidar point cloud frame includes: Determining the average reflection intensity and the maximum reflection intensity of the data points according to the reflection intensity corresponding to each data point in the lidar point cloud frame; Determining an intensity threshold according to the average reflection intensity and the maximum reflection intensity; Filtering the data points in the lidar point cloud frame based on the intensity threshold to obtain a filtered lidar point cloud frame.

5. The method according to claim 1, characterized in that, The marker is a graphic combination composed of at least two graphic elements. The step of clustering the data points in the filtered lidar point cloud frame to obtain point cloud clusters includes: Determining a first clustering threshold and a second clustering threshold according to the distance between the graphic elements in the graphic combination; the first clustering threshold is less than the second clustering threshold; Clustering the data points in the filtered lidar point cloud frame according to the first clustering threshold, and selecting a first cluster that satisfies the first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element; Clustering the data points in the first cluster according to the second clustering threshold, and selecting a second cluster that satisfies the second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination; Filter the second type of clusters to obtain the point cloud clusters.

6. The method according to claim 5, characterized in that, The filtering of the second type of clusters to obtain point cloud clusters includes: According to the first clustering threshold, cluster each of the second type of clusters into a plurality of sub-clusters, and determine the number of sub-clusters in each of the second type of clusters; Filter the second type of clusters based on the number of sub-clusters in the second type of clusters to obtain the point cloud clusters.

7. The method according to any one of claims 1 to 6, characterized in that, The marker is pasted on the edge of the dangerous area or the prohibited area.

8. An apparatus for robot motion control, characterized in that, A lidar is mounted on the robot, and markers are arranged in the operating environment of the robot. The device includes: An acquisition module, configured to acquire a lidar point cloud frame obtained by scanning the operating environment; A filtering module, configured to filter data points in the lidar point cloud frame according to the reflection characteristics corresponding to each data point in the lidar point cloud frame to obtain a filtered lidar point cloud frame; A clustering module, configured to cluster data points in the filtered lidar point cloud frame to obtain point cloud clusters; A control module, configured to select at least two target sub-clusters in the sub-clusters of each point cloud cluster; for each point cloud cluster, determine the ratio between the number of data points of the target sub-clusters; if there is a target ratio that satisfies the ratio condition among the ratios, determine that the lidar point cloud frame contains a marker, and perform motion control on the robot.

9. A robot, comprising a memory and a processor, the memory storing a computer program, characterized in that, A lidar is mounted on the robot, and markers are arranged in the operating environment of the robot. When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images