Robot motion control method, apparatus, robot, and storage medium

Abstract

This application discloses a method, apparatus, robot, and storage medium for controlling the motion of a robot, wherein the robot is equipped with a LiDAR and a marker object is installed in the operating environment of the robot, and the method includes: acquiring a laser point cloud frame obtained by scanning the operating environment (S202); extracting data points in the laser point cloud frame by filtering based on the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame (S204); clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters (S206); and, if it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, performing motion control of the robot (S208).

Description

Technical Field

[0001] (Cross - reference to related applications) This application claims the priority of a Chinese patent application with an application number of 2023106389099 and an application title of "Robot Motion Control Method, Device, Robot, and Storage Medium", which was filed with the China National Intellectual Property Administration on May 31, 2023, and all of its contents are incorporated into this disclosure by reference.

[0002] This application relates to the technical field of mobile robots, and particularly to a robot motion control method, device, robot, storage medium, and computer program product.

Background Art

[0003] With the development of mobile robot technology, the application environment of mobile robots has become increasingly complex, and robots may enter dangerous areas or prohibited areas during the movement process. For example, if a robot enters a dangerous area, there may be a risk of falling. In the prior art, the motion control of a robot can be performed by infrared detection, but due to the short sensing distance of infrared detection, when the moving speed of the robot is fast, the robot's fall cannot be prevented, or the robot may fall due to an emergency brake. In the method of performing robot motion control based on a magnetic tape, the sensing distance of the magnetic tape is short and it is easy to demagnetize. The fall prevention method based on a depth camera has low accuracy and is prone to false detection. Therefore, how to ensure the safe operation of the robot has become an issue to be solved.

Summary of the Invention

[0004] According to each embodiment of this application, a robot motion control method, device, robot, computer - readable storage medium, and computer program product are provided.

[0005] A robot motion control method, wherein a lidar is mounted on the robot, and a marker is installed in the operating environment of the robot, and the method includes: This involves acquiring laser point cloud frames obtained by scanning the operating environment, Based on the reflection characteristics corresponding to each data point in the laser point cloud frame, the data points in the laser point cloud frame are extracted by filtering to obtain the filtered laser point cloud frame. The process involves clustering the data points within the filtered laser point cloud frame to obtain point cloud clusters, The method includes, if it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, then performing motion control of the robot.

[0006] A robot motion control device, wherein the device is An acquisition module configured to acquire a laser point cloud frame obtained by scanning the operating environment, A filtering module configured to extract data points from the laser point cloud frame by filtering them based on the reflection characteristics corresponding to each data point in the laser point cloud frame, thereby obtaining a filtered laser point cloud frame, A clustering module configured to obtain point cloud clusters by clustering data points within the filtered laser point cloud frame, The system includes a control module configured to perform motion control of the robot when it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster.

[0007] A robot comprising memory and a processor, wherein a computer program is stored in the memory, the robot is equipped with a lidar, markers are installed in the robot's operating environment, and when the processor executes the computer program, steps of a method for controlling the robot's motion are realized.

[0008] A computer-readable storage medium is provided, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the robot motion control method are realized.

[0009] A computer program product, the computer program product includes a computer program, and when this computer program is executed by a processor, the steps of the robot motion control method are realized.

[0010] Details of one or more embodiments of the present application are described in the drawings and description below. Other features, purposes and advantages of the present application will be apparent from the specification, drawings and claims.

[0011] To more clearly describe the embodiments of this application or the technical means in the prior art, the drawings that are necessary for use in describing the embodiments or the prior art are briefly introduced below. Clearly, the drawings described below represent only a portion of the embodiments of this application, and those skilled in the art can obtain drawings of other embodiments without any creative work based on these drawings. [Brief explanation of the drawing]

[0012] [Figure 1] This is an application environment diagram of a robot motion control method in one embodiment. [Figure 2] This is a flowchart of a robot motion control method in one embodiment. [Figure 3] This is a schematic diagram of a reflective marker in one embodiment. [Figure 4] This is a flowchart of a distance-based robot motion control method in one embodiment. [Figure 5a] This is a schematic diagram of an area determined based on the distance to a marker object in one embodiment. [Figure 5b] This is a flowchart of the steps for filtering the laser point cloud frame in one embodiment. [Figure 6] This is a flowchart illustrating a method for obtaining point cloud clusters by clustering in one embodiment. [Figure 7] This is a flowchart of the robot motion control method in another embodiment. [Figure 8] This is a block diagram of the configuration of a robot motion control device in one embodiment. [Figure 9] This is an internal configuration diagram of a robot in one embodiment. [Modes for carrying out the invention]

[0013] To facilitate understanding of this application, it will be described more comprehensively with reference to the relevant drawings. The drawings show preferred embodiments of the application. However, the application may be realized in many different forms and is not limited to the embodiments described herein. Rather, the purpose of providing these embodiments is to provide a more complete and thorough understanding of the disclosure of the application.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as that commonly understood by those skilled in the art of the invention. Terms used in the specification of the invention are for illustrative purposes only and are not intended to limit the application. The terms "and / or" used herein include any and all combinations of one or more of the related enumerated items.

[0015] The robot motion control method provided by the embodiments of this application can be applied to an application environment as shown in FIG. 1. The robot 102 acquires a laser point cloud frame obtained by scanning the operating environment, and based on the reflection characteristics corresponding to each data point in the laser point cloud frame, extracts the data points in the laser point cloud frame by filtering to obtain a filtered laser point cloud frame, clusters the data points in the filtered laser point cloud frame to obtain a point cloud cluster, and when it is determined that the laser point cloud frame contains a marker based on the point cloud cluster, performs motion control on the robot 102. The robot 102 may be various delivery robots, working robots, service robots, sorting robots, cleaning robots, etc., but is not limited thereto. The user installs a marker at the edge of a dangerous area or a prohibited area in the operating environment of the robot 102, and this marker has at least one of recognizable structural characteristics, material characteristics, or pattern characteristics. For example, the user can attach markers, such as reflective markers or structured markers, to both sides of the stairs. The reflective marker has a specific shape and a high reflectivity to laser pulses. On the other hand, the structured marker has a specific structure and exhibits specific structural characteristics in the laser point cloud frame formed after reflecting the laser pulses. A lidar is installed on the robot 102, and the operating environment is scanned through the lidar to obtain a laser point cloud frame. Thereby, the reflection intensity or the structural characteristics can be extracted as the reflection characteristics, it can be detected whether there is a marker in the operating environment, and further the motion control of the robot can be performed.

[0016] In one embodiment, as shown in FIG. 2, a robot motion control method is provided. Taking the case of applying this method to the robot in FIG. 1 as an example, the following steps are included.

[0017] S202: Acquire a laser point cloud frame obtained by scanning the operating environment.

[0018] The operating environment is the environment in which the robot operates, which may be an indoor environment or an outdoor environment. For example, the operating environment may be a road environment for delivering goods. Also, for example, the operating environment may be a hotel environment for providing services. Also, for example, the operating environment may be a working environment inside a factory.

[0019] In this operating environment, there are markers installed, and these markers include reflective markers and structured markers. The reflective markers have a high reflection intensity for laser pulses. Based on the reflection intensity corresponding to each data point, data points with a high reflection intensity are filtered, and reflective markers can be identified from the filtered data points. On the other hand, the structured markers have a specific structure and exhibit specific structural characteristics in the laser point cloud frame formed after reflecting laser pulses. Therefore, based on the structural characteristics corresponding to each data point, filtering is performed, and structured markers can be identified from the filtered data points.

[0020] Since this robot is equipped with a lidar, scanning can be performed through the lidar, and a laser point cloud frame can be obtained. This laser point cloud frame is composed of a plurality of data points, and each data point has a corresponding laser intensity. The lidar is an optical sensor that emits laser pulses into the operating environment and receives the laser pulses reflected back from each object in the operating environment, and can generate a laser point cloud frame based on the reflected laser pulses.

[0021] S204: Based on the reflection characteristics corresponding to each data point in the laser point cloud frame, the data points in the laser point cloud frame are extracted by filtering to obtain a filtered laser point cloud frame.

[0022] Data points are points within a laser point cloud frame and are used to describe points in the three-dimensional space scanned by the lidar. Each data point includes multiple attributes such as position coordinates, reflectivity, and scanning angle. Reflectivity may be reflectivity or structural reflectivity. This reflectivity represents the pulse echo intensity of the lidar; the higher the reflectivity of a point in three-dimensional space to the laser pulse, the higher the reflectivity of the data point corresponding to that point in three-dimensional space. This structural reflectivity is structured information exhibited by the pulse echo of the lidar and can reflect the specific structure of each object in the operating environment.

[0023] In one embodiment, when the reflection characteristic is reflection intensity, S204 specifically includes determining an intensity threshold based on the reflection intensity corresponding to each data point in the laser point cloud frame, and filtering out data points from the data points of the laser point cloud frame whose reflection intensity is greater than the intensity threshold to obtain a filtered laser point cloud frame.

[0024] S206: The data points within the filtered laser point cloud frame are clustered to obtain point cloud clusters.

[0025] A point cloud cluster is a cluster composed of data points of the same type. For example, a point cloud cluster may be a cluster composed of data points corresponding to the same object scanned by a lidar. Clustering is an unsupervised learning technique that measures the similarity of the objects to be clustered and groups similar objects into a single cluster.

[0026] In one embodiment, the robot can obtain point cloud clusters by clustering data points within the filtered laser point cloud frame using a K-Means clustering algorithm, a BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) algorithm, or a Gaussian mixed clustering algorithm.

[0027] S208: If it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, the robot's motion control is performed.

[0028] In one embodiment, if the reflection characteristic is reflection intensity, the robot can determine whether or not a reflective marker is included in the laser point cloud frame based on the point cloud cluster. If it is determined that a reflective marker is included in the laser point cloud frame based on the point cloud cluster, the robot performs motion control.

[0029] Reflective markers are markers made of reflective material, affixed to the edges of hazardous or prohibited areas, and may consist of graphic elements of various shapes. For example, a reflective marker may consist of one or more rectangles, circles, ellipses, or triangles. The multiple graphic elements that make up a reflective marker may be similar or different. For example, a reflective marker may consist of two or more rectangles, or one rectangle and one circle, or one circle and a triangle. In one embodiment, as shown in Figure 3, a reflective marker consists of two rectangles arranged in parallel. The size of the rectangles and the spacing between them can be adjusted according to actual needs; for example, the size of the rectangles can be 50 mm x 100 mm, and the spacing between the two rectangles can be 50 mm.

[0030] Motion control involves controlling the robot's motion modes, including controlling the robot's movement speed, direction, or path. For example, this could involve controlling the robot to stop moving, to avoid a dangerous area, or to reduce its movement speed. If the laser point cloud frame is determined to contain reflective markers based on point cloud clusters, it indicates that the robot is moving into a dangerous or prohibited area, and motion control of the robot is necessary. For example, if a user places reflective markers on both sides of stairs or an elevator, and the laser point cloud frame collected by the robot via LiDAR contains these reflective markers, it indicates that the robot is moving near the stairs or elevator and there is a risk of falling, requiring motion control of the robot. Alternatively, if a user places reflective markers on the edge of a prohibited area such as a body of water, and the laser point cloud frame collected by the robot via LiDAR contains these reflective markers, it indicates that the robot is moving into a prohibited area, requiring motion control of the robot. Furthermore, for example, the user can attach reflective markers to the edges of fragile obstacles such as glass. If the laser point cloud frame collected by the robot via LiDAR includes a reflective marker, it indicates that the robot may collide with the fragile obstacle, and the robot's motion is controlled accordingly.

[0031] In another embodiment, if the reflection characteristics are structural characteristics, the robot can determine whether the laser point cloud frame contains structured markers based on the point cloud clusters, and if it is determined that the laser point cloud frame contains structured markers based on the point cloud clusters, the robot performs motion control.

[0032] In the above embodiment, a laser point cloud frame is obtained by scanning the operating environment. Since the reflection characteristics of the marker object to the laser differ from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered based on the reflection characteristics corresponding to each data point in the laser point cloud frame. By filtering, data points with different reflection characteristics from other objects are extracted, and a filtered laser point cloud frame is obtained. By clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters, it is possible to determine whether or not a marker object is included in the laser point cloud frame based on the shape characteristics of the marker object. If it is determined that a marker object is included in the laser point cloud frame based on the point cloud clusters, the robot's motion control is performed. By identifying the marker object in the laser point cloud frame using the different reflection characteristics and shape characteristics of the marker object to the laser, the identification accuracy is higher, and the robot effectively avoids entering dangerous and prohibited areas. Furthermore, the laser has a long detection range, allowing the marker object to be identified from a distance, giving the robot sufficient time to apply the brakes, effectively avoiding falls or tipping over due to emergency braking of the high-speed moving robot, and ensuring the safe operation of the robot.

[0033] In one embodiment, as shown in Figure 4, S208 specifically includes the following steps.

[0034] S402: If it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, the distance between the robot and the marker object is determined.

[0035] Distance is the distance between the robot and the marker object in three-dimensional space. The robot can obtain the time interval between the lidar emitting a laser pulse and receiving the reflected echo, and determine the distance from the robot to the marker object based on this time interval.

[0036] S404: Controls the robot's motion based on distance.

[0037] Since the markers are placed at the edge of the hazardous area, the urgency of a hazard is weak when the robot is far from the markers, and even stronger when the robot is close to the markers. Therefore, the robot's motion is controlled based on the distance between the robot and the markers.

[0038] In one embodiment, S404 specifically includes determining the area where the robot is currently located based on distance, and performing motion control of the robot based on the area where the robot is currently located. Specifically, as shown in Figure 5a, the black rectangular frame is a reflective marker. Area A is a braking area, area B is an avoidance / stopping area, and area C is a deceleration area. When the robot moves into a deceleration area, the robot is controlled to reduce its movement speed, for example, to 0.6 meters / second. When the robot is in an avoidance / stopping area, the robot is controlled to stop moving when it encounters an obstacle. When the robot is in a braking area, the robot is controlled to stop moving. The deceleration area is furthest from the reflective marker, the avoidance / stopping area is located midway between the braking area and the deceleration area, and the braking area is closest to the reflective marker. The distance between each zone and the reflective markers is adjustable. For example, a braking zone can be an area where the distance to the reflective markers is less than 1.2 meters, an avoidance / stopping zone can be an area where the distance to the reflective markers is 1.2 meters or more but less than 1.5 meters, and a deceleration zone can be an area where the distance to the reflective markers is 1.5 meters or more but less than 2 meters.

[0039] In one embodiment, S404 specifically includes controlling the robot's movement path based on distance. For example, if the distance is less than a preset value, the robot is controlled to stop moving and return. Or, if the distance is less than a preset value, the robot is controlled to adjust its direction of movement.

[0040] In the above embodiment, if it is determined that a target object is included in the laser point cloud frame based on the point cloud cluster, the distance between the robot and the target object is determined, and the robot's motion is controlled based on this distance. This allows the robot's motion mode to be adjusted according to the distance, increasing the flexibility of the robot's motion.

[0041] In one embodiment, S208 specifically includes determining the position coordinates corresponding to each marker when it is determined that a marker is included in the laser point cloud frame based on the point cloud cluster, and performing motion control of the robot when the position coordinates corresponding to the markers in at least two laser point cloud frames are the same.

[0042] The position coordinates are the coordinates of the marker object in the world coordinate system. Each data point in the laser point cloud frame contains the position coordinates corresponding to this data point, and the robot can determine the position coordinates of the marker object based on the position coordinates corresponding to each data point. To avoid false detection by the robot, if the robot identifies a marker object in a laser point cloud frame, it is instructed to continue collecting laser point cloud frames via LiDAR. If the position coordinates corresponding to marker objects in multiple laser point cloud frames collected sequentially by LiDAR are similar, it indicates that all of these marker objects correspond to the same marker object at the same location, and the robot determines that it has scanned for marker objects, and then controls the robot's motion.

[0043] In the above embodiment, if it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, the position coordinates of each marker object are determined, and if the position coordinates of the marker objects in at least two laser point cloud frames are the same, the robot's motion control is performed. This effectively avoids false detections by the robot and improves the accuracy of the robot's motion control.

[0044] In one embodiment, the marker includes a reflective marker, and the reflective properties include the reflectivity. As shown in Figure 5b, S204 specifically includes the following steps.

[0045] S502: Based on the reflectance corresponding to each data point in the laser point cloud frame, the average reflectance and maximum reflectance of the data points are determined.

[0046] The average reflectance is the average value of the reflectances corresponding to each data point in the laser point cloud frame. The maximum reflectance is the maximum value of the reflectances corresponding to each data point in the laser point cloud frame. When the robot receives the laser point cloud frame, it statistically analyzes the reflectances corresponding to each data point in the laser point cloud frame, calculates the average reflectance, and searches for the maximum reflectance among the reflectances corresponding to each data point. The robot can search for the maximum reflectance using various search methods such as sequential search, binary search, and binary tree search.

[0047] S504: Determine the intensity threshold based on the average reflectance and the maximum reflectance.

[0048] If the marker is a reflective marker, the reflective marker has a high reflectivity to laser pulses, so the reflection intensity of each data point corresponding to the reflective marker in the laser point cloud frame is high. Therefore, the robot can determine an intensity threshold based on the average reflection intensity and the maximum reflection intensity, and then filter the data points in the laser point cloud frame based on the intensity threshold.

[0049] In one embodiment, S504 specifically includes performing a weighted sum on the average reflectance and the maximum reflectance, and using the resulting sum as the intensity threshold. The weighted values ​​corresponding to the average reflectance and the maximum reflectance may be the same or different. For example, the weighted value corresponding to the average reflectance can be 0.3, and the weighted value corresponding to the maximum reflectance can be 0.7. The developer can set the weighted values ​​corresponding to the average reflectance and the maximum reflectance when shipping the robot, or the user can set them using the configuration interface.

[0050] In one embodiment, S504 specifically includes the robot determining the average of the average reflectance and the maximum reflectance, and using the average value as the intensity threshold.

[0051] S506: Based on the intensity threshold, data points within the laser point cloud frame are extracted by filtering to obtain the filtered laser point cloud frame.

[0052] The robot filters data points within the laser point cloud frame based on an intensity threshold, removing data points with a reflection intensity lower than the intensity threshold and retaining data points with a reflection intensity higher than the intensity threshold.

[0053] In the above embodiment, the average and maximum reflectance of data points are determined based on the reflectance corresponding to each data point in the laser point cloud frame. An intensity threshold is determined based on the average and maximum reflectances. Based on the intensity threshold, data points in the laser point cloud frame are filtered out to obtain a filtered laser point cloud frame. This allows filtering of data points corresponding to the reflectance characteristics of the reflective marker from the laser point cloud frame, enabling the identification of hazardous areas by utilizing the high reflectance characteristic of the reflective marker to laser pulses, thereby improving the accuracy of robot motion control.

[0054] In one embodiment, the sign is a graphic combination consisting of at least two graphic elements. As shown in Figure 6, S206 specifically includes the following steps.

[0055] S602: Based on the distance between each graphic element in the graphic combination, a first clustering threshold and a second clustering threshold are determined, with the first clustering threshold being smaller than the second clustering threshold.

[0056] Graphic elements may be graphics of various shapes, including rectangles, circles, or triangles. The robot can identify graphic combinations in the laser point cloud frame based on the shape characteristics of the graphic combinations. In one embodiment, the first clustering threshold determined by the robot is smaller than the distance between graphic elements, and the second clustering threshold is larger than the distance between graphic elements. This allows the robot to cluster data points corresponding to graphic elements using the first clustering threshold, and cluster data points corresponding to graphic combinations using the second clustering threshold. In one embodiment, assuming the distance between graphic elements is R, the robot can determine the first clustering threshold as 0.5R and the second clustering threshold as 1.5R.

[0057] S604: Based on a first clustering threshold, the data points in the filtered laser point cloud frame are clustered, and a first cluster that satisfies a first shape condition is selected from the clusters obtained by clustering, the first shape condition is determined based on the shape of the graphic element.

[0058] The first shape condition is a selection criterion determined based on the shape of the graphic element and is used to select clusters that conform to the shape characteristics of the graphic element. In one embodiment, the first shape condition may be that the aspect ratio of the minimum circumscribing rectangle of the cluster is within a preset range. The preset range corresponding to the aspect ratio can be determined based on the aspect ratio of the graphic element. For example, if the graphic element is a rectangle with an aspect ratio of 2, the preset range can be a numerical interval of 1.8 to 2.2. In another embodiment, the first shape condition may be that the radius of the minimum circumscribing circle of the cluster is within a preset range. The preset range corresponding to the radius can be determined based on the radius of the graphic element. For example, if the graphic element is a circle with a radius of 3, the preset range can be a numerical interval of 2.5 to 3.5. The robot clusters the data points in the filtered laser point cloud frame based on the first clustering threshold, and groups data points whose distance from each other is less than the first clustering threshold into a single cluster. Next, clusters obtained by clustering are selected based on the first shape condition. Clusters that are too large, too small, or do not resemble the graphic element are discarded, and the resulting first clusters correspond to the geometric properties of a single graphic element.

[0059] S606: Based on the second clustering threshold, the data points in the first cluster are clustered, and a second cluster that satisfies the second shape condition is selected from the clusters obtained by clustering, the second shape condition is determined based on the shape of the graphic combination.

[0060] The second shape condition is a selection criterion determined based on the shape of the graphic combination and is used to select clusters that conform to the shape characteristics of the graphic combination. In one embodiment, the second shape condition may be that the aspect ratio of the minimum circumscribing rectangle of the cluster is within a preset range. The preset range of the aspect ratio can be determined based on the aspect ratio of the entire graphic combination. For example, if the aspect ratio of the graphic combination is 1.5, the preset range can be a numerical interval from 1 to 2. In another embodiment, the second shape condition may be that the radius of the minimum circumscribing circle of the cluster is within a preset range. The preset range of the radius can be determined based on the radius of the entire graphic combination.

[0061] The robot clusters data points in the first cluster based on a second clustering threshold, grouping data points whose distance from each other is less than the second clustering threshold into a single cluster. Next, it selects the clusters obtained by clustering based on a second shape condition, discarding clusters that are too large, too small, or do not have similar shapes compared to the graphic combination, and obtains a second cluster that conforms to the geometric characteristics of the entire graphic combination.

[0062] S608: Filter the second cluster to obtain point cloud clusters.

[0063] To match the number of clusters in the final point cloud cluster with the number of graphic elements in the graphic combination, the robot filters out the second cluster to obtain the point cloud cluster.

[0064] In one embodiment, S608 specifically includes clustering each second cluster into a plurality of subclusters based on a first clustering threshold, determining the number of subclusters in each second cluster, and filtering the second clusters based on the number of subclusters in the second cluster to obtain point cloud clusters.

[0065] The robot clusters each second cluster based on a first clustering threshold, and then clusters each second cluster into multiple subclusters, the shape and size of which are similar to the graphic elements. The robot determines the number of subclusters in each second cluster, compares the number of subclusters in the second cluster with the number of graphic elements in the graphic combination, discards second clusters with too many or too few subclusters, and retains the second clusters as point cloud clusters. For example, if a graphic combination contains two graphic elements, the robot discards second clusters with fewer than two or more than three subclusters, ensuring that the retained point cloud clusters contain two to three subclusters. The robot filters the second clusters based on the number of subclusters in each cluster, ensuring that the filtered point cloud clusters not only conform to the geometric characteristics of the graphic combination but also match the number of subclusters in the included second clusters with the number of graphic elements in the graphic combination, thereby enabling more accurate identification of marked objects and avoiding false detections.

[0066] In the above embodiment, a first clustering threshold and a second clustering threshold are determined based on the distance between each graphic element in the graphic combination. Then, based on the first clustering threshold, the data points in the filtered laser point cloud frame are clustered, and a first cluster that satisfies the first shape condition is selected from the clusters obtained by clustering. Based on the second clustering threshold, the data points in the first cluster are clustered, and a second cluster that satisfies the second shape condition is selected from the clusters obtained by clustering. The second cluster is filtered to obtain a point cloud cluster. This makes it possible to obtain a point cloud cluster that conforms to the geometric characteristics of the graphic combination by clustering, thereby identifying a marked object based on the shape of the graphic combination and improving the accuracy of mark identification.

[0067] In one embodiment, S208 specifically includes selecting at least two target subclusters from among the subclusters of each point cloud cluster, determining the ratio between the data point quantities of the target subclusters for each point cloud cluster, and determining that a target ratio satisfying the ratio condition exists within the ratio, and performing motion control of the robot.

[0068] The target subcluster is a subcluster from all subclusters that satisfies the selection criteria. For example, the selection criterion could be that the number of data points is greater than a preset value. Alternatively, the selection criterion could be that the rank of the number of data points in a subcluster falls within a preset rank, which could be, for example, 2.

[0069] The ratio condition is a condition for determining whether a point cloud cluster is a marker based on its ratio. In one embodiment, the ratio condition can be that the ratio is smaller than a preset value, for example, the preset values ​​can be 0.5, 0.6, etc. In another embodiment, the ratio condition can be that the ratio is within a preset ratio interval. The robot can determine the preset ratio interval based on the size of each graphic element in the graphic combination. For example, if the graphic combination contains two graphic elements, and the two graphic elements are the same size, and the ratio between the sizes of the graphic elements is 1, then if the ratio between the number of data points in the target subclusters is much larger or much smaller than 1, it indicates that the sizes of each target subcluster are very different and do not correspond to the size characteristics of each graphic element in the graphic combination, and therefore the point cloud cluster is not a marker. Thus, the robot can determine the preset ratio interval as [0.5, 1.5]. If there is a target ratio within the ratios that satisfies the ratio condition, it indicates that each target subcluster in the point cloud cluster corresponds to the size characteristics of each graphic element in the graphic combination, and thus the point cloud cluster is determined to be a marker.

[0070] In the above embodiment, at least two target subclusters are selected from the subclusters of each point cloud cluster, and for each point cloud cluster, the ratio between the number of data points of the target subclusters is determined. If a target ratio that satisfies the ratio condition exists within the ratio, it is determined that a marker object is included in the laser point cloud frame, and the robot's motion control is performed. Whether or not a point cloud cluster is a marker object is determined based on whether or not the size of each target subcluster in the point cloud cluster corresponds to the size characteristics of each graphic element in the graphic combination, thereby further improving the accuracy of identifying marker objects.

[0071] In one embodiment, as shown in Figure 7, the robot motion control method includes the following steps.

[0072] S702: Acquire laser point cloud frames obtained by scanning the operating environment.

[0073] Each data point within the laser point cloud frame has a corresponding reflection characteristic, which may be reflection intensity or structural characteristics. If this reflection characteristic is reflection intensity, S704 is executed.

[0074] S704: Based on the reflectance corresponding to each data point in the laser point cloud frame, the average reflectance and maximum reflectance of the data points are determined.

[0075] S706: An intensity threshold is determined based on the average reflection intensity and the maximum reflection intensity, and based on the intensity threshold, data points in the laser point cloud frame are extracted by filtering to obtain the filtered laser point cloud frame.

[0076] S708: If the marker is a graphic combination consisting of at least two graphic elements, a first clustering threshold and a second clustering threshold are determined based on the distance between each graphic element in the graphic combination, wherein the first clustering threshold is smaller than the second clustering threshold.

[0077] S710: Based on a first clustering threshold, the data points in the filtered laser point cloud frame are clustered, and a first cluster that satisfies a first shape condition is selected from the clusters obtained by clustering, the first shape condition is determined based on the shape of the graphic element.

[0078] S712: Based on the second clustering threshold, the data points in the first cluster are clustered, and a second cluster that satisfies the second shape condition is selected from the clusters obtained by clustering, the second shape condition is determined based on the shape of the graphic combination.

[0079] S714: Based on the first clustering threshold, each second cluster is clustered into multiple subclusters, and the number of subclusters in each second cluster is determined.

[0080] S716: The second cluster is filtered based on the number of subclusters in the second cluster to obtain point cloud clusters, and at least two target subclusters are selected from the subclusters of each point cloud cluster.

[0081] S718: For each point cloud cluster, determine the ratio of the number of data points between the target subclusters.

[0082] S720: If a target ratio that satisfies the ratio condition exists within the ratio, it is determined that a target object is included in the laser point cloud frame, the distance between the robot and the target object is determined, and the robot's motion control is performed based on the distance.

[0083] For the specific details of S702 through S720 above, please refer to the detailed implementation process described above.

[0084] While the steps in the flowcharts related to each of the above embodiments are shown sequentially according to the arrows, please understand that these steps are not necessarily performed sequentially in the order indicated by the arrows. Unless otherwise explicitly stated herein, there are no strict order restrictions on the execution of these steps, and they may be performed in other orders. Furthermore, at least some of the steps in the flowcharts related to each of the above embodiments may include multiple steps or stages, and these steps or stages do not necessarily have to be completed simultaneously but may be executed at different times. The execution order of these steps or stages also does not necessarily have to be sequential, but may be performed alternately or in rotation with other steps or at least some of the steps or stages of other steps.

[0085] Based on a similar inventive concept, embodiments of the present application also provide corresponding robot motion control devices for realizing the robot motion control method described above. Since the means for realizing the problem-solving provided by this device are similar to the means for realizing the method described above, specific limitations in the embodiments of one or more robot motion control devices provided below can be referenced to the limitations for the robot motion control method described above, and will not be repeated here.

[0086] In one embodiment, as shown in Figure 8, a robot motion control device is provided, which includes an acquisition module 802, a filtering module 804, a clustering module 806, and a control module 808. The acquisition module 802 is configured to acquire laser point cloud frames obtained by scanning the operating environment. The filtering module 804 is configured to extract data points from the laser point cloud frame by filtering them based on the reflection characteristics corresponding to each data point in the laser point cloud frame, thereby obtaining a filtered laser point cloud frame. The clustering module 806 is configured to cluster data points within the filtered laser point cloud frame to obtain point cloud clusters. The control module 808 is configured to perform motion control of the robot when it is determined, based on the point cloud cluster, that a marked object is included in the laser point cloud frame.

[0087] In the above embodiment, a laser point cloud frame is obtained by scanning the operating environment. Since the reflection characteristics of the marker object to the laser differ from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered based on the reflection characteristics corresponding to each data point in the laser point cloud frame. By filtering, data points with different reflection characteristics from other objects are extracted, and a filtered laser point cloud frame is obtained. By clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters, it is possible to determine whether or not a marker object is included in the laser point cloud frame based on the shape characteristics of the marker object. If it is determined that a marker object is included in the laser point cloud frame based on the point cloud clusters, the robot's motion control is performed. By identifying the marker object in the laser point cloud frame using the different reflection characteristics and shape characteristics of the marker object to the laser, the identification accuracy is higher, and the robot effectively avoids entering dangerous and prohibited areas. Furthermore, the laser has a long detection range, allowing the marker object to be identified from a distance, giving the robot sufficient time to apply the brakes, effectively avoiding falls or tipping over due to emergency braking of the high-speed moving robot, and ensuring the safe operation of the robot.

[0088] In one embodiment, the control module 808 further includes: If it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, the distance between the robot and the marker object is determined. It is configured to control the robot's motion based on distance.

[0089] In one embodiment, the laser point cloud frame includes at least two frames, and the control module 808 further... If it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, the position coordinates corresponding to each marker object are determined. The system is configured to control the robot's motion if the position coordinates corresponding to markers in at least two frames of laser point cloud are similar.

[0090] In one embodiment, the filtering module 804 further includes Based on the reflected intensity corresponding to each data point in the laser point cloud frame, the average and maximum reflected intensity of the data points are determined. The intensity threshold is determined based on the average reflectance and the maximum reflectance. The system is configured to extract data points from the laser point cloud frame by filtering them based on an intensity threshold, thereby obtaining a filtered laser point cloud frame.

[0091] In one embodiment, the marker is a graphic combination consisting of at least two graphic elements, and the clustering module 806 further... A first clustering threshold and a second clustering threshold are determined based on the distance between each graphic element in the graphic combination, and the first clustering threshold is smaller than the second clustering threshold. Based on a first clustering threshold, data points in the filtered laser point cloud frame are clustered, and a first cluster satisfying a first shape condition is selected from the clusters obtained by clustering, the first shape condition being determined based on the shape of the graphic element. Based on a second clustering threshold, data points in the first cluster are clustered, and a second cluster satisfying a second shape condition is selected from the clusters obtained by clustering. The second shape condition is determined based on the shape of the graphic combination. The system is configured to filter the second cluster to obtain a point cloud cluster.

[0092] In one embodiment, the clustering module 806 further includes: Based on the first clustering threshold, each second cluster is clustered into multiple subclusters, and the number of subclusters in each second cluster is determined. The system is configured to filter a second cluster based on the number of subclusters to obtain point cloud clusters.

[0093] In one embodiment, the control module 808 further includes: Select at least two target subclusters from among the subclusters of each point cloud cluster, For each point cloud cluster, determine the ratio of the number of data points between the target subclusters. If a target ratio that satisfies the ratio conditions exists within the ratio, the system determines that a marker object is included in the laser point cloud frame and performs motion control of the robot.

[0094] Each module in the robot motion control system described above may be implemented in whole or in part by software, hardware, or a combination thereof. Each module may be incorporated into a processor in a computer device in hardware form, or it may be independent of the processor, or it may be stored in memory in a computer device in software form, allowing the processor to call and execute the actions corresponding to each module.

[0095] In one embodiment, a robot is provided, the internal structure of which may be as shown in Figure 9. This robot includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The robot's processor is used to provide computing and control capabilities. The robot's memory includes a non-volatile storage medium, internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an execution environment for the operating system and computer programs within the non-volatile storage medium. The robot's input / output interface is used to exchange information between the processor and external devices. The robot's communication interface is used to communicate with external terminals via wired or wireless means, the wireless means may be implemented by Wi-Fi, mobile communication networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, a method for controlling the robot's motion is realized. The robot's display unit is used to form a visually visible screen and may be a display screen, projection device, or virtual reality image device, and the display screen may be a liquid crystal display screen or an electronic ink display screen. The robot's input device may be a touch layer covering the display screen, a button, trackball, or touchpad provided on the robot housing, or an external keyboard, touchpad, or mouse.

[0096] Those skilled in the art will understand that the structure shown in Figure 9 is merely a block diagram showing a part of the structure relating to the means of the present invention, and does not limit the computer devices to which the means of the present invention applies. A particular robot may include more or fewer components than those shown in the figure, and certain components may be combined or their arrangements changed.

[0097] In one embodiment, a robot is provided that includes memory and a processor, a computer program is stored in the memory, a lidar is mounted on the robot, markers are installed in the robot's operating environment, and when the processor executes the computer program, the steps in each embodiment of the above method are realized.

[0098] In one embodiment, a computer-readable storage medium containing a computer program is provided, and when this computer program is executed by a processor, the steps in each embodiment of the above method are realized.

[0099] In one embodiment, a computer program product including a computer program is provided, and when this computer program is executed by a processor, the steps in each embodiment of the above method are realized.

[0100] Furthermore, all user information (including, but not limited to, user device information and user personal information) and data (including, but not limited to, data used for analysis, stored data, and displayed data) related to this application are information and data authorized by the user or fully authorized by all parties involved, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0101] Those skilled in the art will understand that the whole or partial processes of the methods of the above embodiments can be implemented by instructing the relevant hardware with a computer program, which may be stored in a non-volatile computer-readable storage medium, and which, when executed, may include the processes of each embodiment of the above methods. Any reference to memory, database, or other medium used in each embodiment provided by this application may include at least one of non-volatile memory and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. For illustrative purposes only and not limited to, RAM can take various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases related in each embodiment provided by this application may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors related in each embodiment provided by this application may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features of the above embodiments have been described, but any combination of these technical features should be considered to fall within the scope described herein, provided that they do not contradict each other.

[0103] The above embodiments represent only a few embodiments of the present application, and although the descriptions are specific and detailed, they should not be interpreted as limiting the scope of the patent application. It should be noted that a person skilled in the art could make several modifications and improvements without departing from the spirit of the present application, and all of these fall within the scope of protection of the present application. Therefore, the scope of protection of the present application should be determined by the attached claims.

Claims

1. A method for controlling the motion of a robot, wherein the robot is equipped with a LiDAR device, and markers are installed in the operating environment of the robot, and the method is: This involves acquiring laser point cloud frames obtained by scanning the operating environment, Based on the reflection characteristics corresponding to each data point in the laser point cloud frame, the data points in the laser point cloud frame are extracted by filtering to obtain the filtered laser point cloud frame. The process involves clustering the data points within the filtered laser point cloud frame to obtain point cloud clusters, A method comprising: performing motion control of the robot when it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster.

2. If it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, then the robot's motion control is performed accordingly. If it is determined that the laser point cloud frame contains a marker object based on the point cloud cluster, the distance between the robot and the marker object is determined. The method according to claim 1, characterized in that it includes controlling the motion of the robot based on the distance.

3. The laser point cloud frame includes at least two frames, and if it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, the robot's motion control is performed accordingly. If it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, the position coordinates corresponding to each of the marked objects are determined. The method according to claim 1, characterized in that the motion control of the robot is performed such that the position coordinates corresponding to the marker objects in at least two frames of the laser point cloud are the same.

4. The aforementioned marker includes a reflective marker, and the reflective properties include the reflective intensity. Based on the reflection characteristics corresponding to each data point in the laser point cloud frame, the data points in the laser point cloud frame are extracted by filtering to obtain the filtered laser point cloud frame. Determining an intensity threshold based on the reflected intensity corresponding to each data point in the laser point cloud frame, The method according to claim 1, further comprising: filtering out data points from the data points of the laser point cloud frame whose reflection intensity is greater than the intensity threshold to obtain a filtered laser point cloud frame.

5. Determining an intensity threshold based on the reflected intensity corresponding to each data point in the laser point cloud frame means that Based on the reflected intensity corresponding to each data point in the laser point cloud frame, the average reflected intensity and the maximum reflected intensity of the data point are determined. The method according to specification 4, comprising determining an intensity threshold based on the average reflectance and the maximum reflectance.

6. The aforementioned marker is a graphic combination consisting of at least two graphic elements, and obtaining a point cloud cluster by clustering the data points in the filtered laser point cloud frame is, A first clustering threshold and a second clustering threshold greater than the first clustering threshold are determined based on the distance between each graphic element in the aforementioned graphic combination. Based on the first clustering threshold, the data points in the filtered laser point cloud frame are clustered, and from the clusters obtained by clustering, a first cluster that satisfies a first shape condition determined based on the shape of the graphic element is selected. Based on the second clustering threshold, the data points in the first cluster are clustered, and from the clusters obtained by clustering, a second cluster is selected that satisfies the second shape condition determined based on the shape of the graphic combination. The method according to claim 1, characterized by comprising filtering the second cluster to obtain the point cloud cluster.

7. Filtering the second cluster to obtain a point cloud cluster is, Based on the first clustering threshold, each of the second clusters is clustered into multiple subclusters, and the number of subclusters in each of the second clusters is determined. The method according to 6, further comprising: filtering the second cluster based on the number of subclusters in the second cluster to obtain the point cloud cluster.

8. If it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, then the robot's motion control is performed accordingly. Selecting at least two target subclusters from among the subclusters of each point cloud cluster, For each point cloud cluster, determine the ratio between the number of data points of the target subcluster, The method according to any one of claims 1 to 7, characterized in that, if a target ratio that satisfies the ratio condition exists among the ratios, it is determined that a marked object is included in the laser point cloud frame and the motion control of the robot is performed.

9. The above method further, If the reflection characteristics are structural characteristics, the method includes determining whether the laser point cloud frame contains structured labels based on the point cloud clusters. If it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster, then the robot's motion control is performed accordingly. The method according to any one of claims 1 to 8, characterized in that it is determined that the laser point cloud frame contains structured markers based on the point cloud cluster, and the robot's motion is controlled accordingly.

10. Controlling the motion of the aforementioned robot is Controlling the robot to stop moving, or Controlling the robot to avoid the hazardous area, or The method according to any one of claims 1 to 9, characterized in that it includes controlling the robot to reduce its movement speed.

11. A robot motion control device, wherein the device is An acquisition module configured to acquire a laser point cloud frame obtained by scanning the operating environment, A filtering module configured to extract data points from the laser point cloud frame by filtering them based on the reflection characteristics corresponding to each data point in the laser point cloud frame, thereby obtaining a filtered laser point cloud frame, A clustering module configured to cluster data points within the filtered laser point cloud frame to obtain point cloud clusters, The apparatus is characterized by including a control module configured to perform motion control of the robot when it is determined that a marked object is included in the laser point cloud frame based on the point cloud cluster.

12. The apparatus according to claim 11, further comprising the control module, which is configured to determine the distance between the robot and the marker object when it is determined that the laser point cloud frame contains a marker object based on the point cloud cluster, and to perform motion control of the robot based on the distance.

13. The apparatus according to claim 11, further comprising: the control module, when it is determined that a marker object is included in the laser point cloud frame based on the point cloud cluster, determines the position coordinates corresponding to each marker object, and when the position coordinates corresponding to marker objects in at least two frames of the laser point cloud frame are the same, performs motion control of the robot.

14. The aforementioned marker includes a reflective marker, and the reflective properties include the reflective intensity. The apparatus according to claim 11, further comprising: a filtering module that determines an intensity threshold based on the reflection intensity corresponding to each data point in the laser point cloud frame; and filters out data points from the data points of the laser point cloud frame whose reflection intensity is greater than the intensity threshold to obtain a filtered laser point cloud frame.

15. Determining an intensity threshold based on the reflected intensity corresponding to each data point in the laser point cloud frame means that Based on the reflected intensity corresponding to each data point in the laser point cloud frame, the average reflected intensity and the maximum reflected intensity of the data point are determined. The apparatus according to claim 14, comprising determining an intensity threshold based on the average reflectance and the maximum reflectance.

16. The aforementioned sign is a graphic combination consisting of at least two graphic elements, The apparatus according to claim 11, wherein the clustering module is further configured to determine a first clustering threshold and a second clustering threshold greater than the first clustering threshold based on the distance between each graphic element in the graphic combination, cluster the data points in the filtered laser point cloud frame based on the first clustering threshold, and select a first cluster from among the clusters obtained by clustering that satisfies a first shape condition determined based on the shape of the graphic elements, cluster the data points in the first cluster based on the second clustering threshold, and select a second cluster from among the clusters obtained by clustering that satisfies a second shape condition determined based on the shape of the graphic combination, and filter the second cluster to obtain the point cloud cluster.

17. The apparatus according to claim 16, wherein the clustering module is further configured to cluster each of the second clusters into a plurality of subclusters based on the first clustering threshold, determine the number of subclusters in each of the second clusters, and filter the second clusters based on the number of subclusters in the second clusters to obtain the point cloud clusters.

18. The apparatus according to any one of claims 11 to 17, further comprising: a control module which selects at least two target subclusters from among the subclusters of each point cloud cluster; determines the ratio between the data point quantities of the target subclusters for each point cloud cluster; determines that a marked object is included in the laser point cloud frame if a target ratio that satisfies the ratio condition exists among the ratios; and performs motion control of the robot.

19. A robot comprising memory and a processor, wherein a computer program is stored in the memory, a lidar is mounted on the robot, a marker is installed in the operating environment of the robot, and when the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are realized.

20. A computer-readable storage medium wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are realized.

Images