Robot motion control method and apparatus, robot, and storage medium
By equipping robots with lidar, data points are filtered using the reflection and shape features of laser point cloud frames, clustered into point cloud clusters, and then used to identify landmarks and perform motion control. This solves the problem of robots falling or tipping over during movement and achieves higher accuracy and longer-distance identification and avoidance of dangerous areas.
Patent Information
- Application Number
- PCT/CN2024/084443
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2024-03-28
- Publication Date
- 2024-12-05
AI Technical Summary
In existing technologies, robots have difficulty effectively preventing falls or tipping over during movement, especially at high speeds. Infrared detection has a short range, magnetic strips have a short sensing range and are easily demagnetized, and depth cameras have poor accuracy, resulting in poor fall prevention performance.
By equipping robots with lidar, laser point cloud frames of the operating environment are acquired. Data points are filtered using reflection and shape features, clustered into point cloud clusters, and landmarks are identified for motion control, including reducing speed, avoiding dangerous areas, or stopping movement.
This improves the robot's accuracy in identifying dangerous areas and its distance sensing capabilities, ensuring that the robot can identify landmarks at a distance and adjust its movement in a timely manner to avoid falling or tipping over, thus guaranteeing safe operation.
Smart Images

Figure CN2024084443_05122024_PF_FP_ABST
Abstract
Description
Robot motion control method, device, robot and storage medium
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to the Chinese patent application filed with the China Patent Office on May 31, 2023, with application number 2023106389099 and application name “Method, device, robot and storage medium for robot motion control”, the entire contents of which are incorporated by reference into this application. Technical Field
[0003] The present application relates to the technical field of mobile robots, and in particular to a method, device, robot, storage medium and computer program product for robot motion control. Background Art
[0004] With the development of mobile robot technology, the application environment of mobile robots is becoming increasingly complex. Robots may enter dangerous or prohibited areas during movement. For example, entering a dangerous area may put the robot at risk of falling. Traditionally, infrared detection can be used to control the robot's motion. However, due to the short sensing range of infrared detection, when the robot moves quickly, it cannot prevent the robot from falling or may cause the robot to topple over due to emergency braking. In methods that use magnetic strips to control the robot's motion, the magnetic strips have a short sensing range and are easily demagnetized. Anti-fall methods based on depth cameras have poor accuracy and are prone to false detection. Therefore, how to ensure the safe operation of robots has become an urgent problem that needs to be solved.
[0005] Summary of the Invention
[0006] According to various embodiments of the present application, a method, an apparatus, a robot, a computer-readable storage medium, and a computer program product for controlling robot motion are provided.
[0007] A method for controlling robot motion, wherein the robot is equipped with a laser radar and markers are provided in the robot's operating environment, the method comprising:
[0008] Obtaining a laser point cloud frame obtained by scanning the operating environment;
[0009] filtering the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame;
[0010] Clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters;
[0011] If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the robot is motion controlled.
[0012] A device for controlling robot motion, comprising:
[0013] An acquisition module is used to acquire a laser point cloud frame obtained by scanning the operating environment;
[0014] a filtering module, configured to filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame;
[0015] A clustering module, configured to cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters;
[0016] A control module is configured to control the motion of the robot if it is determined that a marker is included in the laser point cloud frame based on the point cloud cluster.
[0017] A robot comprises a memory and a processor, wherein the memory stores a computer program, the robot is equipped with a laser radar, and markers are provided in the operating environment of the robot. When the processor executes the computer program, the steps of the method for controlling the robot motion are implemented.
[0018] A computer-readable storage medium stores a computer program, which implements the steps of the robot motion control method when executed by a processor.
[0019] A computer program product comprises a computer program, which implements the steps of the robot motion control method when executed by a processor.
[0020] The details of one or more embodiments of the present application are set forth in the accompanying drawings and the description below. Other features and advantages of the present application will become apparent from the description, drawings, and claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, without paying any creative work, they can also obtain drawings of other embodiments based on these drawings.
[0022] FIG1 is a diagram illustrating an application environment of a method for controlling robot motion according to an embodiment;
[0023] FIG2 is a schematic flow chart of a method for controlling robot motion according to an embodiment;
[0024] FIG3 is a schematic diagram of a reflective marker in one embodiment;
[0025] FIG4 is a schematic flow chart of a method for controlling the motion of a robot based on distance in one embodiment;
[0026] FIG5 a is a schematic diagram of an area determined based on the distance from a marker in one embodiment;
[0027] FIG5 b is a schematic diagram of a process of filtering a laser point cloud frame in one embodiment;
[0028] FIG6 is a schematic diagram of a process for clustering point cloud clusters according to an embodiment;
[0029] FIG7 is a schematic flow chart of a method for controlling robot motion in another embodiment;
[0030] FIG8 is a block diagram of a robot motion control device according to an embodiment;
[0031] FIG9 is a diagram showing the internal structure of a robot in one embodiment. DETAILED DESCRIPTION
[0032] To facilitate understanding of the present application, a more comprehensive description of the present application will be provided below with reference to the accompanying drawings. The accompanying drawings illustrate preferred embodiments of the present application. However, the present application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of the present application.
[0033] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the invention pertains. The terms used in the specification of the invention herein are for the purpose of describing specific embodiments only and are not intended to limit the present application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0034] The robot motion control method provided in an embodiment of the present application can be applied in the application environment shown in Figure 1. Here, robot 102 obtains a laser point cloud frame obtained by scanning the operating environment; filters the data points in the laser point cloud frame based on the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame; clusters the data points in the filtered laser point cloud frame to obtain point cloud clusters; and if the laser point cloud frame is determined to contain a marker based on the point cloud clusters, motion control of robot 102 is performed. Robot 102 can be, but is not limited to, various delivery robots, work robots, service robots, sorting robots, or cleaning robots. A user can place a marker at the edge of a dangerous area or prohibited area in the operating environment of robot 102. The marker has at least one of a recognizable structural feature, a material feature, or a pattern feature. For example, a user can attach markers, such as reflective markers or structured markers, to both sides of a staircase. Reflective markers have a specific shape and a high reflectivity to laser pulses. Structured markers have a specific structure and exhibit specific structural features in the laser point cloud frame formed after reflecting the laser pulse. The robot 102 is equipped with a laser radar, which scans the operating environment to obtain a laser point cloud frame, and then extracts the reflection intensity or structural characteristics as reflection features to detect whether there are markers in the operating environment, thereby controlling the robot's motion.
[0035] In one embodiment, as shown in FIG2 , a method for robot motion control is provided, which is described by taking the method applied to the robot in FIG1 as an example, and includes the following steps:
[0036] S202: Acquire a laser point cloud frame obtained by scanning the operating environment.
[0037] The operating environment is the environment in which the robot operates, which can be either indoor or outdoor. For example, the operating environment can be a road environment where goods are delivered. Another example is the operating environment in a hotel where services are provided. Another example is the operating environment in a factory.
[0038] The operating environment is equipped with markers, including reflective markers and structured markers. Reflective markers reflect laser pulses with high intensity. Based on the reflection intensity corresponding to each data point, data points with high reflection intensity can be filtered out to identify reflective markers from the filtered data points. Structured markers have a specific structure and exhibit specific structural features in the laser point cloud frame formed after reflecting the laser pulse. Therefore, data points can be filtered based on the structural features corresponding to each data point to identify structured markers from the filtered data points.
[0039] The robot is equipped with a laser radar (LiDAR), which can scan the environment and generate a laser point cloud frame. This laser point cloud frame consists of multiple data points, each with a corresponding laser intensity. LiDAR is an optical sensor that emits laser pulses into the operating environment and receives laser pulses reflected from various objects in the operating environment. The laser point cloud frame is generated based on the reflected laser pulses.
[0040] S204 , filtering the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame.
[0041] Data points are points in the laser point cloud frame, used to describe points in the three-dimensional space scanned by the laser radar. Each data point includes multiple attributes such as position coordinates, reflection characteristics, and scanning angle. Reflection characteristics can be reflection intensity or structural characteristics of reflection. This reflection intensity is used to represent the intensity of the laser radar pulse echo. When the reflectivity of a point in three-dimensional space to the laser pulse is higher, the reflection intensity of the data point corresponding to that point in three-dimensional space is higher. This structural feature is the structured information presented by the laser radar pulse echo, which can reflect the specific structure of each object in the operating environment.
[0042] In one embodiment, when the reflection feature is reflection intensity, S204 specifically includes: determining an intensity threshold based on the reflection intensity corresponding to each data point in the laser point cloud frame; filtering out data points whose reflection intensity is greater than the intensity threshold among the data points in the laser point cloud frame to obtain a filtered laser point cloud frame.
[0043] S206 , clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters.
[0044] A point cloud cluster is a collection of similar data points. For example, a point cloud cluster can be composed of data points corresponding to the same object scanned by a LiDAR. Clustering is an unsupervised learning technique that measures the similarity of clustered objects and groups similar objects into one category.
[0045] In one embodiment, the robot can cluster the data points in the filtered laser point cloud frame using a K-Means (K-means) clustering algorithm, a BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) algorithm, or a Gaussian mixture clustering algorithm to obtain point cloud clusters.
[0046] S208: If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the robot is motion controlled.
[0047] In one embodiment, when the reflection feature is reflection intensity, the robot can determine whether the laser point cloud frame contains a reflective marker based on the point cloud cluster. If it is determined based on the point cloud cluster that the laser point cloud frame contains a reflective marker, the robot is controlled in motion.
[0048] Among them, the reflective marker is a marker made of reflective material, which is pasted on the edge of the dangerous area or prohibited area and can be composed of graphic elements of various shapes. For example, the reflective marker can be composed of one or more rectangles, circles, ovals or triangles. The multiple graphic elements that make up the reflective marker can be the same or different. For example, the reflective marker can be composed of two or more rectangles, or the reflective marker can be composed of a rectangle and a circle, or the reflective marker can be composed of a circle and a triangle. In one embodiment, as shown in Figure 3, the reflective marker is composed of two rectangles arranged side by side. The size of the rectangle and the spacing between the rectangles can be adjusted according to actual needs. For example, the size of the rectangle can be 50 mm × 100 mm, and the spacing between the two rectangles can be 50 mm.
[0049] Motion control involves controlling the robot's motion patterns, including its speed, direction, or path. For example, this can include stopping the robot, circumventing a dangerous area, or reducing its speed. If the point cloud cluster determines that a laser point cloud frame contains reflective markers, this indicates the robot is about to enter a dangerous or prohibited area, requiring motion control. For example, you can place reflective markers on the sides of stairs or elevators. If the laser point cloud frames captured by the robot using LiDAR contain these markers, it indicates the robot is about to enter the staircase or elevator, posing a risk of falling, and motion control can be implemented. For another example, you can place reflective markers on the edge of a restricted area, such as a body of water. If the laser point cloud frames captured by the robot using LiDAR contain these markers, it indicates the robot is about to enter a restricted area, and motion control can be implemented. Alternatively, you can place reflective markers on the edge of fragile obstacles, such as glass. If the laser point cloud frames captured by the robot using LiDAR contain these markers, it indicates the robot may collide with the fragile obstacle, and motion control can be implemented.
[0050] In another embodiment, when the reflection feature is a structural feature, the robot can determine whether the laser point cloud frame contains a structured marker based on the point cloud cluster. If it is determined that the laser point cloud frame contains a structured marker based on the point cloud cluster, the robot is controlled in motion.
[0051] In the above embodiment, a laser point cloud frame is obtained by scanning the operating environment. Because the reflection characteristics of a landmark differ from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered based on the reflection characteristics corresponding to each data point in the laser point cloud frame. This allows the data points with reflection characteristics different from those of other objects to be filtered out, resulting in a filtered laser point cloud frame. The data points in the filtered laser point cloud frame are then clustered to form point cloud clusters. This allows the determination of whether the laser point cloud frame contains a landmark based on the landmark's shape characteristics. If the point cloud clusters determine that the laser point cloud frame contains a landmark, the robot's motion is controlled. By utilizing the landmark's different reflection characteristics and shape characteristics to identify the landmark in the laser point cloud frame, recognition accuracy is improved, effectively preventing the robot from entering dangerous and prohibited areas. Furthermore, the laser's long sensing range allows the landmark to be identified from a distance, giving the robot ample time to brake. This effectively prevents the fast-moving robot from falling or tipping over due to emergency braking, ensuring its safe operation.
[0052] In one embodiment, as shown in FIG4 , S208 specifically includes the following steps:
[0053] S402: If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the distance between the robot and the marker is determined.
[0054] The distance is the distance between the robot and the marker in three-dimensional space. The robot can measure the time interval between the laser radar emitting a laser pulse and receiving the reflected echo, and use this time interval to determine the distance between the robot and the marker.
[0055] S404, controlling the motion of the robot according to the distance.
[0056] Since the marker is attached to the edge of the danger zone, when the robot is far away from the marker, the urgency of the danger is weaker, and when the robot is close to the marker, the urgency of the danger is stronger. Therefore, the robot's motion is controlled according to the distance between the robot and the marker.
[0057] In one embodiment, S404 specifically includes determining the robot's current location based on the distance, and controlling the robot's motion based on the robot's current location. Specifically, as shown in Figure 5a, the black rectangular boxes represent reflective markers. Area A is the braking area, area B is the avoidance area, and area C is the deceleration area. When the robot moves into the deceleration area, the robot is controlled to reduce its movement speed, for example, to 0.6 meters per second. When the robot is in the avoidance area, the robot is controlled to stop upon encountering an obstacle. When the robot is in the braking area, the robot is controlled to stop. The deceleration area is farthest from the reflective marker, the avoidance area is between the braking and deceleration areas, and the braking area is closest to the reflective marker. The distance between each area and the reflective marker can be adjusted. For example, the braking area can be within 1.2 meters of the reflective marker, the avoidance area can be within 1.2 meters or more and less than 1.5 meters, and the deceleration area can be within 1.5 meters or more and less than 2 meters.
[0058] In one embodiment, S404 specifically includes: controlling the robot's movement path based on the distance. For example, when the distance is less than a preset value, controlling the robot to stop moving and return. Alternatively, when the distance is less than a preset value, controlling the robot to adjust its movement direction.
[0059] In the above embodiment, if a marker is determined to be present in the laser point cloud frame based on the point cloud cluster, the distance between the robot and the marker is determined, and the robot's motion is controlled based on the distance. This allows the robot's motion mode to be adjusted based on the distance, improving the robot's motion flexibility.
[0060] In one embodiment, S208 specifically includes: if it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then the position coordinates corresponding to each marker are determined respectively; if the position coordinates corresponding to the markers in at least two frames of laser point cloud frames are the same, then the robot is motion controlled.
[0061] The position coordinates are the coordinates of the marker in the world coordinate system. Each data point in the laser point cloud frame includes the position coordinates corresponding to that data point. The robot can determine the position coordinates of the marker based on the position coordinates corresponding to each data point. To avoid false detection by the robot, when the robot recognizes a marker in a certain laser point cloud frame, the robot continues to collect laser point cloud frames using the laser radar. If the position coordinates corresponding to the markers in multiple laser point cloud frames continuously collected by the laser radar are the same, it means that these markers all correspond to markers in the same position. In this case, it is determined that the robot has scanned the marker and the robot is controlled in motion.
[0062] In the above embodiment, if a laser point cloud frame is determined to contain a marker based on the point cloud cluster, the position coordinates of each marker are determined separately. If the position coordinates of the markers in at least two laser point cloud frames are the same, the robot's motion is controlled. This effectively avoids false detections and improves the accuracy of the robot's motion control.
[0063] In one embodiment, the marker includes a reflective marker, and the reflection characteristic includes reflection intensity. As shown in FIG5b , S204 specifically includes the following steps:
[0064] S502 : Determine the average reflection intensity and the maximum reflection intensity of each data point according to the reflection intensity corresponding to each data point in the laser point cloud frame.
[0065] The average reflection intensity is the average of the reflection intensities corresponding to each data point in the laser point cloud frame. The maximum reflection intensity is the maximum reflection intensity corresponding to each data point in the laser point cloud frame. When the robot receives the laser point cloud frame, it counts the reflection intensities corresponding to each data point in the laser point cloud frame, calculates the average reflection intensity, and searches for the maximum reflection intensity among the reflection intensities corresponding to each data point. The robot can use various search methods, such as sequential search, binary search, or binary tree search, to find the maximum reflection intensity.
[0066] S504: Determine an intensity threshold according to the average reflection intensity and the maximum reflection intensity.
[0067] When the marker is reflective, the reflective intensity of each data point corresponding to the reflective marker in the laser point cloud frame is high due to its high reflectivity to laser pulses. Therefore, the robot can determine the intensity threshold based on the average reflection intensity and the maximum reflection intensity, and then filter the data points in the laser point cloud frame based on the intensity threshold.
[0068] In one embodiment, S504 specifically includes performing a weighted summation of the average reflection intensity and the maximum reflection intensity, and using the resulting sum as the intensity threshold. The weights corresponding to the average reflection intensity and the maximum reflection intensity can be the same or different. For example, the weight corresponding to the average reflection intensity is 0.3, and the weight corresponding to the maximum reflection intensity is 0.7. Developers can configure the weights corresponding to the average and maximum reflection intensities before the robot leaves the factory, or users can set them in the settings interface.
[0069] In one embodiment, S504 specifically includes: the robot determines an average value of the average reflection intensity and the maximum reflection intensity, and uses the average value as the intensity threshold.
[0070] S506 , filtering the data points in the laser point cloud frame based on the intensity threshold to obtain a filtered laser point cloud frame.
[0071] The robot filters the data points in the laser point cloud frame based on the intensity threshold, deletes the data points with reflection intensity lower than the intensity threshold, and retains the data points with reflection intensity higher than the intensity threshold.
[0072] In the above embodiment, the average and maximum reflection intensities of the data points in the laser point cloud frame are determined based on the reflection intensities corresponding to each data point. An intensity threshold is determined based on the average and maximum reflection intensities. Based on the intensity threshold, the data points in the laser point cloud frame are filtered to produce a filtered laser point cloud frame. This allows the laser point cloud frame to be filtered to identify data points that match the reflection characteristics of reflective markers. High laser pulse reflection intensities from reflective markers can be used to identify hazardous areas, thereby improving the accuracy of robot motion control.
[0073] In one embodiment, the marker is a graphic combination consisting of at least two graphic elements; as shown in FIG6 , S206 specifically includes the following steps:
[0074] S602: Determine a first clustering threshold and a second clustering threshold according to the distance between each graphic element in the graphic combination; the first clustering threshold is smaller than the second clustering threshold.
[0075] The graphic elements can be graphics of various shapes, including rectangles, circles, or triangles. The robot can identify the graphic combination in the laser point cloud frame based on the shape characteristics of the graphic combination. In one embodiment, the first clustering threshold determined by the robot is less than the distance between the graphic elements, and the second clustering threshold is greater than the distance between the graphic elements. Thus, the data points corresponding to the graphic elements can be clustered into clusters using the first clustering threshold, and the data points corresponding to the graphic combination can be clustered into clusters using the second clustering threshold. In one embodiment, assuming that the distance between the graphic elements is R, the robot can determine the first clustering threshold to be 0.5R and the second clustering threshold to be 1.5R.
[0076] S604: Clustering the data points in the filtered laser point cloud frame according to a first clustering threshold, and selecting a first cluster that meets a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element.
[0077] Among them, the first shape condition is a screening condition determined based on the shape of the graphic element, which is used to screen out clusters that meet the shape characteristics of the graphic element. In one embodiment, the first shape condition can be that the aspect ratio of the minimum circumscribed rectangle of the cluster is within a preset range. The preset range corresponding to the aspect ratio can be determined based on the aspect ratio of the graphic element. For example, if the graphic element is a rectangle with an aspect ratio of 2, the preset range can be a numerical range of 1.8 to 2.2. In another embodiment, the first shape condition can be that the radius of the minimum circumscribed circle of the cluster is within a preset range. The preset range corresponding to the radius can be determined based on the radius of the graphic element. For example, if the graphic element is a circle with a radius of 3, the preset range can be a numerical range of 2.5 to 3.5. The robot clusters the data points in the filtered laser point cloud frame according to the first clustering threshold, and clusters the data points whose distance between each other is less than the first clustering threshold into a cluster. The clusters obtained by clustering are then screened according to the first shape condition, and clusters that are too large, too small, or have dissimilar shapes compared to the graphic element are discarded. The resulting first cluster meets the geometric characteristics of a single graphic element.
[0078] S606 , clustering the data points in the first cluster according to a second clustering threshold, and selecting a second cluster that meets a second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination.
[0079] Among them, the second shape condition is a screening condition determined based on the shape of the graphic combination, which is used to screen out clusters that meet the shape characteristics of the graphic combination. In one embodiment, the second shape condition can be that the aspect ratio of the minimum circumscribed rectangle of the cluster is within a preset range. The preset range of the aspect ratio can be determined based on the aspect ratio of the entire graphic combination. For example, if the aspect ratio of the graphic combination is equal to 1.5, the preset range can be a numerical interval of 1 to 2. In another embodiment, the second shape condition can be that the radius of the minimum circumscribed circle of the cluster is within a preset range. The preset range of the radius can be determined based on the radius of the entire graphic combination.
[0080] The robot clusters the data points in the first cluster according to the second clustering threshold, grouping the data points whose distance between them is less than the second clustering threshold into a single cluster. The robot then filters the resulting clusters according to the second shape condition, discarding clusters that are too large, too small, or have a dissimilar shape to the graphic combination, to obtain a second cluster that meets the overall geometric characteristics of the graphic combination.
[0081] S608: Filter the second type of clusters to obtain point cloud clusters.
[0082] In order to make the number of clusters contained in the final point cloud cluster consistent with the number of graphic elements contained in the graphic combination, the robot filters the second cluster to obtain the point cloud cluster.
[0083] In one embodiment, S608 specifically includes: clustering each second cluster into multiple subclusters according to a first clustering threshold, and determining the number of subclusters in each second cluster; filtering the second clusters based on the number of subclusters in the second clusters to obtain point cloud clusters.
[0084] The robot clusters each second-category cluster based on a first clustering threshold, dividing each second-category cluster into multiple subclusters whose shapes and sizes are similar to the graphic elements. The robot determines the number of subclusters within each second-category cluster and compares the number of subclusters within the cluster with the number of graphic elements in the graphic combination. Second-category clusters with too many or too few subclusters are discarded, and the remaining second-category clusters are used as point cloud clusters. For example, if the image combination includes two graphic elements, second-category clusters with fewer than two or greater than three subclusters are discarded, ensuring that the remaining point cloud clusters contain two to three subclusters. The robot then filters the second-category clusters based on the number of subclusters within them, ensuring that the filtered point cloud clusters not only conform to the geometric characteristics of the graphic combination but also contain a number of subclusters consistent with the number of graphic elements in the graphic combination. This allows for more accurate landmark identification and avoids false detections.
[0085] In the above embodiment, a first clustering threshold and a second clustering threshold are determined based on the distance between each graphic element in the graphic combination. The data points in the filtered laser point cloud frame are then clustered based on the first clustering threshold, and a first cluster that satisfies the first shape condition is selected from the clustered clusters. The data points in the first cluster are clustered based on the second clustering threshold, and a second cluster that satisfies the second shape condition is selected from the clustered clusters. The second clusters are filtered to obtain point cloud clusters. This allows point cloud clusters that conform to the geometric characteristics of the image combination to be clustered, allowing landmarks to be identified based on the shape of the image combination, thereby improving the accuracy of landmark identification.
[0086] In one embodiment, S208 specifically includes: selecting at least two target subclusters in the subclusters of each point cloud cluster; determining the ratio between the number of data points in the target subclusters for each point cloud cluster; if there is a target ratio in the ratio that meets the ratio condition, determining that a marker is included in the laser point cloud frame, and performing motion control on the robot.
[0087] The target subcluster is the subcluster that satisfies a selection condition among all subclusters. For example, the selection condition may be that the number of data points is greater than a preset value. Another example is that the number of data points ranks within a preset rank among all subclusters, where the preset rank may be 2, for example.
[0088] The ratio condition is a condition used to determine whether a point cloud cluster is a landmark based on a ratio. In one embodiment, the ratio condition can be that the ratio is less than a preset value, such as 0.5, 0.6, or the like. In another embodiment, the ratio condition can be that the ratio is within a preset ratio range. The robot can determine the preset ratio range based on the size of each graphical element in the graphical combination. For example, if the graphical combination includes two graphical elements of the same size, and the ratio between the sizes of the graphical elements is 1, then if the ratio between the number of data points in the target subclusters is much greater than 1 or much less than 1, it indicates that the sizes of the target subclusters differ significantly, do not meet the size characteristics of the graphical elements in the graphical combination, and are therefore not point cloud clusters corresponding to landmarks. Therefore, the robot can determine the preset ratio range to be [0.5, 1.5]. If a target ratio exists in the ratio that meets the ratio condition, it indicates that each target subclusters in the point cloud cluster meets the size characteristics of the graphical elements in the graphical combination, thereby determining that the point cloud cluster is a landmark.
[0089] In the above embodiment, at least two target subclusters are selected from each point cloud cluster's subclusters. For each point cloud cluster, the ratio of the number of data points in the target subclusters is determined. If a target ratio satisfies a ratio condition, the laser point cloud frame is determined to contain a landmark, and the robot's motion is controlled. Whether a point cloud cluster is a landmark is determined based on whether the size of each target subclusters in the point cloud cluster matches the size characteristics of each graphic element in the graphic combination, further improving the accuracy of landmark recognition.
[0090] In one embodiment, as shown in FIG7 , the method for controlling robot motion includes the following steps:
[0091] S702: Acquire a laser point cloud frame obtained by scanning the operating environment.
[0092] Each data point in the laser point cloud frame corresponds to a reflection feature, which may be a reflection intensity or a structural feature. When the reflection feature is reflection intensity, S704 is executed.
[0093] S704 , determining an average reflection intensity and a maximum reflection intensity of the data points according to the reflection intensity corresponding to each data point in the laser point cloud frame.
[0094] S706 , determining an intensity threshold according to the average reflection intensity and the maximum reflection intensity, and filtering the data points in the laser point cloud frame based on the intensity threshold to obtain a filtered laser point cloud frame.
[0095] S708, when the marker is a graphic combination consisting of at least two graphic elements, determine a first clustering threshold and a second clustering threshold according to the distance between the graphic elements in the graphic combination; the first clustering threshold is smaller than the second clustering threshold.
[0096] S710, clustering the data points in the filtered laser point cloud frame according to a first clustering threshold, and selecting a first cluster that meets a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element.
[0097] S712 , clustering the data points in the first cluster according to a second clustering threshold, and selecting a second cluster that meets a second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination.
[0098] S714 , clustering each second cluster into a plurality of subclusters according to the first clustering threshold, and determining the number of subclusters in each second cluster.
[0099] S716 , filtering the second type of clusters based on the number of subclusters in the second type of clusters to obtain point cloud clusters, and selecting at least two target subclusters from the subclusters of each point cloud cluster.
[0100] S718 , for each point cloud cluster, determining the ratio between the number of data points in the target sub-cluster.
[0101] S720: If there is a target ratio that meets the ratio condition in the ratio, it is determined that the laser point cloud frame contains a marker, and the distance between the robot and the marker is determined; the robot is controlled to move according to the distance.
[0102] For the specific contents of S702 to S720 above, reference may be made to the specific implementation process described above.
[0103] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.
[0104] Based on the same inventive concept, embodiments of the present application also provide a robot motion control device for implementing the robot motion control method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more robot motion control device embodiments provided below can be found in the above-mentioned limitations of the robot motion control method and will not be repeated here.
[0105] In one embodiment, as shown in FIG8 , a device for controlling robot motion is provided, including: an acquisition module 802 , a filtering module 804 , a clustering module 806 , and a control module 808 , wherein:
[0106] An acquisition module 802 is used to acquire a laser point cloud frame obtained by scanning the operating environment;
[0107] The filtering module 804 is used to filter the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame;
[0108] A clustering module 806 is used to cluster the data points in the filtered laser point cloud frame to obtain point cloud clusters;
[0109] The control module 808 is configured to control the motion of the robot if it is determined based on the point cloud cluster that the laser point cloud frame contains a marker.
[0110] In the above embodiment, a laser point cloud frame is obtained by scanning the operating environment. Because the reflection characteristics of a landmark differ from those of other objects in the operating environment, the data points in the laser point cloud frame are filtered based on the reflection characteristics corresponding to each data point in the laser point cloud frame. This allows the data points with reflection characteristics different from those of other objects to be filtered out, resulting in a filtered laser point cloud frame. The data points in the filtered laser point cloud frame are then clustered to form point cloud clusters. This allows the determination of whether the laser point cloud frame contains a landmark based on the landmark's shape characteristics. If the point cloud clusters determine that the laser point cloud frame contains a landmark, the robot's motion is controlled. By utilizing the landmark's different reflection characteristics and shape characteristics to identify the landmark in the laser point cloud frame, recognition accuracy is improved, effectively preventing the robot from entering dangerous and prohibited areas. Furthermore, the laser's long sensing range allows the landmark to be identified from a distance, giving the robot ample time to brake. This effectively prevents the fast-moving robot from falling or tipping over due to emergency braking, ensuring its safe operation.
[0111] In one embodiment, the control module 808 is further configured to:
[0112] If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the distance between the robot and the corresponding marker is determined;
[0113] Control the robot's motion based on distance.
[0114] In one embodiment, the laser point cloud frame includes at least two frames; the control module 808 is further configured to:
[0115] If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the position coordinates corresponding to each marker are determined respectively;
[0116] If the position coordinates corresponding to the markers in at least two laser point cloud frames are the same, the robot is controlled to move.
[0117] In one embodiment, the filtering module 804 is further configured to:
[0118] According to the reflection intensity corresponding to each data point in the laser point cloud frame, the average reflection intensity and the maximum reflection intensity of the data point are determined;
[0119] Determine an intensity threshold based on the average reflection intensity and the maximum reflection intensity;
[0120] Based on the intensity threshold, the data points in the laser point cloud frame are filtered to obtain a filtered laser point cloud frame.
[0121] In one embodiment, the marker is a graphic combination consisting of at least two graphic elements; the clustering module 806 is further configured to:
[0122] Determining a first clustering threshold and a second clustering threshold according to the distance between each graphic element in the graphic combination; the first clustering threshold is smaller than the second clustering threshold;
[0123] Clustering the data points in the filtered laser point cloud frame according to a first clustering threshold, and selecting a first cluster that satisfies a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element;
[0124] Clustering the data points in the first cluster according to a second clustering threshold, and selecting a second cluster that satisfies a second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination;
[0125] The second type of clusters are filtered to obtain point cloud clusters.
[0126] In one embodiment, the clustering module 806 is further configured to:
[0127] clustering each second-class cluster into a plurality of subclusters according to a first clustering threshold, and determining the number of subclusters in each second-class cluster;
[0128] The second type of clusters are filtered based on the number of subclusters to obtain point cloud clusters.
[0129] In one embodiment, the control module 808 is further configured to:
[0130] Select at least two target subclusters from each point cloud cluster;
[0131] For each point cloud cluster, determine the ratio between the number of data points in the target sub-cluster;
[0132] If there is a target ratio that meets the ratio condition in the ratio, it is determined that the laser point cloud frame contains a marker, and the robot is controlled to move.
[0133] Each module in the aforementioned robot motion control apparatus may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.
[0134] In one embodiment, a robot is provided, the internal structure of which can be shown in Figure 9. The robot includes a processor, memory, an input / output interface, a communication interface, a display unit, and an input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are connected to the system bus via the input / output interface. The robot's processor is used to provide computing and control capabilities. The robot's memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The robot's input / output interface is used to exchange information between the processor and external devices. The robot's communication interface is used to communicate with external terminals via wired or wireless communication, and the wireless communication can be achieved via Wi-Fi, a mobile cellular network, NFC (near-field communication), or other technologies. When executed by the processor, the computer program implements a method for robot motion control. The display unit of the robot is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the robot can be a touch layer covering the display screen, or a button, trackball or touchpad set on the robot shell, or an external keyboard, touchpad or mouse, etc.
[0135] Those skilled in the art will understand that the structure shown in Figure 9 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific robot may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.
[0136] In one embodiment, a robot is provided, comprising a memory and a processor, wherein a computer program is stored in the memory, the robot is equipped with a laser radar, and markers are set in the operating environment of the robot. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented.
[0137] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.
[0138] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.
[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.
[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.
[0141] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.
Claims
1. A method for controlling robot motion, wherein the robot is equipped with a laser radar and a marker is set in the operating environment of the robot, the method comprising: Acquire a laser point cloud frame obtained by scanning the operating environment; Filtering the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame; Clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters; and If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, the robot is motion controlled.
2. The method according to claim 1, characterized in that If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then controlling the robot to move comprises: If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, determining the distance between the robot and the marker; and The robot is controlled in motion according to the distance.
3. The method according to claim 1, characterized in that The laser point cloud frame includes at least two frames; if it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then the motion control of the robot includes: If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then the position coordinates corresponding to each of the markers are determined respectively; and If the position coordinates corresponding to the markers in at least two frames of the laser point cloud frames are the same, the robot is motion controlled.
4. The method according to claim 1, characterized in that: The marker includes a reflective marker, and the reflection feature includes reflection intensity; filtering the data points in the laser point cloud frame according to the reflection feature corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame includes: Determining an intensity threshold according to the reflection intensity corresponding to each data point in the laser point cloud frame; and Among the data points of the laser point cloud frame, data points whose reflection intensity is greater than the intensity threshold are filtered out to obtain a filtered laser point cloud frame.
5. The method according to claim 4, characterized in that Determining the intensity threshold according to the reflection intensity corresponding to each data point in the laser point cloud frame includes: Determine the average reflection intensity and the maximum reflection intensity of each data point in the laser point cloud frame according to the reflection intensity corresponding to each data point; and An intensity threshold is determined according to the average reflection intensity and the maximum reflection intensity.
6. The method according to claim 1, characterized in that The marker is a graphic combination consisting of at least two graphic elements; clustering the data points in the filtered laser point cloud frame to obtain a point cloud cluster includes: Determine the first clustering threshold and the second clustering threshold according to the distance between each graphic element in the graphic combination; A clustering threshold is smaller than the second clustering threshold; Clustering the data points in the filtered laser point cloud frame according to the first clustering threshold, and selecting a first cluster satisfying a first shape condition from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element; Clustering the data points in the first cluster according to the second clustering threshold, and selecting a second cluster that satisfies a second shape condition from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination; and The second type of clusters is filtered to obtain the point cloud clusters.
7. The method according to claim 6, characterized in that The point cloud clusters obtained by filtering the second type of clusters include: Clustering each of the second-type clusters into a plurality of sub-clusters according to the first clustering threshold, and determining the number of sub-clusters in each of the second-type clusters; and The second type of clusters is filtered based on the number of subclusters in the second type of clusters to obtain the point cloud clusters.
8. The method according to any one of claims 1 to 7, characterized in that: If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then controlling the robot to move comprises: Selecting at least two target subclusters from each subcluster of the point cloud cluster; For each of the point cloud clusters, determining a ratio between the number of data points in the target sub-cluster; and If there is a target ratio satisfying the ratio condition among the ratios, it is determined that a marker is included in the laser point cloud frame, and the robot is motion controlled.
9. The method according to any one of claims 1 to 8, characterized in that: The method further comprises: When the reflection feature is a structural feature, judging whether the laser point cloud frame contains a structured marker according to the point cloud cluster, If it is determined based on the point cloud cluster that the laser point cloud frame contains a marker, then controlling the robot to move comprises: If it is determined based on the point cloud cluster that the laser point cloud frame contains a structured marker, the robot is motion controlled.
10. The method according to any one of claims 1 to 9, characterized in that: The motion control of the robot comprises: Control the robot to stop moving; or, controlling the robot to bypass the danger zone; or, The robot is controlled to reduce the moving speed.
11. A robot motion control device, characterized in that: The device comprises: An acquisition module, used to acquire a laser point cloud frame obtained by scanning the operating environment; A filtering module, used for filtering the data points in the laser point cloud frame according to the reflection characteristics corresponding to each data point in the laser point cloud frame to obtain a filtered laser point cloud frame; A clustering module, used for clustering the data points in the filtered laser point cloud frame to obtain point cloud clusters; A control module is used to control the motion of the robot if it is determined that the laser point cloud frame contains a marker based on the point cloud cluster.
12. The device according to claim 11, characterized in that The control module is also used to determine the distance between the robot and the marker if it is determined based on the point cloud cluster that the laser point cloud frame contains a marker; and to control the motion of the robot according to the distance.
13. The device according to claim 11, characterized in that The control module is also used to determine the position coordinates corresponding to each marker if it is determined based on the point cloud cluster that the laser point cloud frame contains a marker; and to control the motion of the robot if the position coordinates corresponding to the markers in at least two frames of the laser point cloud frames are the same.
14. The device according to claim 11, characterized in that The marker comprises a reflective marker, and the reflection characteristic comprises reflection intensity; The filtering module is also used to determine an intensity threshold according to the reflection intensity corresponding to each data point in the laser point cloud frame; and filter out data points with reflection intensity greater than the intensity threshold among the data points in the laser point cloud frame to obtain a filtered laser point cloud frame.
15. The device according to claim 14, characterized in that Determining the intensity threshold according to the reflection intensity corresponding to each data point in the laser point cloud frame includes: Determine the average reflection intensity and the maximum reflection intensity of each data point in the laser point cloud frame according to the reflection intensity corresponding to each data point; and An intensity threshold is determined according to the average reflection intensity and the maximum reflection intensity.
16. The device according to claim 11, characterized in that The marker is a graphic combination consisting of at least two graphic elements; The clustering module is also used to determine a first clustering threshold and a second clustering threshold according to the distance between each graphic element in the graphic combination; the first clustering threshold is less than the second clustering threshold; according to the first clustering threshold, the data points in the filtered laser point cloud frame are clustered, and a first cluster satisfying a first shape condition is selected from the clusters obtained by clustering; the first shape condition is determined based on the shape of the graphic element; according to the second clustering threshold, the data points in the first cluster are clustered, and a second cluster satisfying a second shape condition is selected from the clusters obtained by clustering; the second shape condition is determined based on the shape of the graphic combination; and the second cluster is filtered to obtain the point cloud cluster.
17. The device according to claim 16, characterized in that The clustering module is further used to cluster each of the second-class clusters into multiple sub-clusters according to the first clustering threshold, and determine the number of sub-clusters in each of the second-class clusters; and filter the second-class clusters based on the number of sub-clusters in the second-class clusters to obtain the point cloud cluster.
18. The device according to any one of claims 11 to 17, characterized in that The control module is also used to select at least two target subclusters in each subcluster of the point cloud cluster; determine the ratio between the number of data points in the target subclusters for each point cloud cluster; and if there is a target ratio that satisfies the ratio condition in the ratio, determine that the laser point cloud frame contains a marker, and control the motion of the robot.
19. A robot, comprising a memory and a processor, wherein the memory stores a computer program, the robot is equipped with a laser radar, and markers are set in the operating environment of the robot, and the processor implements the steps of any one of the methods of claims 1 to 10 when executing the computer program.
20. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.
Citation Information
Patent Citations
Method for quadruped robot to autonomously follow pilot based on three-dimensional laser radar
CN111461023A
Goods shelf detection method and device, electronic equipment and storage medium
CN113253737A
Pose determination method and device of mobile robot
CN114911232A
Method and device for determining obstacle, electronic equipment and storage medium
CN115236696A
Industrial robot system and method for controlling an industrial robot
US20200254610A1
Cited By
Underwater robot course data anomaly monitoring method and system
CN120086776A
