Robot obstacle detection method, device, medium and electronic device

Abstract

This specification discloses a robot obstacle detection method, apparatus, medium, and electronic device, wherein the method includes the steps of acquiring three-dimensional point cloud data of an obstacle in a first coordinate system, the first coordinate system being a coordinate system established by the robot at its current position with the robot as its origin; fusing the three-dimensional point cloud data into a robot-centered grid map to determine obstacle grid points in the grid map; clustering the obstacle grid points to obtain at least one obstacle grid point set; and generating obstacle objects in the grid map based on the obstacle grid point set.

Description

[Technical Field]

[0001] (Related Applications) This application claims priority to a Chinese patent application with application number 202310004467.2, filed on January 3, 2023, the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to the technical field of robot data processing, and in particular to a robot obstacle detection method, device, medium and electronic device. [Background technology]

[0003] Currently, service-oriented robots (such as sweeping robots) typically need to navigate within a certain area. In this case, the robot must sense obstacles within the area and avoid collisions with them. Typically, the robot must perform obstacle avoidance maneuvers based on the obstacles. During obstacle avoidance maneuvers, the robot must determine the location, shape, and size of the obstacles in real time to improve the control accuracy of the robot's obstacle avoidance maneuvers. Existing solutions typically use radar to sense the location, shape, and size of obstacles. However, this obstacle detection method is not sufficiently accurate. Therefore, how to improve the obstacle detection accuracy of robots has become a technical challenge that needs to be resolved. Summary of the Invention

[0004] SUMMARY OF THE INVENTION Embodiments of the present disclosure provide a robotic obstacle detection method, apparatus, medium, and electronic device, which can further improve the obstacle detection accuracy of a robot, at least to some extent.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, there is provided a robot obstacle detection method, the method including the steps of acquiring three-dimensional point cloud data of an obstacle in a first coordinate system, the first coordinate system being a coordinate system established by the robot at a current position with the robot as its origin; fusing the three-dimensional point cloud data into a robot-centered grid map to determine obstacle grid points in the grid map; clustering the obstacle grid points to obtain at least one obstacle grid point set; and generating obstacle objects in the grid map based on the obstacle grid point set.

[0007] According to a second aspect of the present disclosure, there is provided a robotic obstacle detection apparatus, the apparatus comprising: an acquisition unit used to acquire three-dimensional point cloud data of obstacles in a first coordinate system, the first coordinate system being a coordinate system established by the robot at a current position with the robot as its origin; a fusion unit used to fuse the three-dimensional point cloud data into a robot-centered grid map and determine obstacle grid points in the grid map; a clustering unit used to cluster the obstacle grid points to obtain at least one obstacle grid point set; and a generation unit used to generate obstacle objects in the grid map based on the obstacle grid point set.

[0008] According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having at least one computer program instruction stored therein, the at least one computer program instruction being executed by a processor to cause the processor to perform the steps of the method according to any one of the first aspects above.

[0009] According to a fourth aspect of the present disclosure, there is provided an electronic device, the electronic device comprising one or more processors and one or more memories, wherein at least one computer program instruction is stored in the one or more memories, the at least one computer program instruction being executed by the one or more processors to cause the processors to perform the steps of the method according to any one of the first aspect above.

[0010] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the specification, are used to interpret the principles of the present disclosure. Obviously, the accompanying drawings described below are merely some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these accompanying drawings without creative work. [Brief explanation of the drawings]

[0012] [Figure 1] Schematic of a scenario where a robot uses a line laser to detect obstacles [Figure 2] 1 is a flowchart of a robotic obstacle detection method according to some embodiments of the present disclosure. [Figure 3] Details of the flowchart for acquiring 3D point cloud data of obstacles in the first coordinate system in Figure 2 [Figure 4] FIG. 10 is a detailed diagram of another flowchart for acquiring three-dimensional point cloud data of an obstacle in the first coordinate system of FIG. 2. [Figure 5] FIG. 3 is a mode diagram for determining obstacle grid points in the grid map of FIG. [Figure 6] FIG. 3 shows details of the flowchart of FIG. 2 for clustering obstacle grid points. [Figure 7] A mode diagram for determining a set of obstacle grid points in the grid map [Figure 8] FIG. 3 shows details of a flowchart for generating obstacle objects in the grid map based on the obstacle grid point set of FIG. [Figure 9] Mode diagram for generating obstacle objects in the grid map [Figure 10] 1 is a flowchart of obstacle avoidance maneuvers for a controlled robot according to some embodiments of the present disclosure. [Figure 11] 1 is a block diagram of a robotic obstacle detection device according to some embodiments of the present disclosure. [Figure 12] 1 is a schematic structural diagram of an electronic device according to some embodiments of the present disclosure; DETAILED DESCRIPTION OF THE INVENTION

[0013] The technical solutions of the embodiments of the present disclosure will be described below clearly and completely in conjunction with the accompanying drawings of the embodiments of the present disclosure, but obviously, the described embodiments are only some embodiments of the present disclosure, not all embodiments. Based on the embodiments of the present disclosure, other embodiments obtained by those skilled in the art without creative work are all included in the protection scope of the present disclosure.

[0014] Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to thoroughly understand the embodiments of the present disclosure. However, those skilled in the art will understand that it is possible to implement the technical solutions of the present disclosure without one or more of the specific details, or to employ other methods, components, devices, steps, etc. In other instances, well-known methods, devices, implementations, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

[0015] The block diagrams shown in the accompanying drawings are merely functional entities that do not necessarily correspond to physically separate entities, i.e., they may be implemented in software, or in one or more hardware modules or integrated circuits, or in different network and / or processor and / or microcontroller devices.

[0016] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all contents and operations / steps, nor do they necessarily have to be performed in the order described. For example, some operations / steps may be separated, some operations / steps may be combined or partially combined, and therefore the actual execution order may vary depending on the actual situation.

[0017] In describing this disclosure, it should be understood that the terms "first" and "second" are used for descriptive purposes only and do not denote or imply the relative importance or number of such technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more such features. In describing this disclosure, unless otherwise specified, "plurality" means two or more than two.

[0018] To enable those skilled in the art to better understand the present disclosure, a brief description of an application scenario related to the present disclosure will first be given with reference to FIG.

[0019] Referring to Figure 1, a schematic diagram of a scenario in which a robot uses a line laser to detect obstacles is shown.

[0020] The robot 101 according to the present disclosure may be a sweeping cleaning robot, and when the robot 101 performs a cleaning task, it needs to move back and forth within a cleaning area 102. In this case, the robot 101 needs to sense obstacles 103 (such as a bench or a small table on the floor of a room) present in the cleaning area 102 and avoid collisions with the obstacles 103. Typically, in order for the robot 101 to move normally within the cleaning area 102 without colliding with the obstacles 103, it detects the position, shape, and size of the obstacles 103 in real time, and plans its own movement path based on the position, shape, and size of the obstacles 103 to avoid collisions with the obstacles 103 and improve the control accuracy of the obstacle avoidance movement of the robot 101.

[0021] In the present disclosure, the robot 101 can aid its obstacle avoidance movement through a line laser. In some embodiments, the robot 101 projects a line laser externally via one or more line laser modules attached to the robot 101 and collects laser light bars 104 projected on obstacles 103 to aid its obstacle avoidance movement. When the line laser module projects a line laser externally, there are several projection methods. In some embodiments, a vertical line laser projection method may be adopted as shown in FIG. 1( a). In other embodiments, a horizontal line laser projection method may be adopted as shown in FIG. 1( b). In still other embodiments, a method that combines vertical and horizontal line laser projection may be adopted as shown in FIG. 1( c).

[0022] 2, a flowchart of a robot obstacle detection method according to some embodiments of the present disclosure is shown, which may be performed by a device having a computing capability. As shown in FIG. 2, the robot obstacle detection method includes at least steps 110 to 170.

[0023] In step 110, three-dimensional point cloud data of an obstacle in a first coordinate system is obtained, and the first coordinate system is a coordinate system established by the robot at its current position with the robot as the origin.

[0024] In the present disclosure, the three-dimensional point cloud data is used to characterize the position distribution of obstacles in a first coordinate system, where the first coordinate system is a coordinate system established by the robot with the robot as its origin at its current position. It should be understood that as the robot is moving within the cleaning area, the coordinate system established with the robot as its origin (i.e., the robot coordinate system) also changes its position within the cleaning area. Although the obstacles do not move within the cleaning area (i.e., do not move in real time), the robot coordinate system changes along with the robot within the cleaning area, so the three-dimensional point cloud data of the obstacles reflected in the robot coordinate system (i.e., the position coordinates of the obstacles in the robot coordinate system) also change.

[0025] In the present disclosure, when a robot moves from one position to another, it may project a line laser externally via a line laser module attached to the robot, and determine three-dimensional point cloud data in the robot coordinate system of the position projected by the laser on the obstacle based on the laser light bar projected on the obstacle (i.e., coordinates in the robot coordinate system of the position projected by the laser on the obstacle).

[0026] In some embodiments, obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system may be performed according to the steps shown in FIG.

[0027] 3 shows a detailed flowchart for acquiring three-dimensional point cloud data of obstacles in the first coordinate system in FIG. 2. The flowchart may include steps 111 to 113.

[0028] Step 111: The robot acquires light bar information projected on the obstacle by the line laser collected at the current position.

[0029] Step 112: Determine candidate 3D point cloud data of obstacles in the first coordinate system based on the light bar information.

[0030] Step 113: Screening the 3D point cloud data from the candidate 3D point cloud data based on a predetermined height threshold.

[0031] When a robot detects an obstacle, it is not necessary to consider all objects in the cleaning area as obstacles, such as pieces of paper on the floor. Based on this, in the present disclosure, after determining candidate 3D point cloud data for obstacles in the first coordinate system, the 3D point cloud data can be screened from the candidate 3D point cloud data based on a predetermined height threshold. Based on the predetermined height threshold, 3D point cloud data corresponding to some objects on the floor that are not high enough for the robot to perform avoidance can be obtained, thereby not only saving computational resources required for subsequent data processing of the 3D point cloud data but also preventing the robot from performing some unnecessary obstacle avoidance operations in the future.

[0032] In some embodiments, the robot acquiring the light bar information projected on the obstacle by the line laser collected at the current position may be performed according to the following steps 1111 to 1113.

[0033] In step 1111, the robot acquires an obstacle image collected at its current position with the line laser turned on as a first image.

[0034] In step 1112, the robot acquires an obstacle image collected at its current position with the line laser turned off as a second image.

[0035] Step 1113: performing a difference process on the first image and the second image to obtain light bar information projected on the obstacle by a line laser.

[0036] In the present disclosure, the obstacle images under the line laser on state and the obstacle images under the line laser off state can both be collected by a camera mounted on the robot.

[0037] In the present disclosure, a difference image is obtained by performing a difference process on the first image and the second image, and light bars whose brightness values ​​are greater than a predetermined brightness threshold are extracted from the difference image, and information on the light bars projected onto the obstacle by a line laser is obtained.

[0038] In other embodiments, the robot acquiring the light bar information projected onto the obstacle by the line laser collected at its current location may involve acquiring an obstacle image collected by the robot at its current location with the line laser turned on, and then identifying the light bar information projected onto the obstacle from the obstacle image by an image recognition algorithm (in some embodiments, the image recognition algorithm may be an image recognition algorithm based on artificial intelligence).

[0039] In some embodiments, determining candidate 3D point cloud data of an obstacle in the first coordinate system based on the light bar information may be performed according to the following steps 1121 to 1122.

[0040] Step 1121: Determine the obstacle point cloud coordinates of the center of the light bar pixel in the light bar information in the camera coordinate system according to the mapping relationship from the camera coordinate system to the pixel coordinate system.

[0041] Step 1122: Determine candidate 3D point cloud data of an obstacle in the first coordinate system based on the obstacle point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system.

[0042] It should be understood that the camera coordinate system refers to a coordinate system whose origin is the optical center of the camera and whose z-axis coincides with the optical axis of the camera, and the pixel coordinate system refers to a coordinate system whose origin is established at the vertex in the upper left corner of the image and whose unit is the pixel.

[0043] In the present disclosure, after obtaining the light bar information projected on the obstacle, the light bar pixel center p=(u, v) in the light bar information is first determined, and then the obstacle point cloud coordinates P of the light bar pixel center in the camera coordinate system are calculated.C = (x, y, z), where u and v are the coordinate positions of the light bar pixel center on the x-axis and y-axis of the pixel coordinate system, respectively, and x, y, and z are the coordinate positions of the light bar pixel center on the x-axis, y-axis, and z-axis of the camera coordinate system, respectively.

[0044] In some embodiments, the mapping relationship from the camera coordinate system to the pixel coordinate system may be based on the following equation (1):

[0045]

number

[0046] The constraint of the following equation (2) is established.

[0047]

number

[0048] At the same time, the points from the line laser light plane must fall on the line laser light plane, so the following constraint (3) can be established from the line laser light plane equation in the camera coordinate system.

[0049]

number

[0050]

number

[0051] As a result, the obstacle point cloud coordinates P of the light bar pixel center in the camera coordinate system are calculated as shown in the following equations 5 to 7. C =(x,y,z) can be solved.

[0052]

number

[0053]

number

[0054]

number

[0055] Obstacle point cloud coordinates P in the camera coordinate system C After obtaining =(x,y,z), we can obtain the transformation matrix of the following equation 8 from the camera coordinate system to the first coordinate system (i.e., the robot coordinate system when the robot is at its current position).

[0056]

number

[0057] As shown in the following formula 9, the candidate three-dimensional point cloud data P of the obstacle in the first coordinate system R =(x R ,y R ,z R ) can be restored.

[0058]

number

[0059] In some embodiments, obtaining the three-dimensional point cloud data of the obstacle in the first coordinate system may be performed according to the steps shown in FIG.

[0060] 4 shows details of another flowchart for acquiring three-dimensional point cloud data of an obstacle in the first coordinate system in FIG. 2. The flowchart may include steps 114 to 116.

[0061] Step 114: Obtain position change information for the robot moving from the previous position to the current position.

[0062] Step 115: Obtaining 3D point cloud data of the obstacle in a second coordinate system, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as the origin.

[0063] Step 116: converting the three-dimensional point cloud data of the obstacle in the second coordinate system into three-dimensional point cloud data in the first coordinate system based on the position change information.

[0064] In the present disclosure, if an obstacle does not move in the cleaning area but the position of the robot coordinate system changes along with the robot in the cleaning area, the three-dimensional point cloud data of the obstacle reflected in the robot coordinate system (i.e., the position coordinates of the obstacle in the robot coordinate system) also changes. Based on this, it is necessary to convert the three-dimensional point cloud data in the robot coordinate system when the obstacle was in its previous position into the robot coordinate system when it is in its current position. In some embodiments, the three-dimensional point cloud data of the obstacle in the second coordinate system (a coordinate system established with the robot as the origin at the previous position) may be converted into three-dimensional point cloud data in the first coordinate system based on position change information when the robot moves from the previous position to the current position.

[0065] Continuing with reference to FIG. 2, in step 130, the 3D point cloud data is fused into a robot-centered grid map to determine obstacle grid points in the grid map.

[0066] In the present disclosure, to fuse 3D point cloud data of obstacles in the robot coordinate system, a grid map (in some embodiments, the grid map may be a local 2D grid map of obstacles) with the robot center as the origin may be created, and the coordinate positions of obstacle points in the robot coordinate system at the current position observed during the robot's obstacle avoidance maneuver may be updated to this grid map in real time. As the robot moves, the coordinate positions of obstacles observed at past positions are transformed into the robot coordinate system at the current position and subsequently updated to this grid map, and scanning of obstacles, especially floor obstacles, is achieved through multi-frame line laser point cloud fusion.

[0067] In order that those skilled in the art may better understand the present disclosure, reference is now made to FIG.

[0068] Referring to FIG. 5, a mode diagram for determining obstacle grid points 500 in the grid map of FIG. 2 is shown.

[0069] As shown in Figure 5(d), when the robot is at the first position, the robot is located at the center of the grid map. Grid point 501 is a reflection of an obstacle in the grid map observed by the line laser at the robot's previous position, and grid point 502 is a reflection of an obstacle in the grid map observed by the line laser at the robot's first (previous) position. After the robot moves from the first position to the second position, the grid map is updated and the robot is still located at the center of the grid map. As shown in Figure 5(e), as the robot moves, the relative position between the robot and the obstacle changes compared to Figure 5(d). In Figure 5(e), the relative position between the robot and the obstacle grid point in the grid map also changes. When the robot moves to the upper left of the figure, the obstacle grid point in the grid map corresponds to the robot moving to the lower right of the grid map. In Figure 5(e), grid point 503 is a reflection of an obstacle in the grid map observed by the line laser at a past position after the robot has moved to a second position (i.e., the current position), grid point 504 is a reflection of an obstacle in the grid map observed by the line laser at a first position after the robot has moved to a second position (i.e., the current position), and grid point 505 is a reflection of an obstacle in the grid map observed by the line laser at a second position after the robot has moved to the second position (i.e., the current position).

[0070] Still referring to FIG. 2, in step 150, the obstacle grid points are clustered to obtain at least one set of obstacle grid points.

[0071] In the present disclosure, since the obstacle grid points are reflections of obstacles in the grid map, for an obstacle with a certain width and length, multiple obstacle grid points with relatively concentrated position distribution are reflected in the grid map, and based on this, the obstacle grid points can be clustered based on the distances between the obstacle grid points to obtain at least one obstacle grid point set.

[0072] In some embodiments, clustering the obstacle grid points to obtain at least one set of obstacle grid points may be performed according to the steps shown in FIG.

[0073] Referring to Figure 6, there is shown a detailed flowchart of clustering the obstacle grid points in Figure 2. The flowchart may include steps 151 to 155.

[0074] Step 151, optionally select a predetermined number of obstacle grid points as obstacle centroids.

[0075] Step 152 calculates the distance value between each obstacle grid point and each obstacle centroid.

[0076] Step 153: include each of the obstacle grid points in the obstacle grid point set in which the obstacle centroid with the smallest distance value is located.

[0077] Step 154: Determine one obstacle grid point in each obstacle grid point set as the new obstacle centroid.

[0078] Return to step 155, the step of calculating the distance value between each obstacle grid point and each obstacle centroid, and obtain at least one obstacle grid point set until the number of iterations reaches a predetermined number of iterations or the distance between the new obstacle centroid and the old obstacle centroid is less than the first distance threshold.

[0079] In some embodiments, determining one obstacle grid point from each set of obstacle grid points as the new obstacle centroid may be performed according to the following steps 1541-1542.

[0080] Step 1541: calculate the average value of the obstacle grid point coordinates in each obstacle grid point set to obtain the average value coordinates.

[0081] In step 1542, the obstacle grid point in each set of obstacle grid points whose coordinates are closest to the average coordinates is determined as a new obstacle centroid.

[0082] In some embodiments, clustering the obstacle grid points to obtain at least one obstacle grid point set may be performed according to the following steps a, b, c, and d.

[0083] In step a, k points are selected from the set of candidate grid points as the initial obstacle centroid p k =(x k ,y k ) where x k and y k are the coordinate locations of the initial obstacle centroid on the grid map x-axis and y-axis, respectively.

[0084] In step b, each grid point p in the candidate grid point set is i =(x i ,y i ), the Euclidean distance d to the center of gravity of k obstacles is expressed by the following formula (10): ki , and assign this grid point to the obstacle grid point set to which the nearest obstacle centroid belongs, where x i and y i are the coordinate positions of the ith grid point on the grid map x-axis and y-axis, respectively.

[0085]

number

[0086] In step c, for each obstacle grid point set to which each obstacle centroid belongs, the coordinate mean value of all rasters in the corresponding set is calculated, and the coordinate mean value is updated to the new obstacle centroid of the obstacle corresponding to the set.

[0087] In step d, steps b and c are repeated until the number of iterative calculations reaches a predetermined number of iterations or the Euclidean distance between the new obstacle centroid and the old obstacle centroid is less than a first distance threshold, thereby obtaining at least one obstacle grid point set.

[0088] It should be noted that before step 151 shown in FIG. 6, i.e., before optionally selecting a predetermined number of obstacle grid points as obstacle centroids, the following steps 141 to 142 may be performed: Step 141 traverses each obstacle grid point in the grid map and determines a probability value for the obstacle grid point, which is used to characterize the confidence that the obstacle grid point can be used to reflect an obstacle.

[0089] Step 142: if the probability value is less than or equal to a probability threshold, determine the obstacle grid point as a noise grid point and remove the noise grid point from the grid map.

[0090] In this disclosure, when actually observing obstacle detection using a line laser, jumps or noises in distance measurements will accidentally occur due to various uncertainties. If the robot obstacle avoidance accuracy needs to be high and the robot obstacle avoidance motion sensitivity is high, some noise grid points in the grid map need to be filtered out to avoid the impact of obstacle noise on the robot obstacle avoidance motion stability and smoothness.

[0091] In the present disclosure, each obstacle grid point in the grid map corresponds to a probability value that is used to characterize the confidence with which it is used to reflect the obstacle, with larger probability values ​​corresponding to higher confidence. The magnitude of the probability value is related to the number of times each location on the obstacle has been scanned by a line laser, and for a location on the obstacle, the more times that location has been scanned by a line laser, both currently and in the past, the more that location is reflected in the 3D point cloud data, resulting in a larger probability value for the obstacle grid point corresponding to that location in the grid map, and thus a higher confidence for the corresponding obstacle grid point.

[0092] In the present disclosure, by traversing obstacle grid points in the grid map, it is determined whether the probability value of the obstacle grid point is greater than a probability threshold, and the set of candidate grid points that satisfy the obstacle determination condition is filtered (i.e., noise grid points are removed). By removing the noise grid points, the accuracy of the obstacle grid points in the grid map is improved, which subsequently improves the accuracy of obstacle identification, contributing to improving the accuracy and efficiency of the obstacle avoidance movement of the robot.

[0093] In some embodiments, the noise filtering operation may continue to be performed, i.e., for each candidate grid point in the set of candidate grid points, it may be determined whether the magnitude of the number of points determined to be candidate grid points within each adjacent fixed-size window is greater than a number threshold. In some embodiments, for each candidate grid point, it may be determined whether the number of its 3x3 neighbors (i.e., 9 grid points) that are also candidate grid points is greater than 3. If the number condition is met, it moves to the next step; otherwise, it determines that the candidate grid point is an isolated noise grid point and is filtered out of the set of candidate grid points.

[0094] In some embodiments, after obtaining at least one obstacle grid point set, the following steps 161 to 162 may be further executed.

[0095] Step 161: In any obstacle grid point set, if the square deviation of the distance between each obstacle grid point and the obstacle centroid is greater than a square deviation threshold, divide the obstacle grid point set into at least two obstacle grid point sets.

[0096] Step 162: if the distance between the obstacle centroids of any two obstacle grid point sets is less than a second distance threshold, the any two obstacle grid point sets are merged into one obstacle grid point set.

[0097] In the present disclosure, after obtaining at least one obstacle grid point set, the obstacle grid point set can be adjusted to improve the accuracy with which the obstacle grid point set reflects true obstacles, thereby improving the accuracy of subsequent obstacle identification and contributing to improving the accuracy and efficiency of the obstacle avoidance movement of the robot.

[0098] To help those skilled in the art better understand the obstacle grid point set, reference is now made to Figure 7. Referring to Figure 7, a mode diagram for determining the obstacle grid point set in the grid map is shown.

[0099] 7, the grid map includes noise grid points 701, obstacle grid point set 703, and obstacle grid points 705. Grid points 702 and 704 are obstacle centroids of obstacle grid point set 703 and obstacle grid points 705, respectively.

[0100] Continuing with reference to FIG. 2, step 170 generates obstacle objects in the grid map based on the obstacle grid point set.

[0101] In some embodiments, generating obstacle objects in the grid map based on the obstacle grid point set may be performed according to the steps shown in FIG.

[0102] 8 shows a detailed flowchart for generating obstacle objects in the grid map based on the obstacle grid point set in FIG. 2. The flowchart may include steps 171 to 172.

[0103] Step 171: determine the grid area in which each obstacle grid point set is distributed within the grid map.

[0104] Step 172: If the size of the grid area exceeds a predetermined threshold and the density of obstacle grid points within the grid area exceeds a density threshold, determine the grid area as an obstacle object.

[0105] For the obstacle grid point sets obtained by clustering, the number of grid points in each set and the minimum x-coordinate value, maximum x-coordinate value, minimum y-coordinate value, and maximum y-coordinate value of these grid points are counted to obtain the x-coordinate range and y-coordinate range of each set, and the grid area distributed by the set in the grid map, as well as the width and length of the grid area in the grid map are determined. If the width and length of a grid area exceed an edge length threshold (i.e., a predetermined threshold) for obstacle avoidance of the robot navigation module, and the density of obstacle grid points in the grid area (obtained by dividing the number of obstacle grid points in the obstacle grid point set by the area of ​​the corresponding grid area) exceeds a certain threshold (i.e., a density threshold), the grid point in the obstacle grid point set satisfies the obstacle avoidance requirement, and the corresponding grid area is sent to the robot navigation module as an obstacle target for obstacle avoidance.

[0106] To help those skilled in the art better understand obstacle objects, reference is now made to FIG.

[0107] Referring to FIG. 9, a mode diagram for generating obstacle objects in the grid map is shown.

[0108] 9, the grid map includes noise grid point 901, obstacle object 903, and obstacle object 905. Grid point 902 and grid point 904 are the obstacle centroids of obstacle object 903 and obstacle object 905, respectively.

[0109] To help those skilled in the art better understand the present disclosure as a whole, the application process of the present disclosure will be briefly described below through an example with reference to FIG.

[0110] 10, a flowchart for controlling obstacle avoidance motion of a robot according to some embodiments of the present disclosure is shown, which may include steps 1001 to 1009.

[0111] Step 1001, the robot moves from one position to another.

[0112] Step 1002: Obtain an obstacle image under the line laser on state.

[0113] Step 1003: Obtain an obstacle image under the line laser off state.

[0114] Step 1004: The robot position and orientation data is acquired.

[0115] Step 1005: Based on the obstacle images and robot position and orientation data in the line laser on and off states, three-dimensional point cloud data of obstacle candidates is determined.

[0116] Step 1006: Screening the candidate 3D point cloud data for obstacles based on the height threshold.

[0117] Step 1007: A two-dimensional grid map is generated based on the three-dimensional point cloud data and the robot position and orientation data.

[0118] Step 1008: Filter noise grid points in the grid map to generate obstacle objects in the grid map.

[0119] Step 1009: Control the obstacle avoidance motion of the robot according to the obstacle object.

[0120] In the present disclosure, 3D point cloud data of an obstacle is obtained in a coordinate system established with the robot as the origin at the current position, the 3D point cloud data is merged into a grid map centered on the robot, obstacle grid points are determined in the grid map, and then the obstacle grid points are clustered to obtain at least one obstacle grid point set, and obstacle objects are generated in the grid map based on the obstacle grid point set. Because the 3D point cloud data of the obstacle in the robot coordinate system accurately reflects the obstacle position, shape, and size, merging the 3D point cloud data into the grid map centered on the robot ensures that the obstacle grid points in the grid map accurately reflect the obstacle position, shape, and size. Therefore, generating obstacle objects based on the at least one obstacle grid point set obtained by clustering the obstacle grid points can improve the accuracy of obstacle detection and the accuracy of the obstacle avoidance movement of the robot.

[0121] The following describes an embodiment of an apparatus for implementing the obstacle detection method for a robot in the above embodiment of the present disclosure. For details not disclosed in the apparatus embodiment of the present disclosure, please refer to the obstacle detection method for a robot in the above embodiment of the present disclosure.

[0122] Referring to FIG. 11, a block diagram of a robotic obstacle detection device is shown in accordance with some embodiments of the present disclosure.

[0123] As shown in FIG. 11, the robotic obstacle detection apparatus 1100 includes an acquisition unit 1101, a fusion unit 1102, a clustering unit 1103, and a generation unit 1104.

[0124] In some embodiments, the acquisition unit 1101 is used to acquire 3D point cloud data of obstacles in a first coordinate system, which is a coordinate system established by the robot with the robot as the origin at its current position; the fusion unit 1102 is used to fuse the 3D point cloud data into a robot-centered grid map and determine obstacle grid points in the grid map; the clustering unit 1103 is used to cluster the obstacle grid points to obtain at least one obstacle grid point set; and the generation unit 1104 is used to generate obstacle objects in the grid map based on the obstacle grid point set.

[0125] In some embodiments, the acquisition unit 1101 is configured to acquire light bar information projected onto the obstacle by a line laser collected by the robot at its current position, determine candidate 3D point cloud data of the obstacle in the first coordinate system based on the light bar information, and screen the 3D point cloud data from the candidate 3D point cloud data based on a predetermined height threshold.

[0126] In some embodiments, the acquisition unit 1101 is further configured to acquire an obstacle image collected by the robot at its current position when the line laser is on as a first image, and acquire an obstacle image collected by the robot at its current position when the line laser is off as a second image, and perform differential processing on the first image and the second image to acquire light bar information projected on the obstacle by the line laser.

[0127] In some embodiments, the acquisition unit 1101 is further configured to determine obstacle point cloud coordinates of the light bar pixel center in the light bar information in the camera coordinate system based on a mapping relationship from the camera coordinate system to a pixel coordinate system, and determine candidate three-dimensional point cloud data of the obstacle in the first coordinate system based on the obstacle point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system.

[0128] In some embodiments, the acquisition unit 1101 is further configured to acquire position change information when the robot moves from a previous position to a current position, acquire three-dimensional point cloud data of the obstacle in a second coordinate system, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as the origin, and convert the three-dimensional point cloud data of the obstacle in the second coordinate system into three-dimensional point cloud data in the first coordinate system based on the position change information.

[0129] In some embodiments, the clustering unit 1103 is configured to: optionally select a predetermined number of obstacle grid points as obstacle centroids; calculate a distance value between each obstacle grid point and each obstacle centroid; include each obstacle grid point in an obstacle grid point set in which the obstacle centroid with the smallest distance value is located; determine one obstacle grid point in each obstacle grid point set as a new obstacle centroid; return to performing the steps of calculating the distance values ​​between each obstacle grid point and each obstacle centroid; and obtain at least one obstacle grid point set until the number of iterative calculations reaches a predetermined number of iterations or the distance between the new obstacle centroid and the old obstacle centroid is less than a first distance threshold.

[0130] In some examples, the apparatus further comprises a traversal unit used to traverse each obstacle grid point in the grid map before optionally selecting a predetermined number of obstacle grid points as obstacle centroids, to determine a probability value for the obstacle grid point, the probability value being used to characterize a confidence that the obstacle grid point can be used to reflect an obstacle, and to determine the obstacle grid point as a noise grid point and remove the noise grid point from the grid map if the probability value is less than or equal to a probability threshold.

[0131] In some embodiments, the clustering unit 1103 is further configured to calculate an average value of the obstacle grid point coordinates in each set of obstacle grid points to obtain an average value coordinate, and determine the obstacle grid point in each set of obstacle grid points whose coordinate is closest to the average value coordinate as a new obstacle centroid.

[0132] In some embodiments, the apparatus further comprises an adjustment unit that is used, after obtaining at least one obstacle grid point set, to divide any obstacle grid point set into at least two obstacle grid point sets if a square deviation of a distance between each obstacle grid point and an obstacle centroid in any obstacle grid point set is greater than a square deviation threshold, and to combine any two obstacle grid point sets into one obstacle grid point set if a distance between the obstacle centroids of any two obstacle grid point sets is less than a second distance threshold.

[0133] In some embodiments, the generating unit 1104 is configured to determine a grid area in which each set of obstacle grid points is distributed within the grid map, and determine the grid area as an obstacle object if the size of the grid area exceeds a predetermined threshold and the density of obstacle grid points within the grid area exceeds a density threshold.

[0134] Based on the same idea, an embodiment of the present disclosure provides a computer-readable storage medium, wherein at least one computer program instruction is stored in the computer-readable storage medium, and the at least one computer program instruction is loaded and executed by a processor to perform the operations performed by the robot obstacle detection method.

[0135] Based on the same idea, an embodiment of the present disclosure further provides an electronic device, and referring to FIG. 12 , a schematic structural diagram of an electronic device according to some embodiments of the present disclosure is shown, wherein the electronic device comprises one or more memories 1204, one or more processors 1202, and at least one computer program (computer program instructions) stored in the memories 1204 and executable by the processors 1202, and when the processors 1202 execute the computer program, the electronic device performs the robot obstacle detection method.

[0136] 12, in the case of a bus architecture (represented by bus 1200), the bus 1200 may include any number of interconnected buses and bridges, and connects various circuits, such as one or more processors, represented by processor 1202, and memory, represented by memory 1204. The bus 1200 may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be further described herein. A bus interface 1205 provides an interface between the bus 1200 and the receiver 1201 and transmitter 1203. The receiver 1201 and transmitter 1203 may be the same element, i.e., a transceiver, providing a unit for communicating with various other devices over a transmission medium. The processor 1202 is responsible for managing the bus 1200 and normal processing, and the memory 1204 is used to store data used by the processor 1202 in performing operations.

[0137] The functions described herein may be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions may be stored on or transmitted by a computer-readable medium as one or more instructions or code. Other examples and embodiments are within the scope and spirit of this disclosure and the appended claims. For example, due to the nature of software, the functions described above may be performed by software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Furthermore, each functional unit may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit.

[0138] It should be understood that in some embodiments provided by the present disclosure, the disclosed technical content may be realized in other ways. Here, the device embodiments described above are merely examples. For example, the division of the units may be a logical division of functions, or may be divided in other ways when actually implemented. For example, multiple units or assemblies may be combined or combined in another system, or some features may be ignored or not implemented. On the other hand, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection via some interfaces, units, or modules, or may be an electrical or other form of coupling.

[0139] The units described as separate components may or may not be physically separate, and the components as a control device may or may not be physical units, i.e., they may be located in one place or distributed among multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of this embodiment.

[0140] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present disclosure, in practice or as a contribution to the prior art, or all or a part of the technical solution, may be embodied in the form of a software product, and the computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to execute all or a part of the steps of the method described in each embodiment of the present disclosure. The storage medium may include various media capable of storing computer program instructions, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a removable hard drive, a magnetic disk, or a compact disk.

[0141] In the present disclosure, 3D point cloud data of an obstacle is acquired in a coordinate system established with the robot as the origin at the current position, the 3D point cloud data is merged into a grid map centered on the robot, obstacle grid points are determined in the grid map, and then the obstacle grid points are clustered to obtain at least one obstacle grid point set, and obstacle objects are generated in the grid map based on the obstacle grid point set. Because the 3D point cloud data of the obstacle in the robot coordinate system accurately reflects the obstacle position, shape, and size, merging the 3D point cloud data into the grid map centered on the robot ensures that the obstacle grid points in the grid map accurately reflect the obstacle position, shape, and size. Therefore, generating obstacle objects based on at least one obstacle grid point set obtained by clustering the obstacle grid points can improve the accuracy of obstacle detection and the accuracy of the robot's obstacle avoidance movement.

[0142] The above is merely an example of the present disclosure, and does not limit the present disclosure, and those skilled in the art can have various modifications and changes to the present disclosure. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present disclosure shall be included within the scope of the claims of the present disclosure.

Claims

1. acquiring three-dimensional point cloud data of an obstacle in a first coordinate system, the first coordinate system being a coordinate system established by the robot at its current position with the robot as its origin; fusing the 3D point cloud data into a robot-centered grid map and determining obstacle grid points in the grid map; clustering the obstacle grid points to obtain at least one set of obstacle grid points; generating obstacle objects in the grid map based on the obstacle grid point set; Including, Robotic obstacle detection method.

2. The step of acquiring three-dimensional point cloud data of an obstacle in the first coordinate system includes: Acquiring light bar information projected on the obstacle by a line laser collected by the robot at its current position; determining candidate three-dimensional point cloud data of an obstacle in the first coordinate system based on the light bar information; screening the 3D point cloud data from the candidate 3D point cloud data based on a predetermined height threshold; Including, The method of claim 1.

3. Acquiring light bar information projected on the obstacle by a line laser collected by the robot at a current position includes: acquiring, as a first image, an obstacle image collected by the robot at its current position while the line laser is turned on; acquiring, as a second image, an obstacle image collected by the robot at its current position while the line laser is off; performing a differential process on the first image and the second image to obtain light bar information projected onto the obstacle by a line laser; Including, The method of claim 2.

4. Determining candidate three-dimensional point cloud data of an obstacle in the first coordinate system based on the light bar information includes: Determining obstacle point cloud coordinates of the light bar pixel centers of the light bar information in the camera coordinate system based on a mapping relationship from the camera coordinate system to the pixel coordinate system; determining candidate three-dimensional point cloud data of an obstacle in the first coordinate system based on the obstacle point cloud coordinates and a transformation matrix from the camera coordinate system to the first coordinate system; Including, The method of claim 2.

5. The step of acquiring three-dimensional point cloud data of an obstacle in the first coordinate system includes: Obtaining position change information for the robot moving from a previous position to a current position; acquiring three-dimensional point cloud data of the obstacle in a second coordinate system, the second coordinate system being a coordinate system established by the robot at the previous position with the robot as its origin; converting three-dimensional point cloud data of the obstacle in the second coordinate system into three-dimensional point cloud data in the first coordinate system based on the position change information; further comprising: The method of claim 2.

6. The step of clustering the obstacle grid points to obtain at least one set of obstacle grid points comprises: Optionally selecting a predetermined number of obstacle grid points as obstacle centroids; calculating a distance value between each obstacle grid point and each obstacle centroid; Include each of the obstacle grid points in a set of obstacle grid points in which the obstacle centroid with the smallest distance value is located; determining one obstacle grid point from each set of obstacle grid points as a new obstacle centroid; repeatedly performing a step of calculating distance values ​​between each obstacle grid point and each obstacle centroid until a predetermined number of iterations is reached or a distance between the new obstacle centroid and the old obstacle centroid is less than a first distance threshold, thereby obtaining at least one set of obstacle grid points; Including, The method of claim 1.

7. Prior to optionally selecting a predetermined number of obstacle grid points as obstacle centroids, the method comprises: traversing each obstacle grid point in the grid map and determining a probability value for the obstacle grid point, the probability value being used to characterize a confidence that the obstacle grid point can be used to reflect an obstacle; if the probability value is less than or equal to a probability threshold, determining the obstacle grid point as a noise grid point and removing the noise grid point from the grid map; further comprising: The method of claim 6.

8. determining one obstacle grid point from each set of obstacle grid points as a new obstacle centroid, calculating an average of the obstacle grid point coordinates in each obstacle grid point set to obtain an average coordinate; determining an obstacle grid point from each set of obstacle grid points whose coordinates are closest to the average coordinates as a new obstacle center of gravity; Including, The method of claim 6.

9. After obtaining at least one obstacle grid point set, the method comprises: dividing any set of obstacle grid points into at least two sets of obstacle grid points if the squared variance of the distances between each obstacle grid point and the obstacle centroid is greater than a squared variance threshold; 7. The method of claim 6, further comprising: combining any two obstacle grid point sets into one obstacle grid point set if the distance between the obstacle centroids of the two obstacle grid point sets is less than a second distance threshold.

10. generating obstacle objects in the grid map based on the obstacle grid point set, determining a grid area within the grid map in which each obstacle grid point set is distributed; determining the grid area as an obstacle object if the size of the grid area exceeds a predetermined threshold and the density of obstacle grid points within the grid area exceeds a density threshold; Including, The method of claim 6.

11. A robotic obstacle detection device, comprising: an acquisition unit used to acquire three-dimensional point cloud data of an obstacle in a first coordinate system, the first coordinate system being a coordinate system established by the robot at its current position with the robot as its origin; a fusion unit used to fuse the 3D point cloud data into a robot-centric grid map and to determine obstacle grid points in the grid map; a clustering unit used to cluster the obstacle grid points to obtain at least one set of obstacle grid points; a generation unit used to generate obstacle objects in the grid map based on the obstacle grid point set; Equipped with Robotic obstacle detector.

12. computer program instructions stored therein which, when loaded and executed by a processor, cause the processor to perform the operations performed by the method of any one of claims 1 to 10; A computer-readable storage medium.

13. An electronic device comprising a processor and a memory, the memory storing computer program instructions executable by the processor, the computer program instructions, when executed by the processor, performing instructions of the method of any one of claims 1 to 10. electronic equipment.

Images