Slope detection method and device, robot, and storage medium

Abstract

Embodiments of the present application disclose a slope detection method and device, a robot and a storage medium. The robot can include an inertial odometer and a sensor for environment perception. The method comprises the following steps: firstly, determining first pose information of the robot in a world coordinate system at a current time and a first normal vector corresponding to a target body point of the robot according to odometer information collected by the inertial odometer; then, converting point cloud data collected by the sensor to the world coordinate system according to the first pose information to obtain target point cloud data corresponding to the current time, and performing three-dimensional plane fitting according to the target point cloud data to obtain a second normal vector of a fitted plane; and finally, obtaining a slope detection result in the environment around the robot according to an included angle between the first normal vector and the second normal vector. Through the implementation of the method, the accuracy of slope detection can be improved.

Description

Technical Field

[0001] This application relates to the field of robotics, and more particularly to a slope detection method, apparatus, robot, and storage medium. Background Technology

[0002] With the rapid development of technology, robots are gradually appearing in people's field of vision, such as service robots, handling robots, and exploration robots.

[0003] During robot movement, slopes are common due to the diversity of environments, and the quality of slope detection directly impacts the robot's task execution. Therefore, how to accurately detect slopes has become a pressing technical problem for the industry. Summary of the Invention

[0004] This application provides a slope detection method, apparatus, robot, and storage medium, which can improve the accuracy of slope detection.

[0005] The first aspect of this application provides a slope detection method applied to a robot, the robot including an inertial odometer and sensors for environmental perception, the method comprising:

[0006] Based on the odometry information collected by the inertial odometry, the first pose information of the robot in the world coordinate system at the current moment and the first normal vector corresponding to the target body point of the robot are determined; the first normal vector is used to indicate the pose of the robot at the current moment.

[0007] Based on the first pose information, the point cloud data collected by the sensor is converted to the world coordinate system to obtain the target point cloud data corresponding to the current moment;

[0008] Based on the target point cloud data, a three-dimensional plane is fitted to obtain the second normal vector of the fitted plane;

[0009] The slope detection result in the environment surrounding the robot is obtained based on the angle between the first normal vector and the second normal vector.

[0010] A second aspect of this application provides a slope detection device mounted on a robot. The robot includes an inertial odometer and sensors for environmental perception. The device includes:

[0011] The first normal vector determination unit determines the robot's current pose information in the world coordinate system and the first normal vector corresponding to the robot's target body point based on the odometry information collected by the inertial odometry. The first normal vector is used to indicate the robot's current pose.

[0012] The point cloud data processing unit is used to convert the point cloud data collected by the sensor to the world coordinate system according to the first pose information, so as to obtain the target point cloud data corresponding to the current moment.

[0013] The second normal vector determination unit is used to perform three-dimensional plane fitting based on the target point cloud data to obtain the second normal vector of the fitted plane.

[0014] The ramp detection unit is used to obtain the ramp detection result in the environment surrounding the robot based on the angle between the first normal vector and the second normal vector.

[0015] A third aspect of this application provides a robot, comprising:

[0016] Memory containing executable program code;

[0017] and the processor coupled to the memory;

[0018] The processor calls the executable program code stored in the memory, and when the executable program code is executed by the processor, the processor implements the method as described in the first aspect of the embodiments of this application.

[0019] A fourth aspect of this application provides a computer-readable storage medium having executable program code stored thereon, wherein when the executable program code is executed by a processor, it implements the method described in the first aspect of this application.

[0020] The sixth aspect of this application discloses an application publishing platform for publishing computer program products, wherein when the computer program product is run on a computer, the computer executes any of the methods disclosed in the first aspect of this application.

[0021] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0022] In this embodiment, the robot may include an inertial odometry system and sensors for environmental perception. Based on this, firstly, the robot's current pose information in the world coordinate system and the first normal vector corresponding to the robot's target body point can be determined based on the odometry information collected by the inertial odometry system. Then, based on the first pose information, the point cloud data collected by the sensors is converted to the world coordinate system to obtain the target point cloud data corresponding to the current moment. A three-dimensional plane is then fitted based on the target point cloud data to obtain the second normal vector of the fitted plane. Finally, the slope detection result in the robot's surrounding environment is obtained based on the angle between the first and second normal vectors. By implementing this method, by fusing odometry information and point cloud data, the first normal vector corresponding to the robot's target body point and the second normal vector of the three-dimensional plane obtained by fitting the point cloud data can be obtained. Furthermore, the slope detection in the robot's surrounding environment can be achieved based on the angle between the first and second normal vectors. It is evident that this slope detection method eliminates the need for a complex obstacle recognition process, greatly improving the accuracy of slope detection. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments and the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application, and other drawings can be obtained based on these drawings.

[0024] Figure 1 This is a schematic diagram of a legged robot disclosed in an embodiment of this application;

[0025] Figure 2 This is a flowchart of a slope detection method disclosed in an embodiment of this application;

[0026] Figure 3 This is a flowchart of another slope detection method disclosed in the embodiments of this application;

[0027] Figure 4 This is a structural block diagram of a slope detection device disclosed in an embodiment of this application;

[0028] Figure 5 This is a structural block diagram of a robot disclosed in an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] It should be noted that the terms "comprising" and "having," and any variations thereof, in the embodiments and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices.

[0031] It is understood that the terms "first," "second," etc., used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, first pose information may be referred to as second pose information, and similarly, second pose information may be referred to as first pose information. Both first pose information and second pose information are pose information, but they are not the same pose information. Furthermore, it should be noted that the term "multiple" used in the embodiments of this application refers to two or more.

[0032] It is understood that the robot disclosed in this application can be a mobile robot. The main feature of a mobile robot is its mobility, which expands the robot's operational range and can be applied to various industries. For example, mobile robots can perform material handling in industry, combat operations in the military, cleaning in homes, and exploration in environments difficult for humans to reach. In this application, the mobile robot may include, but is not limited to, wheeled robots, tracked robots, and legged robots.

[0033] Current technologies for slope detection often employ the following methods:

[0034] Method 1, which relies solely on ranging sensors (such as LiDAR, binocular cameras, etc.), has poor accuracy.

[0035] Method 2, obstacle category identification method, requires manual labeling of obstacles in advance, increasing labor costs.

[0036] It should be noted that, in the embodiments of this application, the following embodiments are mainly illustrated using a legged robot as an example. Please refer to... Figure 1 , Figure 1 This is a schematic diagram of a legged robot 10 disclosed in an embodiment of this application. In this embodiment, the legged robot may include, but is not limited to, bipedal robots and quadrupedal robots (such as...). Figure 1 (as shown), any one of a 6-legged robot and a 12-legged robot.

[0037] Furthermore, it should be noted that the legged robot disclosed in this application may include an inertial odometry and sensors for environmental perception, wherein the inertial odometry is used to collect the robot's pose information.

[0038] In some embodiments, the inertial odometer may be a motor encoder and / or an inertial measurement unit (IMU).

[0039] Optionally, the motor encoder can be, but is not limited to, a photoelectric encoder, an absolute encoder, an incremental encoder, or a hybrid absolute encoder. The motor encoder can determine the relative change in the pose of the legged robot at different times based on the change in pulses within the sampling period. The IMU is a device that measures the robot's three-axis attitude angles (or angular rates) and acceleration. The IMU may include three single-axis accelerometers and three single-axis gyroscopes. The accelerometers measure the robot's acceleration information, and the gyroscopes measure the robot's angular velocity information. Based on the robot's acceleration and angular velocity information at different times, the relative change in the pose of the legged robot at different times can be calculated.

[0040] In some embodiments, the sensors for environmental perception can be one or more of, but not limited to, multi-line lidar, binocular cameras, and millimeter-wave radar. Such sensors can be used to perceive the environment around the robot and collect environmental information. For example, three-dimensional point cloud data of the surrounding environment can be collected by multi-line lidar or millimeter-wave radar, and images of the surrounding environment can be collected by binocular cameras. Then, the three-dimensional point cloud data of the surrounding environment can be obtained by processing the collected images, thereby realizing the detection of various target objects (such as ground, plants, buildings, walls, obstacles, etc.) in the environment around the robot. Alternatively, a combination of multiple different sensors can be used.

[0041] In some embodiments, the multi-line lidar can be any of 4 lines, 8 lines, 16 lines, 32 lines, 64 lines, and 128 lines.

[0042] In some embodiments, the binocular camera may include, but is not limited to, any one of an RGB (Red, Green, Blue) binocular camera, a TOF (Time of Flight) camera, and a structured light binocular camera.

[0043] The following explanation details the multiple coordinate systems involved in the embodiments of this application:

[0044] World coordinate system; the world coordinate system is the absolute coordinate system of the system.

[0045] Robot coordinate system: A coordinate system established with a point on the robot body as the origin;

[0046] Sensor coordinate system: A coordinate system established with the sensor as the origin.

[0047] It should be noted that, in the embodiments of this application, the robot can detect the slope in the environment around the robot by using the slope detection device installed on the robot, that is, the slope detection method disclosed in this application can be applied to the slope detection device.

[0048] Please see Figure 2 , Figure 2 This is a schematic flowchart of a slope detection method disclosed in an embodiment of this application. It may include the following steps:

[0049] 201. Based on the odometry information collected by the inertial odometry, determine the robot's first pose information in the world coordinate system at the current moment and the first normal vector corresponding to the robot's target body point.

[0050] In this embodiment, the first pose information may include the robot's current position coordinates and pose in the world coordinate system. It is understood that the first position coordinates may be the position coordinates of a target body point of the robot, such as the robot's body center or center of gravity. This embodiment does not impose such limitations. The first normal vector passes through the target body point and is used to indicate the robot's current pose.

[0051] In some embodiments, acquiring odometer information via an inertial odometer can be achieved in ways including, but not limited to, the following:

[0052] Method 1: The inertial odometer can be a motor encoder, which can be set on the robot's walking parts (or connected to the motor of the walking parts). It can be used to collect the rotation angle and speed information of the corresponding motor in real time, and then obtain a pulse signal based on the rotation angle and speed information of the corresponding motor, thereby obtaining the robot's real-time odometer information based on the pulse signal.

[0053] Method 2: The inertial odometry can be an IMU. The IMU can be installed on the robot's walking parts. The IMU can collect the acceleration and angular velocity information of the walking parts in real time, and thus obtain the robot's real-time odometry information based on the acceleration and angular velocity information of the walking parts.

[0054] Method 3: The inertial odometry system can include a motor encoder and an IMU. The motor encoder can acquire pulse signals in real time, and the IMU can acquire acceleration and angular velocity information of the walking components in real time. The pulse signals and acceleration and angular velocity information of the walking components at the same moment can be fused to obtain the odometry information of the robot at each moment. This method of fusing motor encoder and IMU to acquire odometry information in real time is beneficial to improving the accuracy of the first pose information.

[0055] Alternatively, the fusion method may include, but is not limited to, the methods described above.

[0056] 202. Based on the first pose information, convert the point cloud data collected by the sensor to the world coordinate system to obtain the target point cloud data corresponding to the current moment.

[0057] The point cloud data collected by the sensors can be used to characterize the three-dimensional spatial position of each point in the robot's surrounding environment relative to the sensors. Specifically, the three-dimensional spatial position of each point in the robot's surrounding environment relative to the multi-line lidar can be obtained by detecting the reflection duration of the laser beam using a multi-line lidar. Alternatively, the three-dimensional spatial position of each point in the robot's surrounding environment relative to the millimeter-wave radar can be obtained by detecting the reflection duration of the millimeter-wave radar. Furthermore, the three-dimensional spatial position of each point in the robot's surrounding environment relative to the binocular camera can be obtained by simultaneously acquiring left and right eye images using a binocular camera and then using these images. This application does not impose any limitations on the embodiments described herein.

[0058] Furthermore, in this embodiment, the point cloud data acquired by multi-line lidar or millimeter-wave radar may include the three-dimensional position coordinates and laser reflection intensity of each point; the point cloud data acquired by a binocular camera may include the three-dimensional position coordinates, laser reflection intensity, and color information of each point. It should be noted that multi-line lidar and millimeter-wave radar are applicable in both indoor and outdoor environments, while binocular cameras are primarily suitable for indoor environments due to the influence of lighting. It should also be noted that millimeter-wave radar and binocular cameras not only have strong anti-interference capabilities but are also low in cost.

[0059] In this method, the point cloud data acquired by the sensor is in the sensor coordinate system. In some embodiments, based on the first pose information, the point cloud data acquired by the sensor is transformed to the world coordinate system to obtain the target point cloud data corresponding to the current moment. This can include: filtering the original point cloud data acquired by the sensor at the current moment, and transforming the filtered point cloud data from the sensor coordinate system to the world coordinate system based on the first pose information to obtain the target point cloud data corresponding to the current moment. By implementing this method, filtering the original point cloud data acquired by the sensor at the current moment can effectively reduce the amount of point cloud data, which is beneficial to improving the processing efficiency of point cloud data.

[0060] In some embodiments, the filtering of the original point cloud data corresponding to the current time can be performed by, but is not limited to, one or a combination of bilateral filtering, Gaussian filtering, statistical filtering, conditional filtering, pass-through filtering and random sampling consistency filtering.

[0061] 203. Perform 3D plane fitting based on the target point cloud data to obtain the second normal vector of the fitted plane.

[0062] In some embodiments, a least squares function can be used to fit a three-dimensional plane to the target point cloud data to obtain a second normal vector of the fitted plane; wherein, the least squares function is a function of the three-dimensional spatial position of the target point cloud data. The second normal vector of the fitted plane refers to a vector perpendicular to any vector on the fitted plane.

[0063] 204. Based on the angle between the first normal vector and the second normal vector, the slope detection results in the robot's surrounding environment are obtained.

[0064] Understandably, the slope detection result can indicate whether a slope exists in the robot's surrounding environment or not. In some embodiments, determining the slope detection result in the robot's surrounding environment based on the angle between the first normal vector and the second normal vector may include: obtaining the absolute value of the angle between the first normal vector and the second normal vector; and determining the slope detection result in the robot's surrounding environment based on the relationship between the absolute value and an angle threshold.

[0065] It is understandable that when the robot is currently on a horizontal plane, and the fitted 3D plane is an inclined plane, there is an angle between the robot's posture and the normal vector of the 3D plane whose absolute value is greater than the angle threshold. Similarly, when the robot is currently on an inclined plane, and the fitted 3D plane is a horizontal plane, there is also an angle between the robot's posture and the normal vector of the 3D plane whose absolute value is greater than the angle threshold. Therefore, the slope detection result in the robot's surrounding environment can be determined based on the relationship between the absolute value of the angle between the first and second normal vectors and the angle threshold.

[0066] It should be noted that the included angle threshold can be determined by a large number of experiments. For example, the included angle threshold can be 7°, 5°, 3°, 1° or 0°, and there is no limitation here.

[0067] By implementing the above method and fusing odometry information and point cloud data, a first normal vector corresponding to the target body point of the robot and a second normal vector of the three-dimensional plane obtained by fitting the point cloud data can be obtained. Then, the angle between the first and second normal vectors can be used to detect slopes in the robot's surrounding environment. It is evident that this slope detection method eliminates the need for a complex obstacle recognition process, greatly improving the accuracy of slope detection.

[0068] Please see Figure 3 , Figure 3 This is a schematic flowchart of another slope detection method disclosed in the embodiments of this application. It may include the following steps:

[0069] 301. Based on the odometry information collected by the inertial odometry, determine the first pose information of the robot in the world coordinate system at the current moment and the first normal vector corresponding to the target body point of the robot; wherein, the first normal vector is used to indicate the pose of the robot at the current moment.

[0070] In some embodiments, the robot’s current attitude in the world coordinate system may include a first pitch angle and a first yaw angle; wherein, the first pitch angle is the angle between the body axis and the sea level, and the first yaw angle is the angle between the projection of the body axis on the sea level and the earth axis.

[0071] Furthermore, based on the odometry information collected by the inertial odometry, determining the robot's first pose information in the world coordinate system at the current moment and the first normal vector corresponding to the robot's target fuselage point can include: determining the robot's first pose offset between the previous moment and the current moment based on the first odometry information collected by the inertial odometry at the current moment and the second odometry information at the previous moment; determining the robot's first pose information in the world coordinate system at the current moment based on the first pose offset and the robot's second pose information in the world coordinate system at the previous moment; and determining the first normal vector corresponding to the robot's target fuselage point based on the first position coordinates, the first pitch angle, and the first yaw angle.

[0072] The first pose offset can include position offset and attitude offset. The position offset indicates the robot's position offset in three-dimensional space, while the attitude offset indicates the offset of each attitude angle of the robot. Optionally, each attitude angle can include roll angle, pitch angle, and tilt angle in the world coordinate system.

[0073] In some embodiments, the second pose information may include the second position coordinates of the robot's target body point in the world coordinate system at the previous moment and the robot's posture in the world coordinate system at the previous moment. Further, determining the robot's first pose information in the world coordinate system at the current moment based on the first pose offset and the robot's second pose information in the world coordinate system at the previous moment may include: determining the first position coordinates based on the second position coordinates and the aforementioned position offset, and determining the robot's posture in the world coordinate system at the current moment based on the robot's posture in the world coordinate system at the previous moment and the aforementioned posture offset.

[0074] 302. Based on the first pose information, convert the point cloud data collected by the sensor to the world coordinate system to obtain the target point cloud data corresponding to the current moment.

[0075] In some embodiments, converting the point cloud data collected by the sensor to the world coordinate system based on the first pose information to obtain the target point cloud data corresponding to the current moment may include: acquiring the original point cloud data collected by the sensor at the current moment; performing clustering processing on the original point cloud data to obtain the first point cloud data with successful clustering; extracting feature information from the first point cloud data, and extracting edge point cloud and target plane point cloud from the first point cloud data based on the feature information to obtain the second point cloud data; and converting the second point cloud data from the sensor's sensor coordinate system to the world coordinate system based on the first pose information to obtain the target point cloud data.

[0076] In some embodiments, the clustering of the raw point cloud data may include, but is not limited to, any one or a combination of Euclidean clustering, K-Means clustering, density subtraction clustering, and adaptive density clustering.

[0077] In some embodiments, clustering the original point cloud data to obtain the first point cloud data with successful clustering may include: clustering the original point cloud data to obtain point cloud data of multiple successfully clustered clusters; removing invalid clusters from the multiple clusters based on the validity conditions corresponding to the clusters to obtain valid clusters; and obtaining the first point cloud data based on the point cloud data of the valid clusters. It is understood that clusters can represent the categories of point cloud data.

[0078] In some embodiments, clustering objects may include, but are not limited to, combinations of one or more of numbers, letters, and special characters. The first point cloud data that is successfully clustered may include the point cloud data of clustering objects that meet the validity criteria.

[0079] In some embodiments, the validity criteria for a clustering object may include a minimum number of points and a maximum number of points.

[0080] For example, with a minimum number of points of 1000 and a maximum number of points of 25000, it can be understood that if the number of point cloud data corresponding to a cluster object is less than 1000 or greater than 25000, then the cluster object is determined to be an invalid cluster object; if the number of point cloud data corresponding to a cluster object is greater than or equal to 1000 and less than or equal to 25000, then the cluster object is determined to be a valid cluster object. If clustering is performed on the original point cloud data to obtain multiple cluster objects, including cluster object A, cluster object B, cluster object C, cluster object D, cluster object F, and cluster object E, where the number of point cloud data corresponding to cluster object A, cluster object B, cluster object C, and cluster object D is between 1000 and 25000, while the number of point cloud data corresponding to cluster object F is less than 1000, and the number of point cloud data corresponding to cluster object E is greater than 25000, then the first point cloud data can include the point cloud data corresponding to cluster object A, cluster object B, cluster object C, and cluster object D.

[0081] In some embodiments, if Euclidean clustering is used for the original point cloud data, the point cloud data of the original point cloud data is clustered to obtain point cloud data of multiple successfully clustered objects. This may include: for any point O in the original point cloud data, using a nearest neighbor search algorithm to find the n points closest to point O, and obtaining the distances between these n points and point O. From these n points, m points whose distance to point O is less than a second distance threshold can be determined, where m is less than or equal to n. These m points and point O belong to the same cluster object U. Any point can be selected from these m points as a new point O, and the above steps are repeated until no more points are added to the cluster object U, thus obtaining the point cloud data contained in the cluster object U. By traversing all points of the original point cloud data in this way, the point cloud data corresponding to multiple cluster objects can be obtained.

[0082] For example, the aforementioned effective clustering objects can include ground-based clustering objects and non-ground-based clustering objects; that is, the first point cloud data can include both ground-based and non-ground-based point cloud data. By implementing this method, clustering the original point cloud data can treat tiny objects as noise points and remove them. It can be understood that the point cloud data corresponding to tiny objects can be the point cloud data corresponding to invalid clustering objects. For example, these tiny objects can include leaves and grass swaying in the wind, which often appear in the previous frame but not in the next, thus reducing interference caused by the non-repetition of tiny objects between adjacent frames.

[0083] In this embodiment, edge point cloud refers to points on a boundary, and planar point cloud refers to points forming a plane. In some embodiments, extracting feature information from the first point cloud data and extracting edge point cloud and target planar point cloud from the first point cloud data based on the feature information to obtain second point cloud data may include: extracting feature information corresponding to each cluster object from the point cloud data corresponding to each cluster object, and extracting edge point cloud and target planar point cloud corresponding to that cluster object from the point cloud data corresponding to each cluster object based on the feature information corresponding to each cluster object. The feature information corresponding to each cluster object is used to characterize the geometric features of that cluster object. In some embodiments, the feature information corresponding to each cluster object may include a point cloud normal vector, which indicates the direction of the normal.

[0084] In some embodiments, extracting feature information corresponding to each cluster object from the point cloud data corresponding to each cluster object, and extracting edge point clouds and target planar point clouds corresponding to that cluster object from the point cloud data corresponding to each cluster object based on the feature information, may include: fitting the point cloud data corresponding to each cluster object using the least squares method to obtain the point cloud normal vector corresponding to each cluster object; and obtaining the target planar point cloud and edge point cloud based on the point cloud normal vector corresponding to each cluster object and the corresponding point cloud data. It is understood that in the embodiments of this application, the second point cloud data may include planar point cloud data and edge point cloud data of each cluster object. By implementing this method to extract edge point clouds and planar point clouds, the amount of point cloud data can be further reduced.

[0085] In some embodiments, transforming the second point cloud data from the sensor's sensor coordinate system to the world coordinate system based on the first pose information to obtain target point cloud data may include: obtaining the relative positional relationship between the target body point and the sensor in the robot coordinate system; determining the third pose information of the sensor in the world coordinate system based on the relative positional relationship and the first pose information; determining a transformation matrix based on the third pose information and the fourth pose information of the sensor in the robot coordinate system; and using the transformation matrix to transform the second point cloud data from the sensor's sensor coordinate system to the world coordinate system to obtain target point cloud data.

[0086] It should be noted that the relative positional relationship between the target robot point and the sensor in the robot coordinate system is fixed. This relative positional relationship can characterize the distance and orientation angle between the sensor and the target robot point. Optionally, this relative positional relationship may include the distance and orientation angle of the sensor relative to the target robot point, or the distance and orientation angle of the target robot point relative to the sensor; this is not limited here.

[0087] In some embodiments, the third pose information of the sensor in the world coordinate system may include the third position coordinates and attitude of the sensor in the world coordinate system at the current moment. Further, determining the third pose information of the sensor in the world coordinate system based on the aforementioned relative position relationship and the first pose information may include: obtaining the third position coordinates of the sensor in the world coordinate system based on the aforementioned relative position relationship and the first position coordinates of the target body point; and using the robot's current attitude in the world coordinate system as the sensor's current attitude in the world coordinate system.

[0088] In some embodiments, the transformation matrix may include a translation matrix and a rotation matrix. Using the transformation matrix, the second point cloud data is transformed from the sensor coordinate system of the sensor to the world coordinate system to obtain the target point cloud data. This may include multiplying the second point cloud data with the translation matrix and the rotation matrix to transform the second point cloud data from the sensor coordinate system of the sensor to the world coordinate system to obtain the target point cloud data.

[0089] 303. Perform 3D plane fitting based on the target point cloud data to obtain the second normal vector of the fitted plane.

[0090] In some embodiments, performing three-dimensional plane fitting based on target point cloud data to obtain a second normal vector of the fitted plane may include: obtaining a covariance matrix based on the target point cloud data and a least squares function; performing singular value decomposition on the covariance matrix to obtain a singular vector corresponding to the minimum singular value; and obtaining the second normal vector of the fitted plane based on the singular vector.

[0091] It is understandable that the covariance matrix is ​​obtained based on the position of the target point cloud data in three-dimensional space and the least squares function.

[0092] In some embodiments, performing singular value decomposition on the covariance matrix to obtain the singular vector corresponding to the minimum singular value may include: performing singular value decomposition on the covariance matrix to obtain a first unitary matrix and a second unitary matrix; and obtaining the singular vector corresponding to the minimum singular value based on the first unitary matrix and the second unitary matrix.

[0093] Furthermore, the singular vector corresponding to the minimum singular value can be used as the second normal vector of the fitting plane.

[0094] 304. When the absolute value of the angle between the first normal vector and the second normal vector is greater than the angle threshold, it is determined that there is a slope in the environment around the robot.

[0095] It should be noted that, in the embodiments of this application, the absolute value of the angle between the first normal vector and the second normal vector being greater than the angle threshold may include the following situations:

[0096] (1) At the current moment, the robot is on a plane, and the fitted three-dimensional plane has an angle with the horizontal plane;

[0097] (2) At the current moment, the robot is on a slope, and the fitted three-dimensional plane is parallel to the horizontal plane.

[0098] (3) At the current moment, the robot is on a slope, and the angle between the fitted three-dimensional plane and the horizontal plane is greater than or less than the angle between the robot and the slope at the current moment.

[0099] In some embodiments, when the threshold of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, it can be determined that there is no slope in the robot's surrounding environment.

[0100] However, in practice, it has been found that when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, the following situations often occur: (1) At the current moment, the robot is on a slope, and the angle between the fitted three-dimensional plane and the horizontal plane is equal to the angle of the slope where the robot is located at the current moment. (2) At the current moment, the robot is on a plane, and the fitted three-dimensional plane is parallel to the horizontal plane. It can be seen that when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, there may still be a slope in the environment around the robot. Based on this, the embodiments of this application can further analyze the situation where the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold in the following ways:

[0101] In some embodiments, when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, the third normal vector corresponding to the target body point at the previous moment can also be obtained. When the absolute value of the angle between the first normal vector and the third normal vector is greater than the angle threshold, it is determined that there is a slope in the robot's surrounding environment; otherwise, it is determined that there is no slope in the robot's surrounding environment.

[0102] In some embodiments, when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to an angle threshold, the historical climbing posture can also be obtained, and it can be determined whether the robot's posture at the current moment matches the historical climbing posture. If they match, it is determined that a slope exists in the robot's surrounding environment; if they do not match, it is determined that a slope does not exist in the robot's surrounding environment. It should be noted that the historical climbing posture can be obtained by the robot through a large number of climbing experiments or through historical operations, and this application embodiment does not limit it.

[0103] By implementing the above method, when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, the robot's surrounding environment can be further analyzed by using the third normal vector corresponding to the target body point at the previous moment or the historical climbing posture, which helps to improve the detection accuracy of slopes.

[0104] By implementing the above method and fusing odometry information and point cloud data, a first normal vector corresponding to the target body point of the robot and a second normal vector of the three-dimensional plane obtained by fitting the point cloud data can be obtained. Then, the slope detection in the robot's surrounding environment can be achieved based on the angle between the first and second normal vectors. It is evident that this slope detection method eliminates the need for a complex obstacle recognition process, greatly improving the accuracy of slope detection. Furthermore, based on the relationship between the absolute value of the angle between the first and second normal vectors and the angle threshold, the presence of a slope in the robot's surrounding environment can be quickly detected, further improving the real-time performance of slope detection.

[0105] Please see Figure 4 , Figure 4 This is a structural block diagram of a slope detection device disclosed in an embodiment of this application, which may include: a first normal vector determination unit 401, a point cloud data processing unit 402, a second normal vector determination unit 403, and a slope detection unit 404; wherein:

[0106] The first normal vector determination unit 401 determines the robot's first pose information in the world coordinate system at the current moment and the first normal vector corresponding to the robot's target body point based on the odometry information collected by the inertial odometry. The first normal vector is used to indicate the robot's pose at the current moment.

[0107] The point cloud data processing unit 402 is used to convert the point cloud data collected by the sensor to the world coordinate system based on the first pose information to obtain the target point cloud data corresponding to the current moment.

[0108] The second normal vector determination unit 403 is used to perform three-dimensional plane fitting based on the target point cloud data to obtain the second normal vector of the fitted plane.

[0109] The ramp detection unit 404 is used to obtain the ramp detection result in the robot's surrounding environment based on the angle between the first normal vector and the second normal vector.

[0110] In some embodiments, the first pose information includes a first position coordinate, a first pitch angle, and a first yaw angle; the method by which the first normal vector determination unit 401 determines the robot's first pose information in the world coordinate system at the current moment and the first normal vector corresponding to the robot's target fuselage point based on the odometer information collected by the inertial odometry may specifically include: the first normal vector determination unit 401 is used to determine the robot's first pose offset between the previous moment and the current moment based on the first odometer information collected by the inertial odometry at the current moment and the second odometer information at the previous moment; determine the robot's first pose information in the world coordinate system at the current moment based on the first pose offset and the robot's second pose information in the world coordinate system at the previous moment; and determine the first normal vector corresponding to the robot's target fuselage point based on the first position coordinate, the first pitch angle, and the first yaw angle.

[0111] In some embodiments, the point cloud data processing unit 402 is used to convert the point cloud data collected by the sensor to the world coordinate system according to the first pose information to obtain the target point cloud data corresponding to the current moment. Specifically, the point cloud data processing unit 402 is used to acquire the original point cloud data collected by the sensor corresponding to the current moment; perform clustering processing on the original point cloud data to obtain the first point cloud data with successful clustering; extract feature information from the first point cloud data, and extract edge point cloud and target plane point cloud from the first point cloud data according to the feature information to obtain the second point cloud data; and convert the second point cloud data from the sensor coordinate system of the sensor to the world coordinate system according to the first pose information to obtain the target point cloud data.

[0112] Furthermore, in some embodiments, the point cloud data processing unit 402 is used to transform the second point cloud data from the sensor coordinate system to the world coordinate system based on the first pose information to obtain the target point cloud data. Specifically, the point cloud data processing unit 402 is used to obtain the relative positional relationship between the target body point and the sensor in the robot coordinate system; determine the third pose information of the sensor in the world coordinate system based on the relative positional relationship and the first pose information; determine the transformation matrix based on the third pose information and the fourth pose information of the sensor in the robot coordinate system; and use the transformation matrix to transform the second point cloud data from the sensor coordinate system to the world coordinate system to obtain the target point cloud data.

[0113] In some embodiments, the second normal vector determination unit 403 is used to perform three-dimensional plane fitting based on the target point cloud data. The method of obtaining the second normal vector of the fitted plane may specifically include: the second normal vector determination unit 403 is used to obtain the covariance matrix based on the target point cloud data and the least squares function; perform singular value decomposition on the covariance matrix to obtain the singular vector corresponding to the minimum singular value; and obtain the second normal vector of the fitted plane based on the singular vector.

[0114] In some embodiments, the method by which the ramp detection unit 404 obtains the ramp detection result in the robot's surrounding environment based on the angle between the first normal vector and the second normal vector may specifically include: the ramp detection unit 404 determines that a ramp exists in the robot's surrounding environment when the absolute value of the angle between the first normal vector and the second normal vector is greater than an angle threshold.

[0115] In some embodiments, the method by which the ramp detection unit 404 obtains the ramp detection result in the robot's surrounding environment based on the angle between the first normal vector and the second normal vector may specifically include: the ramp detection unit 404 is used to obtain the third normal vector corresponding to the target body point at the previous moment when the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold; and to determine that a ramp exists in the robot's surrounding environment when the absolute value of the angle between the third normal vector and the first normal vector is greater than the angle threshold.

[0116] In some embodiments, the target fuselage point may include the fuselage center point, through which the first normal vector passes.

[0117] Please see Figure 5 , Figure 5 This is a structural block diagram of a robot disclosed in an embodiment of this application. Figure 5 As shown, the robot may include one or more of the following components: processor 501 and memory 502 coupled to processor 501, wherein memory 502 may store one or more computer programs.

[0118] Processor 501 may include one or more processing cores. Processor 501 connects to various parts of the robot via various interfaces and lines, executing instructions, programs, code sets, or instruction sets stored in memory 502, and calling data stored in memory 502 to perform various functions and process data. Optionally, processor 501 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). Processor 501 may integrate one or more of a Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 501 and may be implemented separately through a communication chip.

[0119] The memory 502 may include random access memory (RAM) or read-only memory (ROM). The memory 502 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 502 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch control, sound playback, image playback, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the robot during use.

[0120] It is understood that the slope detection device may include more or fewer structural elements than those shown in the above block diagram, such as a power module, physical buttons, Bluetooth module, etc., which are not limited here.

[0121] In this embodiment of the application, the processor 501 also has the following functions:

[0122] Based on the odometry information collected by the inertial odometry, the first pose information of the robot in the world coordinate system at the current moment and the first normal vector corresponding to the target body point of the robot are determined; the first normal vector is used to indicate the robot's pose at the current moment.

[0123] Based on the first pose information, the point cloud data collected by the sensor is converted to the world coordinate system to obtain the target point cloud data corresponding to the current moment;

[0124] Based on the target point cloud data, a 3D plane is fitted to obtain the second normal vector of the fitted plane;

[0125] The slope detection results in the robot's surrounding environment are obtained based on the angle between the first normal vector and the second normal vector.

[0126] Optionally, the first attitude information includes the first position coordinates, the first pitch angle, and the first yaw angle; the processor 501 also has the following functions:

[0127] Based on the first odometer information at the current moment and the second odometer information at the previous moment collected by the inertial odometry, the first pose offset of the robot between the previous moment and the current moment is determined.

[0128] Based on the first pose offset and the robot's second pose information in the world coordinate system at the previous moment, determine the robot's first pose information in the world coordinate system at the current moment;

[0129] Based on the first position coordinates, the first pitch angle, and the first yaw angle, determine the first normal vector corresponding to the target fuselage point of the robot.

[0130] Optionally, processor 501 also has the following functions:

[0131] Acquire the raw point cloud data corresponding to the current moment collected by the sensor;

[0132] The original point cloud data is clustered to obtain the first point cloud data with successful clustering.

[0133] Extract feature information from the first point cloud data, and extract edge point cloud and target plane point cloud from the first point cloud data based on the feature information to obtain the second point cloud data;

[0134] Based on the first pose information, the second point cloud data is transformed from the sensor coordinate system to the world coordinate system to obtain the target point cloud data.

[0135] Optionally, processor 501 also has the following functions:

[0136] Obtain the relative positional relationship between the target body point and the sensor in the robot coordinate system;

[0137] Based on the relative positional relationship and the first pose information, determine the third pose information of the sensor in the world coordinate system;

[0138] The transformation matrix is ​​determined based on the third pose information and the fourth pose information of the sensor in the robot coordinate system;

[0139] Using a transformation matrix, the second point cloud data is transformed from the sensor's coordinate system to the world coordinate system to obtain the target point cloud data.

[0140] Optionally, processor 501 also has the following functions:

[0141] The covariance matrix is ​​obtained based on the target point cloud data and the least squares function;

[0142] Singular value decomposition is performed on the covariance matrix to obtain the singular vector corresponding to the minimum singular value;

[0143] Based on the singular vectors, the second normal vector of the fitted plane is obtained.

[0144] Optionally, processor 501 also has the following functions:

[0145] When the absolute value of the angle between the first normal vector and the second normal vector is greater than the angle threshold, it is determined that a slope exists in the environment surrounding the robot.

[0146] Optionally, processor 501 also has the following functions:

[0147] When the absolute value of the angle between the first normal vector and the second normal vector is less than or equal to the angle threshold, the third normal vector corresponding to the target body point at the previous moment is obtained; when the absolute value of the angle between the third normal vector and the first normal vector is greater than the angle threshold, it is determined that there is a slope in the environment around the robot.

[0148] This application discloses a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the methods described in the above embodiments.

[0149] This application discloses a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program can be executed by a processor to implement the methods described in the above embodiments.

[0150] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, ROM, etc.

[0151] Any references to memory, storage, databases, or other media used herein may include non-volatile and / or volatile memory. Suitable non-volatile memory may include ROM, Programmable ROM (PROM), Erasable PROM (EPROM), Electrically Erasable PROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which is used as an external cache. By way of illustration and not limitation, RAM may take many forms, such as Static RAM (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus DRAM (RDRAM), and Direct Rambus DRAM (DRDRAM).

[0152] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Those skilled in the art should also recognize that the embodiments described in the specification are optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0153] In the various embodiments of this application, it should be understood that the sequence number of each process does not necessarily imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0154] In the various embodiments of this application, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0155] If the aforementioned integrated units are implemented as software functional units and sold or used as independent products, they can be stored in a computer-accessible memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several requests to cause a computer device (which can be a personal computer, server, or network device, specifically a processor in the computer device) to execute some or all of the steps of the methods described in the various embodiments of this application.

[0156] The foregoing has provided a detailed description of a slope detection method, apparatus, robot, and storage medium disclosed in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A slope detection method characterized by, The method is applied to a robot comprising an inertial odometer and a sensor for environment perception, and the method comprises: determining, according to odometer information collected by the inertial odometer, first pose information of the robot at a current time in a world coordinate system and a first normal vector corresponding to a target body point of the robot; the first normal vector is used to indicate a pose of the robot at the current time, the world coordinate system is an absolute coordinate system, the first pose information comprises a first position coordinate and a pose, and the first position coordinate is a position coordinate of the target body point; the target body point comprises a body center point, and the first normal vector passes through the body center point; converting, according to the first pose information, point cloud data collected by the sensor to the world coordinate system to obtain target point cloud data corresponding to the current time; performing three-dimensional plane fitting according to the target point cloud data to obtain a second normal vector of a fitted plane; obtaining a slope detection result in an environment around the robot according to an included angle between the first normal vector and the second normal vector; the method of obtaining the slope detection result in the environment around the robot according to the included angle between the first normal vector and the second normal vector comprises: when an absolute value of the included angle between the first normal vector and the second normal vector is less than or equal to an included angle threshold, obtaining a third normal vector corresponding to the target body point at a previous time; when an absolute value of an included angle between the third normal vector and the first normal vector is greater than the included angle threshold, determining that there is a slope in the environment around the robot.

2. The method of claim 1, wherein, the first pose information comprises a first position coordinate, a first pitch angle and a first yaw angle; and the method of determining, according to odometer information collected by the inertial odometer, first pose information of the robot at a current time in a world coordinate system and a first normal vector corresponding to a target body point of the robot comprises: determining, according to first odometer information collected by the inertial odometer at the current time and second odometer information at a previous time, a first pose offset of the robot between the previous time and the current time; determining, according to the first pose offset and second pose information of the robot in the world coordinate system at the previous time, the first pose information of the robot at the current time in the world coordinate system; determining, according to the first position coordinate, the first pitch angle and the first yaw angle, the first normal vector corresponding to the target body point of the robot.

3. The method of claim 1, wherein, the method of converting, according to the first pose information, point cloud data collected by the sensor to the world coordinate system to obtain target point cloud data corresponding to the current time comprises: obtaining original point cloud data corresponding to the current time collected by the sensor; performing clustering processing on the original point cloud data to obtain first point cloud data for which clustering is successful; extracting feature information in the first point cloud data, and extracting edge point cloud and target plane point cloud from the first point cloud data according to the feature information to obtain second point cloud data; converting, according to the first pose information, the second point cloud data from a sensor coordinate system of the sensor to the world coordinate system to obtain target point cloud data.

4. The method of claim 3, wherein, The converting the second point cloud data from the sensor coordinate system of the sensor to the world coordinate system according to the first pose information to obtain target point cloud data comprises: obtaining a relative position relationship between the target body point and the sensor in a robot coordinate system; determining third pose information of the sensor in the world coordinate system according to the relative position relationship and the first pose information; determining a conversion matrix according to the third pose information and fourth pose information of the sensor in the robot coordinate system; converting the second point cloud data from the sensor coordinate system of the sensor to the world coordinate system by using the conversion matrix to obtain target point cloud data.

5. The method according to any one of claims 1 to 4, characterized in that, The three-dimensional plane fitting according to the target point cloud data to obtain a second normal vector of the fitted plane comprises: obtaining a covariance matrix according to the target point cloud data and a least square function; performing singular value decomposition on the covariance matrix to obtain a singular vector corresponding to a minimum singular value; obtaining a second normal vector of the fitted plane according to the singular vector.

6. The method according to any one of claims 1 to 4, characterized in that, The slope detection result in the environment around the robot is obtained according to the included angle between the first normal vector and the second normal vector, comprising: when the absolute value of the included angle between the first normal vector and the second normal vector is greater than an included angle threshold, it is determined that there is a slope in the environment around the robot.

7. The method according to any one of claims 1 to 4, characterized in that, The sensor comprises one or more of a multi-line laser radar, a binocular camera and a millimeter wave radar.

8. A ramp detection device, characterized by The slope detection device is installed on a robot, the robot comprises an inertial odometer and a sensor for environment perception, and the device comprises: a first normal vector determination unit configured to determine first pose information of the robot in a world coordinate system at a current time and a first normal vector corresponding to a target body point of the robot according to odometer information collected by the inertial odometer; the first normal vector is used to indicate the attitude of the robot at the current time, the world coordinate system is an absolute coordinate system, the first pose information comprises a first position coordinate and an attitude, the first position coordinate is a position coordinate of the target body point, and the target body point comprises a body center point; the first normal vector passes through the body center point; a point cloud data processing unit configured to convert point cloud data collected by the sensor to the world coordinate system according to the first pose information to obtain target point cloud data corresponding to the current time; a second normal vector determination unit configured to perform three-dimensional plane fitting according to the target point cloud data to obtain a second normal vector of the fitted plane; a slope detection unit configured to obtain a slope detection result in the environment around the robot according to the included angle between the first normal vector and the second normal vector; The slope detection unit is configured to obtain the slope detection result in the environment around the robot according to the included angle between the first normal vector and the second normal vector in the following manner: The slope detection unit is configured to: when an absolute value of an included angle between the first normal vector and the second normal vector is less than or equal to an included angle threshold, acquire a third normal vector corresponding to the target body point at a previous time; and when an absolute value of an included angle between the third normal vector and the first normal vector is greater than the included angle threshold, determine that a slope exists in the surrounding environment of the robot.

9. A robot, characterized in that The method comprises: a memory storing executable program code; and a processor coupled to the memory; the processor invokes the executable program code stored in the memory, and the executable program code is executed by the processor to enable the processor to implement the method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon an executable program code, characterized in that, The executable program code is executed by the processor to implement the method according to any one of claims 1-7.

Images