Robot, obstacle detection method, device, computer equipment and storage medium

Abstract

The present disclosure provides a robot, obstacle detection method, apparatus, computer equipment, and storage medium. The robot includes: a walking mechanism, a depth camera, and a processor. The depth camera is used to acquire point cloud data of a target space and send the point cloud data to the processor. The processor is used to cluster the point cloud points in the point cloud data based on first attribute information of the point cloud points to form at least one point cloud group. The processor is used to determine the classification of each point cloud group from multiple preset classifications using second attribute information of each point cloud group in the at least one point cloud group. Based on the classification result of each point cloud group, the obstacle detection result corresponding to each point cloud group is determined, and the walking mechanism is controlled to move based on the obstacle detection result. The embodiments of the present disclosure can accurately sense obstacles in the environment during the operation of the robot, thereby improving the operating efficiency of the robot.

Description

Technical Field

[0001] The present disclosure relates to the field of machine vision technology, and in particular to a robot, an obstacle detection method and apparatus, a computer device, and a storage medium. Background Art

[0002] With the rapid development of computer software and hardware, and the increasing maturity of artificial intelligence technology, robots have attracted widespread attention as a key application of artificial intelligence. For example, in the field of smart warehousing, intelligent handling robots are widely used for handling and stacking items, gradually replacing human labor and becoming a key component of the smart warehousing chain.

[0003] To ensure the safety of both the robot and its surroundings, intelligent warehouse robots are typically equipped with devices such as lidar sensors, color cameras, and time-of-flight (TOF) depth cameras to perceive their surroundings. However, using TOF depth cameras to detect obstacles in three-dimensional space can lead to false detections, which can affect the robot's operational efficiency. Summary of the Invention

[0004] The embodiments of the present disclosure provide at least one robot, an obstacle detection method and apparatus, a computer device, and a storage medium.

[0005] In a first aspect, an embodiment of the present disclosure provides a robot, comprising:

[0006] Walking mechanism, depth camera and processor;

[0007] The depth camera is used to obtain point cloud data of the target space and send the point cloud data to the processor;

[0008] The processor is used to cluster the point cloud points in the point cloud data based on the first attribute information of the point cloud points in the point cloud data to form at least one point cloud group; use the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from multiple preset classifications; based on the classification result of each point cloud group, determine the obstacle detection result corresponding to each point cloud group, and control the walking mechanism to move based on the obstacle detection result.

[0009] In an optional implementation, the processor, when acquiring point cloud data of the target space, is configured to:

[0010] Collecting first original point cloud data using a depth camera;

[0011] Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

[0012] In an optional implementation, the processor, when acquiring point cloud data of the target space, is configured to:

[0013] Acquire the second original point cloud data of the target space;

[0014] The second original point cloud data is filtered to obtain point cloud data of the target space.

[0015] In an optional implementation manner, the first attribute information includes at least one of the following:

[0016] The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

[0017] In an optional embodiment, the spatial distance between different point cloud points includes at least one of the following:

[0018] Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

[0019] In an optional implementation, the processor is further configured to obtain the color distance between different point cloud points in the following manner:

[0020] Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data;

[0021] Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

[0022] In an optional implementation, the processor is further configured to obtain the normal vector distance between different point cloud points in the following manner:

[0023] Determine the normal vector corresponding to each point cloud point in the point cloud data;

[0024] The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

[0025] In an optional implementation, the processor is further configured to obtain the density distance between different point cloud points in the following manner:

[0026] Based on the point cloud data, an octree is established;

[0027] Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node;

[0028] Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

[0029] In an optional implementation, the processor is further configured to obtain the gradient distance between different point cloud points in the following manner:

[0030] Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

[0031] In an optional embodiment, the processor, when determining the classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group, is used to

[0032] For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group;

[0033] The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0034] In an optional implementation manner, the second attribute information includes:

[0035] At least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group.

[0036] In an optional implementation, the processor is further configured to obtain the normal vector transformation frequency of the point cloud group in the following manner:

[0037] Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction;

[0038] Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals;

[0039] A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

[0040] In an optional embodiment, after determining the classification of each point cloud group, the processor is further configured to: if the classification result of the first point cloud group in the at least one point cloud group indicates that the surface represented by the first point cloud group has a sudden curvature change,

[0041] The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

[0042] In an optional embodiment, the processor, before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group, is further configured to:

[0043] Update the normal vector of each point cloud point in the first point cloud group.

[0044] In an optional embodiment, the processor, when determining the obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group, is configured to:

[0045] If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle;

[0046] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, a convex connection detection is performed on each two third point cloud groups in the third point cloud groups; when the each two third point cloud groups have a convex connection relationship, the each two third point cloud groups are determined to be obstacles.

[0047] In a second aspect, an embodiment of the present disclosure provides an obstacle detection method, comprising:

[0048] Obtain point cloud data of the target space;

[0049] clustering the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group;

[0050] Determining a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group;

[0051] Based on the classification result of each point cloud group, an obstacle detection result corresponding to each point cloud group is determined.

[0052] In an optional implementation, obtaining point cloud data of the target space includes:

[0053] Collecting first original point cloud data using a depth camera;

[0054] Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

[0055] In an optional implementation, obtaining point cloud data of the target space includes:

[0056] Acquire the second original point cloud data of the target space;

[0057] The second original point cloud data is filtered to obtain point cloud data of the target space.

[0058] In an optional implementation manner, the first attribute information includes at least one of the following:

[0059] The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

[0060] In an optional embodiment, the spatial distance between different point cloud points includes at least one of the following:

[0061] Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

[0062] In an optional implementation, the color distance between different point cloud points is obtained in the following manner:

[0063] Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data;

[0064] Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

[0065] In an optional implementation, the normal vector distance between different point cloud points is obtained in the following manner:

[0066] Determine the normal vector corresponding to each point cloud point in the point cloud data;

[0067] The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

[0068] In an optional implementation, the density distance between different point cloud points is obtained in the following manner:

[0069] Based on the point cloud data, an octree is established;

[0070] Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node;

[0071] Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

[0072] In an optional implementation, the gradient distance between different point cloud points is obtained in the following manner:

[0073] Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

[0074] In an optional embodiment, using the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from a plurality of preset classifications includes:

[0075] For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group;

[0076] The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0077] In an optional implementation manner, the second attribute information includes:

[0078] At least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group.

[0079] In an optional implementation, the normal vector transformation frequency of the point cloud group is obtained in the following manner:

[0080] Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction;

[0081] Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals;

[0082] A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

[0083] In an optional embodiment, after determining the classification of each point cloud group, the method further includes: if the classification result of the first point cloud group in the at least one point cloud group indicates that the surface represented by the first point cloud group has a sudden curvature change,

[0084] The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

[0085] In an optional embodiment, before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group, the method further includes:

[0086] Update the normal vector of each point cloud point in the first point cloud group.

[0087] In an optional implementation, determining an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group includes:

[0088] If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle;

[0089] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, a convex connection detection is performed on each two third point cloud groups in the third point cloud groups; when the each two third point cloud groups have a convex connection relationship, the each two third point cloud groups are determined to be obstacles.

[0090] In a third aspect, an embodiment of the present disclosure further provides an obstacle detection device, comprising:

[0091] The acquisition module is used to obtain the point cloud data of the target space.

[0092] The clustering module is used to cluster the point cloud points in the point cloud data based on the first attribute information of the point cloud points in the point cloud data to form at least one point cloud group.

[0093] The classification module is used to use the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from multiple preset classifications.

[0094] A determination module is used to determine an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group.

[0095] In a possible implementation, when acquiring point cloud data of the target space, the acquisition module is used to:

[0096] Collecting first original point cloud data using a depth camera;

[0097] Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

[0098] In a possible implementation, when acquiring point cloud data of the target space, the acquisition module is used to:

[0099] Acquire the second original point cloud data of the target space;

[0100] The second original point cloud data is filtered to obtain point cloud data of the target space.

[0101] In one possible implementation, the first attribute information includes at least one of the following:

[0102] The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

[0103] In one possible implementation, the spatial distance between different point cloud points includes at least one of the following:

[0104] Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

[0105] In a possible implementation, the clustering module is configured to obtain the color distance between different point cloud points in the following manner:

[0106] Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data;

[0107] Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

[0108] In a possible implementation, the clustering module is configured to obtain the normal vector distances between different point cloud points in the following manner:

[0109] Determine the normal vector corresponding to each point cloud point in the point cloud data;

[0110] The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

[0111] In a possible implementation, the clustering module is configured to obtain the density distance between different point cloud points in the following manner:

[0112] Based on the point cloud data, an octree is established;

[0113] Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node;

[0114] Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

[0115] In one possible implementation, the clustering module is configured to obtain the gradient distance between different point cloud points in the following manner:

[0116] Determining depth values ​​corresponding to the different point cloud points based on position information of the different point cloud points in the target space;

[0117] Based on the depth values ​​corresponding to the different point cloud points, a gradient distance between the different point cloud points is determined.

[0118] In one possible implementation, the classification module, when determining the classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group, is configured to:

[0119] For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group;

[0120] The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0121] In a possible implementation manner, the second attribute information includes:

[0122] At least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group.

[0123] In a possible implementation, the classification module is configured to obtain the normal vector transformation frequency of the point cloud group in the following manner:

[0124] Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction;

[0125] Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals;

[0126] A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

[0127] In one possible implementation, after determining the classification of each point cloud group, the clustering module is further configured to: if the classification result of a first point cloud group in the at least one point cloud group indicates that a sudden curvature exists on the surface represented by the first point cloud group,

[0128] The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

[0129] In one possible implementation, the clustering module is further used to update the normal vector of each point cloud point in the first point cloud group before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group.

[0130] In one possible implementation, the determining module determines the obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group, including:

[0131] If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle;

[0132] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, convex connection detection is performed on every two third point cloud groups in the third point cloud groups; when the every two third point cloud groups have a convex connection relationship, the every two third point cloud groups are determined as obstacles.

[0133] In a fourth aspect, an optional implementation of the present disclosure further provides a computer device, a processor, and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory. When the machine-readable instructions are executed by the processor, the machine-readable instructions perform the steps of the above-mentioned second aspect, or any possible implementation of the second aspect.

[0134] In a fifth aspect, an optional implementation of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed, the steps of the above-mentioned second aspect or any possible implementation of the second aspect are executed.

[0135] For a description of the effects of the above-mentioned obstacle detection device, computer equipment, and computer-readable storage medium, please refer to the description of the above-mentioned obstacle detection method, which will not be repeated here.

[0136] The robot, obstacle detection method, apparatus, computer equipment, and storage medium provided by the embodiments of the present disclosure utilize the principle that point clouds belonging to the same object have similar characteristics and cannot undergo vector mutation to cluster the point clouds in the target space, thereby forming point cloud groups representing different objects or different parts of the same object. The robot then classifies the point cloud groups and utilizes the concavity and convexity between different point cloud groups to determine the obstacle detection results corresponding to each point cloud group. This eliminates the need to consider environmental changes, such as changes in ground shape or changes in the installation posture of the depth camera, that interfere with the detection results. This allows for accurate sensing of the environment during robot operation and improves the robot's operating efficiency.

[0137] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0138] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0139] Figure 1 A schematic diagram of a robot provided by an embodiment of the present disclosure is shown;

[0140] Figure 2 A flowchart of an obstacle detection method provided by an embodiment of the present disclosure is shown;

[0141] Figure 3 A schematic diagram of an obstacle detection device provided by an embodiment of the present disclosure is shown;

[0142] Figure 4 A schematic diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0143] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0144] Research has found that when using a TOF depth camera to detect obstacles in a three-dimensional space, the TOF depth camera is first used to obtain point cloud data in the three-dimensional space to be detected, and then the point cloud data in the three-dimensional space is ground fitted. After removing the ground point cloud, the point cloud data of the obstacles on the ground can be obtained. Then, the relative position of the obstacle and the robot is determined based on the point cloud data of the obstacles on the ground, and the robot is controlled to avoid obstacles based on this relative position. In this method, in order to remove the ground point cloud, the ground needs to be calibrated in advance. However, in actual use, the ground conditions in the warehouse are different. In some warehouses, the ground angle fluctuates, which makes it impossible to accurately calibrate the ground. In addition, as the running time increases, the connection structure between the various components on the robot will undergo a certain deformation due to the vibration during the movement of the robot, which causes the installation pitch angle of the TOF depth camera on the robot to change. After the installation pitch angle of the robot changes, the parameters of the calibration ground need to be adapted and adjusted. The calibrated ground before this can no longer accurately match the real ground, resulting in the inability to completely remove the ground point cloud data when fitting the ground data in the three-dimensional space based on the previous calibrated ground. Then, the robot will identify the point cloud belonging to the ground as an obstacle, causing incorrect obstacle avoidance and affecting the robot's operating efficiency.

[0145] Based on the above research, the present disclosure provides an obstacle detection method, device, computer equipment and storage medium. The point cloud of the target space is clustered by utilizing the principle that the characteristics of point clouds belonging to the same object are similar and the vector mutation of point cloud points is relatively small to form point cloud groups representing different objects or different parts of the same object. Then, the obstacle detection results corresponding to each point cloud group are determined by classifying the point cloud groups and utilizing the concavity and convexity between different point cloud groups. There is no need to consider the interference caused by environmental changes, such as changes in ground shape and changes in the installation posture of the depth camera, on the detection results. The method adapts to the precise sensing of the environment during the operation of the robot and improves the operation efficiency of the robot.

[0146] The defects in the above solutions are the results obtained by the inventors after practice and careful research. Therefore, the process of discovering the above problems and the solutions proposed by this disclosure for the above problems below should be the contributions made by the inventors to this disclosure during the disclosure process.

[0147] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0148] To facilitate understanding of this embodiment, we first provide a detailed introduction to an obstacle detection method disclosed in an embodiment of the present disclosure. The obstacle detection method provided in this embodiment is generally executed by a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing device. The terminal device may be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, or the like. Other devices may include, for example, robots and intelligent vehicles. In some possible implementations, the obstacle detection method may be implemented by a processor invoking computer-readable instructions stored in a memory.

[0149] The robot provided by the embodiment of the present disclosure is first described below.

[0150] See also Figure 1 , which is a schematic diagram of a robot provided by an embodiment of the present disclosure, including: a depth camera 10, a walking mechanism 20 and a processor 30;

[0151] The depth camera 10 is used to obtain point cloud data of the target space and send the point cloud data to the processor;

[0152] The processor 30 is used to cluster the point cloud points in the point cloud data based on the first attribute information of the point cloud points in the point cloud data to form at least one point cloud group; use the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from multiple preset classifications; based on the classification result of each point cloud group, determine the obstacle detection result corresponding to each point cloud group, and control the walking mechanism 20 to move based on the obstacle detection result.

[0153] The above-mentioned robot is described in detail below.

[0154] The depth camera 10 is installed at a preset position of the robot and can collect data of the target space by using, for example, binocular, structured light, or time of flight (TOF) methods.

[0155] The walking mechanism 20, including, for example, a frame, front axle, rear axle, suspension system, and wheels or tracks, is used to support the robot's body and other components and to enable the robot to move within a target space based on the actual walking task. The walking mechanism can be controlled by a controller.

[0156] When performing obstacle detection based on point cloud data, the processor 30 may be, for example, a single-chip microcomputer, a digital signal processing (DSP) chip, or a field programmable gate array (FPGA) chip. The processor may calculate and process the point cloud data acquired by the depth camera to obtain obstacle detection results.

[0157] The target space is, for example, a space of a set size and a set position relative to the robot in the direction of movement of the robot. The spatial range information of the target space is determined by the radial sensing depth and radial sensing range of the robot's sensing device in the direction of movement; the specific size of the space and the position of the space relative to the robot can be specifically set according to the actual sensing range and detection needs.

[0158] After acquiring the point cloud data of the target space, the depth camera 10 sends the point cloud data to the processor 30 .

[0159] For example, when the depth camera 10 is used to acquire point cloud data of the target space, the embodiment of the present disclosure:

[0160] (1): In one possible embodiment, a depth camera 10 may be used to collect first raw point cloud data. The first raw point cloud data includes point cloud data of a target space. In this case, the detection range of the depth camera 10 is greater than or equal to the range of the target space. Then, based on the position information of the target space, the point cloud data of the target space is determined from the first raw point cloud data.

[0161] In this case, the position information of the target space includes, for example, the three-dimensional coordinate value of at least one reference point of the target space in the camera three-dimensional coordinate system established based on the depth camera 10. Then, based on the three-dimensional coordinate value of the reference point of the target space in the camera three-dimensional coordinate system, point cloud points located in the target space are determined from the first original point cloud data, and then based on the point cloud points located in the target space, intermediate point cloud data are determined, and point cloud data of the target space is obtained based on the intermediate point cloud data.

[0162] The reference points of the target space vary depending on the shape of the target space. For example, if the target space is a cube, the reference points may include, for example, the center point of the target space; in this case, the target space's position information may also include, for example, the side length of the cube. For another example, if the reference point is any vertex of the cube, the target space's position information may also include, for example, the coordinate offsets of other vertices adjacent to that vertex relative to that vertex. If the cross-section of the target space is a sector, the reference points of the target space may include, for example, the vertex and radius of the sector.

[0163] (2): In a possible implementation, the space that can be detected by the depth camera 10 can be used as the target space. In this case, the first original point cloud data collected by the depth camera 10 can be directly determined as the point cloud data of the target space.

[0164] In addition, the first original point cloud data obtained by the depth camera 10 detecting the space may also include outliers caused by noise.

[0165] Therefore, it is also possible to obtain second original point cloud data of the target space, and perform filtering processing on the second original point cloud data to obtain point cloud data of the target space.

[0166] At this time, the second point cloud data of the target space may be, for example, the intermediate point cloud data obtained in the above (1), or the first original point cloud data in the above (2).

[0167] After filtering the second original point cloud data, outliers caused by noise in the second original point cloud data can be removed, thereby avoiding interference caused by the outliers on the result of clustering the point cloud points.

[0168] The first attribute information of the point cloud point includes, but is not limited to, one or more of the following a1 to a5:

[0169] a1: The spatial distance between different point cloud points.

[0170] Here, the spatial distance between different point cloud points includes, for example, at least one of the following: Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance. Generally, the smaller the spatial distance between different point cloud points, the greater the probability that the two point cloud points belong to the same object.

[0171] When determining the spatial distance between different point cloud points, the processor 30 can, for example, obtain the coordinate values ​​of each point cloud point in the camera's three-dimensional coordinate system; then calculate the spatial distance between different point cloud points based on the coordinate values ​​of different point cloud points in the camera's three-dimensional coordinate system.

[0172] a2: Color distance between different point cloud points.

[0173] Here, the color distance between different point cloud points is, for example, the color difference between different objects represented by different point cloud points at corresponding spatial points. Generally, the smaller the color difference between point cloud points, the greater the probability that they belong to the same object.

[0174] Specifically, the processor 30 may obtain the color distance between different point cloud points in the following manner, for example:

[0175] Based on the two-dimensional image of the target space and the mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data, the pixel value of each point cloud point is determined; based on the pixel values ​​corresponding to different point cloud points, the color distance between the different point cloud points is determined.

[0176] Here, when the processor 30 acquires the point cloud data of the target space, it can also synchronously acquire a two-dimensional image of the target space; when the positions of the depth camera 10 and the color camera that acquires the two-dimensional image are relatively fixed, each spatial point in the target space and each pixel point in the two-dimensional image have a fixed mapping relationship. Therefore, after determining the point cloud data of the target space, the processor 30 can determine the mapping relationship between each point cloud point in the point cloud data and each pixel point in the two-dimensional image. Based on this mapping relationship and the pixel value of each pixel point in the two-dimensional image, the pixel value of each point cloud point can be determined. Then, based on the pixel values ​​of different point cloud points, the processor 30 determines the color distance between different point cloud points.

[0177] The two-dimensional image of the target space is, for example, an image obtained by a color camera, such as a grayscale image, an RGB image, or an HSB image.

[0178] For example, when the two-dimensional image is a grayscale image, the pixel value of each pixel in the two-dimensional image is the grayscale value of the pixel; the pixel values ​​of different point cloud points are also the grayscale values ​​corresponding to different point cloud points; the difference between the grayscale values ​​corresponding to different point cloud points is determined as the color distance between different point cloud points.

[0179] When the two-dimensional image is an RGB image, the pixel value of any pixel in the two-dimensional image includes the color value under the red (Red, R) channel, the color value under the green (Green, G) channel, and the color value under the blue (Blue, B) channel; the pixel value of the corresponding point cloud point can be expressed as (r, g, b); r, g, and b represent the color values ​​under the R channel, G channel, and B channel, respectively.

[0180] The pixel values ​​of two point cloud points are represented as (r1, g1, b1) and (r2, g2, b2), respectively. The color distance L between the two point cloud points is, for example:

[0181] In the case of an HSB 2D image, the pixel value of any pixel in the 2D image includes a hue (H), a saturation (S), and a brightness (B). That is, the color of the point cloud corresponding to the pixel can also be expressed as: hue value, saturation value, and brightness value. The processor 30 then obtains the color distance between different point cloud points based on the hue value, saturation value, and brightness value corresponding to each point cloud point.

[0182] a3: Normal vector distance between different point cloud points.

[0183] The normal vector distance between different point cloud points can be represented by the angle between the normal vectors corresponding to different point cloud points. Generally, the smaller the angle between the normal vectors of different point cloud points, the greater the probability that the two point cloud points belong to the same object.

[0184] In a specific implementation, for example, the normal vector corresponding to each point cloud point in the point cloud data can be first determined, and then the first angle between the normal vectors corresponding to the different point cloud points can be determined as the normal vector distance between the different point cloud points.

[0185] Here, the processor 30 obtains the normal vector of any point in the point cloud data, for example, by the following method:

[0186] Based on the point cloud data of the target space, any point cloud point in the target space is taken as a first vertex, and other point cloud points whose distance between the target space and the first vertex is less than a preset first distance threshold are taken as second vertices, and at least one polygonal face is formed based on the first vertex and at least two second vertices; and the normal vector of each polygonal face in the at least one polygonal face formed is determined.

[0187] The normal vector of the point cloud point corresponding to the first vertex is determined by taking the average of the normal vectors corresponding to at least one polygonal facet.

[0188] After determining the normal vectors corresponding to different point cloud points, since the normal vector can represent the angle of the normal vector in space, the first angle between the normal vectors corresponding to different point cloud points, that is, the normal vector distance between different point cloud points, is determined based on the normal vectors corresponding to different point cloud points.

[0189] a4: Density distance between different point cloud points.

[0190] Generally speaking, the closer the density distance between different point cloud points is, the greater the probability that the different point cloud points belong to the same object.

[0191] The processor 30 may obtain the density distance between different point cloud points in any of the following two ways, for example:

[0192] First, for any point in the point cloud data, a subspace of a certain size is defined with that point as the center. Based on the point cloud data, the number of points within this subspace is determined. The number of points in the subspace is then used as the density corresponding to that point. For different points, the difference in density corresponding to each point is calculated to obtain the density distance between them.

[0193] Second: based on the point cloud data, establish an octree; based on the octree, determine the density value of the target leaf node where each point cloud point is located in the octree; wherein, the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node; based on the density values ​​corresponding to the different point cloud points, determine the density distance between the different point cloud points.

[0194] At this time, the difference in density values ​​corresponding to different point cloud points can be determined as the density distance between different point cloud points.

[0195] a5: Gradient distance between different point cloud points.

[0196] The gradient of a point cloud point includes, for example, the depth value of the point cloud point in the camera's three-dimensional coordinate system. Generally, the closer the gradient distance between different point cloud points is, the greater the probability that the different point cloud points belong to the same object.

[0197] In a specific implementation, the processor 30 may, for example, obtain the gradient distance between different point cloud points in the following manner:

[0198] Based on the position information of the different point cloud points in the target space, the gradient value of each point cloud point is determined, and then based on the gradient values ​​corresponding to the different point cloud points, the gradient distance between the different point cloud points is calculated.

[0199] In a specific implementation, the gradient value of each point cloud point, for example, is to calculate the gradients s1 and s2 in the x-direction and y-direction for each point cloud point according to its corresponding domain point cloud point; then calculate the square root of the sum of the squares of s1 and s2 to obtain the gradient value of the point cloud point.

[0200] After determining the first attribute information of each point cloud point in the point cloud data, the processor 30 can cluster the point cloud points in the point cloud data based on the first attribute information of each point cloud point.

[0201] The clustering method can adopt at least one of the following, for example: partitioning method (such as K-MEANS algorithm, K-MEDOIDS algorithm, or CLARANS algorithm), hierarchical method (BIRCH algorithm, CURE algorithm, or CHAMELEON algorithm), density-based method (DBSCAN algorithm, OPTICS algorithm, DENCLUE algorithm), grid-based method (STING algorithm, CLIQUE algorithm, WAVE-CLUSTER algorithm), model-based method, etc., and the specific selection can be made according to actual needs.

[0202] Exemplarily, when clustering based on the first attribute information of the point cloud points, the processor 30 can, for example, determine the difference between different first point cloud points based on the corresponding first attribute information of different point cloud points, and then cluster the point cloud points based on the difference.

[0203] The difference Diff satisfies the following formula (1):

[0204] Diff=W dis *D dis +W c *D c +W n *D n +W g *D g +W grad *D grad (1)

[0205] Among them, W i ,i∈[dis,c,n,g,grad] represents the weight.

[0206] D dis Indicates the spatial distance between different point cloud points; D c Indicates the color distance between different point cloud points; D n Represents the normal vector distance between different point cloud points; D g Indicates the density distance between different point cloud points; D grad Represents the gradient distance between different point cloud points.

[0207] Through the above process, at least one point cloud group is formed.

[0208] When the processor 30 classifies the point cloud groups based on the second attribute information of each point cloud group, for example, the following method may be used:

[0209] For each point cloud group, feature data used to characterize the point cloud group is determined based on the second attribute information of the point cloud points in each point cloud group; and the feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0210] Here, the pre-trained classifier is, for example, any one of a logistic regression classifier, a support vector machine classifier, a classifier using activation functions such as softmax and sigmoid, and a random forest, and can be specifically set according to actual needs.

[0211] The classifier can be trained using pre-labeled samples, for example, and can classify the point cloud group and determine the category to which the point cloud group belongs among multiple preset categories.

[0212] The second attribute information of the point cloud group includes, but is not limited to, at least one of the following b1 to b3:

[0213] b1: The normal vector transformation frequency of the point cloud group.

[0214] In a specific implementation, the smaller the frequency of normal vector transformation of the point cloud group, the smaller the change in curvature of the surface represented by the point cloud group, and the higher the probability that the point cloud group belongs to the ground.

[0215] Specifically, the processor 30 may determine the normal vector transformation frequency of the point cloud group in the following manner, for example:

[0216] Determine the normal vector of each first point cloud point among the multiple first point cloud points of the point cloud group, and determine the second angle between the normal vector of each first point cloud point and the preset direction; based on the second angle corresponding to each first point cloud point, determine the target angle interval corresponding to the first point cloud point from multiple preset angle intervals; based on the number of target angle intervals corresponding to the multiple first point cloud points, determine the normal vector transformation frequency of the point cloud group.

[0217] In a specific implementation, the normal vector of the first point cloud point can represent its direction in the camera's 3D coordinate system; that is, the angle between the normal vector and each axis in the camera's 3D coordinate system. The preset direction is, for example, the direction pointed by at least one axis in the camera's 3D coordinate system. Once the preset direction is determined, the second angle between the normal vector and the preset direction can be determined.

[0218] Then, the target angle interval to which the second angle corresponding to each of the first point cloud points of the point cloud group belongs is determined, and the number of target angle intervals corresponding to the first point cloud points is counted, and the number is determined as the normal vector transformation frequency of the point cloud group.

[0219] The greater the number of target angle intervals corresponding to the multiple first point cloud points, the higher the frequency of normal vector transformation representing the corresponding point cloud group, the greater the change in curvature of the corresponding surface, the smaller the probability of belonging to the ground, and the greater the probability of the corresponding obstacle on the ground.

[0220] b2: density information of the point cloud group.

[0221] Here, the density information of the point cloud group includes, for example, density values ​​corresponding to a plurality of first point cloud points in the point cloud group.

[0222] The density value corresponding to the first point cloud point can be obtained by the processor 30, for example, by the method of determining the density of the point cloud point in the above a4, which will not be repeated here.

[0223] b3: Texture information of the point cloud group.

[0224] The texture information of the point cloud group, for example, includes pixel values ​​corresponding to each of the first point cloud points in the point cloud group.

[0225] The pixel value of the first point cloud point can be obtained by the processor 30, for example, by the method of determining the pixel value of the point cloud point in the above a2, which will not be repeated here.

[0226] After determining the second attribute information of the point cloud group, the processor 30 constructs feature data of the point cloud group based on the second attribute information.

[0227] For example, for a certain point cloud group, the number of the first point cloud points in the point cloud group is n, the normal vector transformation frequency corresponding to the point cloud group is a1, and the density information of the point cloud group is: (b1, b2, ..., b n ), the pixel value of the first point cloud point in the point cloud group is grayscale value, and the corresponding texture information is: (c1, c2, ..., c n ); then the characteristic data can be, for example: (a1, b1, b2, ..., b n ,c1,c2,…,c n ).

[0228] The feature data is input into the classifier to obtain the classification corresponding to the point cloud group.

[0229] In the embodiments of the present disclosure, a specific example of multiple classification categories is also provided, including:

[0230] Category 1: Flat surfaces, such as point cloud groups representing the ground, textbooks, walls, etc.

[0231] Category 2: Surfaces with stable curvature transformations, such as point cloud groups representing cups, mice, shelves, and other robots.

[0232] Category 3: Surfaces with abrupt changes in curvature, such as a point cloud representing the base of a cup containing the ground.

[0233] When the category corresponding to a point cloud group is Classification 3, the point clouds corresponding to the ground and obstacles are not completely distinguished. Therefore, in this case, that is, when the classification result of the first point cloud group in the point cloud group indicates that the surface represented by the point cloud group has a sudden change in curvature, the first point cloud group can be used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data can be clustered to form at least one new point cloud group.

[0234] Here, it should be noted that before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group, the processor 30 will also update the normal vector of each point cloud point in the first point cloud group.

[0235] When updating the normal vectors of each point cloud point in the first point cloud group, the processor 30 sets a second distance threshold smaller than the first distance threshold for each point cloud point in the first point cloud group, and re-determines the normal vectors for each point cloud point in the first point cloud group based on the second distance threshold. This allows for finer-grained clustering of each point cloud point in the first point cloud group, making it easier to group point cloud points representing different objects into different new point cloud groups.

[0236] Then, for the new point cloud group, the processor 30 uses the second attribute information of the new point cloud group to classify the new point cloud group again until there is no point cloud group of category 3 in the obtained point cloud group, or the classification result no longer changes.

[0237] When the processor 30 determines the obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group,

[0238] When the classification result of the second point cloud group in the at least one point cloud group indicates that the curvature of the surface represented by the second point cloud group has a stable change, for the second point cloud group with the classification result of category 2, the object represented by the second point cloud group is determined to be an obstacle.

[0239] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, that is, for the third point cloud group whose classification result is classification 1, convex connection detection is performed on every two third point cloud groups in the third point cloud group; when every two third point cloud groups have a convex connection relationship, every two third point cloud groups are determined to be obstacles.

[0240] Here, for example, convex connection detection may be performed on every two third point cloud groups based on a Sanity criterion (SC) algorithm.

[0241] Through this process, the point cloud group representing a flat surface such as a wall and the point cloud group representing a flat surface such as the ground can be distinguished from the point cloud group in the third point cloud group classified as 1.

[0242] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0243] Based on the same inventive concept, an obstacle detection method corresponding to the robot is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the method in the embodiment of the present disclosure is similar to that of the above-mentioned robot in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0244] See also Figure 2 FIG. 2 is a flow chart of an obstacle detection method provided by an embodiment of the present disclosure, wherein the method includes steps S201 to S204, wherein:

[0245] S201: Acquire point cloud data of the target space;

[0246] S202: clustering the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group;

[0247] S203: Determine a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group;

[0248] S204: Determine an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group.

[0249] The disclosed embodiment utilizes the first attribute information of the point cloud points in the point cloud data of the target space to cluster the point cloud points in the point cloud data to form at least one point cloud group, and utilizes the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from a plurality of preset classifications, and determines the detection result of the obstacle corresponding to each point cloud group based on the classification result of each point cloud group. This embodiment utilizes the similarity between the point cloud points to realize the classification of point clouds representing different objects or different parts of the same object, without considering the interference of environmental changes, such as changes in the shape of the ground, changes in the installation posture of the depth camera, etc. on the obstacle detection results, adapts to the precise sensing of the environment during the operation of the robot, and improves the operation efficiency of the robot.

[0250] The obstacle detection method provided by the embodiment of the present disclosure has been described in detail in the description of the above-mentioned robot and will not be repeated here.

[0251] In an optional implementation, obtaining point cloud data of the target space includes:

[0252] Collecting first original point cloud data using a depth camera;

[0253] Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

[0254] In an optional implementation, obtaining point cloud data of the target space includes:

[0255] Acquire the second original point cloud data of the target space;

[0256] The second original point cloud data is filtered to obtain point cloud data of the target space.

[0257] In an optional implementation manner, the first attribute information includes at least one of the following:

[0258] The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

[0259] In an optional embodiment, the spatial distance between different point cloud points includes at least one of the following:

[0260] Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

[0261] In an optional implementation, the color distance between different point cloud points is obtained in the following manner:

[0262] Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data;

[0263] Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

[0264] In an optional implementation, the normal vector distance between different point cloud points is obtained in the following manner:

[0265] Determine the normal vector corresponding to each point cloud point in the point cloud data;

[0266] The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

[0267] In an optional implementation, the density distance between different point cloud points is obtained in the following manner:

[0268] Based on the point cloud data, an octree is established;

[0269] Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node;

[0270] Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

[0271] In an optional implementation, the gradient distance between different point cloud points is obtained in the following manner:

[0272] Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

[0273] In an optional embodiment, using the second attribute information of each point cloud group in the at least one point cloud group to determine the classification of each point cloud group from a plurality of preset classifications includes:

[0274] For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group;

[0275] The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0276] In an optional implementation manner, the second attribute information includes:

[0277] At least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group.

[0278] In an optional implementation, the normal vector transformation frequency of the point cloud group is obtained in the following manner:

[0279] Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction;

[0280] Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals;

[0281] A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

[0282] In an optional embodiment, after determining the classification of each point cloud group, the method further includes: if the classification result of the first point cloud group in the at least one point cloud group indicates that the surface represented by the first point cloud group has a sudden curvature change,

[0283] The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

[0284] In an optional embodiment, before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group, the method further includes:

[0285] Update the normal vector of each point cloud point in the first point cloud group.

[0286] In an optional implementation, determining an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group includes:

[0287] If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle;

[0288] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, a convex connection detection is performed on each two third point cloud groups in the third point cloud groups; when the each two third point cloud groups have a convex connection relationship, the each two third point cloud groups are determined to be obstacles.

[0289] Reference Figure 3 FIG. 1 is a schematic diagram of an obstacle detection device provided by an embodiment of the present disclosure, wherein the device includes: an acquisition module, a clustering module, a classification module, and a determination module; wherein,

[0290] The acquisition module 31 is used to acquire point cloud data of the target space.

[0291] The clustering module 32 is configured to cluster the point cloud points in the point cloud data based on the first attribute information of the point cloud points in the point cloud data to form at least one point cloud group.

[0292] The classification module 33 is configured to determine a classification of each point cloud group from a plurality of preset classifications by using the second attribute information of each point cloud group in the at least one point cloud group.

[0293] The determination module 34 is configured to determine an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group.

[0294] In a possible implementation, when acquiring point cloud data of the target space, the acquisition module 31 is used to:

[0295] Collecting first original point cloud data using a depth camera;

[0296] Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

[0297] In a possible implementation, when acquiring the point cloud data of the target space, the acquisition module 31 is configured to:

[0298] Acquire the second original point cloud data of the target space;

[0299] The second original point cloud data is filtered to obtain point cloud data of the target space.

[0300] In one possible implementation, the first attribute information includes at least one of the following:

[0301] The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

[0302] In one possible implementation, the spatial distance between different point cloud points includes at least one of the following:

[0303] Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

[0304] In a possible implementation, the clustering module 32 is configured to obtain the color distance between different point cloud points in the following manner:

[0305] Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data;

[0306] Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

[0307] In a possible implementation, the clustering module 32 is configured to obtain the normal vector distances between different point cloud points in the following manner:

[0308] Determine the normal vector corresponding to each point cloud point in the point cloud data;

[0309] The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

[0310] In a possible implementation, the clustering module 32 is configured to obtain the density distance between different point cloud points in the following manner:

[0311] Based on the point cloud data, an octree is established;

[0312] Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node;

[0313] Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

[0314] In a possible implementation, the clustering module 32 is configured to obtain the gradient distance between different point cloud points in the following manner:

[0315] Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

[0316] In one possible implementation, the classification module 33, when determining the classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group, is configured to:

[0317] For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group;

[0318] The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

[0319] In a possible implementation manner, the second attribute information includes:

[0320] At least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group.

[0321] In a possible implementation, the classification module 33 is configured to obtain the normal vector transformation frequency of the point cloud group in the following manner:

[0322] Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction;

[0323] Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals;

[0324] A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

[0325] In one possible implementation, after determining the classification of each point cloud group, the clustering module 32 is further configured to: if the classification result of a first point cloud group in the at least one point cloud group indicates that the surface represented by the first point cloud group has a sudden curvature change,

[0326] The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

[0327] In a possible implementation, the clustering module 32 is further used to update the normal vector of each point cloud point in the first point cloud group before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group.

[0328] In a possible implementation, the determining module 34 determines the obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group, including:

[0329] If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle;

[0330] When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, convex connection detection is performed on every two third point cloud groups in the third point cloud groups; when the every two third point cloud groups have a convex connection relationship, the every two third point cloud groups are determined as obstacles.

[0331] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference can be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0332] The present disclosure also provides a computer device, such as Figure 4 FIG. 1 is a schematic diagram of a computer device structure provided by an embodiment of the present disclosure, including:

[0333] Processor 41 and memory 42; the memory 42 stores machine-readable instructions executable by the processor 41, and the processor 41 is configured to execute the machine-readable instructions stored in the memory 42. When the machine-readable instructions are executed by the processor 41, the processor 41 performs the following steps:

[0334] Obtain point cloud data of the target space;

[0335] clustering the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group;

[0336] Determining a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group;

[0337] Based on the classification result of each point cloud group, an obstacle detection result corresponding to each point cloud group is determined.

[0338] The above-mentioned memory 42 includes internal memory 421 and external memory 422; the memory 421 here is also called internal memory, which is used to temporarily store the calculation data in the processor 41 and the data exchanged with the external memory 422 such as the hard disk. The processor 41 exchanges data with the external memory 422 through the memory 421.

[0339] The specific execution process of the above instructions can refer to the steps of the obstacle detection method described in the embodiment of the present disclosure, which will not be repeated here.

[0340] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the obstacle detection method described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0341] The computer program product of the obstacle detection method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the steps of the obstacle detection method described in the above method embodiments. For details, please refer to the above method embodiments and will not be repeated here.

[0342] The present disclosure also provides a computer program that, when executed by a processor, implements any of the methods of the aforementioned embodiments. The computer program product can be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK).

[0343] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems and devices described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0344] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0345] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0346] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0347] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A robot, characterized in that: include: Walking mechanism, depth camera and processor; The depth camera is used to obtain point cloud data of the target space and send the point cloud data to the processor; The processor is configured to cluster the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group; Determining a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group; determining an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group, and controlling the walking mechanism to move based on the obstacle detection result; After determining the classification of each point cloud group, the processor is further configured to: if the classification result of a first point cloud group in the at least one point cloud group indicates that a sudden curvature exists on the surface represented by the first point cloud group, The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

2. The robot according to claim 1, characterized in that The processor, when acquiring point cloud data of the target space, is used to: Collecting first original point cloud data using a depth camera; Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

3. The robot according to claim 1 or 2, characterized in that: The processor, when acquiring point cloud data of the target space, is used to: Acquire the second original point cloud data of the target space; The second original point cloud data is filtered to obtain point cloud data of the target space.

4. The robot according to claim 1, characterized in that The first attribute information includes at least one of the following: The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

5. The robot according to claim 4, characterized in that The spatial distance between different point cloud points includes at least one of the following: Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

6. The robot according to claim 4, characterized in that The processor is further configured to obtain the color distance between different point cloud points in the following manner: Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data; Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

7. The robot according to claim 4, characterized in that The processor is further configured to obtain the normal vector distance between different point cloud points in the following manner: Determine the normal vector corresponding to each point cloud point in the point cloud data; The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

8. The robot according to claim 4, characterized in that The processor is further configured to obtain the density distance between different point cloud points in the following manner: Based on the point cloud data, an octree is established; Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node; Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

9. The robot according to claim 4, characterized in that The processor is further configured to obtain the gradient distance between different point cloud points in the following manner: Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

10. The obstacle detection robot according to claim 1, characterized in that: The processor, when determining the classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group, is used to For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group; The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

11. The obstacle detection robot according to claim 10, characterized in that: The second attribute information includes: at least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group; The normal vector transformation frequency represents the degree of change in curvature of the surfaces of different point cloud groups.

12. The obstacle detection robot according to claim 11, characterized in that: The processor is further configured to obtain the normal vector transformation frequency of the point cloud group in the following manner: Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction; Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals; A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

13. The robot according to claim 1, wherein: The processor is further configured to, before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group: Update the normal vector of each point cloud point in the first point cloud group.

14. The obstacle detection robot according to claim 1, characterized in that: The processor is configured to, when determining the obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group,: If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle; When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, a convex connection detection is performed on each two third point cloud groups in the third point cloud groups; when the each two third point cloud groups have a convex connection relationship, the each two third point cloud groups are determined to be obstacles.

15. An obstacle detection method, characterized in that: include: Obtain point cloud data of the target space; clustering the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group; Determining a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group; Determining an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group; After determining the classification of each point cloud group, the method further includes: if the classification result of the first point cloud group in the at least one point cloud group indicates that the surface represented by the first point cloud group has a sudden curvature change, The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

16. The method according to claim 15, characterized in that The step of obtaining point cloud data of the target space includes: Collecting first original point cloud data using a depth camera; Based on the spatial range information of the target space, point cloud data of the target space is determined from the first original point cloud data.

17. The method according to claim 15 or 16, characterized in that The step of obtaining point cloud data of the target space includes: Acquire the second original point cloud data of the target space; The second original point cloud data is filtered to obtain point cloud data of the target space.

18. The method according to claim 17, characterized in that The first attribute information includes at least one of the following: The spatial distance between different point cloud points, the color distance between different point cloud points, the normal vector distance between different point cloud points, the density distance between different point cloud points, and the gradient distance between different point cloud points.

19. The method according to claim 18, characterized in that The spatial distance between different point cloud points includes at least one of the following: Euclidean distance, Mahalanobis distance, Hamming distance, and Manhattan distance.

20. The method according to claim 18, wherein The color distance between different point cloud points is obtained in the following way: Determining a pixel value of each point cloud point based on the two-dimensional image of the target space and a mapping relationship between each pixel point in the two-dimensional image and each point cloud point in the point cloud data; Based on the pixel values ​​corresponding to the different point cloud points, the color distance between the different point cloud points is determined.

21. The method according to claim 18, wherein The normal vector distance between different point cloud points is obtained in the following way: Determine the normal vector corresponding to each point cloud point in the point cloud data; The first angle between the normal vectors corresponding to the different point cloud points is determined as the normal vector distance between the different point cloud points.

22. The method according to claim 18, wherein The density distance between different point cloud points is obtained in the following way: Based on the point cloud data, an octree is established; Based on the octree, determining a density value of a target leaf node where each point cloud point is located in the octree; wherein the density value of the target leaf node where the point cloud point is located in the octree is the number of leaf nodes owned by the parent node of the target leaf node; Based on the density values ​​corresponding to the different point cloud points, the density distance between the different point cloud points is determined.

23. The method according to claim 18, wherein The gradient distance between different point cloud points is obtained in the following way: Based on the position information of the different point cloud points in the target space, the gradient distance between the different point cloud points is determined.

24. The obstacle detection method according to claim 15, characterized in that: Determining a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group includes: For each point cloud group, determining feature data for characterizing the point cloud group according to second attribute information of the point cloud points in each point cloud group; The feature data is classified using a pre-trained classifier to obtain a classification corresponding to each point cloud group.

25. The obstacle detection method according to claim 24, characterized in that: The second attribute information includes: at least one of a normal vector transformation frequency of the point cloud group, density information of the point cloud group, and texture information of the point cloud group; The normal vector transformation frequency represents the degree of change in curvature of the surfaces of different point cloud groups.

26. The obstacle detection method according to claim 25, characterized in that: The normal vector transformation frequency of the point cloud group is obtained in the following way: Determining a normal vector of each first point cloud point in the plurality of first point cloud points of the point cloud group, and determining a second angle between the normal vector of each first point cloud point and a preset direction; Based on the second angle corresponding to each first point cloud point, determining a target angle interval corresponding to the first point cloud point from a plurality of preset angle intervals; A normal vector transformation frequency of the point cloud group is determined based on the number of target angle intervals corresponding to the plurality of first point cloud points.

27. The method according to claim 15, wherein Before clustering the point cloud points in the new point cloud data based on the first attribute information of the point cloud points in the new point cloud group, the method further includes: Update the normal vector of each point cloud point in the first point cloud group.

28. The obstacle detection method according to claim 15, characterized in that: Determining an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group includes: If the classification result of a second point cloud group in the at least one point cloud group indicates that a surface represented by the second point cloud group has a stable change in curvature, determining the object represented by the second point cloud group as an obstacle; When the classification result of the third point cloud group in the at least one point cloud group indicates that the surface represented by the third point cloud group has a gentle curvature, a convex connection detection is performed on each two third point cloud groups in the third point cloud groups; when the each two third point cloud groups have a convex connection relationship, the each two third point cloud groups are determined to be obstacles.

29. An obstacle detection device, characterized in that: include: Acquisition module, used to obtain point cloud data of the target space; a clustering module, configured to cluster the point cloud points in the point cloud data based on first attribute information of the point cloud points in the point cloud data to form at least one point cloud group; a classification module, configured to determine a classification of each point cloud group from a plurality of preset classifications using the second attribute information of each point cloud group in the at least one point cloud group; a determination module, configured to determine an obstacle detection result corresponding to each point cloud group based on the classification result of each point cloud group; The clustering module, after determining the classification of each point cloud group, is further configured to: if the classification result of a first point cloud group in the at least one point cloud group indicates that a sudden curvature exists on the surface represented by the first point cloud group, The first point cloud group is used as new point cloud data, and based on the first attribute information of the point cloud points in the new point cloud group, the point cloud points in the new point cloud data are clustered to form at least one new point cloud group.

30. A computer device, characterized in that: include: A processor and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the processor is used to execute the machine-readable instructions stored in the memory, and when the machine-readable instructions are executed by the processor, the obstacle detection method according to any one of claims 15 to 28 is executed by the processor.

31. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed, executes the obstacle detection method according to any one of claims 15 to 28.

Images