An obstacle recognition method, device, and robot based on deep point cloud data

Abstract

This invention provides an obstacle recognition method based on depth point cloud data, comprising: S1, acquiring depth point cloud data from a structured light camera; S2, converting the depth point cloud data to a robot coordinate system, and acquiring protruding obstacles and concave obstacles based on the maximum threshold height of the highest protruding obstacle and the minimum threshold height of the lowest concave obstacle, respectively; S3, acquiring initial ground points, and acquiring the intercept of the ground equation, as well as the upper and lower thresholds of the obstacles segmented from the point cloud, based on the initial ground point cloud data and the dot product of the normal vector, and acquiring the type of each point in the initial ground point cloud data; S4, using the protruding and concave obstacles acquired in step S2 and step S3 as the protruding and concave obstacles of the entire point cloud. The method of this invention can accurately acquire obstacles in the point cloud and reduce computational load.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically, to an obstacle recognition method, apparatus, and robot based on depth point cloud data. Background Technology

[0002] During autonomous navigation, robots typically perform path planning tasks from the current point to the target point. This requires them to perceive information about the surrounding environment and select a safe and passable path.

[0003] Robots are typically equipped with sensors such as LiDAR (generally, robots use LiDAR with no more than 32 lines, as higher line count LiDAR is more expensive), IMU, and speed encoders. However, mechanical LiDAR has a relatively large vertical angular resolution, and the line count is sparse when it is below 128 lines. It is mostly used for mapping or localization, and due to blind spots caused by the installation angle, it has few or no scanning points for small objects, making it unsuitable for detecting relatively small obstacles. However, in most scenarios where robots operate, such as parks, factories, and high-speed rail stations, there may be some low obstacles on the robot's patrol route that LiDAR cannot detect. In these cases, other sensors are needed to detect and avoid these obstacles.

[0004] Currently, there are three main types of sensors used by robots to detect obstacles:

[0005] 1. Ultrasonic Sensors: Ultrasonic sensors measure the distance to obstacles in front of them based on the principle of ultrasonic ranging. They are generally single-point measurements. However, ultrasonic sensors are limited by the characteristics of sound waves. Besides being limited to single-point measurements, their ranging distance is small, and they have a divergence angle; the larger the distance, the larger the divergence angle, making it impossible to obtain detailed environmental data. Furthermore, to cover the robot's blind spots, they need to be arranged in an array, which leads to interference between the ultrasonic waves and increases the robot's cost.

[0006] 2. Solid-state or semi-solid-state LiDAR: Compared to mechanical LiDAR, solid-state or semi-solid-state LiDAR has a smaller horizontal field of view (mechanical LiDAR can generally reach 360 degrees), but its vertical angle resolution is smaller than that of mechanical LiDAR. The resulting point cloud is denser and can capture more detailed information, but it is limited by the laser power and is easily affected by ambient light, resulting in poor performance outdoors.

[0007] 3. Structured Light Cameras: Compared to ordinary RGB cameras, which have strict requirements for environmental texture, structured light cameras can perform matching and depth calculations in environments with weak or no texture by actively emitting light with encoded information. This makes structured light cameras suitable for most scenarios, both indoors and outdoors. Furthermore, the point cloud density of structured light sensors is also at the pixel level, resulting in a relatively dense point cloud.

[0008] The background description provided herein is for the purpose of generally presenting the context of this disclosure. Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application and should not be acknowledged as prior art by virtue of its inclusion in this section. Summary of the Invention

[0009] To address the aforementioned technical problems in related technologies, this invention proposes an obstacle recognition method based on deep point cloud data, which includes the following steps:

[0010] S1, acquire depth point cloud data from the structured light camera;

[0011] S2, the depth point cloud data is converted to the robot coordinate system, and the protruding obstacles and the concave obstacles are obtained according to the maximum threshold height of the protruding obstacles and the minimum threshold height of the concave obstacles, respectively.

[0012] S3, obtain the initial ground point, and obtain the intercept d of the ground equation based on the initial ground point, as well as the upper and lower thresholds of the obstacle in the point cloud segmentation. Based on the dot product result of the initial ground point cloud data and the normal vector, obtain the type of each point in the initial ground point cloud data, wherein the type is ground point, raised obstacle, or recessed obstacle.

[0013] S4, take the raised obstacles and recessed obstacles obtained in step S2 and the raised obstacles and recessed obstacles obtained in step S3 as the raised obstacles and recessed obstacles of the entire point cloud.

[0014] 2. The method according to claim 1, wherein step S3 further comprises:

[0015] S31. Obtain the initial ground point P based on the vertical distance of the ground elevation of the points in the preliminary ground point cloud data. ground ;

[0016] S32. Obtain the ground normal vector and the intercept d of the ground equation based on the initial ground points;

[0017] S33. Obtain the upper and lower threshold values ​​of obstacles in the point cloud segmentation based on the intercept d of the ground equation.

[0018] S34, perform a dot product between each point in the preliminary ground point cloud data and the normal vector to obtain the dot product result, and obtain the type of each point in the preliminary ground point cloud data based on the dot product result, wherein the type is ground point, raised obstacle, or recessed obstacle;

[0019] S35. Transfer the ground point P from step S34. ground As the initial ground point in step S31, repeat steps S31-S34 until the preset number of iterations is reached.

[0020] Specifically, step 31 includes: sorting the points of the preliminary ground point cloud data in ascending order of their vertical distance from the ground elevation, and selecting the first N points as the initial ground points P. ground , where N is a positive integer.

[0021] Specifically, the upper limit threshold of the obstacle is:

[0022] d threhold_up = d + preset minimum obstacle height + point cloud data measurement noise;

[0023] The lower threshold of the obstacle is:

[0024] d threhold_down =-d-preset minimum obstacle height-point cloud data measurement noise.

[0025] Specifically, based on the dot product result d n To obtain the type of each point in the preliminary ground point cloud data, specifically:

[0026] d n ≥d threhold_up This point belongs to the raised obstacle P. convex ;

[0027] d n ≤d threhold_down This point belongs to the concave obstacle P. concave ;

[0028] d n ≥d threhold_down &&d n ≤d threhold_up This point belongs to ground point P. ground .

[0029] Secondly, another embodiment of the present invention discloses an obstacle recognition device based on deep point cloud data, which includes the following units:

[0030] The depth point cloud data acquisition unit is used to acquire depth point cloud data from the structured light camera.

[0031] The preliminary obstacle classification unit is used to convert the depth point cloud data to the robot coordinate system and obtain the protruding obstacles and the concave obstacles according to the maximum threshold height of the protruding obstacles and the minimum threshold height of the concave obstacles, respectively.

[0032] The obstacle classification unit is used to obtain initial ground points, and obtain the intercept d of the ground equation based on the initial ground points, as well as the upper and lower thresholds of the obstacles in the point cloud segmentation. Based on the dot product result of the initial ground point cloud data and the normal vector, the type of each point in the initial ground point cloud data is obtained, wherein the type is ground point, raised obstacle, and recessed obstacle.

[0033] An obstacle classification unit is used to classify the raised and recessed obstacles obtained by the preliminary obstacle classification unit and the raised and recessed obstacles obtained by the subdivision obstacle classification unit as raised and recessed obstacles for the entire point cloud.

[0034] Specifically, the subdivided obstacle classification unit further includes:

[0035] The initial ground point acquisition unit is used to acquire initial ground point P based on the vertical distance of the ground elevation of points in the preliminary ground point cloud data. ground ;

[0036] The ground normal vector and intercept acquisition unit is used to obtain the ground normal vector and the intercept d of the ground equation based on the initial ground point;

[0037] The obstacle upper and lower limit threshold acquisition unit is used to obtain the upper and lower limit thresholds of the obstacle segmentation of the point cloud based on the intercept d of the ground equation.

[0038] The obstacle type classification unit is used to perform a dot product between each point in the preliminary ground point cloud data and the normal vector to obtain the dot product result, and to obtain the type of each point in the preliminary ground point cloud data based on the dot product result, wherein the type is ground point, raised obstacle, or recessed obstacle;

[0039] Iterative unit, used to classify ground points P in obstacle type classification unit ground The initial ground point is used as the initial ground point acquisition unit. The initial ground point acquisition unit, ground normal vector and intercept acquisition unit, obstacle upper and lower limit threshold acquisition unit, and obstacle type classification unit are repeatedly executed until the preset number of iterations is reached.

[0040] Specifically, the initial ground point acquisition unit includes: sorting the points of the preliminary ground point cloud data in ascending order of their vertical distance from the ground elevation, and selecting the first N points as the initial ground point P. ground , where N is a positive integer.

[0041] Specifically, the upper limit threshold of the obstacle is:

[0042] d threhold_up = d + preset minimum obstacle height + point cloud data measurement noise;

[0043] The lower threshold of the obstacle is:

[0044] d threhold_down =-d-preset minimum obstacle height-point cloud data measurement noise.

[0045] Specifically, based on the dot product result d n To obtain the type of each point in the preliminary ground point cloud data, specifically:

[0046] d n ≥d threhold_up This point belongs to the raised obstacle P. convex ;

[0047] d n ≤d threhold_down This point belongs to the concave obstacle P. concave ;

[0048] d n ≥d threhold_down &&d n ≤d threhold_up This point belongs to ground point P. ground .

[0049] Thirdly, another embodiment of the present invention discloses a robot, the robot comprising: a processor, a memory, a structured light camera, wherein the memory stores instructions, and the processor, when executing the instructions, implements an obstacle recognition method based on depth point cloud data.

[0050] Fourthly, another embodiment of the present invention discloses a non-volatile storage medium storing instructions that, when executed by a processor, are used to implement an obstacle recognition method based on depth point cloud data.

[0051] This invention first divides the point cloud into two categories based on maximum and minimum thresholds: a point cloud containing obvious obstacles and a point cloud to be further subdivided. Then, by iteratively calculating the normal vector and the distance from a point to a plane, the point cloud to be subdivided is categorized into ground point clouds, concave point clouds, or convex point clouds. Finally, the categories are merged and summarized to obtain the classification of each point in the point cloud. This method can accurately identify obstacles in the point cloud while reducing computational complexity. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a schematic diagram of the robot provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of an obstacle recognition method based on deep point cloud data provided in an embodiment of the present invention;

[0055] Figure 3 This is a schematic diagram of the initial extraction of uneven obstacles provided in an embodiment of the present invention;

[0056] Figure 4 This is a schematic diagram of an obstacle recognition device based on deep point cloud data provided in an embodiment of the present invention;

[0057] Figure 5 This is a schematic diagram of an obstacle recognition device based on deep point cloud data provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0059] Example 1

[0060] refer to Figure 1 , Figure 1 This is a schematic diagram of the robot in this embodiment. A structured light camera 2 is installed on the robot body 1, and the structured light camera 2 can acquire depth point cloud data. (See diagram O) r and O c The symbols represent the robot's body coordinate system and the structured light camera coordinate system, respectively.

[0061] The specific robot includes a central processing unit (CPU) for controlling the robot and processing data.

[0062] The external parameters for the structured light camera's installation on the robot in this embodiment, including installation height and tilt angle, can be measured at the robot's factory and built into the robot's application or operating system. Therefore, the relative positional relationship between the robot's coordinate system and the structured light camera's coordinate system can be determined based on the structured light camera's external parameters. This allows the data measured by the structured light camera to be transferred to the robot's body coordinate system.

[0063] refer to Figure 2 This embodiment discloses an obstacle recognition method based on deep point cloud data, which includes the following steps:

[0064] S1, acquire depth point cloud data from the structured light camera;

[0065] Specifically, in this embodiment, the structured light camera acquires depth point cloud data of the environment and transmits the acquired depth point cloud data to the central processing unit.

[0066] Furthermore, in this embodiment, the original depth point cloud data is downsampled to obtain downsampled point cloud data.

[0067] Generally, the raw depth point cloud data output by a structured light camera is quite dense and has a large data volume. In order to reduce the data volume, this embodiment first downsamples the raw depth point cloud to reduce the number of sampling points and reduce the computational load. More generally, the downsampling method in this embodiment can adopt a general point cloud downsampling method. This embodiment does not make further limitations, as long as it can achieve downsampling of the raw depth point cloud data.

[0068] S2, the depth point cloud data is converted to the robot coordinate system, and the protruding obstacles and the concave obstacles are obtained according to the maximum threshold height of the protruding obstacles and the minimum threshold height of the concave obstacles, respectively.

[0069] like Figure 3 As shown, the point cloud in the camera coordinate system is first transformed into the robot coordinate system. Assume the transformation matrix between the camera coordinate system and the robot coordinate system is... Then, a measurement point P in the camera coordinate system c The representation in robot coordinates is:

[0070]

[0071] After conversion according to formula (1), the measurement data in the robot body coordinate system is obtained, and then the measurement points are initially segmented.

[0072] like Figure 3 As shown, preliminary obstacle segmentation is performed based on the highest and lowest obstacle distances that the robot can pass through. It is assumed that the maximum threshold height of the raised obstacle is... The minimum threshold height for low-lying obstacles is

[0073] The maximum threshold height of the protruding obstacle in this embodiment is: The minimum threshold height for low-lying obstacles is The settings are based on the robot's obstacle-crossing ability. For example, indoors, obstacles taller than 3cm can be considered low obstacles, while outdoors, obstacles taller than 8cm can be considered low obstacles. Similarly, the maximum threshold height is also set based on the robot's actual obstacle-crossing ability.

[0074] Specifically, the maximum threshold height above the protruding obstacle is set as... The point is classified as a raised obstacle P. convex ;

[0075] Points below the minimum threshold are classified as concave obstacles P. concave .

[0076] S3, obtain the initial ground point, and obtain the intercept d of the ground equation based on the initial ground point, as well as the upper and lower thresholds of the obstacle in the point cloud segmentation. Based on the dot product result of the preliminary ground point cloud data and the normal vector, obtain the type of each point in the preliminary ground point cloud data, wherein the type is ground point, raised obstacle, or recessed obstacle.

[0077] After obtaining the protruding and recessed obstacles in step S2, this embodiment removes the points of the protruding and recessed obstacles to obtain preliminary ground point cloud data.

[0078] The specific step S3 includes:

[0079] Step S31: Obtain the initial ground point P based on the vertical distance of the ground height of the points in the preliminary ground point cloud data. ground ;

[0080] The points in the preliminary ground point cloud data are sorted in ascending order of their vertical distance from the ground elevation, and the first N points are selected as the initial ground points P. ground , where N is a positive integer.

[0081] Specifically, in this embodiment, a preferred value for N is 100 or 200.

[0082] S32. Obtain the ground normal vector and the intercept d of the ground equation based on the initial ground points;

[0083] Based on the initial ground point P ground Obtain the initial ground point P ground The mean [x] mean y mean zmean and covariance matrix;

[0084] The covariance matrix is ​​decomposed using SVD; the eigenvector corresponding to the minimum eigenvalue is the ground orientation variable [abc] for this fitting. The intercept d of the ground equation is calculated from the mean point and normal vector obtained from the initial ground points, where d = -[abc]. T *[x mean y mean z mean ];

[0085] Therefore, the ground equation for this fitting is ax + by + cz + d = 0;

[0086] S33. Obtain the upper and lower threshold values ​​of obstacles in the point cloud segmentation based on the intercept d of the ground equation.

[0087] Based on the noise error range of the sensor's point cloud data measurement, the upper and lower thresholds for obstacles during point cloud segmentation are obtained:

[0088] d threhold_up = d + preset minimum obstacle height + point cloud data measurement noise;

[0089] The lower threshold of the obstacle is:

[0090] d threhold_down = -d - preset minimum obstacle height - point cloud data measurement noise. When the point cloud data measurement error is ±2.5cm,

[0091] d threhold_up =d + 2.5cm

[0092] d threhold_down =-d-2.5cm

[0093] When the preset minimum obstacle height is 5cm:

[0094] d threhold_up = d + 5cm + 2.5cm

[0095] d threhold_down = -d-5cm-2.5cm.

[0096] S34, perform a dot product between each point in the preliminary ground point cloud data and the normal vector to obtain the dot product result, and obtain the type of each point in the preliminary ground point cloud data based on the dot product result, wherein the type is ground point, raised obstacle, or recessed obstacle;

[0097] After obtaining the normal vector of the ground equation from step S32, calculate the [x] of each point. n y n zn ] and plane normal vector [x mean y mean z mean The dot product of ], i.e., d n =-[abc] T *[x n y n z n The dot product result is compared with the upper and lower thresholds to obtain the specific type of the point. In this embodiment, the specific types are ground point, raised obstacle, and recessed obstacle.

[0098] d n ≥d threhold_up This point belongs to the raised obstacle P. convex ;

[0099] d n ≤d threhold_down This point belongs to the concave obstacle P. concave ;

[0100] d n ≥d threhold_down &&d n ≤d threhold_up This point belongs to ground point P. ground ;

[0101] S35. Transfer the ground point P from step S34. ground As the initial ground point in step S31, repeat steps S31-S34 until the preset number of iterations is reached;

[0102] Step S35, once again take the ground point P from step S34. ground Using the initial ground points from step S31, the ground normal vector is re-estimated and the ground equation is fitted. After iterating a specified number of times, a more accurate segmentation type is obtained.

[0103] S4, take the raised obstacles and recessed obstacles obtained in step S2 and the raised obstacles and recessed obstacles obtained in step S3 as the raised obstacles and recessed obstacles of the entire point cloud.

[0104] After step S3 is completed, steps S2 and S3 are combined to summarize the convex and concave obstacles initially segmented in step S2, the ground points further subdivided in step S3, and the convex and concave obstacles, thus obtaining the overall point cloud segmentation result.

[0105] This embodiment first divides the point cloud into obvious obstacle point clouds and point clouds to be further subdivided based on maximum and minimum thresholds. Then, by iteratively calculating the normal vector and the distance from the point to the plane, the point clouds to be subdivided are classified into ground point clouds, concave or convex point clouds. Finally, the data is merged and summarized to obtain the classification of each point in the point cloud. The method of this embodiment can accurately identify obstacles in the point cloud and reduce the amount of computation.

[0106] Example 2

[0107] refer to Figure 4 This embodiment discloses an obstacle recognition device based on deep point cloud data, which includes the following units:

[0108] The depth point cloud data acquisition unit is used to acquire depth point cloud data from the structured light camera.

[0109] Specifically, in this embodiment, the structured light camera acquires depth point cloud data of the environment and transmits the acquired depth point cloud data to the central processing unit.

[0110] Furthermore, in this embodiment, the original depth point cloud data is downsampled to obtain downsampled point cloud data.

[0111] Generally, the raw depth point cloud data output by a structured light camera is quite dense and has a large data volume. In order to reduce the data volume, this embodiment first downsamples the raw depth point cloud to reduce the number of sampling points and reduce the computational load. More generally, the downsampling method in this embodiment can adopt a general point cloud downsampling method. This embodiment does not make further limitations, as long as it can achieve downsampling of the raw depth point cloud data.

[0112] The preliminary obstacle classification unit is used to convert the depth point cloud data to the robot coordinate system and obtain the protruding obstacles and the concave obstacles according to the maximum threshold height of the protruding obstacles and the minimum threshold height of the concave obstacles, respectively.

[0113] like Figure 3 As shown, the point cloud in the camera coordinate system is first transformed into the robot coordinate system. Assume the transformation matrix between the camera coordinate system and the robot coordinate system is... Then, a measurement point P in the camera coordinate system c The representation in robot coordinates is:

[0114]

[0115] After conversion according to formula (1), the measurement data in the robot body coordinate system is obtained, and then the measurement points are initially segmented.

[0116] like Figure 3As shown, preliminary obstacle segmentation is performed based on the highest and lowest obstacle distances that the robot can pass through. It is assumed that the maximum threshold height of the raised obstacle is... The minimum threshold height for low-lying obstacles is

[0117] The maximum threshold height of the protruding obstacle in this embodiment is: The minimum threshold height for low-lying obstacles is The settings are based on the robot's obstacle-crossing ability. For example, indoors, obstacles taller than 3cm can be considered low obstacles, while outdoors, obstacles taller than 8cm can be considered low obstacles. Similarly, the maximum threshold height is also set based on the robot's actual obstacle-crossing ability.

[0118] Specifically, the maximum threshold height above the protruding obstacle is set as... The point is classified as a raised obstacle P. convex ;

[0119] Points below the minimum threshold are classified as concave obstacles P. concave .

[0120] The obstacle classification unit is used to obtain initial ground points, and to obtain the intercept d of the ground equation based on the initial ground points, as well as the upper and lower thresholds of the obstacles in the point cloud segmentation. Based on the dot product of the initial ground point cloud data and the normal vector, the type of each point in the initial ground point cloud data is obtained, wherein the type is ground point, raised obstacle, or recessed obstacle.

[0121] After obtaining protruding and concave obstacles in the preliminary obstacle classification unit, this embodiment removes the points of the protruding and concave obstacles to obtain preliminary ground point cloud data.

[0122] Specific sub-categories of obstacles also include:

[0123] The initial ground point acquisition unit is used to acquire an initial ground point P based on the vertical distance of the ground elevation of the points in the preliminary ground point cloud data. ground ;

[0124] The points in the preliminary ground point cloud data are sorted in ascending order of their vertical distance from the ground elevation, and the first N points are selected as the initial ground points P. ground , where N is a positive integer.

[0125] Specifically, in this embodiment, a preferred value for N is 100 or 200.

[0126] Ground normal vector and intercept acquisition unit, used to obtain the ground normal vector and the intercept d of the ground equation based on the initial ground point;

[0127] Based on the initial ground point P ground Obtain the initial ground point P ground The mean [x] mean y mean z mean and covariance matrix;

[0128] The covariance matrix is ​​decomposed using SVD; the eigenvector corresponding to the minimum eigenvalue is the ground orientation variable [abc] for this fitting. The intercept d of the ground equation is calculated from the mean point and normal vector obtained from the initial ground points, where d = -[abc]. T *[x mean y mean z mean ];

[0129] Therefore, the ground equation for this fitting is ax + by + cz + d = 0;

[0130] An obstacle upper and lower limit threshold acquisition unit is used to obtain the upper and lower limit thresholds of obstacles in the point cloud segmentation based on the intercept d of the ground equation.

[0131] Based on the noise error range of the sensor's point cloud data measurement, the upper and lower thresholds for obstacles during point cloud segmentation are obtained:

[0132] d threhold_up = d + preset minimum obstacle height + point cloud data measurement noise;

[0133] The lower threshold of the obstacle is:

[0134] d threhold_down = -d - preset minimum obstacle height - point cloud data measurement noise. When the point cloud data measurement error is ±2.5cm,

[0135] d threhold_up =d + 2.5cm

[0136] d threhold_down =-d-2.5cm

[0137] When the preset minimum obstacle height is 5cm:

[0138] d threhold_up = d + 5cm + 2.5cm

[0139] d threhold_down = -d-5cm-2.5cm.

[0140] The obstacle type classification unit is used to perform a dot product between each point in the preliminary ground point cloud data and the normal vector to obtain the dot product result, and to obtain the type of each point in the preliminary ground point cloud data based on the dot product result, wherein the type is ground point, raised obstacle, or recessed obstacle;

[0141] After obtaining the normal vector of the ground equation from step S32, calculate the [x] of each point. n y n z n ] and plane normal vector [x mean y mean z mean The dot product of ], i.e., d n =-[abc] T *[x n y n z n The dot product result is compared with the upper and lower thresholds to obtain the specific type of the point. In this embodiment, the specific types are ground point, raised obstacle, and recessed obstacle.

[0142] d n ≥d threhold_up This point belongs to the raised obstacle P. convex ;

[0143] d n ≤d threhold_down This point belongs to the concave obstacle P. concave ;

[0144] d n ≥d threhold_down &&d n ≤d threhold_up This point belongs to ground point P. ground ;

[0145] Iterative unit, used to classify ground point P in the obstacle type classification unit ground The initial ground point is used as the initial ground point acquisition unit. The initial ground point acquisition unit, ground normal vector and intercept acquisition unit, obstacle upper and lower limit threshold acquisition unit, and obstacle type classification unit are repeatedly executed until the preset number of iterations is reached.

[0146] The iterative unit in this embodiment is used to further classify the ground point P in the obstacle type classification unit. ground The initial ground points of the unit are obtained by setting initial ground points, and the ground normal vector estimation and ground equation fitting are performed again.

[0147] After iterating a specified number of times, a more accurate segmentation type is obtained.

[0148] An obstacle classification unit is used to classify the raised and recessed obstacles obtained by the preliminary obstacle classification unit and the raised and recessed obstacles obtained by the subdivision obstacle classification unit as raised and recessed obstacles for the entire point cloud.

[0149] After the initial obstacle classification unit is completed, the initial obstacle classification unit and the subdivision obstacle classification unit are combined. The raised and lower obstacles initially segmented in the initial obstacle classification unit and the ground points further subdivided in the subdivision obstacle classification unit are summarized to obtain the overall point cloud segmentation result.

[0150] This embodiment first divides the point cloud into obvious obstacle point clouds and point clouds to be further subdivided based on maximum and minimum thresholds. Then, by iteratively calculating the normal vector and the distance from the point to the plane, the point clouds to be subdivided are classified into ground point clouds, concave or convex point clouds. Finally, the data is merged and summarized to obtain the classification of each point in the point cloud. The method of this embodiment can accurately identify obstacles in the point cloud and reduce the amount of computation.

[0151] Example 3

[0152] refer to Figure 1 This embodiment discloses a robot, which includes: a processor, a memory, a structured light camera, and instructions stored in the memory. When the processor executes the instructions, it implements the obstacle recognition method based on depth point cloud data of Embodiment 1.

[0153] Example 3

[0154] refer to Figure 5 , Figure 5 This is a schematic diagram of the structure of an image enhancement device according to this embodiment. The image enhancement device 20 of this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above device embodiments.

[0155] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the image enhancement device 20. For example, the computer program can be divided into the modules shown in Embodiment 2. The specific functions of each module are described in the working process of the device described in the above embodiments, and will not be repeated here.

[0156] The image enhancement device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the image enhancement device 20 and does not constitute a limitation on the image enhancement device 20. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the image enhancement device 20 may also include input / output devices, network access devices, buses, etc.

[0157] The processor 21 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the image enhancement device 20, connecting all parts of the image enhancement device 20 via various interfaces and lines.

[0158] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the image enhancement device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0159] If the modules / units integrated in the image enhancement device 20 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0160] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for obstacle recognition based on depth point cloud data, comprising the following steps: S1, obtaining depth point cloud data of a structured light camera; S2, converting the depth point cloud data to a robot coordinate system, and obtaining raised obstacles and sunken obstacles according to a maximum threshold height of the raised obstacles and a minimum threshold height of the sunken obstacles; S3, obtaining preliminary ground point cloud data by removing points of the raised obstacles and the sunken obstacles, obtaining initial ground points from the preliminary ground point cloud data, obtaining an intercept d of a ground equation according to the initial ground points, and obtaining an upper threshold and a lower threshold of obstacle segmentation according to a dot product result of the initial ground point cloud and a normal vector, and obtaining a type of each point in the initial ground point cloud according to the dot product result, wherein the type is a ground point, a raised obstacle or a sunken obstacle; S4, taking the raised obstacles and the sunken obstacles obtained in the step S2 and the raised obstacles and the sunken obstacles obtained in the step S3 as the raised obstacles and the sunken obstacles of the entire point cloud. 2.The method according to claim 1, wherein the step S3 further comprises: S31, obtaining an initial ground point P according to the vertical distance of the ground height of the point of the initial ground point cloud ground ; S32, obtaining a ground normal vector and the intercept d of the ground equation according to the initial ground points; S33, obtaining the upper threshold and the lower threshold of the obstacle segmentation according to the intercept d of the ground equation; S34, obtaining the dot product result of each point in the initial ground point cloud and the normal vector, and obtaining the type of each point in the initial ground point cloud according to the dot product result, wherein the type is the ground point, the raised obstacle or the sunken obstacle. S35, the ground points in step S34 are combined to generate a 3D point cloud of the ground surface. As the initial ground points of step S31, steps S31-S34 are repeatedly executed until a preset iteration number is reached.

3. The method of claim 2, said step 31 specifically comprising: Sort the points of the initial ground point cloud data in ascending order according to the vertical distance of the ground height, and select the first N points as the initial ground points where N is a positive integer. 4.The method according to claim 2, wherein the upper threshold of the obstacle is: ； and the lower threshold of the obstacle is: 。 5. The method of claim 4, wherein the dot product result d n Obtaining the type of each point in the initial ground point cloud data, specifically: , the point belongs to the raised obstacle ; , the point belongs to the lower concave obstacle ; , the point belongs to ground points . 6.An apparatus for obstacle recognition based on depth point cloud data, comprising the following units: a depth point cloud data obtaining unit, configured to obtain depth point cloud data of a structured light camera; an initial obstacle classification unit, configured to convert the depth point cloud data to a robot coordinate system, and obtain raised obstacles and sunken obstacles according to a maximum threshold height of the raised obstacles and a minimum threshold height of the sunken obstacles; a subdivided obstacle classification unit, configured to obtain preliminary ground point cloud data by removing points of the raised obstacles and the sunken obstacles, obtain initial ground points from the preliminary ground point cloud data, obtain an intercept d of a ground equation according to the initial ground points, obtain an upper threshold and a lower threshold of obstacle segmentation, obtain a type of each point in the initial ground point cloud according to a dot product result of the initial ground point cloud and a normal vector, and wherein the type is a ground point, a raised obstacle or a sunken obstacle; an obstacle classification unit, configured to take the raised obstacles and the sunken obstacles obtained by the initial obstacle classification unit and the raised obstacles and the sunken obstacles obtained by the subdivided obstacle classification unit as the raised obstacles and the sunken obstacles of the entire point cloud. 7.The apparatus according to claim 6, wherein the subdivided obstacle classification unit further comprises: An initial ground point acquisition unit is configured to acquire an initial ground point P according to a vertical distance of a ground height of a point of initial ground point cloud data ground ; a ground normal vector and intercept obtaining unit, configured to obtain a ground normal vector and the intercept d of the ground equation according to the initial ground points. An obstacle upper and lower threshold obtaining unit is configured to obtain an upper threshold and a lower threshold of the obstacle segmented from the point cloud according to the intercept d of the ground equation; An obstacle type classification unit is configured to perform a dot product of each point in the initial ground point cloud data and a normal vector to obtain a dot product result, and obtain a type of each point in the initial ground point cloud data according to the dot product result, wherein the type is a ground point, a convex obstacle or a concave obstacle; an iteration unit for classifying the ground points in the obstacle type classification unit the initial ground point as the initial ground point acquisition unit, the ground normal vector and the intercept acquisition unit, the upper and lower threshold value acquisition unit of the obstacle, and the obstacle type classification unit are repeatedly executed until a preset number of iterations is reached.

8. The apparatus of claim 7, the initial ground point acquisition unit specifically comprising: Sort the points of the initial ground point cloud data in ascending order according to the vertical distance of the ground height, and select the first N points as the initial ground points where N is a positive integer.

9. The apparatus of claim 8, wherein the upper threshold of the obstacle is: ； The lower threshold of the obstacle is: 。 10. The apparatus of claim 9, wherein the dot product result d n Obtaining the type of each point in the initial ground point cloud data, in particular: , the point belongs to the raised obstacle ; , the point belongs to the lower concave obstacle ; , the point belongs to ground points .

Images