Obstacle detection method, device and detection system

Abstract

The present application provides an obstacle detection method, device, and detection system. The method includes: obtaining a depth image and the pixel coordinates of each pixel point in the depth image to obtain multiple pixel coordinates; converting the multiple pixel coordinates to a world coordinate system to obtain multiple point clouds, wherein the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane; and obtaining a target obstacle point cloud based on at least the normal vectors of the multiple point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has one and only one coordinate axis perpendicular to the horizontal plane, which reduces the computational complexity of the process of converting multiple pixel coordinates to the world coordinate system, thereby reducing the overall computational complexity of the obstacle detection method, shortening the obstacle detection time, and thus solving the problem of the lack of a lightweight obstacle detection algorithm in the prior art.

Description

Technical Field

[0001] The present application relates to the field of obstacle detection, and more specifically, to an obstacle detection method, device, computer-readable storage medium, processor, and detection system. Background Art

[0002] Automated Guided Vehicles (AGVs) are increasingly being used in industrial production and commercial services to improve productivity and reduce labor costs. AGVs can travel along planned paths. To prevent safety issues caused by environmental changes, AGVs must monitor their surroundings in real time and adjust their paths and strategies accordingly. Obstacle avoidance is a fundamental and critical issue in AGV operation, requiring the AGV to detect obstacles along the planned path in real time. However, computing resources on AGVs are relatively limited.

[0003] Therefore, a lightweight obstacle detection algorithm is urgently needed.

[0004] The above information disclosed in the background technology section is only used to enhance the understanding of the background technology of the technology described in this article. Therefore, the background technology may contain certain information that does not form the prior art known in this country to those skilled in the art. Summary of the Invention

[0005] The main purpose of this application is to provide an obstacle detection method, device, computer-readable storage medium, processor and detection system to solve the problem of lack of a lightweight obstacle detection algorithm in the prior art.

[0006] According to one aspect of an embodiment of the present invention, a method for detecting an obstacle is provided, comprising: acquiring a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle; converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system; and obtaining a point cloud of the target obstacle based at least on normal vectors of the plurality of point clouds.

[0007] Optionally, a target obstacle point cloud is obtained at least based on the normal vectors of the multiple point clouds, including: determining a bearing surface point cloud in the multiple point clouds based on the normal vectors of the multiple point clouds, the bearing surface being used to bear the target obstacle; and removing the bearing surface point cloud from the multiple point clouds to obtain the target obstacle point cloud.

[0008] Optionally, based on the normal vectors of the multiple point clouds, the bearing surface point clouds in the multiple point clouds are determined, including: calculating the angle between each normal vector and the Y-axis of the world coordinate system to obtain multiple first angles; when the first angle is less than a first threshold, at least part of the point clouds corresponding to the first angle that is less than the first threshold are used as the bearing surface point clouds.

[0009] Optionally, when the first angle is less than a first threshold, at least part of the point cloud corresponding to the first angle less than the first threshold is used as the bearing surface point cloud, including: when the first angle is less than the first threshold, at least part of the point cloud corresponding to the first angle less than the first threshold is used as an alternative bearing surface point cloud; calculating the curvature of each alternative bearing surface point cloud to obtain multiple curvatures; when the curvature is less than the second threshold, the alternative bearing surface point cloud corresponding to the curvature less than the second threshold is used as the bearing surface point cloud.

[0010] Optionally, when the first angle is less than a first threshold, the point cloud corresponding to at least a portion of the first angle less than the first threshold is used as an alternative bearing surface point cloud, including: when the first angle is less than the first threshold, the point cloud corresponding to the first angle less than the first threshold is used as an initial alternative bearing surface point cloud; calculating the angle between the normal vectors of two adjacent initial alternative bearing surface point clouds to obtain multiple second angles; when the second angle is greater than a third threshold, the two initial bearing surface alternative point clouds corresponding to the second angle greater than the third threshold are used as the alternative bearing surface point cloud.

[0011] Optionally, based on the normal vectors of the multiple point clouds, the bearing surface point clouds in the multiple point clouds are determined, including: obtaining an initial bearing surface point cloud based on the normal vectors of the multiple point clouds; using a density clustering algorithm to divide the initial bearing surface point cloud into multiple point cloud clusters to obtain multiple point cloud clusters; calculating the average height and the number of point clouds of each point cloud cluster; obtaining a target probability based on the average height and the number of point clouds, and the target probability is used to characterize the probability that the point cloud cluster is the bearing surface; when the target probability is greater than a fourth threshold, determining that the multiple point clouds in the point cloud cluster corresponding to the target probability are the bearing surface point clouds.

[0012] Optionally, an initial bearing surface point cloud is obtained based on the normal vectors of multiple point clouds, including: calculating the angle between each normal vector and the Y-axis of the world coordinate system to obtain multiple first angles; when the first angle is less than the first threshold, the point clouds corresponding to some of the first angles that are less than the first threshold are used as the initial bearing surface point clouds.

[0013] Optionally, the target probability is obtained according to the average height and the number of point clouds, including: calculating the target probability according to g=λ1h+λ2n, wherein g is the target probability, λ1 is a first parameter, h is the average height, λ2 is a second parameter, and n is the number of point clouds.

[0014] Optionally, converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds includes: converting the plurality of pixel coordinates into a camera coordinate system to obtain a plurality of camera coordinates; and converting the plurality of camera coordinates into the world coordinate system to obtain a plurality of point clouds.

[0015] According to another aspect of an embodiment of the present invention, an obstacle detection device is also provided, including: an acquisition unit, configured to acquire a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle; a conversion unit, configured to convert the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system; and a determination unit, configured to obtain a point cloud of a target obstacle based at least on normal vectors of the plurality of point clouds.

[0016] According to yet another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein the program executes any one of the methods described above.

[0017] According to yet another aspect of the embodiments of the present invention, a processor is provided, wherein the processor is configured to run a program, wherein any one of the methods is executed when the program is run.

[0018] According to another aspect of an embodiment of the present invention, a detection system is also provided, comprising: a camera, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the methods described.

[0019] In an embodiment of the present invention, a depth image and the pixel coordinates of each pixel in the depth image are first acquired to obtain multiple pixel coordinates. Next, the multiple pixel coordinates are converted to a world coordinate system to obtain multiple point clouds, where the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane. Finally, a target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has only one coordinate axis perpendicular to the horizontal plane, thereby reducing the computational complexity of converting the multiple pixel coordinates to the world coordinate system. Subsequently, the target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. Because the computational complexity of the coordinate conversion process is relatively low, the overall computational complexity of the obstacle detection method is reduced, shortening the obstacle detection time and thereby addressing the lack of a lightweight obstacle detection algorithm in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings that constitute part of this application are used to provide a further understanding of this application. The illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation on this application. In the drawings:

[0021] Figure 1 A schematic flow chart of a method for detecting an obstacle according to an embodiment of the present application is shown;

[0022] Figure 2 A schematic structural diagram of an obstacle detection device according to an embodiment of the present application is shown;

[0023] Figure 3 A schematic diagram showing a flow chart of a method for detecting an obstacle according to an embodiment of the present application is shown;

[0024] Figure 4 shows an image before recognition using an obstacle detection method according to an embodiment of the present application;

[0025] Figure 5 The figure shows an image after recognition using the obstacle detection method according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element may be directly on the other element or intervening elements may be present. Moreover, in the specification and claims, when it is described that an element is "connected to" another element, the element may be "directly connected to" the other element or "connected to" the other element through a third element.

[0030] As mentioned in the background technology, the prior art lacks a lightweight obstacle detection algorithm. In order to solve the above problem, in a typical embodiment of the present application, an obstacle detection method, device, computer-readable storage medium, processor and detection system are provided.

[0031] According to an embodiment of the present application, a method for detecting an obstacle is provided.

[0032] Figure 1 FIG. 1 is a flow chart of a method for detecting obstacles according to an embodiment of the present application. Figure 1 As shown, the method includes the following steps:

[0033] Step S101, acquiring a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of the pixel coordinates, wherein the depth image includes an image of a target obstacle;

[0034] Step S102: converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system;

[0035] Step S103 : obtaining a target obstacle point cloud based on at least the normal vectors of the plurality of point clouds.

[0036] In the above method, first, a depth image and the pixel coordinates of each pixel in the depth image are acquired to obtain multiple pixel coordinates. Second, the multiple pixel coordinates are converted to a world coordinate system to obtain multiple point clouds, where the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane. Finally, a target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has one and only one coordinate axis perpendicular to the horizontal plane, thereby reducing the computational complexity of converting the multiple pixel coordinates to the world coordinate system. Subsequently, the target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. Because the computational complexity of the coordinate conversion process is relatively low, the overall computational complexity of the obstacle detection method is reduced, shortening the obstacle detection time and thereby addressing the lack of a lightweight obstacle detection algorithm in the prior art.

[0037] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] In one embodiment of the present application, obtaining a target obstacle point cloud based at least on the normal vectors of the plurality of point clouds comprises: determining a bearing surface point cloud from the plurality of point clouds based on the normal vectors of the plurality of point clouds, the bearing surface being used to bear the target obstacle; and removing the bearing surface point cloud from the plurality of point clouds to obtain the target obstacle point cloud. In this embodiment, the bearing surface point cloud is first determined based on the normal vectors of the point clouds, and then the bearing surface point cloud is removed from the plurality of point clouds to obtain an accurate obstacle point cloud.

[0039] Specifically, there are multiple situations for the above-mentioned bearing surface. When the AGV device is placed on the ground, the bearing surface is the ground; when the AGV device is placed on a table, the bearing surface is the table.

[0040] In order to further reduce the computational complexity of the detection method, in another embodiment of the present application, based on the normal vectors of the multiple point clouds, the bearing surface point clouds in the multiple point clouds are determined, including: calculating the angle between each of the normal vectors and the Y axis of the world coordinate system to obtain multiple first angles; when the first angle is less than a first threshold, at least part of the point clouds corresponding to the first angles less than the first threshold are used as the bearing surface point clouds. In this embodiment, it is only necessary to calculate the first angle between each of the normal vectors and the Y axis of the world coordinate system. The smaller the first angle, the more likely the point cloud corresponding to the first angle is to be the bearing surface point cloud. Therefore, the first angle is compared with the first threshold, and the point cloud corresponding to the first angle less than the first threshold is used as the bearing surface point cloud. This detection method is relatively simple and has a relatively small computational complexity.

[0041] The smaller the first threshold, the more parallel the normal vector is to the Y-axis, and the more likely it is a load-bearing surface. When the first threshold is 0, the judgment of the load-bearing surface point cloud is the most stringent. However, in actual applications, slopes do not meet this condition, so slopes are not considered load-bearing surfaces, which can lead to detection errors. Therefore, in actual applications, the first threshold is usually 30 degrees, which means that when the slope angle is less than 30 degrees, it can also be judged as a load-bearing surface.

[0042] In another embodiment of the present application, when the first angle is less than a first threshold, at least a portion of the point cloud corresponding to the first angle less than the first threshold is used as the bearing surface point cloud, including: when the first angle is less than the first threshold, at least a portion of the point cloud corresponding to the first angle less than the first threshold is used as an alternative bearing surface point cloud; calculating the curvature of each of the alternative bearing surface point clouds to obtain a plurality of curvatures; when the curvature is less than a second threshold, using the alternative bearing surface point cloud corresponding to the curvature less than the second threshold as the bearing surface point cloud. If only the angle between the normal vector and the Y-axis of the world coordinate system is used to determine the bearing surface point cloud, some smaller protruding obstacles on the bearing surface may not be detected, resulting in detection errors. Therefore, in this embodiment, the first angle is first used to screen out the alternative bearing surface point clouds, and then the curvature of each of the alternative bearing surface point clouds is calculated. If the curvature of the alternative bearing surface point cloud is less than the second threshold, the alternative bearing surface point cloud is determined to be the bearing surface point cloud, making the detection result more accurate, thereby further improving the accuracy of obstacle detection.

[0043] The smaller the above-mentioned second threshold is, the more likely the alternative bearing surface point cloud is the bearing surface point cloud. The above-mentioned second threshold can be taken as 0, and the judgment condition at this time is the most stringent. Of course, in actual applications, the above-mentioned second threshold can also be other values, and technical personnel in this field can choose according to actual conditions.

[0044] In order to further improve the accuracy of obstacle detection in the present application, in another embodiment of the present application, when the above-mentioned first angle is less than the first threshold, the above-mentioned point cloud corresponding to at least part of the above-mentioned first angle less than the above-mentioned first threshold is used as an alternative bearing surface point cloud, including: when the above-mentioned first angle is less than the above-mentioned first threshold, the above-mentioned point cloud corresponding to the above-mentioned first angle less than the above-mentioned first threshold is used as the initial alternative bearing surface point cloud; calculating the angle between the above-mentioned normal vectors of two adjacent above-mentioned initial alternative bearing surface point clouds to obtain multiple second angles; when the above-mentioned second angle is greater than a third threshold, the two above-mentioned initial alternative point clouds corresponding to the above-mentioned second angle greater than the third threshold are used as the above-mentioned alternative bearing surface point clouds. In this embodiment, after taking the point cloud corresponding to the first angle smaller than the first threshold as the initial candidate bearing surface point cloud, it is also necessary to calculate the angle between the normal vectors of the two adjacent initial candidate bearing surface point clouds to obtain multiple second angles. When the second angle is larger, the two initial candidate bearing surface point clouds are more likely to be bearing surface point clouds. Therefore, the second angle is compared with the third threshold, and the two initial candidate bearing surface point clouds corresponding to the second angle greater than the third threshold are taken as the alternative bearing surface point clouds, thereby further eliminating smaller protrusions on the bearing surface.

[0045] In another embodiment of the present application, based on the above-mentioned normal vectors of the above-mentioned multiple point clouds, the bearing surface point clouds in the above-mentioned multiple point clouds are determined, including: obtaining an initial bearing surface point cloud based on the above-mentioned normal vectors of the above-mentioned multiple point clouds; adopting a density clustering algorithm to divide the above-mentioned initial bearing surface point cloud into the above-mentioned multiple point cloud clusters to obtain multiple point cloud clusters; calculating the average height and the number of point clouds of each of the above-mentioned point cloud clusters; obtaining a target probability based on the above-mentioned average height and the above-mentioned number of point clouds, and the above-mentioned target probability is used to characterize the probability that the above-mentioned point cloud cluster is the above-mentioned bearing surface; when the above-mentioned target probability is greater than a fourth threshold value, determining that the above-mentioned multiple point clouds in the above-mentioned point cloud cluster corresponding to the above-mentioned target probability are the above-mentioned bearing surface point clouds. In this embodiment, the initial bearing surface point cloud is first obtained based on the normal vector of the point cloud. Because the initial bearing surface point cloud includes the bearing surface and objects (or parts of objects) with similar features to the bearing surface, such as a shorter table top, the initial bearing surface point cloud needs to be further screened, so the density clustering method is used to divide the initial bearing surface point cloud into multiple point cloud clusters. Because the point cloud clusters of the bearing surface have the following three characteristics: low height, large number of point clouds, and occupy most of the initial bearing surface point cloud, it is necessary to calculate the average height and number of point clouds of each of the above point cloud clusters, and obtain the above target probability based on the average height and number of point clouds. Then, the above target probability is compared with the fourth threshold to obtain an accurate bearing surface point cloud, thereby further improving the accuracy of obstacle detection.

[0046] The density clustering algorithm may adopt the dbscan algorithm. Of course, in practical applications, other algorithms may also be adopted, and those skilled in the art may select one according to the actual situation.

[0047] In another embodiment of the present application, an initial bearing surface point cloud is obtained based on the normal vectors of the plurality of point clouds, including: calculating the angle between each of the normal vectors and the Y-axis of the world coordinate system to obtain the plurality of first angles; when the first angle is less than the first threshold, the point clouds corresponding to the portion of the first angles less than the first threshold are used as the initial bearing surface point clouds. In this embodiment, the method for obtaining the initial bearing surface point cloud is also to calculate the first angle, and then use the point clouds corresponding to the portion of the first angles less than the first threshold as the initial bearing surface point clouds.

[0048] Of course, in actual applications, after calculating the first angle, the point cloud corresponding to the portion of the first angle less than the first threshold value may be used as the first initial bearing surface point cloud. The curvature or second angle of the first initial bearing surface point cloud may then be calculated, and then the first initial bearing surface point cloud that meets the conditions may be selected as the initial bearing surface point cloud. The specific calculation process is the same as the above method and will not be further described here.

[0049] In order to obtain an accurate target probability, in another embodiment of the present application, the target probability is obtained according to the above-mentioned average height and the above-mentioned number of point clouds, including: calculating the above-mentioned target probability according to g=λ1h+λ2n, wherein g is the above-mentioned target probability, λ1 is the first parameter, h is the above-mentioned average height, λ2 is the second parameter, and n is the above-mentioned number of point clouds.

[0050] The first parameter and the second parameter need to be set according to actual conditions, and the fourth threshold value needs to be set by the first parameter and the second parameter.

[0051] In a specific embodiment of the present application, after calculating the average height and number of points in each of the above-mentioned point cloud clusters, it is necessary to sort and label the multiple point cloud clusters from largest to smallest according to the number of points. For example, if a point cloud cluster is labeled k, it means that the number of points in this point cloud cluster accounts for the kth largest number of all point cloud clusters. The smaller k is, the more points in the point cloud cluster, and the more likely it is to be a load-bearing surface. The above-mentioned target probability is then calculated according to g = λ1h + λ2n + λ3k, where λ3 is the third parameter.

[0052] In another embodiment of the present application, the plurality of pixel coordinates are converted into a world coordinate system to obtain a plurality of point clouds, including: converting the plurality of pixel coordinates into a camera coordinate system to obtain a plurality of camera coordinates; converting the plurality of camera coordinates into the world coordinate system to obtain a plurality of point clouds.

[0053] The specific calculation process of converting the above pixel coordinates into the world coordinate system is as follows:

[0054] In the pinhole imaging model, the relationship for converting pixel points from world coordinates to pixel coordinates is as follows:

[0055]

[0056] Among them, K and R are the intrinsic and extrinsic parameter matrices of the camera respectively; u and v represent the horizontal and vertical coordinates of the pixel coordinate system respectively; X w 、Y w and Z w Respectively represent the x-axis, y-axis, and z-axis coordinate values ​​in the world coordinate system.

[0057] Assume that pixel d is a specific point in the depth image, and the corresponding pixel coordinate system is (u, v). Then the coordinate value of d in the camera coordinate system can be obtained by the following formula (X c , Y c , Z c ).

[0058]

[0059] Among them, u0, v0, f x and f y is the camera internal parameter, and the internal parameter value is determined by the physical structure and material of the camera; Z c This is the depth value, which can be read directly. Typical depth cameras provide intrinsic parameters when operating. If the camera does not actively publish these values, you can obtain them using the Zhang Zhengyou calibration method. In short, the camera's intrinsic parameters are readily available and can be obtained, so we will not elaborate on them here.

[0060] Then transform the point in the camera coordinate system to the world coordinate system:

[0061] Note the camera extrinsic matrix The inverse matrix of The corresponding coordinates of point d in the world coordinate system can be obtained according to the following formula

[0062]

[0063] Thus, the point cloud data in the world coordinate system is obtained, and the point cloud is obtained based on the above point cloud data.

[0064] The present application also provides an obstacle detection device. It should be noted that the obstacle detection device of the present application can be used to execute the obstacle detection method provided in the present application. The obstacle detection device provided in the present application is introduced below.

[0065] Figure 2 Schematic diagram of an obstacle detection device according to an embodiment of the present application. Figure 2 As shown, the device includes:

[0066] An acquisition unit 10 is configured to acquire a depth image and pixel coordinates of each pixel in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle;

[0067] a conversion unit 20 for converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system;

[0068] The determination unit 30 is configured to obtain a target obstacle point cloud based at least on the normal vectors of the plurality of point clouds.

[0069] The above-mentioned device includes an acquisition unit, a conversion unit, and a determination unit. The acquisition unit is configured to acquire a depth image and pixel coordinates of each pixel in the depth image to obtain a plurality of pixel coordinates. The conversion unit is configured to convert the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds. The predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane. The determination unit is configured to obtain a target obstacle point cloud based at least on the normal vectors of the plurality of point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, constraining the establishment of the world coordinate system. The coordinate system has only one axis perpendicular to the horizontal plane, thereby reducing the computational complexity of converting the plurality of pixel coordinates into the world coordinate system. Subsequently, the target obstacle point cloud is obtained based at least on the normal vectors of the plurality of point clouds. Because the computational complexity of the coordinate conversion process is relatively low, the overall computational complexity of the obstacle detection method is reduced, shortening the obstacle detection time and thereby addressing the lack of a lightweight obstacle detection algorithm in the prior art.

[0070] In one embodiment of the present application, the determination unit includes a determination subunit and a removal subunit, wherein the determination subunit is configured to determine a bearing surface point cloud from the plurality of point clouds based on the normal vectors of the plurality of point clouds, wherein the bearing surface is configured to bear the target obstacle; and the removal subunit is configured to remove the bearing surface point cloud from the plurality of point clouds to obtain the target obstacle point cloud. In this embodiment, the bearing surface point cloud is first determined based on the normal vectors of the point clouds, and then the bearing surface point cloud is removed from the plurality of point clouds to obtain an accurate obstacle point cloud.

[0071] Specifically, there are multiple situations for the above-mentioned bearing surface. When the AGV device is placed on the ground, the bearing surface is the ground; when the AGV device is placed on a table, the bearing surface is the table.

[0072] In order to further reduce the computational complexity of the detection method, in another embodiment of the present application, the above-mentioned determination subunit includes a first calculation module and a first determination module, wherein the above-mentioned first calculation module is used to calculate the angle between each of the above-mentioned normal vectors and the Y-axis of the above-mentioned world coordinate system to obtain multiple first angles; the above-mentioned first determination module is used to, when the above-mentioned first angle is less than a first threshold, use at least part of the above-mentioned point clouds corresponding to the above-mentioned first angles that are less than the above-mentioned first threshold as the above-mentioned bearing surface point clouds. In this embodiment, it is only necessary to calculate the first angle between each of the above-mentioned normal vectors and the Y-axis of the above-mentioned world coordinate system. The smaller the above-mentioned first angle is, the more likely the point cloud corresponding to the above-mentioned first angle is to be the bearing surface point cloud. Therefore, the above-mentioned first angle is compared with the above-mentioned first threshold, and the above-mentioned point cloud corresponding to the above-mentioned first angle that is less than the above-mentioned first threshold is used as the bearing surface point cloud. This detection method is relatively simple and has a relatively small computational complexity.

[0073] The smaller the first threshold, the more parallel the normal vector is to the Y-axis, and the more likely it is a load-bearing surface. When the first threshold is 0, the judgment of the load-bearing surface point cloud is the most stringent. However, in actual applications, slopes do not meet this condition, so slopes are not considered load-bearing surfaces, which can lead to detection errors. Therefore, in actual applications, the first threshold is usually 30 degrees, which means that when the slope angle is less than 30 degrees, it can also be judged as a load-bearing surface.

[0074] In another embodiment of the present application, the above-mentioned first determination module includes a first determination submodule, a first calculation submodule, and a second determination submodule, wherein the above-mentioned first determination submodule is used to, when the above-mentioned first angle is less than the above-mentioned first threshold, use the above-mentioned point cloud corresponding to the above-mentioned first angle that is at least partially less than the above-mentioned first threshold as the alternative bearing surface point cloud; the above-mentioned first calculation submodule is used to calculate the curvature of each of the above-mentioned alternative bearing surface point clouds to obtain multiple above-mentioned curvatures; the above-mentioned second determination submodule is used to, when the above-mentioned curvature is less than the second threshold, use the above-mentioned alternative bearing surface point cloud corresponding to the above-mentioned curvature that is less than the above-mentioned second threshold as the above-mentioned bearing surface point cloud. If only the angle between the normal vector and the Y-axis of the world coordinate system is used to determine the bearing surface point cloud, some smaller protruding obstacles on the bearing surface may not be detected, resulting in detection errors. Therefore, in this embodiment, the alternative bearing surface point clouds are first screened out by the first angle, and then the curvature of each of the above alternative bearing surface point clouds is calculated. If the curvature of the above alternative bearing surface point cloud is less than the second threshold, the above alternative bearing surface point cloud is determined to be the bearing surface point cloud, so that the detection result is more accurate, thereby further improving the accuracy of obstacle detection.

[0075] The smaller the above-mentioned second threshold is, the more likely the alternative bearing surface point cloud is the bearing surface point cloud. The above-mentioned second threshold can be taken as 0, and the judgment condition at this time is the most stringent. Of course, in actual applications, the above-mentioned second threshold can also be other values, and technical personnel in this field can choose according to actual conditions.

[0076] In order to further improve the accuracy of obstacle detection in the present application, in another embodiment of the present application, the above-mentioned first determination module includes a third determination submodule, a second calculation submodule and a fourth determination submodule, wherein the above-mentioned third determination submodule is used to use the above-mentioned point cloud corresponding to the above-mentioned first angle smaller than the above-mentioned first threshold as the initial alternative bearing surface point cloud when the above-mentioned first angle is smaller than the above-mentioned first threshold; the above-mentioned second calculation submodule is used to calculate the angle between the above-mentioned normal vectors of two adjacent above-mentioned initial alternative bearing surface point clouds to obtain multiple second angles; the above-mentioned fourth determination submodule is used to use the two above-mentioned initial alternative bearing surface point clouds corresponding to the above-mentioned second angle larger than the above-mentioned third threshold as the above-mentioned alternative bearing surface point cloud when the above-mentioned second angle is greater than the third threshold. In this embodiment, after taking the point cloud corresponding to the first angle smaller than the first threshold as the initial candidate bearing surface point cloud, it is also necessary to calculate the angle between the normal vectors of the two adjacent initial candidate bearing surface point clouds to obtain multiple second angles. When the second angle is larger, the two initial candidate bearing surface point clouds are more likely to be bearing surface point clouds. Therefore, the second angle is compared with the third threshold, and the two initial candidate point clouds corresponding to the second angle greater than the third threshold are taken as the alternative bearing surface point clouds, thereby further eliminating smaller protrusions on the bearing surface.

[0077] In another embodiment of the present application, the above-mentioned determination subunit includes a second determination module, a second calculation module, a third calculation module, a fourth calculation module and a third determination module, wherein the above-mentioned second determination module is used to obtain the initial bearing surface point cloud based on the above-mentioned normal vectors of the multiple point clouds; the above-mentioned second calculation module is used to divide the above-mentioned initial bearing surface point cloud into multiple point cloud clusters using a density clustering algorithm to obtain the multiple point cloud clusters; the above-mentioned third calculation module is used to calculate the average height and the number of point clouds of each of the above-mentioned point cloud clusters; the above-mentioned fourth calculation module is used to obtain the target probability based on the above-mentioned average height and the above-mentioned number of point clouds, and the above-mentioned target probability is used to characterize the probability that the above-mentioned point cloud cluster is the above-mentioned bearing surface; the above-mentioned third determination module is used to determine that the multiple point clouds in the above-mentioned point cloud cluster corresponding to the above-mentioned target probability are the above-mentioned bearing surface point clouds when the above-mentioned target probability is greater than a fourth threshold. In this embodiment, the initial bearing surface point cloud is first obtained based on the normal vector of the point cloud. Because the initial bearing surface point cloud includes the bearing surface and objects (or parts of objects) with similar features to the bearing surface, such as a shorter table top, the initial bearing surface point cloud needs to be further screened, so the density clustering method is used to divide the initial bearing surface point cloud into multiple point cloud clusters. Because the point cloud clusters of the bearing surface have the following three characteristics: low height, large number of point clouds, and occupy most of the initial bearing surface point cloud, it is necessary to calculate the average height and number of point clouds of each of the above point cloud clusters, and obtain the above target probability based on the average height and number of point clouds. Then, the above target probability is compared with the fourth threshold to obtain an accurate bearing surface point cloud, thereby further improving the accuracy of obstacle detection.

[0078] The density clustering algorithm may adopt the dbscan algorithm. Of course, in practical applications, other algorithms may also be adopted, and those skilled in the art may select one according to the actual situation.

[0079] In another embodiment of the present application, the second determination module includes a third calculation submodule and a fifth determination submodule, wherein the third calculation submodule is used to calculate the angle between each of the normal vectors and the Y-axis of the world coordinate system to obtain multiple first angles; and the fifth determination submodule is used to, when the first angle is less than the first threshold, use the point cloud corresponding to the portion of the first angle less than the first threshold as the initial bearing surface point cloud. In this embodiment, the method for obtaining the initial bearing surface point cloud is also to calculate the first angle, and then use the point cloud corresponding to the portion of the first angle less than the first threshold as the initial bearing surface point cloud.

[0080] Of course, in actual applications, after calculating the first angle, the point cloud corresponding to the portion of the first angle less than the first threshold value may be used as the first initial bearing surface point cloud. The curvature or second angle of the first initial bearing surface point cloud may then be calculated, and then the first initial bearing surface point cloud that meets the conditions may be selected as the initial bearing surface point cloud. The specific calculation process is the same as the above method and will not be further described here.

[0081] In order to obtain accurate target probability, in another embodiment of the present application, the above-mentioned fourth calculation module includes a fourth calculation submodule, wherein the above-mentioned fourth calculation submodule is used to calculate the above-mentioned target probability according to g=λ1h+λ2n, wherein g is the above-mentioned target probability, λ1 is the first parameter, h is the above-mentioned average height, λ2 is the second parameter, and n is the number of the above-mentioned point clouds.

[0082] The first parameter and the second parameter need to be set according to actual conditions, and the fourth threshold value needs to be set by the first parameter and the second parameter.

[0083] In a specific embodiment of the present application, after calculating the average height and number of points in each of the above-mentioned point cloud clusters, it is necessary to sort and label the multiple point cloud clusters from largest to smallest according to the number of points. For example, if a point cloud cluster is labeled k, it means that the number of points in this point cloud cluster accounts for the kth largest number of all point cloud clusters. The smaller k is, the more points in the point cloud cluster, and the more likely it is to be a load-bearing surface. The above-mentioned target probability is then calculated according to g = λ1h + λ2n + λ3k, where λ3 is the third parameter.

[0084] In another embodiment of the present application, the above-mentioned conversion unit includes a first conversion module and a second conversion module, wherein the above-mentioned first conversion module is used to convert the multiple pixel coordinates into the camera coordinate system to obtain multiple camera coordinates; the above-mentioned second conversion module is used to convert the multiple camera coordinates into the above-mentioned world coordinate system to obtain multiple point clouds.

[0085] The specific calculation process of converting the above pixel coordinates into the world coordinate system is as follows:

[0086] In the pinhole imaging model, the relationship for converting pixel points from world coordinates to pixel coordinates is as follows:

[0087]

[0088] Among them, K and R are the intrinsic and extrinsic parameter matrices of the camera respectively; u and v represent the horizontal and vertical coordinates of the pixel coordinate system respectively; X w 、Y w and Z w Respectively represent the x-axis, y-axis, and z-axis coordinate values ​​in the world coordinate system.

[0089] Assume that pixel d is a specific point in the depth image, and the corresponding pixel coordinate system is (u, v). Then the coordinate value of d in the camera coordinate system can be obtained by the following formula (X c , Y c , Z c ).

[0090]

[0091] Among them, u0, v0, f x and f y is the camera internal parameter, and the internal parameter value is determined by the physical structure and material of the camera; Z c This is the depth value, which can be read directly. Typical depth cameras provide intrinsic parameters when operating. If the camera does not actively publish these values, you can obtain them using the Zhang Zhengyou calibration method. In short, the camera's intrinsic parameters are readily available and can be obtained, so we will not elaborate on them here.

[0092] Then transform the point in the camera coordinate system to the world coordinate system:

[0093] Note the camera extrinsic matrix The inverse matrix of The corresponding coordinates of point d in the world coordinate system can be obtained according to the following formula

[0094]

[0095] Thus, the point cloud data in the world coordinate system is obtained, and the point cloud is obtained based on the above point cloud data.

[0096] The above-mentioned obstacle detection device includes a processor and a memory. The above-mentioned acquisition unit, conversion unit and determination unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0097] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and a lightweight obstacle detection algorithm can be provided by adjusting the kernel parameters.

[0098] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0099] An embodiment of the present invention provides a computer-readable storage medium having a program stored thereon, which implements the above-mentioned obstacle detection method when executed by a processor.

[0100] An embodiment of the present invention provides a processor, which is used to run a program, wherein the obstacle detection method is executed when the program is run.

[0101] An embodiment of the present invention provides a detection system, comprising a camera, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the above methods.

[0102] The above-mentioned detection system includes a camera, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the above-mentioned memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing any of the above-mentioned methods. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has one and only one coordinate axis perpendicular to the horizontal plane, so that the computational complexity of the process of converting multiple pixel coordinates into the world coordinate system is reduced. Thereafter, the target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. Because the computational complexity of the coordinate conversion process is small, the overall computational complexity of the obstacle detection method is small, shortening the obstacle detection time, thereby solving the problem of the lack of a lightweight obstacle detection algorithm in the prior art.

[0103] An embodiment of the present invention provides a device, comprising a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, at least the following steps are performed:

[0104] Step S101, acquiring a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle;

[0105] Step S102: converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system;

[0106] Step S103 : obtaining a target obstacle point cloud based on at least the normal vectors of the plurality of point clouds.

[0107] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0108] The present application also provides a computer program product, which, when executed on a data processing device, is adapted to execute a program for initializing at least the following method steps:

[0109] Step S101, acquiring a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle;

[0110] Step S102: converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system;

[0111] Step S103 : obtaining a target obstacle point cloud based on at least the normal vectors of the plurality of point clouds.

[0112] In order to make those skilled in the art more clearly understand the technical solution of this application, the following will be described in conjunction with specific embodiments:

[0113] Example

[0114] The flowchart of the obstacle detection method is as follows: Figure 3 As shown, the method includes the following steps:

[0115] Step 1: Establish a world coordinate system. The predetermined plane formed by the X-axis and Z-axis of the world coordinate system of this application is parallel to the horizontal plane.

[0116] In the pinhole imaging model, the relationship for converting pixel points from world coordinates to pixel coordinates is as follows:

[0117]

[0118] Among them, K and R are the intrinsic and extrinsic parameter matrices of the camera respectively; u and v represent the horizontal and vertical coordinates of the pixel coordinate system respectively; X w 、Y w and Z w Respectively represent the x-axis, y-axis, and z-axis coordinate values ​​in the world coordinate system.

[0119] Step 2: Get the depth image and the pixel coordinates of each pixel in the depth image

[0120] Step 3: Convert pixel coordinates to world coordinates

[0121] Assume that pixel d is a specific point in the depth image, and the corresponding pixel coordinate system is (u, v). Then the coordinate value of d in the camera coordinate system can be obtained by the following formula (X c , Y c , Z c ).

[0122]

[0123] Among them, u0, v0, fx and f y is the camera internal parameter, and the internal parameter value is determined by the physical structure and material of the camera; Z c This is the depth value, which can be read directly. Typical depth cameras provide intrinsic parameters when operating. If the camera does not actively publish these values, you can obtain them using the Zhang Zhengyou calibration method. In short, the camera's intrinsic parameters are readily available and can be obtained, so we will not elaborate on them here.

[0124] Step 4: Convert the point in the camera coordinate system to the world coordinate system:

[0125] Note the camera extrinsic matrix The inverse matrix of The corresponding coordinates of point d in the world coordinate system can be obtained according to the following formula

[0126]

[0127] Thus, the point cloud data in the world coordinate system is obtained, and the point cloud is obtained based on the above point cloud data.

[0128] Step 5: Roughly select the bearing surface point cloud

[0129] Traverse the point cloud in the world coordinate system. For each point cloud, those that meet the following three conditions can be preliminarily determined to be the bearing surface:

[0130] (1) The normal vector of the point cloud is nearly parallel to the Y axis of the world coordinate system; (2) The normal vectors of two adjacent point clouds are nearly parallel; (3) The curvature of the point cloud is less than the third threshold.

[0131] Based on the above three conditions, the candidate bearing surface point cloud can be selected.

[0132] Step 6: Select the bearing surface point cloud to determine the target obstacle point cloud

[0133] The point cloud obtained by rough selection in step 5 contains the bearing surface and some objects (or parts of objects) with similar features to the bearing surface, such as a shorter table top. Therefore, step 6 performs density clustering on all the point clouds in step 5, and the algorithm used is dbscan (other clustering algorithms can also be used). After dbscan clustering, the point cloud can be divided into several point cloud clusters. In most scenes, the bearing surface point cloud cluster has the following three characteristics: (1) relatively low height; (2) a large number of point clouds; (3) accounting for the majority of the candidate bearing surface point clouds;

[0134] Let all point cloud clusters output by dbscan be called P, where a specific point cloud cluster is p. Traverse P and calculate the average height h and the number of point clouds n for each p. In addition, sort all point cloud clusters p in P from large to small according to n. For example, if the number of point clouds of p is ranked k, it means that p has the kth largest number of point clouds in P. The smaller k is, the more point clouds p has, that is, the more likely p is a bearing surface. Then, use the following formula 4 to estimate the probability g that the point cloud cluster belongs to the bearing surface:

[0135] g=λ1h+λ2n+λ3k

[0136] When g is greater than or equal to the threshold Tg, the point cloud cluster is considered to be a load-bearing surface point cloud. By setting appropriate parameters λ1, λ2, and λ3 and the corresponding threshold Tg, it can be determined whether the point cloud cluster p is truly a load-bearing surface. The point cloud clusters of all non-load-bearing surfaces are output as the target obstacle point cloud.

[0137] Figure 4 is the image before the obstacle detection method is used, and the image after the obstacle detection method is used is as follows: Figure 5 As shown, the coordinate system in the figure is the camera coordinate system.

[0138] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0139] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the above-mentioned units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0140] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0141] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0142] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the above-mentioned methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0143] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0144] 1) The obstacle detection method of the present application first obtains a depth image and the pixel coordinates of each pixel in the depth image to obtain multiple pixel coordinates. Second, the multiple pixel coordinates are converted to a world coordinate system to obtain multiple point clouds. The predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane. Finally, a target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has only one coordinate axis perpendicular to the horizontal plane, which reduces the computational complexity of converting the multiple pixel coordinates to the world coordinate system. Subsequently, the target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. Because the computational complexity of the coordinate conversion process is relatively low, the overall computational complexity of the obstacle detection method is reduced, shortening the obstacle detection time, thereby solving the problem of the lack of a lightweight obstacle detection algorithm in the prior art.

[0145] 2) The obstacle detection device of the present application includes an acquisition unit, a conversion unit, and a determination unit. The acquisition unit is configured to acquire a depth image and pixel coordinates of each pixel in the depth image to obtain a plurality of pixel coordinates. The conversion unit is configured to convert the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds. The predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane. The determination unit is configured to obtain a target obstacle point cloud based at least on the normal vectors of the plurality of point clouds. In this method, the predetermined plane formed by the X-axis and Z-axis of the world coordinate system is parallel to the horizontal plane, constraining the establishment of the world coordinate system. The coordinate system has only one coordinate axis perpendicular to the horizontal plane, thereby reducing the computational complexity of converting the plurality of pixel coordinates into the world coordinate system. Subsequently, the target obstacle point cloud is obtained based at least on the normal vectors of the plurality of point clouds. Because the computational complexity of the coordinate conversion process is relatively low, the overall computational complexity of the obstacle detection method is reduced, shortening the obstacle detection time and thereby addressing the problem of the lack of a lightweight obstacle detection algorithm in the prior art.

[0146] 3) The detection system of the present application includes a camera, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include a method for executing any of the above methods. In this method, the predetermined plane formed by the X-axis and the Z-axis of the world coordinate system is parallel to the horizontal plane, which constrains the establishment of the world coordinate system. The coordinate system has one and only one coordinate axis perpendicular to the horizontal plane, so that the computational complexity of the process of converting multiple pixel coordinates to the world coordinate system is reduced. Afterwards, the target obstacle point cloud is obtained based on at least the normal vectors of the multiple point clouds. Because the computational complexity of the coordinate conversion process is small, the overall computational complexity of the obstacle detection method is small, which shortens the obstacle detection time, thereby solving the problem of the lack of a lightweight obstacle detection algorithm in the prior art.

[0147] The above description is merely a preferred embodiment of the present application and is not intended to limit the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A method for detecting an obstacle, characterized in that: include: Acquire a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of the target obstacle; Converting the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system; Obtaining a target obstacle point cloud based on at least a plurality of normal vectors of the point clouds; At least according to the normal vectors of the plurality of point clouds, a target obstacle point cloud is obtained, including: according to the normal vectors of the plurality of point clouds, an initial bearing surface point cloud is obtained; a density clustering algorithm is used to divide the initial bearing surface point cloud into a plurality of point cloud clusters to obtain a plurality of point cloud clusters; an average height and a number of point clouds of each point cloud cluster are calculated; according to the average height and the number of point clouds, a target probability is obtained, the target probability being used to characterize the probability that the point cloud cluster is the bearing surface; when the target probability is greater than a fourth threshold, a plurality of point clouds in the point cloud cluster corresponding to the target probability are determined to be the bearing surface point clouds, the bearing surface being used to bear the target obstacle; and the bearing surface point cloud is removed from the plurality of point clouds to obtain the target obstacle point cloud. After calculating the average height and the number of point clouds of each point cloud cluster, the plurality of point cloud clusters are sorted and numbered from large to small according to the number of point clouds. , calculate the target probability, where g is the target probability, is the first parameter, h is the average height, is the second parameter, n is the number of point clouds, is the third parameter, k is the label corresponding to the point cloud cluster, and the fourth threshold is determined according to the first parameter and the second parameter.

2. The method according to claim 1, characterized in that The pixel coordinates are converted into a world coordinate system to obtain a plurality of point clouds, including: Converting the plurality of pixel coordinates into a camera coordinate system to obtain a plurality of camera coordinates; The plurality of camera coordinates are converted into the world coordinate system to obtain the plurality of point clouds.

3. An obstacle detection device, characterized in that: include: an acquisition unit, configured to acquire a depth image and pixel coordinates of each pixel point in the depth image to obtain a plurality of pixel coordinates, wherein the depth image includes an image of a target obstacle; a conversion unit, configured to convert the plurality of pixel coordinates into a world coordinate system to obtain a plurality of point clouds, wherein a predetermined plane of the world coordinate system is parallel to a horizontal plane, and the predetermined plane is a plane formed by an X-axis and a Z-axis of the world coordinate system; a determination unit, configured to obtain a target obstacle point cloud based at least on normal vectors of the plurality of point clouds; The determination unit includes a determination subunit and a removal subunit, wherein the determination subunit is used to obtain an initial bearing surface point cloud based on the normal vectors of the plurality of point clouds; the initial bearing surface point cloud is divided into a plurality of point cloud clusters using a density clustering algorithm to obtain a plurality of point cloud clusters; the average height and the number of point clouds of each point cloud cluster are calculated; the target probability is obtained based on the average height and the number of point clouds, and the target probability is used to characterize the probability that the point cloud cluster is the bearing surface; when the target probability is greater than a fourth threshold, the plurality of point clouds in the point cloud cluster corresponding to the target probability are determined to be the bearing surface point clouds, and the bearing surface is used to carry the target obstacle; the removal subunit is used to remove the bearing surface point cloud from the plurality of point clouds to obtain the target obstacle point cloud, The device is further configured to, after calculating the average height and the number of point clouds of each point cloud cluster, sort and label the plurality of point cloud clusters according to the number of point clouds from large to small, and , calculate the target probability, where g is the target probability, is the first parameter, h is the average height, is the second parameter, n is the number of point clouds, is the third parameter, k is the label corresponding to the point cloud cluster, the fourth threshold is determined according to the first parameter and the second parameter, and the fourth threshold is determined according to the first parameter and the second parameter.

4. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method of claim 1 or 2.

5. A processor, characterized in that: The processor is configured to run a program, wherein the method according to claim 1 or 2 is executed when the program is run.

6. A detection system, characterized in that: include: A camera, one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for executing the method of claim 1 or 2.

Images