A point cloud data extraction method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a point cloud data extraction method and device, electronic equipment and a storage medium. Embodiments of the present application can obtain first point cloud data in a preset coordinate system; in the first coordinate axis direction of the preset coordinate system, point cloud deduplication is performed on the first point cloud data to obtain second point cloud data after deduplication processing; the second point cloud data is projected to the first preset plane of the preset coordinate system to obtain third point cloud data after projection; the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction is determined; based on the statistical distribution value, projection point cloud filtering processing is performed on the third point cloud data to obtain fourth point cloud data after filtering; and the extracted stockpile point cloud data is obtained according to the fourth point cloud data, so as to extract the stockpile point cloud data from the first point cloud data. The present application can effectively extract the stockpile point cloud data to be extracted from the first point cloud data, and the extraction precision is very high.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of visual recognition, in particular to a point cloud data extraction method and device, electronic equipment and storage medium. BACKGROUND

[0002] In the volume measurement scene of warehouse stacking objects, a scanning system such as a laser radar is often used to generate 3D point cloud data of the warehouse scene. The stacking object (such as bulk grain) point cloud data is first extracted from the 3D point cloud data, and then the stacking object surface point cloud is filled, and then the volume of the stacking object is calculated.

[0003] However, when extracting the stacking object point cloud data from the point cloud data, the walls, columns and other objects in the warehouse directly contact the stacking object, and it is not easy to extract the stacking object surface point cloud data. The mechanical vehicles, grain conveying equipment and the like near or above the stacking object are not easy to filter out. The manual labeling method for extracting the grain surface point cloud is time-consuming and labor-intensive and is not automated. Therefore, there is an urgent need for a point cloud data extraction method that can effectively and accurately extract the stacking object point cloud data from the 3D point cloud data. SUMMARY

[0004] The embodiments of the present application provide a point cloud data extraction method and device, electronic equipment and storage medium, which can effectively extract the stacking object point cloud data from the 3D point cloud data and have high extraction accuracy.

[0005] The embodiments of the present application provide a point cloud data extraction method, which comprises:

[0006] Obtain first point cloud data in a preset coordinate system, the first point cloud data comprising stacking object point cloud data to be extracted;

[0007] In the first coordinate axis direction of the preset coordinate system, the first point cloud data is subjected to point cloud deduplication to obtain second point cloud data after deduplication processing;

[0008] Project the second point cloud data onto a first preset plane of the preset coordinate system to obtain third point cloud data after projection;

[0009] Determine a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction;

[0010] Based on the statistical distribution value, the third point cloud data is subjected to projection point cloud filtering processing to obtain fourth point cloud data after filtering;

[0011] According to the fourth point cloud data, the extracted stacking object point cloud data is obtained.

[0012] The embodiments of the present application also provide a point cloud data extraction device, which comprises:

[0013] The point cloud data acquisition unit is configured to acquire first point cloud data in a preset coordinate system, and the first point cloud data includes point cloud data of the heap to be extracted.

[0014] The de-duplication processing unit is configured to perform point cloud de-duplication on the first point cloud data in a first coordinate axis direction of the preset coordinate system to obtain second point cloud data after de-duplication processing.

[0015] The projection processing unit is configured to project the second point cloud data onto a first preset plane of the preset coordinate system to obtain third point cloud data after projection.

[0016] The distribution value determination unit is configured to determine a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0017] The filtering processing unit is configured to perform projection point cloud filtering processing on the third point cloud data based on the statistical distribution value to obtain fourth point cloud data after filtering.

[0018] The point cloud extraction unit is configured to obtain extracted point cloud data of the heap according to the fourth point cloud data.

[0019] In some embodiments, the point cloud data extraction apparatus further includes a down-sampling unit, which is configured to:

[0020] The down-sampling unit is configured to perform down-sampling on the first point cloud data based on a preset spatial range to obtain point cloud data within the preset spatial range from the first point cloud data.

[0021] In some embodiments, the down-sampling unit includes a conditional filtering unit, which is configured to:

[0022] The conditional filtering unit is configured to determine a first coordinate value, a second coordinate value and a third coordinate value of the point cloud data in the first point cloud data.

[0023] When the first coordinate value meets a first preset condition, the second coordinate value meets a second preset condition and the third coordinate value meets a third preset condition, the point cloud data in the first point cloud data that meets the preset conditions is obtained.

[0024] In some embodiments, in the conditional filtering unit, the first preset condition is that the first coordinate value is greater than a first minimum threshold value and less than a first maximum threshold value; the second preset condition is that the second coordinate value is greater than a second minimum threshold value and less than a second maximum threshold value; and the third preset condition is that the third coordinate value is greater than a third minimum threshold value and less than a third maximum threshold value.

[0025] In some embodiments, the de-duplication processing unit includes a de-duplication processing subunit, which includes:

[0026] The three-dimensional grid is established based on a preset coordinate system, and the three-dimensional grid includes a plurality of voxel grids with a preset three-dimensional range, and there is at most one point cloud data in each voxel grid;

[0027] Based on the preset three-dimensional range of the voxel grid, a first plane grid corresponding to a first preset plane of the preset coordinate system is obtained from the three-dimensional grid; the first plane grid has a first grid;

[0028] A first quantity of point cloud data corresponding to the first grid in the direction of the first coordinate axis is determined;

[0029] When the first quantity is greater than a first preset quantity threshold, point cloud data with a first coordinate value greater than a preset first coordinate threshold is removed;

[0030] The point cloud data after the first coordinate value deduplication processing is obtained, and the point cloud data after the first coordinate value deduplication processing is taken as second point cloud data after deduplication processing.

[0031] In some embodiments, the distribution value determination unit includes a distribution value determination subunit, and the distribution value determination subunit is configured to:

[0032] The first preset plane is divided into a second plane grid, and the second plane grid has a second grid;

[0033] A second quantity of point cloud data corresponding to the second grid in the direction of the first coordinate axis is determined;

[0034] When the second quantity is greater than a second preset quantity threshold, a first coordinate value average and a first coordinate value standard deviation of the point cloud data corresponding to the second grid in the direction of the first coordinate axis are determined;

[0035] The first coordinate value average and the first coordinate value standard deviation are taken as statistical distribution values of third point cloud data in the direction of the first coordinate axis.

[0036] In some embodiments, the filtering processing unit includes a filtering processing subunit, and the filtering processing subunit includes:

[0037] When the first coordinate value standard deviation is not less than a standard deviation threshold, point cloud data with a first coordinate value less than an average threshold corresponding to the second grid is obtained;

[0038] The point cloud data with the first coordinate value less than the average threshold is taken as fourth point cloud data after filtering;

[0039] When the first coordinate value standard deviation is less than the standard deviation threshold, all point cloud data corresponding to the second grid is taken as the fourth point cloud data after filtering.

[0040] In some embodiments, the point cloud extraction unit includes a statistical filtering unit, and the statistical filtering unit is configured to:

[0041] determine a preset neighborhood of the point cloud data in the fourth point cloud data;

[0042] obtain distances between the point cloud data in the fourth point cloud data and other point cloud data within the preset neighborhood;

[0043] calculate an average distance according to the distances between the point cloud data in the fourth point cloud data and other point cloud data within the preset neighborhood;

[0044] remove the point cloud data in the fourth point cloud data with the average distance greater than a distance threshold;

[0045] obtain the point cloud data after statistical filtering;

[0046] use the point cloud data after statistical filtering as the extracted point cloud data of the heap.

[0047] The embodiment of the present application also provides an electronic device, comprising a processor and a memory, the memory stores a plurality of instructions, and the processor loads the instructions to execute the steps in any point cloud data extraction method provided by the embodiment of the present application.

[0048] The embodiment of the present application also provides a storage medium, the storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in any point cloud data extraction method provided by the embodiment of the present application.

[0049] The embodiment of the present application can obtain first point cloud data in a preset coordinate system through a scanning system; the first point cloud data is subjected to point cloud deduplication in a first coordinate axis direction of the preset coordinate system to obtain second point cloud data after deduplication processing; the second point cloud data is projected onto a first preset plane of the preset coordinate system to obtain third point cloud data after projection; a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction is determined; and the third point cloud data is subjected to projection point cloud filtering processing based on the statistical distribution value to obtain fourth point cloud data after filtering. In this way, the extracted point cloud data of the heap can be obtained from the fourth point cloud data, so that the point cloud data of the heap is extracted from the first point cloud data.

[0050] When the point cloud data is extracted, the first point cloud data is subjected to point cloud deduplication in the first coordinate axis direction, so that other objects of the heap in the first coordinate axis direction can be removed, then the projection point cloud filtering processing is performed, so that the point cloud data of the heap and other point cloud data can be better distinguished, and effective heap information can be retained, the point cloud data of the heap to be extracted can be effectively extracted from the first point cloud data, and the extraction precision is high. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0052] Figure 1a is a scene schematic diagram of the point cloud data extraction system provided by the embodiment of the present application;

[0053] Figure 1b is a flow schematic diagram of the point cloud data extraction method provided by the embodiment of the present application;

[0054] Figure 1c is a preset coordinate system schematic diagram provided by the embodiment of the present application;

[0055] Figure 1d is a first point cloud data schematic diagram provided by the embodiment of the present application;

[0056] Figure 1e is a point cloud deduplication flow schematic diagram provided by the embodiment of the present application;

[0057] Figure 1f is a mechanical equipment point cloud data schematic diagram provided by the embodiment of the present application;

[0058] Figure 1g is a discrete noise schematic diagram provided by the embodiment of the present application;

[0059] Figure 2 is a structure schematic diagram of the point cloud data extraction device provided by the embodiment of the present application.

[0060] Figure 3 is a structure schematic diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0062] The embodiment of the present application provides a point cloud data extraction method, device, electronic device and storage medium.

[0063] The point cloud data extraction device can be integrated in an electronic device, which can be a terminal, a server, or the like. The terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, or a personal computer (PC), or the like. The server can be a single server or a server cluster composed of multiple servers.

[0064] In some embodiments, the point cloud data extraction device can also be integrated in multiple electronic devices, for example, the point cloud data extraction device can be integrated in multiple servers to implement the point cloud data extraction method of the present application.

[0065] In some embodiments, the server can also be implemented in the form of a terminal.

[0066] For example, when the point cloud data extraction device is integrated in an electronic device, the electronic device can obtain first point cloud data in a preset coordinate system through a scanning system, perform point cloud deduplication on the first point cloud data in a first coordinate axis direction of the preset coordinate system to obtain second point cloud data after deduplication processing, project the second point cloud data to a first preset plane of the preset coordinate system to obtain third point cloud data after projection, determine a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction, perform projection point cloud filtering processing on the third point cloud data based on the statistical distribution value to obtain fourth point cloud data after filtering, and obtain extracted heap object point cloud data according to the fourth point cloud data, so as to extract the heap object point cloud data from the first point cloud data.

[0067] Please refer to Figure 1a , Figure 1a The point cloud data extraction system provided by the embodiment of the present application is shown in the scene diagram. The system can include a server 10, a storage terminal 11, and a scanning system 12. The scanning system 12 is used to obtain point cloud data, the storage terminal 11 is used to store point cloud data, the server 10 is used to extract heap object point cloud data from the point cloud data, the scanning system 12 is in communication connection with the server 10 and the storage terminal 11 respectively, and the server 10 and the storage terminal 11 are in communication connection with each other, which will not be described here.

[0068] The server 10 can include a processor and a memory, etc. The storage terminal 11 can include a cloud server, etc. The scanning system 12 can include a laser radar, etc.

[0069] It should be noted that Figure 1aThe system scenario diagram shown is only an example, and the server and scenario described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, as the system evolves and new business scenarios appear, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems. The following will be described in detail. It should be noted that the order of description of the following embodiments does not limit the preferred order of the embodiments.

[0070] As shown in Figure 1b The specific process of the point cloud data extraction method can be as follows:

[0071] 110、Obtain first point cloud data in a preset coordinate system, the first point cloud data including point cloud data of the stockpile to be extracted.

[0072] In the embodiments of the present application, the point cloud data in the preset coordinate system can be obtained by a scanning system, and the point cloud data in the preset coordinate system obtained by the scanning system is taken as the first point cloud data. The scanning system can be a laser radar, which can include a three-dimensional scanner (3D scanner) and the like. The scanning system is used to detect and analyze the shape (geometric structure) and appearance data (such as color, surface albedo, etc.) of objects or environment in the real world. The data collected by the scanning system is used to create a point cloud of the geometric surface of the object, and these points can be used to interpolate the surface shape of the object and used for three-dimensional reconstruction calculation, to create a digital model of the actual object in the virtual world, and a more accurate model can be created by a denser point cloud.

[0073] As shown in Figure 1c The preset coordinate system is a coordinate system established based on the stockpile where the stockpile is located, the first lower right corner of the stockpile is the origin O, the height of the stockpile is the first coordinate axis, i.e. the z coordinate axis; the wide side of the stockpile is the second coordinate axis, i.e. the x coordinate axis; the long side of the stockpile is the third coordinate axis, i.e. the y coordinate axis; the plane formed by the second coordinate axis and the third coordinate axis corresponds to the first preset plane of the preset coordinate system, and the first preset plane is the xOy plane.

[0074] Figure 1d The first point cloud data is a schematic diagram, and the present application can use the first point cloud data for three-dimensional reconstruction, wherein the first point cloud data includes not only the point cloud data of the stockpile to be extracted, but also the point cloud data of the roof of the stockpile, the point cloud data of the noise outside the stockpile, and the like. The first point cloud data can have geometric structure information and appearance data information, wherein the geometric structure information can include a first coordinate value, a second coordinate value and a third coordinate value, and the appearance data information can include surface albedo and the like.

[0075] 120. In a first coordinate axis direction of the preset coordinate system, the first point cloud data is subjected to point cloud deduplication to obtain second point cloud data after deduplication processing.

[0076] In the embodiment of the present application, the point cloud data obtained after the first point cloud data is subjected to point cloud deduplication can be used as the second point cloud data.

[0077] The first coordinate axis direction of the preset coordinate system is the z coordinate axis, and there are various non-pile material point cloud data above the pile material. For example, the non-pile material point cloud data can include mechanical equipment, roofs, columns and the like above the pile material. By subjecting the first point cloud data to point cloud deduplication, the embodiment of the present application can filter out the non-pile material point cloud data above the pile material, thereby further obtaining the second point cloud data after deduplication processing.

[0078] In an embodiment, before the first point cloud data is subjected to point cloud deduplication in the first coordinate axis direction of the preset coordinate system, the method further comprises:

[0079] Based on the preset spatial range, the first point cloud data is subjected to down-sampling to obtain point cloud data within the preset spatial range from the first point cloud data.

[0080] In the embodiment of the present application, the preset spatial range can be set according to the length, width and height of the storage bin.

[0081] After the point cloud data is collected by the scanning system, the embodiment of the present application can down-sample the point cloud data, thereby eliminating redundant point cloud data. For example, the pile material of the embodiment of the present application is located in the storage bin, and the point cloud data collected outside the storage bin is redundant data. By eliminating the point cloud data outside the preset spatial range, the present application removes the redundant data and speeds up the extraction rate of the effective point cloud data, i.e. the pile material point cloud data.

[0082] In an embodiment, based on the preset spatial range, the first point cloud data is subjected to down-sampling, comprising:

[0083] The first coordinate value, the second coordinate value and the third coordinate value of the point cloud data in the first point cloud data are determined.

[0084] When the first coordinate value satisfies the first preset condition, the second coordinate value satisfies the second preset condition and the third coordinate value satisfies the third preset condition, the point cloud data in the first point cloud data that satisfies the preset condition is obtained.

[0085] The first preset condition is that the first coordinate value is greater than the first coordinate minimum threshold and less than the first coordinate maximum threshold; the second preset condition is that the second coordinate value is greater than the second coordinate minimum threshold and less than the second coordinate maximum threshold; and the third preset condition is that the third coordinate value is greater than the third coordinate minimum threshold and less than the third coordinate maximum threshold.

[0086] In the embodiment of the present application, the minimum threshold and the maximum threshold of the point cloud data in the X coordinate axis, the Y coordinate axis and the Z coordinate axis can be respectively limited according to the actual size of the storage bin, so that conditional filtering is performed, and the embodiment of the present application can filter out noise points, overexposed points, roof points and the like in the point cloud data through conditional filtering.

[0087] In an embodiment, the first point cloud data is subjected to point cloud deduplication in the direction of the first coordinate axis of the preset coordinate system, as shown in the following formula: Figure 1e

[0088] 1201. Establish a three-dimensional grid based on a preset coordinate system.

[0089] The three-dimensional grid includes a plurality of voxel grids with a preset three-dimensional range, and at most one point cloud data in the voxel grid.

[0090] In the embodiment of the present application, the concept of voxel is similar to that of pixel, and pixel is a point in a two-dimensional image, while voxel is a small space in a three-dimensional coordinate system. The embodiment of the present application can regard the voxel grid as a micro 3D small space in the space based on the preset coordinate system, and create a plurality of 3D voxel grids on the input point cloud data to establish a three-dimensional grid. The embodiment of the present application limits the three-dimensional size of the voxel grid, so that at most one point cloud data can be included in the voxel grid. For example, the three-dimensional range of the voxel grid can be set to 0.1m*0.1m*0.1m, and at most one point cloud data can be included in each voxel grid with a size of 0.1m*0.1m*0.1m.

[0091] 1202. Obtain a first plane grid corresponding to a first preset plane of the preset coordinate system from the three-dimensional grid based on the preset three-dimensional range of the voxel grid.

[0092] In the embodiment of the present application, the first plane grid has a plurality of first grids, and the first preset plane of the preset coordinate system is a plane formed by the second coordinate axis and the third coordinate axis, i.e. the xOy plane.

[0093] The embodiment of the present application can obtain the plane grid corresponding to the three-dimensional grid in the xOy plane, and record the plane grid corresponding to the three-dimensional grid as the first plane grid, and the first plane grid has a plurality of first grids. For example, the embodiment of the present application can set the plane range of the first grid to 0.1m*0.1m according to the three-dimensional range 0.1m*0.1m*0.1m of the voxel grid.

[0094] 1203. Determine the first number of point cloud data corresponding to the first grid in the direction of the first coordinate axis.

[0095] ​In the embodiment of the present application, each grid (i.e., the first grid) in the first plane grid can have no point cloud data in the z coordinate axis direction, can have one point cloud data, or can have multiple point cloud data.

[0096] 1204、When the first quantity is greater than the first preset quantity threshold, the point cloud data with the first coordinate value greater than the preset first coordinate threshold is removed.

[0097] In the embodiment of the present application, when there are multiple point cloud data in the first grid, the point cloud data with the first coordinate value greater than the preset first coordinate threshold can be removed, so as to remove the mechanical equipment, roof, column, etc. above the bulk material. Figure 1f A schematic diagram of the point cloud of the mechanical equipment above the grain surface is shown, wherein the point cloud A of the mechanical equipment is located above the point cloud data (i.e., the point cloud data of the bulk material) B of the grain surface.

[0098] For example, the first preset quantity threshold can be set to 1, and when the quantity of the point cloud data in the first grid is greater than 1, only the point cloud data with the minimum z coordinate value in the first grid is retained, so as to remove other point cloud data that does not belong to the bulk material.

[0099] 1205、The point cloud data after the first coordinate value deduplication processing is obtained, and the point cloud data after the first coordinate value deduplication processing is taken as the second point cloud data after the deduplication processing.

[0100] In the embodiment of the present application, by performing the deduplication processing in the z coordinate axis direction on the first point cloud data, the second point cloud data after the deduplication processing is obtained, so as to further filter out the non-bulk material point cloud data such as the mechanical equipment, roof, column, etc. above the bulk material.

[0101] 130、Project the second point cloud data to a first preset plane of a preset coordinate system to obtain third point cloud data after projection.

[0102] In the embodiment of the present application, the third point cloud data is the point cloud data after the projection of the second point cloud data to the first preset plane of the preset coordinate system.

[0103] The first preset plane of the preset coordinate system is the xOy plane, and after the projection of the second point cloud data to the first preset plane of the preset coordinate system in the embodiment of the present application, the third point cloud data after the projection can be obtained.

[0104] 140、Determine a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0105] The statistical distribution value is a measurement basis index in the statistical distribution degree, and the statistical distribution value includes the average value, the standard deviation, etc.

[0106] The average value represents a quantity of a trend in a data set, is a sum of all data in a data set divided by the number of data in the data set, and reflects a trend in the data set.

[0107] In an embodiment, determining the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction includes:

[0108] Dividing the first preset plane into a second plane grid, the second plane grid having a second grid;

[0109] Determining a second number of point cloud data corresponding to the second grid in the first coordinate axis direction;

[0110] When the second number is greater than a second preset number threshold, determining a first coordinate value average and a first coordinate value standard deviation of the point cloud data corresponding to the second grid in the first coordinate axis direction;

[0111] Taking the first coordinate value average and the first coordinate value standard deviation as the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0112] In an embodiment, the point cloud can be projected on an xOy plane (i.e., a first preset plane) first, the first preset plane is divided into a second plane grid, the second plane grid having a second grid, for example, the second plane grid can be evenly divided into a second grid with a size of 0.5m*0.5m.

[0113] Each second grid may have no point, one point, or multiple points in the vertical z coordinate axis direction, and when there are multiple points, the z coordinate values of all points in each second grid range are averaged and the standard deviation is determined. For example, the second preset number threshold can be set to 1, when the second number is greater than 1, the first coordinate value average and the first coordinate value standard deviation of the point cloud data corresponding to the second grid in the first coordinate axis direction can be determined, and then the first coordinate value average and the first coordinate value standard deviation are taken as the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0114] 150, based on the statistical distribution value, performing a projection point cloud filtering process on the third point cloud data to obtain fourth point cloud data after filtering.

[0115] In an embodiment, the fourth point cloud data is the point cloud data after the projection point cloud filtering process on the third point cloud data.

[0116] In order to better distinguish the stacking object point cloud data (such as grain surface point cloud) from other point clouds, the embodiment of the present application can perform ground projection (i.e., projection to the xOy plane) on the third point cloud data, then determine the statistical distribution value corresponding to the third point cloud data in the direction of the first coordinate axis, and then perform point cloud filtering processing according to the statistical distribution value.

[0117] In an embodiment, based on the statistical distribution value, the projection point cloud filtering processing is performed on the third point cloud data, including:

[0118] When the first coordinate value standard deviation is not less than the standard deviation threshold, the point cloud data corresponding to the first coordinate value less than the average value threshold of the second grid is obtained;

[0119] The point cloud data with the first coordinate value less than the average value threshold is taken as the filtered fourth point cloud data;

[0120] When the first coordinate value standard deviation is less than the standard deviation threshold, all point cloud data corresponding to the second grid is taken as the filtered fourth point cloud data.

[0121] In the embodiment of the present application, the first coordinate value standard deviation and the first coordinate value average value can be calculated in each second grid, if the first coordinate value standard deviation is less than the standard deviation threshold, all points in the second grid are retained; if the first coordinate value standard deviation is greater than or equal to the standard deviation threshold, only the point with the z coordinate value less than the average value is retained. In the embodiment of the present application, each second grid is traversed, when there are multiple points in the second grid, the average value mean_z and the standard deviation std_z of the z coordinate values of all points are calculated, if: the standard deviation std_z< the standard deviation threshold THR_Z, all points are retained; otherwise: all points in the second grid are traversed, when the z coordinate value point.z of a point in the second grid is less than the average value mean_z, the point is retained.

[0122] In the embodiment of the present application, because the curvature change of the point cloud of the stacking object (such as the grain surface) in the z direction is much smaller than that of the wall surface, column and mechanical equipment in the z direction, the z coordinate value standard deviation of the second grid can distinguish the grain surface. It is not excluded that there are both grain surface point clouds and other object point clouds in the second grid, so when the standard deviation is greater than or equal to the standard deviation threshold, the point with the z coordinate value less than the average value is retained, which can retain effective grain surface information.

[0123] 160、According to the fourth point cloud data, the extracted stacking object point cloud data is obtained.

[0124] After the projection point cloud filtering processing, the present application can obtain relatively accurate stacking object point cloud data, so as to realize the extraction of the stacking object point cloud data from the first point cloud data.

[0125] In addition, since most irrelevant point clouds are filtered out from the point cloud, the point cloud of the grain surface is retained. However, there will still be some discrete noise that is not filtered out completely, such as Figure 1g As shown in FIG. 6, the discrete noise C is far away from other adjacent point clouds. At this time, the embodiment of the present application can use statistical filtering to filter out the discrete noise for one time or multiple times.

[0126] In an embodiment, after the projection point cloud filtering processing is performed on the third point cloud data based on the statistical distribution value to obtain the fourth point cloud data after filtering, the method further includes:

[0127] determining a preset neighborhood of point cloud data in the fourth point cloud data;

[0128] obtaining distances between the point cloud data in the fourth point cloud data and other point cloud data in the preset neighborhood;

[0129] calculating an average distance according to the distances between the point cloud data and the other point cloud data in the preset neighborhood;

[0130] removing point cloud data in the fourth point cloud data with an average distance greater than a distance threshold value;

[0131] obtaining point cloud data after statistical filtering;

[0132] taking the point cloud data after statistical filtering as extracted heap object point cloud data.

[0133] In the embodiment of the present application, statistical filtering can be performed on the fourth point cloud data obtained after the projection point cloud filtering processing, that is, a neighborhood of each point in the fourth point cloud data is statistically analyzed, and some points that do not meet the standard are pruned. Specifically, the average distance of each point to all adjacent points can be calculated, the points with an average distance outside the standard range are defined as discrete noise, and the points are removed from the point cloud data.

[0134] For example, the embodiment of the present application can first determine a preset neighborhood of a point, then obtain distances between the point cloud data in the fourth point cloud data and other point cloud data in the preset neighborhood, and then calculate an average distance. When the average distance is greater than a distance threshold value, the point is regarded as an outlier (i.e., discrete noise), and the point is removed. Then, other remaining points in the third point cloud data are traversed.

[0135] This invention provides a point cloud data extraction method. The method first acquires first point cloud data in a preset coordinate system using a scanning system; then, deduplicating the first point cloud data along the first coordinate axis of the preset coordinate system yields deduplicated second point cloud data; the second point cloud data is projected onto a first preset plane of the preset coordinate system to obtain projected third point cloud data; the statistical distribution value corresponding to the third point cloud data along the first coordinate axis is determined; and based on the statistical distribution value, the third point cloud data undergoes projection point cloud filtering to obtain filtered fourth point cloud data. Thus, this invention enables the extraction of aggregate point cloud data from the first point cloud data.

[0136] When extracting point cloud data, this invention can remove duplicate points in the first point cloud data along the first coordinate axis by deduplicating the points in the first point cloud data. This removes other objects in the first coordinate axis direction from the accumulated object. Then, this invention can perform projection point cloud filtering to not only better distinguish the accumulated object point cloud data from other point cloud data, but also retain effective accumulated object information. This invention can effectively extract the accumulated object point cloud data to be extracted from the first point cloud data with high extraction accuracy.

[0137] To better implement the above methods, this embodiment of the invention also provides a point cloud data extraction device, which can be integrated into an electronic device, such as a terminal or a server. The terminal can be a mobile phone, tablet computer, smart Bluetooth device, laptop computer, or personal computer; the server can be a single server or a server cluster composed of multiple servers.

[0138] For example, in this embodiment, the method of the present invention will be described in detail by taking the point cloud data extraction device specifically integrated into the server as an example.

[0139] For example, such as Figure 2 As shown, the point cloud data extraction device may include a point cloud data acquisition unit 201, a deduplication processing unit 202, a projection processing unit 203, a distribution value determination unit 204, a filtering processing unit 205, and a point cloud extraction unit 206, as follows:

[0140] (I) Point Cloud Data Acquisition Unit 201

[0141] The point cloud data acquisition unit 201 is used to acquire the first point cloud data in a preset coordinate system. The first point cloud data includes the point cloud data of the aggregate to be extracted.

[0142] In some embodiments, the point cloud data extraction apparatus further includes a downsampling unit, which is used for:

[0143] The first point cloud data is down-sampled based on a preset spatial range, and point cloud data within the preset spatial range is obtained from the first point cloud data.

[0144] In some embodiments, the down-sampling unit comprises a conditional filtering unit, and the conditional filtering unit is configured to:

[0145] determine first, second, and third coordinate values of the point cloud data in the first point cloud data;

[0146] When the first coordinate value meets a first preset condition, the second coordinate value meets a second preset condition, and the third coordinate value meets a third preset condition, the point cloud data in the first point cloud data that meets the preset conditions is obtained.

[0147] In some embodiments, in the conditional filtering unit, the first preset condition is that the first coordinate value is greater than a first minimum coordinate threshold and less than a first maximum coordinate threshold; the second preset condition is that the second coordinate value is greater than a second minimum coordinate threshold and less than a second maximum coordinate threshold; and the third preset condition is that the third coordinate value is greater than a third minimum coordinate threshold and less than a third maximum coordinate threshold.

[0148] (ii) De-duplication processing unit 202

[0149] The de-duplication processing unit 202 is configured to perform point cloud de-duplication on the first point cloud data in a first coordinate axis direction of a preset coordinate system to obtain second point cloud data after de-duplication processing.

[0150] In some embodiments, the de-duplication processing unit 202 comprises a de-duplication processing subunit, and the de-duplication processing subunit comprises:

[0151] A three-dimensional grid is established based on the preset coordinate system, the three-dimensional grid comprises a plurality of voxel grids with a preset three-dimensional range, and there is at most one point cloud data in each voxel grid;

[0152] Based on the preset three-dimensional range of the voxel grid, a first plane grid corresponding to a first preset plane of the preset coordinate system is obtained from the three-dimensional grid; the first plane grid has a first grid;

[0153] The first number of point cloud data corresponding to the first grid in the first coordinate axis direction is determined;

[0154] When the first number is greater than a first preset number threshold, point cloud data with a first coordinate value greater than a preset first coordinate threshold is removed;

[0155] Point cloud data after de-duplication processing of the first coordinate value is obtained, and the point cloud data after de-duplication processing of the first coordinate value is taken as the second point cloud data after de-duplication processing.

[0156] (iii) Projection processing unit 203

[0157] The projection processing unit 203 is configured to project the second point cloud data onto a first preset plane of a preset coordinate system to obtain third point cloud data after projection.

[0158] (iv) The distribution value determination unit 204

[0159] The distribution value determination unit 204 is configured to determine a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0160] In some embodiments, the distribution value determination unit 204 comprises a distribution value determination subunit, which is configured to:

[0161] divide the first preset plane into a second plane grid, the second plane grid having a second grid;

[0162] determine a second number of point cloud data corresponding to the second grid in the first coordinate axis direction;

[0163] when the second number is greater than a second preset number threshold, determine a first coordinate value average and a first coordinate value standard deviation of the point cloud data corresponding to the second grid in the first coordinate axis direction;

[0164] take the first coordinate value average and the first coordinate value standard deviation as the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction.

[0165] (v) The filtering processing unit 205

[0166] The filtering processing unit 205 is configured to perform projection point cloud filtering processing on the third point cloud data based on the statistical distribution value to obtain fourth point cloud data after filtering.

[0167] In some embodiments, the filtering processing unit 205 comprises a filtering processing subunit, which comprises:

[0168] when the first coordinate value standard deviation is not less than a standard deviation threshold, obtain point cloud data corresponding to the second grid and having a first coordinate value less than an average value threshold;

[0169] take the point cloud data having the first coordinate value less than the average value threshold as the fourth point cloud data after filtering;

[0170] when the first coordinate value standard deviation is less than the standard deviation threshold, take all point cloud data corresponding to the second grid as the fourth point cloud data after filtering.

[0171] (vi) The point cloud extraction unit 206

[0172] The point cloud extraction unit 206 is configured to obtain extracted heap object point cloud data according to the fourth point cloud data.

[0173] In some embodiments, the point cloud extraction unit 206 comprises a statistical filtering unit, which is configured to:

[0174] determine a preset neighborhood of the point cloud data in the fourth point cloud data;

[0175] obtain distances between the point cloud data in the fourth point cloud data and other point cloud data within the preset neighborhood;

[0176] calculate an average distance according to the distances between the point cloud data in the fourth point cloud data and other point cloud data within the preset neighborhood;

[0177] remove the point cloud data in the fourth point cloud data whose average distance is greater than a distance threshold;

[0178] obtain the point cloud data after statistical filtering;

[0179] use the point cloud data after statistical filtering as the extracted stockpile point cloud data.

[0180] In implementation, the above various modules can be implemented as independent entities, or can be combined as the same or several entities, and the specific implementation of the above various modules can be referred to the method embodiments above, which will not be described here.

[0181] As can be seen from the above, the point cloud data extraction device of the present embodiment can obtain the first point cloud data in the preset coordinate system through the scanning system; the first point cloud data is subjected to point cloud deduplication in the direction of the first coordinate axis of the preset coordinate system to obtain the second point cloud data after deduplication processing; the second point cloud data is projected onto the first preset plane of the preset coordinate system to obtain the third point cloud data after projection; the statistical distribution value corresponding to the third point cloud data in the direction of the first coordinate axis is determined; and the third point cloud data is subjected to projection point cloud filtering processing based on the statistical distribution value to obtain the fourth point cloud data after filtering. In this way, the present application can obtain the extracted stockpile point cloud data according to the fourth point cloud data, so as to extract the stockpile point cloud data from the first point cloud data.

[0182] When extracting the point cloud data, the present application can eliminate other objects of the stockpile in the direction of the first coordinate axis by performing point cloud deduplication in the direction of the first coordinate axis of the first point cloud data, and then the present application can perform projection point cloud filtering processing, so as to not only better distinguish the stockpile point cloud data from other point cloud data, but also retain the effective stockpile information. The present application can effectively extract the stockpile point cloud data to be extracted from the first point cloud data, and the extraction precision is high.

[0183] Correspondingly, the embodiment of the present application also provides an electronic device, which can be a terminal or a server. The terminal can be a terminal device such as a smart phone, a tablet computer, a notebook computer, a touch screen, a game machine, a personal computer, a personal digital assistant (PDA), etc. The server can be a single server or a server cluster composed of multiple servers.

[0184] As shown in Figure 3 , Figure 3 The electronic device structure schematic diagram provided by the embodiment of the present application includes a memory 301, a processor 302 and a communication module 303.

[0185] The memory 301 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read only memory (PROM), an erasable programmable read only memory (EPROM), an electric erasable programmable read only memory (EEPROM), a magnetic disk or a solid state disk, etc. The memory 301 is used to store a program, and the processor 302 executes the program after receiving an execution instruction.

[0186] The processor 302 can be an integrated circuit chip with data processing capability. The processor 302 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor 302 can implement or execute the methods, steps and logic block diagrams in the present application. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0187] The communication module 303 is used for communication connection between the electronic device and an external device, and realizes the transceiving operation of network signals and data. The network signals can include wireless signals or wired signals.

[0188] The specific implementation of each module can refer to the above embodiments, which will not be described here.

[0189] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described in detail in a certain embodiment can refer to the relevant description of other embodiments.

[0190] As can be seen from the above, the electronic device provided in the embodiment can effectively extract the to-be-extracted pile point cloud data from other point cloud data, and the extraction precision is high.

[0191] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling relevant hardware, which can be stored in a computer readable storage medium and loaded and executed by a processor.

[0192] To this end, the embodiment of the present application provides a computer readable storage medium, which stores a plurality of computer programs capable of being loaded by a processor to execute the steps in any of the point cloud data extraction methods provided by the embodiments of the present application.

[0193] The specific implementation of each operation can refer to the foregoing embodiments, which will not be described here.

[0194] The storage medium can include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0195] Since the computer program stored in the storage medium can execute the steps in any of the point cloud data extraction methods provided by the embodiments of the present application, the beneficial effects of any of the point cloud data extraction methods provided by the embodiments of the present application can be achieved, which will be described in detail in the foregoing embodiments, and will not be described here.

[0196] The above describes in detail the point cloud data extraction method, device, electronic device and storage medium provided by the embodiments of the present application. The principle and implementation manner of the present application are described by applying specific examples in this paper. The above embodiment is only used to help understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A point cloud data extraction method, characterized by, The method comprises: acquiring first point cloud data under a preset coordinate system, the first point cloud data comprising point cloud data of a stockpile to be extracted; performing point cloud deduplication on the first point cloud data in a first coordinate axis direction of the preset coordinate system to obtain second point cloud data after deduplication processing; projecting the second point cloud data onto a first preset plane of the preset coordinate system to obtain third point cloud data after projection; determining a statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction; performing projection point cloud filtering processing on the third point cloud data based on the statistical distribution value to obtain fourth point cloud data after filtering; obtaining extracted point cloud data of the stockpile according to the fourth point cloud data; wherein the preset coordinate system is a coordinate system established with a stockpile in a storage bin, the height of the storage bin being the first coordinate axis, and the point cloud deduplication on the first point cloud data in the first coordinate axis direction of the preset coordinate system comprises: establishing a three-dimensional grid based on the preset coordinate system, the three-dimensional grid comprising a plurality of voxel grids with a preset three-dimensional range, and there being at most one point cloud data in each voxel grid; acquiring a first plane grid corresponding to the first preset plane of the preset coordinate system based on the preset three-dimensional range of the voxel grid; the first plane grid has a first grid; determining a first number of point cloud data corresponding to the first grid in the first coordinate axis direction; when the first number is greater than a first preset number threshold, removing point cloud data with a first coordinate value greater than a preset first coordinate threshold; obtaining point cloud data after first coordinate value deduplication processing, and taking the point cloud data after first coordinate value deduplication processing as the second point cloud data after deduplication processing.

2. The point cloud data extraction method of claim 1, wherein, Before the point cloud deduplication on the first point cloud data in the first coordinate axis direction of the preset coordinate system, the method further comprises: based on a preset spatial range, down-sampling the first point cloud data to obtain point cloud data within the preset spatial range from the first point cloud data.

3. The point cloud data extraction method of claim 2, wherein, Based on a preset spatial range, down-sampling the first point cloud data comprises: determining a first coordinate value, a second coordinate value and a third coordinate value of point cloud data in the first point cloud data; when the first coordinate value meets a first preset condition, the second coordinate value meets a second preset condition and the third coordinate value meets a third preset condition, obtaining point cloud data in the first point cloud data that meets the preset conditions.

4. The point cloud data extraction method of claim 3, wherein, The first preset condition is that the first coordinate value is greater than a first coordinate minimum threshold and less than a first coordinate maximum threshold; the second preset condition is that the second coordinate value is greater than a second coordinate minimum threshold and less than a second coordinate maximum threshold; and the third preset condition is that the third coordinate value is greater than a third coordinate minimum threshold and less than a third coordinate maximum threshold.

5. The point cloud data extraction method of claim 1, wherein, The determination of the statistical distribution value corresponding to the third point cloud data in the first coordinate axis direction comprises: dividing the first preset plane into a second plane grid, the second plane grid having a second grid; determining a second number of point cloud data corresponding to the second grid in the first coordinate axis direction; when the second number is greater than a second preset number threshold, determining a first coordinate value average and a first coordinate value standard deviation of point cloud data corresponding to the second grid in the first coordinate axis direction; taking the first coordinate value average and the first coordinate value standard deviation as statistical distribution values corresponding to the third point cloud data in the first coordinate axis direction.

6. The point cloud data extraction method of claim 5, wherein, the projection point cloud filtering processing of the third point cloud data based on the statistical distribution values includes: when the first coordinate value standard deviation is not less than a standard deviation threshold, obtaining point cloud data corresponding to the second grid and less than an average value threshold; taking the point cloud data less than the average value threshold as the fourth point cloud data after filtering; when the first coordinate value standard deviation is less than the standard deviation threshold, taking all point cloud data corresponding to the second grid as the fourth point cloud data after filtering.

7. The point cloud data extraction method of claim 1 or 6, wherein, after the projection point cloud filtering processing of the third point cloud data based on the statistical distribution values to obtain the fourth point cloud data after filtering, the method further includes: determining a preset neighborhood of point cloud data in the fourth point cloud data; obtaining distances between the point cloud data in the fourth point cloud data and other point cloud data in the preset neighborhood; calculating an average distance according to the distances between the point cloud data and the other point cloud data in the preset neighborhood; removing point cloud data in the fourth point cloud data with an average distance greater than a distance threshold; obtaining statistical filtered point cloud data; taking the statistical filtered point cloud data as extracted stockpile point cloud data.

8. A point cloud data extraction apparatus characterized by comprising: The device includes: a point cloud data acquisition unit configured to acquire first point cloud data in a preset coordinate system, the first point cloud data including stockpile point cloud data to be extracted; a de-duplication processing unit configured to perform point cloud de-duplication on the first point cloud data in a first coordinate axis direction of the preset coordinate system to obtain second point cloud data after de-duplication processing; a projection processing unit configured to project the second point cloud data onto a first preset plane of the preset coordinate system to obtain third point cloud data after projection; a distribution value determination unit configured to determine statistical distribution values corresponding to the third point cloud data in the first coordinate axis direction; a filtering processing unit configured to perform projection point cloud filtering processing on the third point cloud data based on the statistical distribution values to obtain fourth point cloud data after filtering; a point cloud extraction unit configured to obtain extracted stockpile point cloud data according to the fourth point cloud data to extract the stockpile point cloud data from the first point cloud data. The preset coordinate system is a coordinate system established based on a stockpile, the height of the stockpile is the first coordinate axis, and the de-duplication processing unit includes a de-duplication processing subunit, which includes: establishing a three-dimensional grid based on the preset coordinate system, the three-dimensional grid including a plurality of voxel grids with a preset three-dimensional range, and there being at most one point cloud data in each voxel grid; Based on the preset three-dimensional range of the voxel grid, a first plane grid corresponding to a first preset plane of a preset coordinate system is obtained from the three-dimensional grid; the first plane grid has a first grid; A first quantity of point cloud data corresponding to the first grid in a first coordinate axis direction is determined; when the first quantity is greater than a first preset quantity threshold, point cloud data with a first coordinate value greater than a preset first coordinate threshold is removed; Point cloud data after the first coordinate value is de-duplicated is obtained, and the point cloud data after the first coordinate value is de-duplicated is taken as second point cloud data after de-duplication.

9. An electronic device, comprising: The device comprises a processor and a memory, the memory stores a plurality of instructions, and the processor loads the instructions to execute the steps in the point cloud data extraction method of any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the point cloud data extraction method of any one of claims 1 to 7.

Images