A method, device, and medium for identifying sparse point cloud data

Abstract

The application discloses a kind of identification method, device, equipment and medium of sparse point cloud data, the method includes: obtaining denoising point cloud data and voxelization processing to obtain each three-dimensional voxel matrix, difference processing is carried out to three-dimensional voxel matrix to obtain difference matrix, mean filter processing or low-pass filter processing is used to determine periodic motion voxel and general motion voxel in difference matrix, based on motion keeping algorithm, update the motion state corresponding to general motion voxel and stationary voxel, based on clustering algorithm and iterative clustering algorithm processing, the target point cloud data corresponding to the periodic motion voxel, general motion voxel and stationary voxel determined are used to obtain segmentation point cloud data, based on Pointnet + + algorithm or convolutional neural network, the identification result of sparse point cloud data is obtained by identifying segmentation point cloud data, by difference combination filtering algorithm of voxel matrix, improve the data processing speed, the ability of identifying sparse point cloud and the efficiency of entire algorithm procedure.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer science, and in particular to a sparse point cloud data recognition method, device, equipment and medium. BACKGROUND

[0002] In the safety monitoring and management of power operation sites, laser radar technology is widely used due to its high precision, long-distance measurement and three-dimensional modeling capabilities. Especially in complex and variable operating environments, laser radars can generate detailed point cloud data, providing strong data support for on-site safety monitoring and early warning.

[0003] In related technologies, object recognition and safety risk assessment for power operation sites have the shortcomings of long data processing time, poor small object recognition ability, and strong noise sensitivity, which leads to limitations in processing large-scale, sparse and noisy point cloud data.

[0004] Therefore, how to improve the training efficiency, and at the same time enhance the small object recognition ability and noise robustness, is of great significance to improve the safety monitoring level of power operation sites. SUMMARY

[0005] In view of the above problems, the present application provides a sparse point cloud data recognition method, device, electronic equipment and storage medium to at least solve the problems in related technologies.

[0006] In a first aspect, the embodiments of the present application provide a sparse point cloud data recognition method, the sparse point cloud data recognition method comprising:

[0007] Obtaining denoised point cloud data and performing voxelization processing to obtain a plurality of three-dimensional voxel matrices;

[0008] Difference processing the three-dimensional voxel matrices based on a preset number of consecutive frames to obtain a difference matrix, and determining moving voxels and stationary voxels in the difference matrix;

[0009] Performing mean filtering processing or low-pass filtering processing on the moving voxels to determine periodic motion voxels and general motion voxels;

[0010] Updating the motion states corresponding to the stationary voxels and general motion voxels based on a motion preserving algorithm;

[0011] Processing the target point cloud data corresponding to the periodic motion voxels, general motion voxels and stationary voxels determined based on a DBSCAN clustering algorithm and an iterative clustering algorithm to obtain segmented point cloud data;

[0012] Performing recognition on the segmented point cloud data based on a Pointnet++ algorithm or a convolutional neural network to obtain a sparse point cloud data recognition result.

[0013] In some embodiments, updating the motion states corresponding to the stationary voxels and the general moving voxels based on the motion-preserving algorithm includes:

[0014] Detect whether the matrix corresponding to the stationary voxel and the general moving voxel is in a stationary state and the number of points inside the matrix;

[0015] If it is determined that the matrix is ​​not in a static state and the number of points inside the matrix is ​​not zero, then the static voxel and the generally moving voxel are in a generally moving state.

[0016] If it is determined that the matrix is ​​in a static state and the number of points inside the matrix is ​​not zero, a static countdown segment is set for the matrix;

[0017] When the matrix exceeds the time threshold of the static countdown segment or the number of points inside the matrix is ​​zero, it is determined that the static voxel and the general moving voxel are in a static state.

[0018] In some embodiments, applying mean filtering or low-pass filtering to the motion voxels to determine periodic motion voxels and general motion voxels includes:

[0019] Determine the voxel matrix that changes across consecutive time frames for the moving voxels;

[0020] A discrete-time signal sequence is established based on the voxel matrix;

[0021] Given that mean filtering is to be used, the average value of the discrete-time signal sequence is calculated.

[0022] The average value is stored in the filtered voxel matrix;

[0023] After mean filtering, if the average value is greater than a first set threshold, the voxel matrix is ​​marked as periodic motion voxels; otherwise, the voxel matrix is ​​marked as general motion voxels.

[0024] In some embodiments, the step of applying mean filtering or low-pass filtering to the motion voxels to determine periodic motion voxels and general motion voxels further includes:

[0025] Determine the voxel matrix that changes across consecutive time frames for the moving voxels;

[0026] A discrete-time signal sequence is established based on the voxel matrix;

[0027] Given that low-pass filtering is to be used, a calculation formula is constructed based on the discrete-time signal sequence to determine the assumed calculated value after low-pass filtering.

[0028] The assumed calculated value is iteratively processed to obtain an iterative value, and the iterative value is stored in the filtered voxel matrix;

[0029] After low-pass filtering, when the iteration value is greater than the second set threshold, the voxel matrix is ​​marked as a periodic motion voxel; otherwise, the voxel matrix is ​​marked as a general motion voxel.

[0030] In some embodiments, the step of processing the target point cloud data corresponding to the periodic moving voxels, general moving voxels, and stationary voxels determined by the DBSCAN clustering algorithm and iterative clustering algorithm to obtain segmented point cloud data includes:

[0031] Based on the DBSCAN clustering algorithm, outliers are removed from the center points of the target point cloud data corresponding to each voxel.

[0032] The clustering results of each type of center point are back-mapped to the target point cloud data and labeled;

[0033] An iterative clustering algorithm is used to establish a polymorphic body with the center point of the same label, and each face of the polymorphic body is expanded sequentially to obtain the expanded region.

[0034] When the point density of the expansion region is greater than the target threshold, the expansion requirement is met;

[0035] The segmentation is considered complete and segmentation point cloud data is obtained when all faces of the polymorph are amplified ineffectively.

[0036] In some embodiments, updating the motion states corresponding to the stationary voxels and the general moving voxels based on the motion-preservation algorithm further includes:

[0037] Detect changes in the number of points within the matrix;

[0038] If the change in quantity exceeds a preset quantity, the static countdown segment is reset.

[0039] In some embodiments, acquiring the denoised point cloud data includes:

[0040] Acquire sparse point cloud data collected by lidar;

[0041] The sparse point cloud data is preprocessed based on distance denoising to obtain denoised point cloud data.

[0042] Secondly, embodiments of this application provide a device for identifying sparse point cloud data, including:

[0043] The acquisition module is used to acquire denoised point cloud data and perform voxelization processing to obtain various three-dimensional voxel matrices.

[0044] The first processing module is used to perform subtraction processing on the three-dimensional voxel matrix based on a preset number of consecutive frames to obtain a difference matrix, and to determine the moving voxels and stationary voxels in the difference matrix.

[0045] The second processing module is used to apply mean filtering or low-pass filtering to the motion voxels to determine periodic motion voxels and general motion voxels.

[0046] The update module is used to update the motion state corresponding to the stationary voxel and the general moving voxel based on the motion preservation algorithm;

[0047] The third processing module is used to process the target point cloud data corresponding to the periodic moving voxels, general moving voxels and stationary voxels determined by the DBSCAN clustering algorithm and the iterative clustering algorithm to obtain segmented point cloud data.

[0048] The recognition module is used to identify the segmented point cloud data based on the Pointnet++ algorithm or a convolutional neural network to obtain the recognition result of the sparse point cloud data.

[0049] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores program code that can run on the processor, and when the program code is executed by the processor, it implements the sparse point cloud data identification method as described in any embodiment of the first aspect.

[0050] Fourthly, embodiments of this application provide a computer storage medium storing one or more programs, which can be executed by an electronic device as described in the third aspect to implement the sparse point cloud data identification method as described in any embodiment of the first aspect.

[0051] This application provides a method, apparatus, device, and medium for identifying sparse point cloud data. The method involves acquiring denoised point cloud data and performing voxelization to obtain three-dimensional voxel matrices; performing subtraction on the three-dimensional voxel matrices based on a preset number of consecutive frames to obtain a difference matrix, and determining moving and stationary voxels in the difference matrix; applying mean filtering or low-pass filtering to the moving voxels to determine periodic moving voxels and general moving voxels; updating the motion states corresponding to stationary and general moving voxels based on a motion-preserving algorithm; processing the determined periodic moving voxels, general moving voxels, and stationary voxels using DBSCAN clustering and iterative clustering algorithms to obtain segmented point cloud data; and identifying the segmented point cloud data using Pointnet++ algorithm or convolutional neural network to obtain the identification result of sparse point cloud data. By using the voxel matrix subtraction method combined with filtering algorithms, the data processing speed is improved, effectively enhancing the ability to identify sparse point cloud objects and segment sparse moving point cloud scenes, thus improving the efficiency of the entire algorithm process. Combined with DBSCAN clustering, Pointnet++, and iterative clustering algorithms, the method further enhances the sparse point cloud data identification result. The Net++ algorithm and iterative clustering algorithm can identify clusters of specific shapes, making the cluster feature values ​​distinct and effectively realizing the segmentation capability of sparse moving point cloud scenes, ultimately improving the computational accuracy.

[0052] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0053] The present application will be described in more detail below based on embodiments and with reference to the accompanying drawings.

[0054] Figure 1 This paper shows a schematic flowchart of a sparse point cloud data identification method proposed in one embodiment of the present application;

[0055] Figure 2 An exemplary overall process flowchart according to an embodiment of this application is shown;

[0056] Figure 3 A flowchart illustrating an exemplary motion-maintaining algorithm processing method proposed in one embodiment of this application is shown.

[0057] Figure 4 This paper shows a structural block diagram of a sparse point cloud data identification device proposed in one embodiment of the present application;

[0058] Figure 5 A structural block diagram of an electronic device for performing a method for identifying sparse point cloud data according to an embodiment of this application is shown.

[0059] Figure 6 A computer-readable storage medium for storing or carrying a method for identifying sparse point cloud data according to an embodiment of this application is shown. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0061] In the research on related technologies, the traditional industry commonly uses the PointNet++ principle as the core algorithm for processing these point cloud data to achieve object recognition and safety risk assessment at power operation sites. However, the traditional PointNet++ has limitations when processing large-scale, sparse, and noisy point cloud data due to its long data processing time, poor ability to recognize small objects, and high sensitivity to noise.

[0062] To address the aforementioned issues, the applicant proposes a method, apparatus, electronic device, and storage medium for identifying sparse point cloud data, as provided in the embodiments of this application. Specifically, after collecting point cloud data from a power operation site using LiDAR, to address the poor scene segmentation and object clustering classification capabilities of the Point Net++ algorithm for sparse point clouds, voxelization processing is introduced. The deviation between voxels in two consecutive frames is used to distinguish moving points from temporarily stationary points in the point cloud data. For periodic moving points and other moving points, mean filtering or low-pass filtering can be used for segmentation. A motion-preserving algorithm is employed for moving points. An iterative clustering method is used, which can identify cluster shapes with special characteristics, improving the algorithm's ability to identify and segment sparse moving point cloud scenes. The method for identifying sparse point cloud data will be described in detail in subsequent embodiments.

[0063] The following describes the application scenarios of the sparse point cloud data identification method provided in the embodiments of this application:

[0064] Please see Figure 1 , Figure 1 This is a schematic flowchart illustrating a method for identifying sparse point cloud data provided in this application embodiment. In this embodiment, the method for identifying sparse point cloud data can be applied to, for example... Figure 4 The sparse point cloud data identification device 300 shown is... Figure 5 The electronic device 200 shown may include one or more devices. For example, the electronic device may include a mobile terminal, computer, tablet, etc., and this application does not limit it. The following describes... Figure 1 The process shown is described in detail, and the method for identifying sparse point cloud data may include S110 to S160.

[0065] S110: Acquire denoised point cloud data and perform voxelization to obtain each three-dimensional voxel matrix.

[0066] In some implementations, obtaining denoised point cloud data may include steps S101 to S102.

[0067] S101: Acquire sparse point cloud data collected by lidar.

[0068] In this embodiment, a solid-state lidar can be connected to a computer. When collecting data at a power operation site, the lidar can be fixed on the ground. A level must be used to ensure that the lidar lens is perpendicular to the ground. The lidar must remain stationary throughout the test to collect sparse point cloud data.

[0069] S102: Preprocess the sparse point cloud data based on distance denoising to obtain denoised point cloud data.

[0070] In this embodiment, considering the large scale of power operation sites, the distance between point clouds and lidar, and the low density of collected point clouds, a distance-based denoising method is adopted. For each point, if the distance to neighboring points within a certain radius (which can be the minimum or average of the nearest 3-5 points) is greater than a certain threshold, the point is considered an isolated point and removed, thus obtaining denoised point cloud data.

[0071] In this embodiment, noise reduction is achieved by removing isolated points.

[0072] In this embodiment, the denoised point cloud data is voxelized. At the beginning of point cloud acquisition, a set of consecutive preset frames of point cloud data (e.g., 5 frames) are input. The left range of the point cloud data on the x, y, and z axes is calculated for each frame. The union of these preset frame ranges is taken as the initial point cloud range. A certain distance (0.5 meters or 5% of the axis length, depending on the actual situation) is added to each side and direction of this initial range as a boundary to construct the range of the point cloud recognition scene. For example, if the initial range is x: 1–5 meters, it will become x: 0.5–5.5 meters, and so on for other axes. To reduce the impact of dynamic factors such as obstacle movement on scene edge judgment, the point cloud data is divided into several three-dimensional voxels (e.g., cube voxels) by setting a division precision (generally 0.1 meters). The number of points within each voxel is calculated, and a three-dimensional voxel matrix is ​​created according to the voxel scale. Each element in the voxel matrix stores the number of points within its voxel range.

[0073] For example, the final scene's range is (with 0.5 meters added to each side and direction) x: 0~5 meters, y: 0~5 meters, z: 0~5 meters, with a division precision of 0.1 meters, i.e., divided into small cubes of 0.1*0.1*0.1 (unit: meters). A three-dimensional matrix VOXEL (assuming this is a C++ three-dimensional array) stores this voxel. If there are 3 points within the range of x: 0~0.1 meters, y: 0~0.1 meters, z: 0~0.1 meters, then VOXEL[0][0][0] = 3, and so on.

[0074] S120: Based on a preset number of consecutive frames, perform a difference operation on the three-dimensional voxel matrix to obtain a difference matrix, and determine the moving voxels and stationary voxels in the difference matrix.

[0075] In this embodiment, the voxel matrices of two consecutive frames can be subtracted to obtain the difference matrix between the two voxel matrices. A non-zero difference matrix indicates a moving voxel, while a zero difference matrix indicates a stationary voxel. Furthermore, voxels whose absolute values ​​in the difference matrix are greater than a certain threshold are marked as changing voxels, and the absolute value of the change in these changing voxels is stored in the changing voxel matrix.

[0076] It should be noted that variable voxels can be considered as motion voxels.

[0077] S130: The motion voxels are subjected to mean filtering or low-pass filtering to determine periodic motion voxels and general motion voxels.

[0078] In the embodiments of this application, mean filtering or low-pass filtering can be applied to each element of the changing voxel matrix (moving voxels) over time.

[0079] In some embodiments, S130 may include S131 to S135.

[0080] S131: Determine the voxel matrix that changes over consecutive time frames for the moving voxels.

[0081] In the embodiments of this application, mean filtering or low-pass filtering can be applied to each element of the changing voxel matrix (moving voxels) over time.

[0082] S132: Establish a discrete-time signal sequence based on the voxel matrix.

[0083] In the embodiments of this application, the voxel matrices VOXEL_D1, VOXEL_D2, ..., which change continuously over several frames in time, can be filtered by mean filtering or low-pass filtering. Taking the element at [0][0][0] of each matrix as an example, VOXEL_D1[0][0][0], VOXEL_D2[0][0][0], ..., can be formed into a discrete-time signal sequence.

[0084] S133: If mean filtering is used, calculate the average value of the discrete-time signal sequence.

[0085] S134: Store the average value into the filtered voxel matrix.

[0086] In this embodiment of the application, if mean filtering is used, the average value of this discrete-time signal sequence is calculated and stored in the element VOXEL_D[0][0][0] of the filtered matrix (assuming it is VOXEL_D). The other elements are processed in the same way.

[0087] S135: After mean filtering, when the average value is greater than the first set threshold, the voxel matrix is ​​marked as periodic motion voxels; otherwise, the voxel matrix is ​​marked as general motion voxels.

[0088] In this embodiment of the application, the method for determining periodic motion points and general motion points is as follows: if mean filtering is used, voxels that are greater than a certain threshold after filtering are considered as voxels of periodic motion, and the rest are voxels of general motion.

[0089] In some embodiments, S130 may further include S136 to S138.

[0090] S136: Given that low-pass filtering is to be used, a calculation formula is constructed based on the discrete-time signal sequence to determine the assumed calculated value after low-pass filtering.

[0091] In this embodiment of the application, if a low-pass filter is used, taking a first-order low-pass filter as an example, the first term of the discrete-time signal sequence can be taken as the initial value x(1), the second term as x(2), and so on. The filtered result is assumed to be y(n).

[0092] S137: Iterate the assumed calculated value to obtain the iterative value, and store the iterative value in the filtered voxel matrix.

[0093] In this embodiment of the application, y(n) is specifically calculated as follows:

[0094] y(n) = a*x(n) + (1-a)*y(n-1);

[0095] Where y(0)=0, y(1)=a*x(1)+(1-a)*y(0)=a*x(1), and a is a coefficient determined by the cutoff frequency, which can be obtained through testing in actual engineering. The remaining terms of y(n) can be obtained through this iterative formula, and finally the last term of y(n) is obtained, which is stored in the element VOXEL_D[0][0][0] in the filtered matrix (assuming it is VOXEL_D). The other elements are obtained in the same way.

[0096] S138: After low-pass filtering, when the iteration value is greater than the second set threshold, the voxel matrix is ​​marked as a periodic motion voxel; otherwise, the voxel matrix is ​​marked as a general motion voxel.

[0097] In this embodiment, if a low-pass filter is used, voxels with minimal changes (i.e., values ​​greater than a certain threshold after low-pass filtering) can be marked as periodically moving voxels. These marked periodically moving voxels are stored in another matrix, and the periodically moving voxels in the original matrix are removed. The remaining moving voxels are tentatively designated as representing workers, construction vehicles, etc.

[0098] S140: Update the motion state corresponding to the stationary voxel and the general moving voxel based on the motion-preserving algorithm.

[0099] S140 may include S141 to S144.

[0100] S141: Detect whether the matrix corresponding to the stationary voxel and the general moving voxel is in a stationary state and the number of points inside the matrix.

[0101] S142: If it is determined that the matrix is ​​not in a static state and the number of points inside the matrix is ​​not zero, then the static voxel and the general moving voxel are in a general moving state.

[0102] S143: If it is determined that the matrix is ​​in a static state and the number of points inside the matrix is ​​not zero, a static countdown segment is set for the matrix.

[0103] S144: When the matrix exceeds the time threshold of the static countdown segment or the number of points inside the matrix is ​​zero, it is determined that the static voxel and the general moving voxel are in a static state.

[0104] In the embodiments of this application, please refer to Figure 3 , Figure 3 This application provides an exemplary motion-maintaining algorithm processing flowchart, combining the above steps S141 to S144 and... Figure 3For the unmarked part (called temporarily stationary voxels), the division between the stationary part and the normal moving part can be done by using the motion preservation method. If a voxel was in normal motion in the previous frame and becomes temporarily stationary in this frame, and the value (representing the number of points in the voxel) is not 0, then its motion state is maintained for a certain period of time. When its continuous temporary stationary state exceeds the time threshold or the value is 0, it immediately becomes stationary.

[0105] In some implementations, S140 further includes S145 to S146.

[0106] S145: Detect the change in the number of points inside the matrix.

[0107] S146: If the change in quantity exceeds a preset quantity, reset the static countdown segment.

[0108] In the embodiments of this application, combined with Figure 2 In steps S145 to S146, when a significant change occurs in the value of a temporarily stationary voxel that is considered to be in motion, it is considered to be in motion and its countdown is reset.

[0109] It should be noted that, generally speaking, only non-periodic moving voxels such as staff and vehicles need to be moved to maintain operation.

[0110] In this embodiment, when judging the motion state of parts other than periodic motion, a motion-maintaining method is used for the moving parts. A time threshold is set and the number of points inside the voxel is calculated to determine whether it has become stationary. This provides a double guarantee for improving accuracy.

[0111] S150: The target point cloud data corresponding to the periodic moving voxels, general moving voxels and stationary voxels determined by the DBSCAN clustering algorithm and iterative clustering algorithm are processed to obtain segmented point cloud data.

[0112] In this embodiment of the application, step S150 includes S151 to S155.

[0113] S151: Based on the DBSCAN clustering algorithm, outliers are removed from the center points of the target point cloud data corresponding to each voxel.

[0114] In the embodiments of this application, for the three types of voxels (stationary, periodic motion, and general motion), density-based DBSCAN clustering is first used on the center point of each type of voxel (representing the approximate location of the voxel) to remove outliers.

[0115] For each cluster, the center coordinates of all points are calculated. These center coordinates are only used as reference points for the position of each object, not as the actual center coordinates.

[0116] S152: Back-map the clustering results of each type of center point onto the target point cloud data and label it.

[0117] In this embodiment of the application, the clustering results of each type of voxel center point are back-mapped onto the point cloud and labeled (including whether it is moving and the clustering category).

[0118] For points with the same label, the detailed segmentation can be achieved in the following two ways: First, the point cloud can be clustered using DBSCAN based on the reference center point to obtain the true segmentation result.

[0119] S153: An iterative clustering algorithm is used to establish a polymorphic body with the center point of the same label, and each face of the polymorphic body is expanded sequentially to obtain an expanded region.

[0120] In the embodiments of this application, secondly, cuboid iterative expansion clustering can be adopted. A cuboid is established with the reference center point as the center, and the six faces of the cuboid are expanded. The size of each expansion distance can be smaller than the size of the voxel mesh.

[0121] S154: When the point density of the expansion region is greater than the target threshold, the expansion requirement is met.

[0122] S155: When the amplification of each face of the polymorph is invalid, the segmentation is determined to be complete, and the segmentation point cloud data is obtained.

[0123] In this embodiment, if the point density of the newly expanded region is greater than a certain threshold, the expansion of that surface is considered valid. Each expansion rotates through all six surfaces. If an expansion in any of the six directions is ineffective in a given round, no further expansion is made, and the partitioning is considered complete.

[0124] S160: The segmented point cloud data is identified based on the Pointnet++ algorithm or a convolutional neural network to obtain the identification result of sparse point cloud data.

[0125] In this application embodiment, there are three identification methods for points that have completed clustering:

[0126] In some implementations, the Pointnet++ algorithm can be used to identify the segmented point cloud. Since the above point cloud segmentation algorithm has already achieved the segmentation of point cloud data, directly using its classification method will have a better effect.

[0127] In other implementations, if iterative expansion clustering of polyhedra (cubic prisms) is used, a rough classification can be made based on the shape characteristics of the clustering results. For example, the polyhedral (cubic prism) characteristics of a standing person and a moving vehicle will be quite different.

[0128] In some other implementations, each cluster of point clouds is divided using denser voxels, and then a convolutional neural network is used for identification. Since the volume of the point cloud segmented by the above method is not too large, using a convolutional neural network to classify it will have good results and high efficiency.

[0129] Furthermore, in some possible implementations, the classification results need to take into account the motion state of the point cloud itself (stationary, periodic motion, general motion).

[0130] For example, periodic moving points are likely to be swaying power lines or trees, general moving points are likely to be moving vehicles or workers, and stationary points are generally buildings, utility poles, etc.

[0131] In the process of identification, the motion state of the point cloud can be given a relatively large weight. At the same time, the number of classification categories can be reduced according to the different motion states, such as recognizing that a moving point cannot be a building.

[0132] The three implementation methods described above are not isolated and can be combined as needed, using a weighted evaluation method to obtain the final identification result.

[0133] In this embodiment, the identification of the motion state of points in the point cloud includes voxel subtraction, which pre-segments periodically moving voxels through mean filtering or low-pass filtering, improving the accuracy of subsequent clustering identification within each voxel class and reducing error sources. When identifying points in the point cloud, the motion state of the points is comprehensively considered, reducing the number of classification categories and improving the accuracy and speed of identification.

[0134] In this application, the implementation methods in the above embodiments can be referred to Figure 2 A sample overall flowchart is provided for reference, demonstrating how point cloud identification and analysis at a work site can be achieved by combining the existing Point Net++ algorithm. In the algorithm implementation, outlier identification based on distance is first used for noise reduction. Then, the concept of "voxelization" is introduced, and a voxel matrix is ​​constructed. The moving and temporarily stationary parts in the work site are identified by subtracting the voxel matrices of two consecutive frames and filtering, thus dividing the site into different regions to prepare for subsequent early warning systems. Next, DBSCAN clustering is used to remove outliers and calculate a reference center point. Then, for example, cuboid clustering is used to alternately expand two opposing faces until the point cloud density of the expanded portion falls below a certain threshold, at which point the operation stops, thus completing the segmentation of the sparse point cloud scene.

[0135] Please see Figure 4 , Figure 4This application provides a structural block diagram of a sparse point cloud data identification device 300, which includes: an acquisition module 310, a first processing module 320, a first processing module 330, an update module 340, a third processing module 350, and an identification module 360, wherein:

[0136] The acquisition module 310 is used to acquire denoised point cloud data and perform voxelization processing to obtain various three-dimensional voxel matrices.

[0137] The first processing module 320 is used to perform difference processing on the three-dimensional voxel matrix based on a preset number of consecutive frames to obtain a difference matrix, and to determine the moving voxels and stationary voxels in the difference matrix.

[0138] The second processing module 330 is used to apply mean filtering or low-pass filtering to the motion voxels to determine periodic motion voxels and general motion voxels.

[0139] The update module 340 is used to update the motion state corresponding to the stationary voxel and the general moving voxel based on the motion-maintaining algorithm.

[0140] The third processing module 350 is used to process the target point cloud data corresponding to the periodic moving voxels, general moving voxels and stationary voxels determined by the DBSCAN clustering algorithm and the iterative clustering algorithm to obtain segmented point cloud data.

[0141] The recognition module 360 ​​is used to recognize the segmented point cloud data based on the Pointnet++ algorithm or a convolutional neural network to obtain the recognition result of the sparse point cloud data.

[0142] The device embodiments in this application may also include other modules, specifically corresponding to the above-described method components.

[0143] It should be noted that the device embodiments in this application correspond to the aforementioned method embodiments. The specific principles in the device embodiments can be found in the content of the aforementioned method embodiments, and will not be repeated here.

[0144] In the several embodiments provided in this example, the coupling between modules can be electrical, mechanical, or other forms of coupling.

[0145] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0146] Please see Figure 5 , Figure 5The present application provides a structural block diagram of an electronic device 200 that can perform the above-described method for identifying sparse point cloud data. The electronic device 200 may be a smartphone, tablet computer, computer, or portable computer.

[0147] The electronic device 200 also includes a processor 202 and a memory 204. The memory 204 stores programs that can execute the contents of the foregoing embodiments, and the processor 202 can execute the programs stored in the memory 204.

[0148] The processor 202 may include one or more cores for data processing and message matrix units. The processor 202 connects to various parts of the electronic device 200 using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 204, and by calling data stored in the memory 204. Optionally, the processor 202 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 202 may integrate one or more of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem / decoder. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem is used for wireless communication. It is understood that the modem / decoder may also not be integrated into the processor and may be implemented separately through a communication chip.

[0149] Memory 204 may include random access memory (RAM) or read-only memory (ROM). Memory 204 can be used to store instructions, programs, code, code sets, or instruction sets. Memory 204 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (e.g., instructions for a user to obtain random numbers), instructions for implementing the various method embodiments described below, etc. The data storage area may also store data (e.g., random numbers) created by the terminal during use.

[0150] Electronic device 200 may also include a network module and a screen. The network module is used to receive and transmit electromagnetic waves, converting electromagnetic waves into electrical signals, thereby enabling communication with communication networks or other devices, such as audio playback devices. The network module may include various existing circuit elements used to perform these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, SIM cards, memory, etc. The network module can communicate with various networks such as the Internet, corporate intranets, and wireless networks, or communicate with other devices via wireless networks. The aforementioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The screen can display interface content and facilitate data interaction.

[0151] Please refer to Figure 6 , Figure 6 This diagram illustrates a structural block diagram of a computer-readable storage medium according to an embodiment of this application. The computer-readable storage medium 400 stores program code 410, which can be called by a processor to execute the methods described in the above method embodiments.

[0152] The computer-readable storage medium 400 may be an electronic memory such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium 400 has storage space for program code 410 that performs any of the method steps described above. This program code 410 can be read from or written to one or more computer program products. The program code 410 may, for example, be compressed in a suitable form.

[0153] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the sparse point cloud data identification method described in the various optional implementations above.

[0154] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for identifying sparse point cloud data, characterized in that, The method includes: Acquire denoised point cloud data and perform voxelization to obtain each three-dimensional voxel matrix; The difference matrix is ​​obtained by performing a subtraction operation on the three-dimensional voxel matrix based on a preset number of consecutive frames, and the moving voxels and stationary voxels in the difference matrix are determined. The moving voxels are subjected to mean filtering or low-pass filtering to determine periodic moving voxels and general moving voxels. Specifically, determining periodic moving voxels and general moving voxels using mean filtering includes: determining the voxel matrix of the moving voxels changing over consecutive time frames; establishing a discrete-time signal sequence based on the voxel matrix; calculating the average value of the discrete-time signal sequence when mean filtering is applied; storing the average value in the filtered voxel matrix; and after mean filtering, marking the voxel matrix as periodic moving voxels if the average value is greater than a first preset threshold, otherwise marking the voxel matrix as general moving voxels. The process involves using low-pass filtering to determine periodic and general motion voxels, including: determining a voxel matrix that varies across consecutive time frames; establishing a discrete-time signal sequence based on the voxel matrix; if low-pass filtering is used, constructing a calculation formula based on the discrete-time signal sequence to determine a hypothetical calculated value after low-pass filtering; iteratively processing the hypothetical calculated value to obtain an iterative value, and storing the iterative value in the filtered voxel matrix; after low-pass filtering, if the iterative value is greater than a second preset threshold, marking the voxel matrix as a periodic motion voxel; otherwise, marking the voxel matrix as a general motion voxel. Updating the motion state corresponding to the stationary voxel and the general moving voxel based on the motion-preserving algorithm includes: detecting whether the matrix corresponding to the stationary voxel and the general moving voxel is in a stationary state and the number of points inside the matrix; if it is determined that the matrix is ​​not in a stationary state and the number of points inside the matrix is ​​not zero, determining that the stationary voxel and the general moving voxel are in a general moving state; if it is determined that the matrix is ​​in a stationary state and the number of points inside the matrix is ​​not zero, setting a stationary countdown segment for the matrix; when the matrix exceeds the time threshold of the stationary countdown segment or the number of points inside the matrix is ​​zero, determining that the stationary voxel and the general moving voxel are in a stationary state. The target point cloud data corresponding to the periodic moving voxels, general moving voxels and stationary voxels determined by the DBSCAN clustering algorithm and iterative clustering algorithm are processed to obtain segmented point cloud data. The segmented point cloud data is identified using the Pointnet++ algorithm or a convolutional neural network to obtain the identification results of sparse point cloud data.

2. The method for identifying sparse point cloud data according to claim 1, characterized in that, The process of processing the target point cloud data corresponding to the periodic moving voxels, general moving voxels, and stationary voxels determined by the DBSCAN clustering algorithm and iterative clustering algorithm to obtain segmented point cloud data includes: Based on the DBSCAN clustering algorithm, outliers are removed from the center points of the target point cloud data corresponding to each voxel. The clustering results of each type of center point are back-mapped to the target point cloud data and labeled; An iterative clustering algorithm is used to establish a polymorphic body with the center point of the same label, and each face of the polymorphic body is expanded sequentially to obtain the expanded region. When the point density of the expansion region is greater than the target threshold, the expansion requirement is met; The segmentation is considered complete and segmentation point cloud data is obtained when all faces of the polymorph are amplified ineffectively.

3. The method for identifying sparse point cloud data according to claim 1, characterized in that, The step of updating the motion states corresponding to the stationary voxels and the general moving voxels based on the motion-preservation algorithm also includes: Detect changes in the number of points within the matrix; If the change in quantity exceeds a preset quantity, the static countdown segment is reset.

4. The method for identifying sparse point cloud data according to claim 1, characterized in that, The acquisition of denoised point cloud data includes: Acquire sparse point cloud data collected by lidar; The sparse point cloud data is preprocessed based on distance denoising to obtain denoised point cloud data.

5. A device for identifying sparse point cloud data, characterized in that, The device includes: The acquisition module is used to acquire denoised point cloud data and perform voxelization processing to obtain various three-dimensional voxel matrices. The first processing module is used to perform subtraction on the three-dimensional voxel matrix based on a preset number of consecutive frames to obtain a difference matrix, and to determine the moving voxels and stationary voxels in the difference matrix. The second processing module is used to apply mean filtering or low-pass filtering to the moving voxels to determine periodic moving voxels and general moving voxels. Specifically, applying mean filtering to determine periodic moving voxels and general moving voxels includes: determining the voxel matrix that changes across consecutive time frames; establishing a discrete-time signal sequence based on the voxel matrix; calculating the average value of the discrete-time signal sequence when mean filtering is applied; storing the average value in the filtered voxel matrix; and marking the voxel matrix as a periodic moving voxel when the average value is greater than a first preset threshold after mean filtering, otherwise marking the voxel matrix as a periodic moving voxel. The process involves: defining periodic motion voxels and general motion voxels using low-pass filtering, including: determining the voxel matrix of the motion voxels changing over consecutive time frames; establishing a discrete-time signal sequence based on the voxel matrix; constructing a calculation formula based on the discrete-time signal sequence to determine the hypothetical calculated value after low-pass filtering, when low-pass filtering is determined; iteratively processing the hypothetical calculated value to obtain an iterative value, and storing the iterative value in the filtered voxel matrix; after low-pass filtering, if the iterative value is greater than a second set threshold, marking the voxel matrix as a periodic motion voxel; otherwise, marking the voxel matrix as a general motion voxel. The update module is used to update the motion state corresponding to the stationary voxel and the general motion voxel based on the motion-maintaining algorithm, including: detecting whether the matrix corresponding to the stationary voxel and the general motion voxel is in a stationary state and the number of points inside the matrix; if it is determined that the matrix is ​​not in a stationary state and the number of points inside the matrix is ​​not zero, determining that the stationary voxel and the general motion voxel are in a general motion state; if it is determined that the matrix is ​​in a stationary state and the number of points inside the matrix is ​​not zero, setting a stationary countdown segment for the matrix; when the matrix exceeds the time threshold of the stationary countdown segment or the number of points inside the matrix is ​​zero, determining that the stationary voxel and the general motion voxel are in a stationary state. The third processing module is used to process the target point cloud data corresponding to the periodic moving voxels, general moving voxels and stationary voxels determined by the DBSCAN clustering algorithm and the iterative clustering algorithm to obtain segmented point cloud data. The recognition module is used to identify the segmented point cloud data based on the Pointnet++ algorithm or a convolutional neural network to obtain the recognition result of the sparse point cloud data.

6. An electronic device, characterized in that, The electronic device includes a memory and a processor. The memory stores program code that can run on the processor. When the program code is executed by the processor, it implements the method for identifying sparse point cloud data as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code that can be called by one or more processors to perform the method for identifying sparse point cloud data as described in any one of claims 1-4.

Images