Small target point cloud segmentation method based on mixed features

Abstract

The invention relates to a small target point cloud segmentation method based on mixed features, and provides a small-scale target point cloud segmentation network SSSNet. According to the method, a mixed attention mechanism of geometric features and multi-source features is combined, and the complex challenge of small target point cloud segmentation in an industrial scene is solved. The method mainly comprises the following steps: (1) extracting point features and geometric features based on sparse convolution and extracting high-dimensional shape features based on edge convolution; (2) an ROI extraction module based on a balloon simulation algorithm is adopted, a target area is accurately separated, and interference of a complex scene on segmentation precision is reduced; (3) introducing a mixed attention module, fusing the correlation between the point features and the shape features, and improving the performance of small target point cloud segmentation; and (4) a multi-feature fusion deep learning model is adopted, global and local information is combined, and the segmentation precision is improved. According to the method, the problems of point cloud shielding, object proximity, point cloud hollowing and the like in an industrial scene can be effectively solved, and high-precision identification and segmentation of key components are realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional point cloud object segmentation, and particularly relates to a small object point cloud segmentation method based on hybrid features. Background Art

[0002] In recent years, with the development of deep learning technology, many problems in the field of point clouds have been effectively solved, and there have been many research results in the fields of small-scale object recognition, fine-grained classification, point cloud segmentation, etc. Point cloud segmentation methods based on deep learning are divided into projection-based, voxel-based, and point-based methods.

[0003] Projection-based methods convert a 3D scene into a 2D image through mapping. However, during the projection process, the folds of the point cloud will destroy the inherent geometric relationship, resulting in information loss. In contrast, voxel-based methods train regular voxelized grids transformed from point clouds and can better maintain the inherent geometric relationship of the point cloud. However, this type of method is sensitive to voxel granularity and consumes a large amount of computing and memory.

[0004] Point-based methods directly perform convolution operations on the original point cloud without using intermediate variables. For example, PointNet directly processes point cloud data, individually processes each point to obtain its spatial encoding, and autonomously learns the critical points in the discrete state of the point cloud data. First, random sampling is performed on the input point cloud to reduce the computational complexity, and then local feature enhancement and attention mechanisms are combined to enhance local feature extraction.

[0005] The above methods have made certain progress in point cloud segmentation. However, they focus more on extracting global or local features, ignoring the spatial information and geometric structure features of the point cloud, and failing to fully explore the context relationships at different levels and scales, thus limiting the performance level of the network in small-scale object point cloud segmentation in industrial scenarios. Specifically, the small object point cloud segmentation in industrial scenarios has the following challenges:

[0006] (1) Complex scene: Many multi-class components are distributed in the scene in various poses, stacked or interlaced with each other, resulting in serious component occlusion and a complex scene structure.

[0007] (2) Object proximity: The components are placed closely, with only a centimeter-level distance, which blurs the boundaries of the point clouds belonging to different semantics and makes point cloud segmentation difficult.

[0008] (3) Point cloud hollowing: Due to the obstruction of nearby components and the influence of the scanning angle, the point cloud of the object is missing, resulting in poor contour integrity. Summary of the Invention

[0009] To address the above technical problems, the objective of the present invention is to propose a small-scale target point cloud segmentation method based on hybrid features - SSSNet (Small-Scale Segmentation Network). This method automatically extracts regions of interest by combining geometric features to improve the learning ability. This fully considers the characteristics of point clouds in industrial scenarios and improves the segmentation accuracy.

[0010] The technical solution of the method of the present invention is as follows:

[0011] A small-scale target point cloud segmentation method based on hybrid features, comprising the following steps:

[0012] (1) Taking the original point cloud as input, the feature information of the target point cloud is obtained through the ROI extraction module, reducing the influence of the cluttered scene on the segmentation accuracy of the small target point cloud.

[0013] (2) The ROI extraction module based on the Balloon Simulation Algorithm (BSA) adaptively extracts the target region by constructing a particle spring model to simulate the interaction between particles and calculating the position, velocity, and acceleration of particles at each time step.

[0014] (3) The multi-source features are fused through the Hybrid Attention (HA) module, including: generating weighted vectors for the point-wise features and shape features respectively, and calculating their attention score matrices. The Softmax function is applied to normalize the score matrix to measure the similarity between points. Through summation and matrix multiplication, the enhanced point-wise features and shape features are output.

[0015] (4) A deep learning model SSSNet based on multi-feature fusion is constructed to fuse global and local features and improve the segmentation accuracy of small target point clouds.

[0016] (1) Taking the original point cloud as input, the feature information of the target point cloud is obtained by using an adaptive region of interest (ROI) extraction module, reducing the influence of the cluttered scene on the segmentation accuracy of the small-scale target point cloud. Specifically, it includes the following steps:

[0017] (1.1) Sparse convolution operations are used to extract point-wise features, improving the efficiency and accuracy of the per-point feature extraction process.

[0018] (1.2) Analyze the local structure and neighboring points of each point to obtain the spatial distribution features of each point, fuse two-dimensional features, and obtain 26-dimensional geometric features.

[0019] (1.3) Edge convolution operations are used to map the 26-dimensional geometric features to a high-dimensional feature space to obtain richer shape information and high-dimensional shape features with shape-aware discrimination ability.

[0020] (2) In the structural features of the hollow target, an ROI extraction module based on the balloon simulation algorithm (BSA) is adopted to map the potential area point cloud to a higher-dimensional feature space, reduce the segmentation accuracy of the small-scale target point cloud caused by the cluttered point cloud, and achieve high-precision adaptive extraction of the target area.

[0021] (2.1) In BSA, the balloon model is modeled by constructing a grid of interconnected particles with mass, called the particle spring model. According to Newton's second law, the position of the particles can be calculated:

[0022] (2.2) During the expansion of the balloon nodes, collision point detection is performed on the nodes at the end of each iteration to determine the final position of the balloon nodes.

[0023] (2.3) After BSA, the in-plane point cloud of the cavity object, that is, the ROI area, is separated to reduce the influence of the cluttered scene point cloud on the segmentation of the small-scale object point cloud.

[0024] (3) Construct a hybrid attention (HA) module that fuses multi-source features to enable the model to better learn the correlation between shape features and point orientation features, provide a better performance level, and improve the segmentation accuracy of small-scale target point clouds.

[0025] (3.1) Use six independent weighted multi-layer perceptrons (MLPs) to generate two sets of Q, K, V vectors respectively. Q1, K1, V1 are used for the point feature f p , and Q2, K2, V2 are used for the shape feature f s .

[0026] (3.2) Calculate the attention score matrix. Calculate the attention score matrix S of the point feature f p for the shape feature f s , and calculate the attention score matrix S of the shape feature f 1 for the point feature f s p . 2

[0027] (3.3) Normalize the attention score matrix. HM applies the Softmax function to S 1 and S 2 to obtain the normalized results softmax(S 1 ), softmax(S 2 ). The normalized S 1 and S 2 measure the similarity between the i-th point and the j-th point. The larger the value, the higher the similarity, which reflects the correlation between points to a certain extent.

[0028] (3.4) HM performs a weighted summation operation on V1 and V2, and multiplies them with the normalized S 1 and S 2 for matrix multiplication to output the enhanced fp and fs.

[0029] (4) Adopt a deep learning model based on multi-feature fusion (SSSNet) to improve the accuracy of small-scale object point cloud segmentation.

[0030] The beneficial effects of the system of the present invention are: (1) A coarse-to-fine small-scale object point cloud segmentation structure model SSSNet is proposed for identifying key components in a cluttered industrial scene; (2) By fusing geometric structure features and introducing an aggregated descriptor vector to fuse features, the accuracy of small-scale object point cloud segmentation is improved. Brief Description of the Drawings

[0031] Figure 1 It is a schematic diagram of the method steps of the present invention

[0032] Figure 2 It is a schematic diagram of the structure of the feature extraction module

[0033] Figure 3 It is a schematic diagram of the principle of the hybrid attention module

[0034] Figure 4 It is a schematic diagram of the deep learning model based on multi-feature fusion (SSSNet) Detailed Embodiments

[0035] The method and system of the present invention will be further described below with reference to the accompanying drawings: Specific Embodiment

[0037] As Figure 1 shown, a small-scale object point cloud segmentation method based on hybrid features includes the following steps:

[0038] (1) Taking the original point cloud as the input, an adaptive region of interest (ROI) extraction module is used to obtain the feature information of the target point cloud, reducing the influence of the cluttered scene on the accuracy of small-scale object point cloud segmentation. Specifically, it includes the following steps:

[0039] (1.1) Sparse convolution operation is used to extract point-wise features, improving the efficiency and accuracy of the per-point feature extraction process.

[0040] (1.2) Analyze the local structure and neighboring points of each point to obtain the spatial distribution feature of each point, and fuse two-dimensional features to obtain 26-dimensional geometric features.

[0041] (1.3) Edge convolution operation is used to map the 26-dimensional geometric features to a high-dimensional feature space to obtain richer shape information and high-dimensional shape features with shape-aware discrimination ability.

[0042] (2) In the structural features of the hollow target, an ROI extraction module based on the Balloon Simulation Algorithm (BSA) is adopted to map the potential region point cloud to a higher-dimensional feature space, reduce the segmentation accuracy of the small-scale target point cloud caused by the cluttered point cloud, and achieve high-precision adaptive extraction of the target region.

[0043] (2.1) In BSA, the balloon model is modeled by constructing an interconnected particle grid with mass, called the particle spring model. According to Newton's second law, the position of the particle can be calculated as follows:

[0044]

[0045] where m is the particle mass; X(t) is the position of the particle at time t, F ext (X,t) is the external force acting on the particle in the direction of motion, F in (X,t) is the internal force generated by the interaction between particles, and X, t are the position and time at which the internal force between particles is generated, respectively.

[0046] (2.2) During the expansion of the balloon node, collision point detection is performed on the node at the end of each iteration to determine the final position of the balloon node. Assuming the collision state is defined as C, the collision constraint is determined by the position and velocity constraints of the balloon node and is defined as follows within two time steps:

[0047]

[0048] where d t is the time step, X C (t), X C (t + d t ), and X C (t + 2d t ) are the collision positions at times t, t + d, and t + 2d, respectively; X′ C (t), X′ C (t + d t ) and X′ C (t + 2d t )) are the collision velocities at times t, t + d, and t + 2d, respectively, while X″ C (t) and X″ C (t + d t ) are the collision accelerations at times t and t + d, respectively; ΔX″ C (t) and ΔX″ C (t + d t ) are the corrected velocities of the collision at times t and t + d, respectively

[0049] (2.3) After BSA, the inner plane point cloud of the cavity object, i.e., the ROI region, is separated to reduce the influence of the cluttered scene point cloud on the segmentation of the small-scale object point cloud.

[0050] (3) Construct a hybrid attention (HA) module that fuses multi-source features, enabling the model to better learn the correlation between shape features and point orientation features, providing a better performance level and improving the accuracy of small-scale target point cloud segmentation.

[0051] (3.1) Use six independent weighted multi-layer perceptrons (MLPs) to generate two sets of Q, K, V vectors respectively. Q1, K1, V1 are used for the point feature f p , and Q2, K2, V2 are used for the shape feature f s .

[0052] (3.2) Calculate the attention score matrix. Calculate the attention score matrix S of the point feature f p for the shape feature f s , and the specific calculation formula is as follows: 1

[0053] S 1 = ω 1 f p · (ω 5 f s ) T (3)

[0054] Calculate the attention score matrix S of the shape feature f s for the point feature f p 2 , and the specific calculation formula is as follows:

[0055] S 2 = ω 4 f s · (ω 2 f p ) T (4)

[0056] where ω i is the learnable parameter of the i-th MLP.

[0057] (3.3) Normalize the attention score matrix. HM applies the Softmax function to S 1 and S 2 to obtain the normalized results softmax(S 1 ), softmax(S 2 ). The normalized S 1 and S 2 measure the similarity between the i-th point and the j-th ​​

[0058] The similarity between points. The larger its value, the higher the similarity, which to a certain extent reflects the correlation between points.

[0059] (3.4) HM performs a weighted summation operation on V1 and V2, and multiplies them with the normalized S 1 and S 2 to perform matrix multiplication and output the enhanced fp and fs. The specific calculation formula is as follows:

[0060] f p = softmax(S 1 )·(ω 3 f p + ω 6 f s ) (5)

[0061] f s = softmax(S 2 )·(ω 3 f p + ω 6 f s ) (6) (4) Adopt a deep learning model based on multi-feature fusion (SSSNet) to improve the accuracy of small-scale target point cloud segmentation.

[0062] The above are only the preferred embodiments of the present invention. It should be understood that those of ordinary skill in the art can make many modifications and changes according to the concept of the present invention without creative labor. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field of the present invention based on the concept of the present invention through logical analysis, reasoning, or limited experiments should be within the protection scope determined by the claims.

Claims

1. A small target point cloud segmentation method based on hybrid features, characterized in that: The method comprises the following steps: (1) Taking the original point cloud as input, the feature information of the target point cloud is obtained through the ROI extraction module to reduce the impact of cluttered scenes on the segmentation accuracy of small target point clouds. (2) The ROI extraction module based on the balloon simulation algorithm (BSA) constructs a particle spring model to simulate the interaction between particles and calculates the position, velocity and acceleration of particles at each time step to achieve adaptive extraction of the target area. (3) The hybrid attention (HA) module is used to fuse multi-source features, including generating weighted vectors for point features and shape features respectively, and calculating their attention score matrices. The score matrix is ​​normalized using the Softmax function to measure the similarity between points. The enhanced point features and shape features are output through summation and matrix multiplication. (4) Construct a deep learning model SSSNet based on multi-feature fusion to fuse global and local features and improve the accuracy of small target point cloud segmentation.

2. The method according to claim 1, characterized in that The sparse convolution improves computational efficiency by reducing computational redundancy in sparsely distributed areas of the point cloud while maintaining high-resolution feature extraction capabilities.

3. The method according to claim 1, characterized in that: The collision detection of balloon nodes in the BSA model is based on Newton's second law to calculate particle motion, and the node position is dynamically adjusted in combination with the time step to improve the adaptability to the target area.

4. The method according to claim 1, characterized in that The hybrid attention module generates multiple weighted vectors through independent multi-layer perceptrons (MLPs) to process point features and shape features respectively to improve the correlation and distinguishability between features.

5. The method according to claim 1, characterized in that The SSSNet model combines geometric structure features with contextual information and fuses multi-scale features by aggregating descriptor vectors, thereby effectively dealing with occlusion and hollowing problems in complex scenes.

6. The method according to claim 1, characterized in that This method is suitable for small target point cloud segmentation tasks with tightly stacked components and fuzzy boundaries in industrial scenes.

7. The method according to claim 1, characterized in that This method can significantly reduce the interference of complex background in point cloud on the accuracy of small target segmentation and improve the segmentation effect of key components.

Images