Anisotropic point cloud convolution method based on local geometry self-adaption

Abstract

This application belongs to the field of computer vision and 3D point cloud processing technology. It provides a convolution method for anisotropic point clouds based on local geometric adaptation. The embodiments of this disclosure estimate the principal orientation of the local geometric structure by performing principal component analysis on the neighborhood point cloud and constructing an affine transformation matrix. This matrix is ​​then used to dynamically deform the basic kernel points of the convolution operator, dynamically aligning the shape and orientation of the convolution kernel with the local structure of the point cloud. Integrating this convolution operator into a point cloud registration network allows for feature extraction from 3D point clouds, particularly those from underwater multibeam echo sounders. The convolution operator constructed using this method effectively captures the structural priors of strongly anisotropic point clouds, solving the problems of structural information loss and limited feature extraction capabilities caused by isotropic kernels when processing such data.

Description

Technical Field

[0001] This disclosure relates to the fields of computer vision and 3D point cloud processing technology, and in particular to an anisotropic point cloud convolution method based on local geometry adaptation. Background Technology

[0002] 3D point cloud registration is one of the key technologies in computer vision, robotics and remote sensing. It aims to estimate the rigid transformation between different point clouds and is the foundation for tasks such as autonomous driving, 3D reconstruction and underwater topographic mapping.

[0003] In recent years, deep learning-based methods have made significant progress in point cloud registration. Among them, kernel-based convolution (KPConv), with its intuitive, concise, and efficient characteristics inspired by 2D image convolution, has become the foundational backbone of many advanced point cloud matching network architectures. However, point clouds acquired by underwater multibeam echo sounding (MBES) exhibit characteristics drastically different from those of land-based LiDAR point clouds. Due to the limitations of the sonar scanning mechanism, MBES point clouds typically display a sparse, non-uniform "strip-like" distribution with extremely strong anisotropy.

[0004] Existing technologies suffer from serious drawbacks: most mainstream point cloud convolution operators assume that point cloud data is locally isotropic. For example, KPConv uses a set of rigidly distributed kernel points in space to aggregate neighborhood information. This isotropic design fails to align the convolution kernel with the local geometry of MBES point clouds, making it difficult to effectively capture their potential geometric priors and potentially leading to the loss of structural information and a significant decrease in feature extraction capabilities. Therefore, achieving accurate registration of strongly anisotropic point clouds using traditional isotropic convolution operators is extremely difficult. A new anisotropic method is needed to enable the algorithm model to achieve high performance when processing anisotropic point clouds.

[0005] Therefore, it is necessary to improve one or more of the problems existing in the above-mentioned related technical solutions.

[0006] It should be noted that this section is intended to provide background or context for the technical solutions of this disclosure as set forth in the claims. The description herein does not constitute an admission that it is prior art simply because it is included in this section. Summary of the Invention

[0007] The purpose of this disclosure is to provide an anisotropic point cloud convolution method based on local geometric adaptation, thereby overcoming, to at least a certain extent, the problem of limited feature extraction capability and loss of structural information caused by structural mismatch when the isotropic kernels of traditional convolution operators such as KPConv process strongly anisotropic data such as MBES point clouds.

[0008] According to embodiments of this disclosure, an anisotropic point cloud convolution method based on local geometry adaptation is provided, including:

[0009] In the point cloud, a convolution center point is set, and its corresponding local neighborhood point set is determined;

[0010] Based on the local neighborhood point set, the covariance matrix is ​​calculated and eigenvalue decomposition is performed to obtain local geometric features;

[0011] Based on local geometric features, a dynamic affine transformation matrix is ​​constructed;

[0012] By using an affine transformation matrix to deform the preset basic kernel points, a set of anisotropic kernel points adapted to the local geometry is generated;

[0013] Based on the anisotropic kernel, the features of each point in the local neighborhood point set are weighted and aggregated to obtain the output features of the convolution center point.

[0014] Furthermore, the step of setting the convolution center point in the point cloud and determining its corresponding local neighborhood point set includes:

[0015] Set the center point of convolution Its coordinates are three-dimensional vectors, passing through the PCA radius. Determine its local neighborhood set:

[0016]

[0017] in, For all neighboring points.

[0018] Furthermore, the step of calculating the covariance matrix and performing eigenvalue decomposition based on the local neighborhood point set to obtain local geometric features includes:

[0019] Calculate the centroid of the local neighborhood point set ;

[0020] All neighboring points By centralizing the points and combining the centroids of the local neighborhood point set, we obtain the centralized point set. ;in, ;

[0021] Based on the centralized point set Calculate its covariance matrix:

[0022]

[0023] For covariance matrix Eigenvalue decomposition is performed to obtain local geometric features; these local geometric features include three eigenvalues. and its three corresponding orthogonal eigenvectors ; Corresponding largest eigenvalue , which represents the direction with the largest variance in the local neighborhood data.

[0024] Furthermore, the step of constructing a dynamic affine transformation matrix based on local geometric features includes:

[0025] Based on three orthogonal eigenvectors Construct rotation matrix ;

[0026] Based on three eigenvalues and eigenvalue-based scaling functions Construct a scaling matrix ;

[0027] According to the rotation matrix and scaling matrix Construct the affine transformation matrix .

[0028] Furthermore, the scaling function is specifically defined as follows: .

[0029] Furthermore, the step of deforming the preset basic kernel points using an affine transformation matrix to generate a set of anisotropic kernel points adapted to the local geometry includes:

[0030] Set the basic core points All the basic core points are within a sphere of a preset radius;

[0031] Using affine transformation matrices to apply to the fundamental kernel points Dynamic deformation is performed to generate a set of anisotropic core points. :

[0032]

[0033] Among them, the new core point Spatial distribution and convolution center point The local geometric features are matched.

[0034] Furthermore, the step of weighted aggregation of features of each point in the local neighborhood point set based on the anisotropic kernel point to obtain the output features of the convolution center point includes:

[0035] Using anisotropic kernel points For the center point of convolution neighborhood points within and its input features Perform feature aggregation:

[0036]

[0037] in, In order to be with the first The learnable weight matrix associated with each kernel point. This is the influence function between the core point and its neighboring points.

[0038] Furthermore, the influence function Linear interpolation is used, and it is defined as follows:

[0039]

[0040] in, The radius of influence of the core.

[0041] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0042] In the embodiments of this disclosure, the anisotropic point cloud convolution method based on local geometric adaptation described above, on the one hand, estimates the principal orientation of the local geometric structure by performing principal component analysis (PCA) on the neighborhood point cloud and constructs an affine transformation matrix; this matrix is ​​then used to dynamically deform the basic kernel points of the convolution operator, so that the shape and orientation of the convolution kernel are dynamically aligned with the local structure of the point cloud. Integrating this convolution operator into a point cloud registration network can be used for feature extraction of 3D point clouds, especially multibeam echo sounder (MBES) point clouds. On the other hand, by introducing a kernel point adaptive deformation mechanism based on principal component analysis, the convolution kernel can be dynamically adjusted according to the local geometric features of the point cloud, so that its shape and orientation are dynamically aligned with the local structure. Furthermore, the convolution operator constructed by this method can effectively capture the structural prior of strongly anisotropic point clouds, solving the problems of structural information loss and limited feature extraction capabilities caused by isotropic kernels when processing such data. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0044] Figure 1 The diagram illustrates the steps of an anisotropic point cloud convolution method based on local geometry adaptation in an exemplary embodiment of this disclosure.

[0045] Figure 2 This diagram illustrates the overall architecture of the A-Predator system in an exemplary embodiment of this disclosure.

[0046] Figure 3 This diagram illustrates the operation of the A-KPConv module in an exemplary embodiment of this disclosure.

[0047] Figure 4 This diagram shows a comparison of the registration performance of various algorithms under different effective overlap rates in an exemplary embodiment of this disclosure.

[0048] Figure 5 This diagram shows a comparison of point cloud registration visualization results in a low overlap rate scenario according to an exemplary embodiment of this disclosure. Detailed Implementation

[0049] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0050] Furthermore, the accompanying drawings are merely illustrative diagrams of embodiments of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

[0051] This example implementation provides an anisotropic point cloud convolution method based on local geometry adaptation. (See reference...) Figure 1 As shown, this anisotropic point cloud convolution method based on local geometry adaptation may include:

[0052] Step S101: Set the convolution center point in the point cloud and determine its corresponding local neighborhood point set;

[0053] Step S102: Based on the local neighborhood point set, calculate the covariance matrix and perform eigenvalue decomposition to obtain local geometric features;

[0054] Step S103: Construct a dynamic affine transformation matrix based on local geometric features;

[0055] Step S104: Use an affine transformation matrix to deform the preset basic kernel points to generate a set of anisotropic kernel points that adapt to the local geometry;

[0056] Step S105: Based on the anisotropic kernel point, the features of each point in the local neighborhood point set are weighted and aggregated to obtain the output features of the convolution center point.

[0057] The aforementioned anisotropic point cloud convolution method based on local geometric adaptation addresses two main issues. First, it estimates the principal orientation of the local geometry by performing principal component analysis (PCA) on the neighborhood point cloud and constructs an affine transformation matrix. This matrix is ​​then used to dynamically deform the base kernel of the convolution operator, aligning its shape and orientation with the local structure of the point cloud. Integrating this convolution operator into a point cloud registration network allows for feature extraction from 3D point clouds, particularly those from multibeam echo sounders (MBES). Second, by introducing a kernel-point adaptive deformation mechanism based on PCA, the convolution kernel can dynamically adjust its shape and orientation according to the local geometric features of the point cloud, aligning it with the local structure. Furthermore, the convolution operator constructed in this method effectively captures the structural priors of strongly anisotropic point clouds, solving the problems of structural information loss and limited feature extraction capabilities caused by isotropic kernels when processing such data.

[0058] Below, we will refer to Figures 1 to 5 The steps of the anisotropic point cloud convolution method based on local geometry adaptation described in this example implementation will be explained in more detail.

[0059] In steps S101 and S102, a convolution center point is set in the point cloud, and its corresponding local neighborhood point set is determined; based on the local neighborhood point set, the covariance matrix is ​​calculated and eigenvalue decomposition is performed to obtain local geometric features.

[0060] Specifically, local geometric feature analysis: First, for a given convolution center point... Its coordinates are three-dimensional vectors, which determine its local neighborhood.

[0061] Preferably, the neighborhood It is through principal component analysis (PCA) radius Defined by, that is:

[0062] .

[0063] Then calculate the neighborhood. center of mass Subsequently, in order to eliminate the influence of location, all neighboring points... Centralization, to obtain Based on the centralized point set Calculate its covariance matrix:

[0064] .

[0065] For covariance matrix Perform eigenvalue decomposition to obtain three eigenvalues. and its three corresponding orthogonal eigenvectors .in, (corresponding to the largest eigenvalue) The direction representing the largest variance in the local neighborhood data is the main direction of the local geometry.

[0066] In step S103, a dynamic affine transformation matrix is ​​constructed based on local geometric features.

[0067] Specifically, construct the adaptive affine transformation matrix: utilize local geometric features for each center point. Construct a unique affine transformation matrix Then construct the rotation matrix. This is used to align the coordinate system with the principal directions of the local geometry. A scaling matrix is ​​constructed using the eigenvalues ​​obtained in step two:

[0068]

[0069] in This is used to scale the kernel points on the aligned coordinate axes, adapting the kernel point distribution to the flattening or stretching of the local structure. The final result is the kernel point transformation affine matrix. .

[0070] The matrix Preferably, it consists of a rotation matrix. and a scaling matrix Composition, namely:

[0071]

[0072] Construct using feature vectors The purpose of this matrix is ​​to align the coordinate system with the principal directions of the local geometry.

[0073] The scaling matrix is ​​constructed as follows: .in It is an eigenvalue-based scaling function used to scale on aligned coordinate axes to accommodate local anisotropy. In a preferred embodiment, the scaling function is specifically defined as... This design allows for adjustments to the scaling ratio based on the flatness or linearity of the local structure.

[0074] In step S104, the preset basic core points are deformed using an affine transformation matrix to generate a set of anisotropic core points that adapt to the local geometry.

[0075] Specifically, core point deformation: This application provides a set of basic core points. All core points are located within a sphere bounded by a defined radius. The position of each point is determined roughly as follows: the center of the sphere serves as a core point; the center exerts an attractive force on other points, drawing them closer; the other core points repel each other, causing them to move away from one another; once this system of attraction and repulsion stabilizes, the position of each core point can be determined. Then, the center point is used... The calculated affine transformation matrix For this group of basic core points Dynamic deformation is performed to generate a set of anisotropic core points. :

[0076]

[0077] This new set of core points The spatial distribution is no longer fixed, but rather related to... The local geometry of the points matches.

[0078] In step S105, based on the anisotropic kernel point, the features of each point in the local neighborhood point set are weighted and aggregated to obtain the output features of the convolution center point.

[0079] Specifically, anisotropic feature aggregation: using anisotropic kernel points For the center point neighborhood points within (and its input features) Feature aggregation is performed. The aggregation formula is expressed as:

[0080]

[0081] in, These are the coordinates of the centralized neighborhood points. Is with the first The learnable weight matrix associated with each kernel point. It is the influence function between the core point and its neighboring points. Preferably, the influence function... Linear interpolation is used, and it is defined as follows:

[0082]

[0083] in It is the radius of influence of the nucleus.

[0084] Through the above steps, the A-KPConv operator of this application can adaptively adjust its receptive field (by the deformed kernel point). The definition is aligned with the local structure of the point cloud (such as "strips") to extract structured features more accurately.

[0085] In one specific embodiment, this embodiment provides a point cloud registration system (A-Predator) based on a local geometry-adaptive anisotropic point cloud convolution method. This system employs an encoder-decoder architecture and integrates an overlapping attention module. For example... Figure 2 As shown, the system mainly consists of three parts: a feature encoder, an overlapping attention module, and a feature decoder.

[0086] 1. Feature Encoder

[0087] The encoder is used to extract multi-scale features from point clouds. In this embodiment, the encoder consists of a series of stacked residual modules. Unlike traditional methods, this application introduces an A-KPConv-based residual module (RBAKPC) in the shallow layer of the encoder (i.e., the critical stage for extracting geometric features), while continuing to use the standard KPConv residual module (RBKPC) in the deeper layers. A schematic diagram of the A-KPConv module's operation is shown below. Figure 3 As shown.

[0088] Specifically, the structure of the improved RBAKPC (Residual Block with Anisotropic Kernel PointConvolution) module is as follows: Figure 2 As shown in the "Model Details" section, the specific processing procedure is as follows:

[0089] Input: The input to this module includes the point cloud coordinates of the current level. and corresponding point features .

[0090] Processing procedure:

[0091] (1) Feature preprocessing: Input features First, a multilayer perceptron is used for dimensionality reduction or feature transformation to adjust the number of channels and add nonlinear features.

[0092] (2) Anisotropic Convolution (A-KPConv): The transformed features are fed into the A-KPConv layer. At this point, the method described in Example 1 is used to calculate the features based on the point cloud. The covariance matrix of the local neighborhood is calculated and an affine transformation matrix is ​​generated. The convolution kernel is dynamically deformed to extract features with anisotropic geometric awareness.

[0093] (3) Standardization and activation: After batch normalization, the convolution output is again processed by a multilayer perceptron for feature integration and non-linear activation is performed using the Leaky ReLU activation function.

[0094] (4) Residual connection: The input characteristics of the module are... (Or features whose dimensions have been adjusted by 1x1 convolution) are added element-wise to the output of step 3 to form a residual structure, which alleviates the gradient vanishing problem in deep networks.

[0095] (5) Downsampling: If the layer is a downsampling layer, then max pooling is performed in parallel on the residual path to reduce the point cloud size and increase the receptive field.

[0096] Output: The module outputs the updated point features. , as input for the next layer.

[0097] 2. Overlapping Attention Module

[0098] This module, located between the encoder and decoder, is designed to facilitate information exchange between the source and target point clouds and predict overlapping regions. For example... Figure 2 As shown, the specific processing procedure is as follows:

[0099] Input: High-dimensional sparse features (i.e., bottleneck layer features) of the source point cloud and target point cloud output by the encoder.

[0100] Processing procedure:

[0101] (1) Graph Neural Network Aggregation: First, local context information is aggregated within their respective point clouds through graph neural network layers to enhance the saliency of features.

[0102] (2) Multi-head attention interaction: Using a multi-head attention module, cross-attention calculation is performed between the features of the source point cloud and the target point cloud. This step enables each point to perceive the potential matching information in another point cloud.

[0103] (3) Overlap rate prediction: The features after attention interaction are passed through a multilayer perceptron, and the overlap score of each point is output. .

[0104] Output: The output is a feature vector that integrates interaction information and the overlap probability score of each point, which is used to guide subsequent decoding and registration.

[0105] 3. Feature Decoder

[0106] The decoder aims to restore low-resolution deep features to the resolution of the original point cloud and generate the final dense feature descriptor. For example... Figure 2 As shown, the specific processing procedure is as follows:

[0107] Input: Features output by the attention module and skip connection features at each level of the encoder.

[0108] Processing procedure:

[0109] (1) Nearest neighbor upsampling: perform nearest neighbor upsampling on low-resolution features to make their number of points consistent with the point cloud of the next higher resolution level.

[0110] (2) Feature concatenation: The upsampled features are concatenated with the corresponding layer output features in the encoder along the channel dimension. This design can integrate deep semantic features and shallow geometric detail features.

[0111] Feature fusion: The concatenated features are fused and dimensionality reduced by a multilayer perceptron to obtain the refined features of the current layer.

[0112] The above process is repeated until the resolution of the feature map is restored to the scale of the original input point cloud.

[0113] Output: The final output of the decoder consists of two parts: one is a dense feature descriptor used to calculate the matching relationships. Second, the significance score is used to assess the importance of a point.

[0114] In one specific embodiment, this embodiment verifies the performance of the A-Predator system proposed in this application based on the publicly available Dotson-east multibeam point cloud dataset, and compares and analyzes it with existing mainstream point cloud registration methods.

[0115] 1. Experimental Environment and Dataset Setup

[0116] All experiments were conducted on workstations equipped with an Intel Xeon Platinum 8255C CPU (2.50GHz) and an Nvidia GeForce RTX3090 graphics card (24GB VRAM), based on the Ubuntu 20.04 operating system and the PyTorch deep learning framework.

[0117] The Dotson-east dataset—a semi-synthetic multibeam echo sounder (MBES) registration dataset built based on an autonomous underwater vehicle (AUV) mission in the West Antarctic Ocean—was selected. The dataset contains 9415 sub-maps. Experiments divided it into training, validation, and test sets, containing 7263, 1206, and 946 sub-maps, respectively. Based on the original 80% overlap of the continuous sub-maps, sub-map pairs with different overlap rates (20%, 40%, 60%, 80%, and 100%) were derived, generating a total of 36300 training pairs, 6015 validation pairs, and 4715 test pairs.

[0118] 2. Data Preprocessing and Augmentation

[0119] To simulate the challenges of real underwater environments and enhance the robustness of the model, the data in this embodiment underwent the following preprocessing:

[0120] Simulated drift: A random rigid transformation is applied to the submap pair. Considering the characteristics of sonar acquisition, a rotation of [0°, 10°] is applied only in the Z-axis direction; a translation of [-40, 40] meters is applied in the X and Y-axis directions, and a translation of [-2, 2] meters is applied in the Z-axis direction.

[0121] Low overlap simulation: During the training and testing phases, each subgraph is independently cropped by 70% randomization. This operation halves the effective overlap of subgraph pairs (e.g., a subgraph pair with an original 20% overlap drops to approximately 10% after cropping), thus creating an extreme low overlap test scenario.

[0122] 3. Comparative Experiment and Evaluation Indicators

[0123] The A-Predator of this application is compared with the following mainstream methods:

[0124] (1) Predator: The baseline model of this application, designed for low overlap.

[0125] (2) FCGF: Fully Convolutional Geometric Feature Extractor.

[0126] (3) DGR: A deep learning-based outlier removal method.

[0127] (4) BathyNN: A network designed specifically for loop closure detection of multibeam point clouds.

[0128] (5) GICP: The classic generalized iterative nearest point algorithm.

[0129] (6) FPFH+RANSAC: Traditional handmade feature descriptors combined with random sampling consensus algorithm.

[0130] Registration recall, relative translation error (RTE), and relative rotation error (RRE) were used as evaluation metrics. Successful registration was defined as: relative rotation error RRE ≤ 5° and relative translation error RTE ≤ 10m.

[0131] 4. Experimental Results and Analysis

[0132] Experimental data results are as follows Figure 4 As shown, under test conditions with different effective overlap rates, the A-Predator proposed in this application outperforms all comparable methods in three key metrics: recall, translation accuracy, and rotation accuracy, achieving state-of-the-art performance. The visualization of the matching results is shown below. Figure 5 As shown, in the selection examples with low overlap, A-Predator achieved optimal matching in all cases.

[0133] Especially in the highly challenging 10% effective overlap scenario, A-Predator demonstrated significant advantages. Compared to the baseline model Predator, A-Predator's registration recall rate increased dramatically from 31.63% to 59.55%. Furthermore, A-Predator's translation and rotation errors were significantly lower than Predator's across all overlap settings.

[0134] The main reason for the performance improvement is the introduction of anisotropic kernel convolution (A-KPConv). MBES point clouds have a unique "strip-like" distribution and anisotropic features. Traditional isotropic convolutions (such as the standard KPConv used in Predator) are difficult to effectively capture this structure. However, A-Predator, through the A-KPConv module, can extract richer lateral broadening features with anisotropic perception capabilities even with low ping numbers (less vertical overlap), thus more sensitively perceiving the undulations of the underwater terrain.

[0135] In summary, this embodiment demonstrates that A-Predator has significant advantages in accuracy and robustness compared to existing technologies in handling underwater multibeam point cloud registration tasks with strong anisotropy and low overlap.

[0136] The aforementioned anisotropic point cloud convolution method based on local geometric adaptation addresses two main issues. First, it estimates the principal orientation of the local geometry by performing principal component analysis (PCA) on the neighborhood point cloud and constructs an affine transformation matrix. This matrix is ​​then used to dynamically deform the base kernel of the convolution operator, aligning its shape and orientation with the local structure of the point cloud. Integrating this convolution operator into a point cloud registration network allows for feature extraction from 3D point clouds, particularly those from multibeam echo sounders (MBES). Second, by introducing a kernel-point adaptive deformation mechanism based on PCA, the convolution kernel can dynamically adjust its shape and orientation according to the local geometric features of the point cloud, aligning it with the local structure. Furthermore, the convolution operator constructed in this method effectively captures the structural priors of strongly anisotropic point clouds, solving the problems of structural information loss and limited feature extraction capabilities caused by isotropic kernels when processing such data.

[0137] It should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise" in the above description indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of this disclosure and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of this disclosure.

[0138] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, "a plurality of" means two or more, unless otherwise explicitly specified.

[0139] In the embodiments of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0140] In embodiments of this disclosure, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0141] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0142] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A method of locally geometry-adaptive anisotropic point cloud convolution, comprising: The application relates to a method for point cloud convolution, comprising the following steps: Setting a convolution center point in a point cloud and determining a corresponding local neighborhood point set; Based on the local neighborhood point set, a covariance matrix is calculated and feature decomposition is performed to obtain local geometric features; Based on the local geometric features, a dynamic affine transformation matrix is constructed; The preset basic kernel points are deformed by using the affine transformation matrix to generate a set of anisotropic kernel points adapted to local geometry; Based on the anisotropic kernel points, the features of each point in the local neighborhood point set are weighted and aggregated to obtain the output features of the convolution center point; wherein In the step of setting a convolution center point in a point cloud and determining a corresponding local neighborhood point set, the following steps are included: Setting the convolution center point whose coordinates are three-dimensional vectors, by PCA radius determining its local neighborhood point set: In the formula, are all neighboring points; In the step of calculating a covariance matrix based on the local neighborhood point set and performing feature decomposition to obtain local geometric features, the following steps are included: Computing a centroid of a local neighborhood point set ; all the neighborhood points centralization, combined with the centroid of the local neighborhood point set, to get the centralized point set ; wherein, ; Based on the centralized point set , compute its covariance matrix: For covariance matrix Eigenvalue decomposition is performed to obtain local geometric features; these local geometric features include three eigenvalues. and its three corresponding orthogonal eigenvectors ; Corresponding to the largest eigenvalue , which represents the direction with the largest variance in the local neighborhood data.

2. The method of claim 1, wherein, In the step of constructing a dynamic affine transformation matrix based on the local geometric features, the following steps are included: According to three orthogonal eigenvectors , a rotation matrix is constructed According to three eigenvalues and a scaling function based on the eigenvalues , a scaling matrix is constructed ; According to the rotation matrix and the scaling matrix , an affine transformation matrix is constructed.

3. The method of claim 2, wherein, The scaling function is defined in particular as .

4. The method of claim 3, wherein, In the step of deforming the preset basic kernel points by using the affine transformation matrix to generate a set of anisotropic kernel points adapted to local geometry, the following steps are included: Setting base core points wherein all the base core points are within a sphere of a preset radius; Applying an affine transformation matrix to the base kernel point Performing dynamic morphing to generate a set of anisotropic kernel points : Wherein the new core point matches the local geometric features of the convolution center point in the spatial distribution.

5. The method of claim 4, wherein, In the step of weighting and aggregating the features of each point in the local neighborhood point set based on the anisotropic kernel points to obtain the output features of the convolution center point, the following steps are included: Using anisotropic kernel points Feature aggregation is performed on points in the neighborhood of the convolution center point and their input features ​​​ wherein, is a matrix of learnable weights associated with the first kernel point, is an influence function between the kernel point and the neighborhood point.

6. The method of claim 5, wherein, influence function With linear interpolation, which is defined as: wherein is the influence radius of the nucleus.

Images