Training method of three-dimensional shape correspondence model, three-dimensional shape correspondence method and system

Abstract

The application discloses a three-dimensional shape correspondence model training method, a three-dimensional shape correspondence method and a system, and relates to the technical field of computer vision and graphics, and comprises the following steps: acquiring the geometric structure and two-dimensional semantic multi-source feature descriptor of source and target three-dimensional shapes; obtaining shape features through joint optimization of a feature extraction network and constructing a bilateral soft point-to-point correspondence matrix; iteratively refining a functional mapping matrix by using a multi-scale filtering operator in a mixed base function space; and calculating a reliability calibration contrast learning loss and a mapping consistency loss based on the bilateral soft point-to-point correspondence matrix and the refined functional mapping matrix to update network parameters. The application effectively improves the precision, robustness and cross-dataset generalization capability of complex non-rigid three-dimensional shape correspondence.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and graphics technology, specifically relating to a training method for a three-dimensional shape correspondence model, a three-dimensional shape correspondence method, and a system. Background Technology

[0002] In the field of 3D shape analysis, establishing semantically consistent point-by-point correspondences between different 3D shapes is fundamental to applications such as 3D reconstruction, animation transfer, and shape retrieval. Existing deep functional mapping methods typically rely on single geometric features for shape representation and lack effective structural consistency constraints during unsupervised learning. This results in insufficient feature discrimination capabilities when facing scenarios with complex non-rigid deformations, missing local structures, or topological noise. Furthermore, the contrastive learning mechanism is susceptible to interference from unreliable samples, making it difficult to obtain high-precision and robust correspondence results. Summary of the Invention

[0003] To address the limitations of existing 3D shape correspondence methods in terms of feature representation capabilities and reliability of correspondence relationships under complex non-rigid deformation scenarios, the present invention aims to provide a training method, a 3D shape correspondence method, and a system for a 3D shape correspondence model, achieving high-precision 3D shape correspondence through joint optimization of multi-source feature fusion and reliability calibration.

[0004] This invention provides a method for training a three-dimensional shape correspondence model, comprising:

[0005] Obtain multi-source feature descriptors of the source 3D shape and the target 3D shape. The multi-source feature descriptors include geometric structural features and 2D semantic features.

[0006] By jointly optimizing the multi-source feature descriptors through a feature extraction network, the first shape feature of the source 3D shape and the second shape feature of the target 3D shape are obtained.

[0007] Based on the first and second shape features, a bilateral soft pointwise correspondence matrix is ​​constructed; in the mixed basis function space, the functional mapping matrix is ​​iteratively refined using a multi-scale filtering operator to obtain the refined functional mapping matrix; the mixed basis function space is composed of the first and second basis functions.

[0008] Based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, reliability calibration contrastive learning loss and mapping consistency loss are calculated to update the parameters of the feature extraction network. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space.

[0009] The process of obtaining multi-source feature descriptors for source and target 3D shapes includes: performing feature decomposition based on the Laplacian matrices of the source and target 3D shapes to obtain heat kernel feature descriptors as geometric structural features; extracting features from multi-view rendered images of the source and target 3D shapes using a 2D vision foundation model, and inversely projecting the extracted 2D features onto the vertices of the 3D surface for aggregation to obtain semantic descriptors as 2D semantic features; and concatenating the heat kernel feature descriptors and semantic descriptors to obtain multi-source feature descriptors.

[0010] Feature extraction is performed on multi-view rendered images of source and target 3D shapes using a 2D vision foundation model. The extracted 2D features are then back-projected onto 3D surface vertices and aggregated to obtain semantic descriptors. This process includes: rendering source and target 3D shapes from multiple preset virtual camera positions to obtain a set of 2D images and corresponding depth and normal maps; inputting the 2D image set, depth maps, and normal maps into a frozen image generation model to generate 2D features under the guidance of category prompts; using recorded depth information and camera parameters, back-projecting the 2D features onto the 3D surface vertices of the source and target 3D shapes; and averaging the 2D features projected from multiple viewpoints for each 3D surface vertex to form a semantic descriptor.

[0011] In the mixed basis function space, the functional mapping matrix is ​​iteratively refined using a multi-scale filtering operator to obtain the refined functional mapping matrix. This includes: concatenating the iso-invariant Laplacian function and the elastic basis function that handles non-iso-dimensional deformation to construct the mixed basis function space, where the first basis function is the Laplacian function and the second basis function is the elastic basis function; calculating the feature similarity matrix based on the first and second shape features, and multiplying the rows and columns of the feature similarity matrix after normalization to obtain the bilateral soft pointwise correspondence matrix; and calculating the initial functional mapping sub-matrix based on the hard pointwise correspondence matrix in the mixed basis function space, and iteratively updating the initial functional mapping sub-matrix using a multi-scale filtering operator to obtain the refined functional mapping matrix.

[0012] In a mixed basis function space, an initial functional mapping submatrix is ​​calculated based on the hard point-by-point correspondence matrix, and a multi-scale filtering operator is used to iteratively update the initial functional mapping submatrix to obtain a refined functional mapping matrix. This process includes: using the bilateral soft point-by-point correspondence matrix as the initial input to the hard point-by-point correspondence matrix; in each iteration, calculating the first functional mapping submatrix in the Laplacian function space and the second functional mapping submatrix in the elastic basis function space; filtering the first and second functional mapping submatrixes using a multi-scale filtering operator to obtain updated first and second functional mapping submatrixes; recalculating the hard point-by-point correspondence matrix based on the updated first and second functional mapping submatrixes; and after reaching a preset number of iterations, combining the finally updated first and second functional mapping submatrixes to output the refined functional mapping matrix.

[0013] Based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, the reliability calibration contrastive learning loss is calculated, including: calculating the cross-contrast learning loss: introducing learnable scaling parameters and offsets to calibrate feature similarity; calculating the cross-contrast loss value through activation functions based on the calibrated feature similarity, the set of pseudo-positive samples, and the set of negative samples to enhance the optimization ability of hard negative samples; calculating the self-contrast learning loss: calculating the self-contrast loss value through activation functions based on the pseudo-negative sample sets of the source 3D shape and the target 3D shape to penalize unreliable soft correspondences; and weighted summing the cross-contrast learning loss and the self-contrast learning loss to obtain the reliability calibration contrastive learning loss.

[0014] The reliability calibration contrast learning loss and mapping consistency loss are calculated to update the parameters of the feature extraction network. The mapping consistency loss is calculated as follows: In the Laplacian function space, the first residual matrix between the first Laplacian function matrix of the source 3D shape and the bilateral soft pointwise correspondence matrix, the second Laplacian function matrix of the target 3D shape, and the transpose of the corresponding first mapping submatrix in the refined functional mapping matrix is ​​calculated, and the Frobenius norm of the first residual matrix is ​​calculated as the first consistency loss; In the elastic basis function space, the second residual matrix between the first elastic basis function matrix of the source 3D shape and the bilateral soft pointwise correspondence matrix, the second elastic basis function matrix of the target 3D shape, and the transpose of the corresponding second mapping submatrix in the refined functional mapping matrix is ​​calculated, and the Frobenius norm of the second residual matrix is ​​calculated as the second consistency loss; The first consistency loss and the second consistency loss are added together to obtain the mapping consistency loss; The reliability calibration contrast learning loss and the mapping consistency loss are weighted and summed to obtain the total loss function to update the parameters of the feature extraction network.

[0015] Furthermore, this invention also provides a three-dimensional shape correspondence method, comprising: acquiring multi-source feature descriptors of a source three-dimensional shape and a target three-dimensional shape to be matched, wherein the multi-source feature descriptors include geometric structural features and two-dimensional semantic features; jointly optimizing the multi-source feature descriptors through a feature extraction network in a pre-trained three-dimensional shape correspondence model to obtain a first shape feature of the source three-dimensional shape and a second shape feature of the target three-dimensional shape, wherein the three-dimensional shape correspondence model is trained based on reliability calibration contrastive learning loss and mapping consistency loss, wherein the reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space; calculating feature similarity based on the first shape feature and the second shape feature, and determining the pointwise correspondence matrix between the source three-dimensional shape and the target three-dimensional shape based on feature similarity.

[0016] Furthermore, this invention also provides a three-dimensional shape correspondence system, comprising: a feature acquisition module for acquiring multi-source feature descriptors of a source three-dimensional shape and a target three-dimensional shape, wherein the multi-source feature descriptors include geometric structural features and two-dimensional semantic features; a feature extraction module for jointly optimizing the multi-source feature descriptors through a feature extraction network to obtain a first shape feature of the source three-dimensional shape and a second shape feature of the target three-dimensional shape; a mapping construction and refinement module for constructing a bilateral soft pointwise correspondence matrix based on the first shape feature and the second shape feature; and iteratively refining the functional mapping matrix using a multi-scale filtering operator in a mixed basis function space to obtain a refined functional mapping matrix, wherein the mixed basis function space is composed of the first basis function and the second basis function; and a loss calculation module for calculating a reliability calibration contrastive learning loss and a mapping consistency loss based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix to update the parameters of the feature extraction network; wherein the reliability calibration contrastive learning loss includes a cross-contrast learning loss and a self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space.

[0017] In addition, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the training method for the three-dimensional shape correspondence model as described in any of the preceding claims, or implements the three-dimensional shape correspondence method as described above.

[0018] Beneficial effects:

[0019] The technical solution provided by this invention constructs a multi-source feature descriptor by fusing geometric structural features and two-dimensional semantic features. It leverages the strong semantic understanding capability of the two-dimensional visual foundation model to compensate for the insufficient discriminative power of pure geometric features in structurally ambiguous regions, enhancing the expressive power of shape features and cross-class recognition capabilities. Furthermore, by iteratively refining the functional mapping matrix using multi-scale filtering operators in a hybrid basis function space composed of Laplacian functions and elastic basis functions, it balances the representation requirements of near-isometric and non-isometric deformations, improving the structural fidelity and numerical stability of the mapping matrix. Further, a reliability calibration contrastive learning loss, incorporating cross-contrast learning and self-contrast learning, is designed. This loss dynamically calibrates the similarity distribution using learnable parameters and penalizes unreliable samples, effectively suppressing the interference of erroneous correspondences on network training. Simultaneously, a mapping consistency loss is introduced to enforce the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the hybrid basis space, ensuring the synergy between feature learning and mapping optimization. The above mechanisms work together to enable this application to achieve high-precision, high-robust point-to-point correspondence of complex non-rigid 3D shapes without manual annotation, and to have excellent computational efficiency and cross-dataset generalization ability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the network structure of the three-dimensional shape correspondence model training method according to an embodiment of this application;

[0021] Figure 2 This is a comparison chart of the color and texture mapping results of the embodiments of this application on the SHREC'19 dataset;

[0022] Figure 3 This is a comparison diagram of the color and texture mapping results of the embodiments of this application on the SMAL dataset;

[0023] Figure 4 This is a visualization of the non-rigid shape correspondence results of the embodiments of this application on the DT4D dataset;

[0024] Figure 5 This is a visualization of the corresponding results of the embodiments of this application on the shape of the partial missing parts;

[0025] Figure 6 This is a visualization of the corresponding results of the embodiments of this application in terms of topological noise shape. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0028] like Figure 1 As shown, this embodiment provides a training method for a 3D shape correspondence model. This method achieves high-precision shape correspondence learning under unsupervised conditions by fusing multi-source features and a dual optimization mechanism. The training method mainly includes the following steps:

[0029] Step S100: Obtain multi-source feature descriptors for the source 3D shape and the target 3D shape. The multi-source feature descriptors include geometric structural features and 2D semantic features.

[0030] Specifically, the source 3D shape and the target 3D shape are typically input in the form of triangular meshes or point clouds. Geometric structural features are used to characterize the shape's intrinsic geometric properties such as local curvature and geodesic distance; for example, they can be Laplacian spectrum-based descriptors such as thermal kernel signatures (HKS) or wave kernel signatures (WKS). Two-dimensional semantic features are used to supplement the shape's high-level semantic information, compensating for the insufficient discriminative power of pure geometric features in structurally ambiguous regions; for example, they can be feature vectors extracted from a pre-trained visual model and projected back onto the 3D surface. By splicing or fusing these two types of heterogeneous features, a multi-source feature descriptor with both geometric accuracy and semantic richness is constructed, providing high-quality input for subsequent feature learning. It should be understood that although this embodiment lists specific feature types, in other embodiments, other combinations of descriptors capable of characterizing geometric and semantic information can also be used, as long as the function of multi-source complementarity is satisfied.

[0031] Step S200: The multi-source feature descriptors are jointly optimized through a feature extraction network to obtain the first shape feature of the source 3D shape and the second shape feature of the target 3D shape.

[0032] Specifically, the feature extraction network can be a graph neural network, a diffuse network, or other deep learning architecture suitable for non-Euclidean data. During the training phase, the network receives the multi-source feature descriptors generated in step S100 as input and maps the original low-level features to a high-dimensional embedding space through multi-layer nonlinear transformations and message passing mechanisms. This process is called "joint optimization" because the network not only learns how to extract features but also how to adjust the feature representation according to subsequent corresponding tasks, making corresponding points closer together in the feature space and non-corresponding points farther apart. The final output, the first and second shape features, are task-driven, highly discriminative, dense feature representations.

[0033] Step S300: Construct a bilateral soft point-to-point correspondence matrix based on the first shape feature and the second shape feature.

[0034] Specifically, the bilateral soft pointwise correspondence matrix describes the probability distribution of establishing correspondences between the vertices of the source and target shapes. Unlike traditional unidirectional nearest neighbor search, "bidirectional" means that the matrix considers bidirectional consistency from source to target and from target to source simultaneously. It is typically obtained by multiplying the rows and columns of the feature similarity matrix after normalization (e.g., using a softmax operation). This bidirectional constraint effectively suppresses outliers and asymmetric errors common in unidirectional matching, providing a more reliable initial estimate for subsequent functional mapping refinement. During training, the matrix is ​​differentiable, allowing gradient backpropagation to the feature extraction network.

[0035] Step S400: In the mixed basis function space, the functional mapping matrix is ​​iteratively refined using a multi-scale filtering operator to obtain a refined functional mapping matrix. The mixed basis function space consists of the first basis function and the second basis function.

[0036] Specifically, the functional mapping matrix is ​​a corresponding representation established in a function space rather than a point space, exhibiting better continuity and robustness. The hybrid basis function space is designed to handle complex deformations. The first basis function (such as the eigenfunctions of the Laplace-Beltrami operator) excels at handling near-isotropic deformations, maintaining geometric stability; the second basis function (such as elastic basis functions) is specifically designed to capture topological or metric changes caused by non-isotropic deformations. The hybrid basis space, formed by concatenating these two, can simultaneously accommodate the expressive power of both types of deformations. The multi-scale filtering operator smooths and denoises the functional mapping matrix in the spectral domain, gradually eliminating high-frequency noise interference through iterative updates, enhancing the global smoothness and structural fidelity of the mapping. This step achieves the transformation from coarse point-to-point probabilities to fine-grained function space mappings.

[0037] In step S500, based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, the reliability calibration contrastive learning loss and mapping consistency loss are calculated to update the parameters of the feature extraction network.

[0038] Specifically, since this application adopts an unsupervised training paradigm and lacks true ground truth values ​​as supervisory signals, two complementary loss functions are designed to guide the network learning. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss. Its core function is to calibrate the reliability of sample pairs from an optimization perspective. By introducing learnable parameters to dynamically adjust the similarity distribution, it enhances the ability to distinguish hard-negative samples and penalizes spurious negative samples in its own structure, thereby avoiding erroneous correspondences that mislead gradient updates. The mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space. It forces that point-level probability mappings and function-level linear mappings remain mathematically equivalent, preventing drift or contradictions between the two during training. By minimizing the weighted sum of these two loss functions, the parameters of the feature extraction network are continuously updated until convergence.

[0039] Through the closed-loop training process described in steps S100 to S500, this embodiment establishes a complete mechanism of "multi-source feature input - joint optimization - bilateral soft correspondence - hybrid basis refinement - dual loss update". This mechanism not only solves the problem of weak expressive power of single geometric features, but also effectively overcomes the pain points of unreliable samples and inconsistent mappings in unsupervised learning through reliability calibration and structural consistency constraints, significantly improving the model's correspondence accuracy and robustness in complex non-rigid deformation scenarios. It should be noted that this embodiment focuses on the overall data flow and optimization logic during the training phase; the specific implementation details of each module will be further elaborated in subsequent embodiments.

[0040] In some embodiments, the process of obtaining multi-source feature descriptors of the source 3D shape and the target 3D shape can be further specified to construct an input representation that combines geometric accuracy and semantic richness. Specifically, obtaining multi-source feature descriptors of the source 3D shape and the target 3D shape includes: performing feature decomposition based on the Laplacian matrix of the source 3D shape and the target 3D shape to obtain a heat kernel feature descriptor as a geometric structure feature; extracting features from the multi-view rendered images of the source 3D shape and the target 3D shape using a 2D vision basic model, and inversely projecting the extracted 2D features onto the vertices of the 3D surface for aggregation to obtain a semantic descriptor as a 2D semantic feature; and concatenating the heat kernel feature descriptor and the semantic descriptor to obtain a multi-source feature descriptor.

[0041] For acquiring geometric structural features, this embodiment employs the HeatKernel Signature (HKS) based on spectral graph theory. The specific calculation process is as follows: First, the Laplace-Beltrami operator discrete matrices of the source and target 3D shapes are constructed, and then generalized eigenvalue decomposition is performed on these matrices to obtain the first K eigenvalues ​​and their corresponding eigenvectors. Subsequently, the heatkernel signature of each vertex at different time scales is calculated based on the eigenvalues ​​and eigenvectors. The formula for calculating HKS is as follows: ;in, Represents discrete vertices on a three-dimensional shape. This represents the thermal diffusion time parameter. Denotes the first Laplace matrix 1 eigenvalue, Indicates the first The eigenvectors at the vertices The value at that location, This indicates the number of basis functions involved in the calculation. The thermal kernel signature essentially simulates the diffusion process of heat on the surface of a shape, possessing inherent isometry invariance. It can effectively capture the shape's intrinsic geometric structure information, such as local curvature and geodesic distance, and exhibits good robustness to near-isometry deformation. It should be understood that although this embodiment uses HKS as an example, in other implementations, wave kernel signature (WKS), shape DNA, or other Laplacian spectrum-based geometric descriptors can also be used, as long as they can characterize the shape's intrinsic geometric properties.

[0042] To address the issue of insufficient discriminative power of pure geometric features in structurally ambiguous regions or cross-class matching, this embodiment introduces a semantic extraction mechanism based on a two-dimensional vision model. As a preferred implementation, the two-dimensional vision model is used to extract features from multi-view rendered images of the source and target three-dimensional shapes. The extracted two-dimensional features are then inversely projected onto the vertices of the three-dimensional surface for aggregation to obtain a semantic descriptor. This process includes the following four sub-steps:

[0043] Step S121: Render the source 3D shape and the target 3D shape from multiple preset virtual camera positions to obtain a set of 2D images and corresponding depth and normal maps. Specifically, set up several uniformly distributed or specific-viewpoint virtual cameras around the 3D shape and perform rasterization rendering on the shape. In addition to the regular RGB color image, a depth map and a normal map are generated simultaneously. The depth map records the distance information from the pixels to the camera, which is the key geometric basis for subsequent inverse projection; the normal map provides surface orientation information, which helps to enhance the robustness of features to changes in lighting. The number and distribution angle of the virtual cameras can be adjusted according to actual accuracy requirements, for example, 8, 12 or more viewpoints can be used to ensure full coverage of the 3D shape surface.

[0044] Step S122 involves inputting the 2D image set, depth map, and normal map into the frozen image generation model, generating 2D features under the guidance of category hints. "Frozen" here refers to keeping the parameters of the image generation model fixed and not updating gradients during 3D shape correspondence training. The reason for adopting the freezing strategy is that 2D vision foundational models (such as Stable Diffusion, ControlNet, etc.) are typically pre-trained on massive amounts of internet image data, containing extremely rich general semantic knowledge and texture understanding capabilities. Fine-tuning them is prone to catastrophic forgetting or overfitting due to the relatively small amount of 3D training data, thus losing their generalized semantic representation ability. In a specific implementation, Stable Diffusion in the Diff3F framework combined with ControlNet can be used as a concrete example of an image generation model. Depth maps and normal maps are used as conditional control inputs, and simple category text hints (such as "a cat," "ahuman") are combined to guide the model to generate intermediate layer feature maps consistent with the semantics of the current shape. It is important to emphasize that this application is not limited to a specific generative model architecture; any frozen vision model with strong semantic extraction capabilities can be applied.

[0045] Step S123 involves using the recorded depth information and camera parameters to back-project the 2D features onto the 3D surface vertices of the source and target 3D shapes. Since 2D features are pixel-based, while 3D shapes require vertex-level features, a precise mapping from 2D pixels to 3D vertices must be established. Using the depth map obtained in step S121, combined with the camera's intrinsic and extrinsic parameters during rendering, the 3D spatial coordinates of each visible pixel can be calculated and then associated with the nearest 3D mesh vertex. For occluded or invisible vertices, feature assignment cannot be obtained from the current viewpoint; this will be addressed in the next step through multi-view fusion.

[0046] Step S124 involves averaging the 2D features projected from multiple viewpoints onto each 3D surface vertex to form a semantic descriptor. Since occlusion is inevitable in a single viewpoint, only by integrating information from multiple viewpoints can a complete and consistent 3D semantic representation be obtained. For each vertex, the feature vectors projected from all unoccluded viewpoints within its view frustum are collected, and their arithmetic mean or weighted average is calculated as the final semantic descriptor for that vertex. This multi-viewpoint inverse projection aggregation mechanism not only solves the information loss problem caused by single-viewpoint occlusion but also eliminates the ambiguity of 2D features in 3D space through multi-viewpoint consistency constraints, ensuring strict alignment between semantic features and the 3D geometric structure.

[0047] Finally, the obtained heat kernel feature descriptors and semantic descriptors are concatenated along their feature dimensions to form the final multi-source feature descriptor. For example, if the heat kernel features are 16-dimensional and the semantic features are 256-dimensional, the concatenated multi-source features will be 272-dimensional. This fusion of heterogeneous features allows the subsequent feature extraction network to simultaneously utilize the stability of the geometric structure and the discriminativeness of semantic information, significantly improving the model's performance in handling extreme scenarios such as complex non-rigid deformations, topological noise, and local missing values.

[0048] In some embodiments, the feature extraction network in step S200 uses the existing DiffusionNet network to perform point-by-point feature extraction on the input three-dimensional shape to obtain a three-dimensional shape feature description with local geometric information and global intrinsic geometric information.

[0049] The DiffusionNet network comprises layers 1, 2, 3, 4, 5, and 6 connected sequentially. Layer 1 is the input feature mapping layer, layers 2 through 5 are diffusion feature extraction layers, and layer 6 is the output feature mapping layer. Layers 1, 2, 3, 4, 5, and 6 are connected sequentially according to the data flow direction; that is, the output of layer 1 serves as the input of layer 2, the output of layer 2 serves as the input of layer 3, the output of layer 3 serves as the input of layer 4, the output of layer 4 serves as the input of layer 5, and the output of layer 5 serves as the input of layer 6.

[0050] In some embodiments, to further improve the model's accuracy and robustness in complex deformation scenarios, the construction and refinement process of the functional mapping matrix is ​​specifically designed. Specifically, in the mixed basis function space, a multi-scale filtering operator is used to iteratively refine the functional mapping matrix to obtain a refined functional mapping matrix. The mixed basis function space consists of the first basis function and the second basis function.

[0051] The fundamental reason for constructing a hybrid basis function space is that a single basis function cannot simultaneously handle both near-isometric and non-isometric deformations. Traditional Laplace-Beltrammé (LBO) characteristic functions possess isometry invariance and perform excellently in handling near-isometric deformations such as attitude changes, but their expressive power significantly decreases when facing non-isometric deformations such as topological changes or non-uniform scaling. Therefore, this embodiment introduces a second basis function, namely the elastic basis function. This type of basis function is based on the definition of elastic deformation energy and can effectively capture non-isometric geometric changes. The isometry-invariant Laplacesky function and the elastic basis function that handles non-isometric deformations are concatenated to construct a hybrid basis function space, where the first basis function is the Laplacesky function and the second basis function is the elastic basis function. This hybrid basis structure allows subsequent functional mappings to be expressed simultaneously in two complementary subspaces, preserving the intrinsic geometric stability of the shape while possessing adaptability to severe non-rigid deformations. As a specific implementation method, the first 128 Laplacian functions and the first 200 elastic basis functions can be selected and concatenated. However, it should be understood that the specific number and type of basis functions can be adjusted according to the complexity of the actual data, as long as the function of equidistant and non-equidistant representation is satisfied.

[0052] Based on the construction of the hybrid basis space, a reliable initial correspondence needs to be established first. A feature similarity matrix is ​​calculated based on the first and second shape features. After normalizing the rows and columns of the feature similarity matrix, they are multiplied to obtain a bilateral soft point-by-point correspondence matrix. Specifically, the feature similarity matrix reflects the distance metric between the vertices of the source and target shapes in the embedding space. To obtain a probabilistic soft correspondence, this embodiment employs a bidirectional Softmax normalization strategy: first, a Softmax operation is performed on each row of the similarity matrix to obtain the one-way matching probability from the source to the target; simultaneously, a Softmax operation is performed on each column to obtain the one-way matching probability from the target to the source; finally, the two are multiplied element-by-element to obtain the bilateral soft point-by-point correspondence matrix. This bilateral constraint mechanism forces the correspondence to remain consistent in both directions, effectively filtering out outliers and fuzzy matches common in one-way matching, providing high-quality initial input for subsequent iterative refinement.

[0053] Given two 3D shapes and The feature similarity matrix The following formula is used to express this: ;in, The first shape feature learned The L2 row normalized feature matrix; This is the learned second shape feature. The L2 row normalized feature matrix; This is a temperature parameter used to control the hardness or softness of the generated pointwise mapping matrix;

[0054] The bilateral soft point-by-point correspondence matrix Calculate using the following formula: ;in, For matrix The element located in the i-th row and j-th column; For matrix The element in the i-th row; For matrix The element in the j-th column; k represents the index value of the row or column number; This is a Softmax normalization operation;

[0055] Subsequently, the core iterative refinement stage begins. In the mixed basis function space, the initial functional mapping submatrix is ​​calculated based on the hard pointwise correspondence matrix, and then iteratively updated using a multi-scale filtering operator to obtain the refined functional mapping matrix. This process aims to gradually eliminate noise and enhance structural consistency by alternately optimizing point-level correspondences and function-level mappings. The specific iterative refinement process includes the following steps:

[0056] Step S410: Use the bilateral soft pointwise correspondence matrix as the initial input to the hard pointwise correspondence matrix. At the beginning of the first iteration, directly use the bilateral soft pointwise correspondence matrix obtained in the previous steps as the current hard correspondence estimate, and use it as the starting point for subsequent calculations.

[0057] Step S420: In each iteration, calculate the first functional mapping submatrix in the Laplacian function space and the second functional mapping submatrix in the elastic basis function space. Specifically, using the current hard pointwise correspondence matrix, solve the least squares problem or a similar projection problem in the two subspaces to obtain the corresponding functional mapping coefficient matrices. Since the mixed basis is composed of two sets of basis concatenations, the functional mapping matrix is ​​structurally represented in a block diagonal form or a concatenated form, corresponding to the mapping relationship between the LBO subspace and the ELA subspace, respectively.

[0058] Step S430 involves using a multi-scale filtering operator to filter the first and second functional mapping sub-matrices, resulting in updated first and second functional mapping sub-matrices. The multi-scale filtering operator is a key component ensuring the smoothness and continuity of the mapping. In the spectral domain, high-frequency components typically correspond to noise or local detail perturbations, while low-frequency components correspond to the global structure. By applying attenuation weights (such as Gaussian filters or Tikhonov regularization terms) in the frequency domain, high-frequency oscillations caused by inaccurate initial correspondences or data noise can be effectively suppressed, while preserving the main geometric structure information. For the LBO and ELA subspaces, the same or different filtering parameters can be used to accommodate the different spectral characteristics of the two types of basis functions.

[0059] Step S440: Based on the updated first and second functional mapping sub-matrices, recalculate the hard point-to-point correspondence matrix. The filtered and smoothed functional mapping matrix contains more reliable structural information. Transforming it back to the point space (e.g., by restoring vertex features through a linear combination of basis functions and performing nearest neighbor search) yields a more accurate hard point-to-point correspondence matrix than the previous round. This step achieves information feedback from the function space to the point space, forming a positive loop of "point correspondence → functional mapping → filtering and smoothing → new point handling".

[0060] In step S450, after reaching the preset number of iterations, the final updated first functional mapping submatrix and the final updated second functional mapping submatrix are combined to output the refined functional mapping matrix. The preset number of iterations can be set based on convergence or experience, for example, 5 to 10 times. When the iteration ends, the final optimized submatrices in the two subspaces are combined according to the concatenation order of the mixed basis to obtain the final refined functional mapping matrix. This matrix not only mathematically satisfies the linear mapping constraint of the function space, but also physically maintains the geometric continuity between shapes, significantly outperforming the coarse correspondence calculated directly based on the original features.

[0061] It should be noted that although the mathematical calculation logic in the above iterative refinement process involves complex matrix operations, its core idea lies in utilizing the complementarity of mixed bases and the regularization effect of multi-scale filtering to achieve self-correction of correspondences under unsupervised conditions. Even if the specific formula expression may vary depending on the implementation method, as long as it follows the technical concept of "mixed base decomposition - spectral domain filtering - alternating update", it falls within the protection scope of this application. Through this refined iterative optimization mechanism, this embodiment can effectively solve the technical problems of mapping distortion under non-equidistant deformation and correspondence instability under noise interference in traditional methods.

[0062] In some embodiments, to obtain high-precision and robust correspondences under unsupervised conditions, this application designs a specialized loss function system to guide the optimization of the feature extraction network. Specifically, based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, reliability calibration contrastive learning loss and mapping consistency loss are calculated to update the parameters of the feature extraction network. These two loss functions provide supervision signals from the dimensions of sample reliability and structural consistency, respectively, and together constitute the core driving force of unsupervised training.

[0063] The calculation of the comparative learning loss for reliability calibration mainly includes two parts: cross-comparison learning loss and self-comparison learning loss.

[0064] First, when calculating the cross-contrast learning loss, a learnable scaling parameter and offset are introduced to calibrate the feature similarity. Based on the calibrated feature similarity, the set of pseudo-positive samples, and the set of negative samples, the cross-contrast loss value is calculated through an activation function to enhance the optimization capability for hard negative samples. Specifically, in traditional contrastive learning, feature similarity is usually scaled directly using a fixed temperature coefficient. However, in 3D shape correspondence tasks, the distribution of the feature space changes significantly as training progresses, and a fixed scaling factor is difficult to maintain the optimal gradient flow. This embodiment introduces a learnable scaling parameter... and offset , original feature similarity Linear transformation to calibrated similarity This dynamic calibration mechanism allows the network to adaptively adjust the steepness of the similarity distribution. Especially in the later stages of training, when most simple samples have been correctly classified, the learnable parameters automatically amplify subtle differences between hard negative samples, thus avoiding gradient vanishing and ensuring a continuously effective optimization signal. The set of pseudo-positive samples typically consists of vertex pairs with higher probabilities in the current bilateral soft-point correspondence matrix, while the set of negative samples contains all vertices except for the pseudo-positive samples. The activation function can be the sigmoid function or other non-linear mapping functions, used to convert the calibrated similarity into a probabilistic form for calculating the loss.

[0065] Calculate the similarity matrix: ;make This matrix represents The element in the i-th row and j-th column represents the similarity score between point i on the source shape and point j on the target shape. Let the `topk` function represent the function that takes the top k values ​​of the independent variable, and `\` represent the difference operation between sets. Then, the index set of positive correspondence is defined as... The index set of negative correspondences is ;

[0066] The cross-comparison learning loss Express it using the following formula: ;in, The similarity after calibration; Similarity of original features; ; These are the learnable parameters and the offset, respectively.

[0067] Secondly, when calculating the self-contrast learning loss, a self-contrast loss value is calculated using an activation function based on the pseudo-negative sample sets of the source and target 3D shapes themselves, to penalize unreliable soft correspondences. The core motivation for this design is to address the symmetry or repetitive structure problem prevalent in 3D shapes. For example, the limbs of a human model or the legs of an animal are geometrically highly similar, and relying solely on cross-shape cross-contrast loss can easily lead to "self-confusion" errors such as swapping left and right limbs. The self-contrast learning loss forces the network to learn feature representations that can distinguish its own symmetrical parts by constructing negative sample pairs (i.e., points on the same shape that should not be matched) within a single shape. When the network incorrectly treats its own symmetrical points as highly similar matches, this loss term generates a large penalty gradient, forcing the feature space to separate these easily confused regions, thereby significantly improving the structural reliability of the correspondence results.

[0068] The self-comparative learning loss Express it using the following formula: ;in, and Representing shapes and The set of negative sample indices for anchor point i, i.e. ; ;

[0069] Finally, the cross-contrast learning loss and the self-contrast learning loss are weighted and summed to obtain the reliability calibration contrastive learning loss. The weight coefficients can be adjusted according to actual training needs to balance the relationship between cross-shape matching accuracy and self-structure recognition ability.

[0070] The calculation of mapping consistency loss aims to constrain the mathematical equivalence between point-level correspondences and function-level mappings. Specifically, the calculation of mapping consistency loss includes the following steps:

[0071] In the Laplacian function space, the first residual matrix is ​​calculated between the first Laplacian function matrix of the source 3D shape and the transpose of the bilateral soft pointwise correspondence matrix, the second Laplacian function matrix of the target 3D shape, and the corresponding first mapping submatrix in the refined functional mapping matrix. The Frobenius norm of this first residual matrix is ​​then calculated as the first consistency loss. Physically, if the bilateral soft pointwise correspondence matrix accurately describes the mapping relationship between vertices, it should be able to correctly transform the basis functions of the target shape into the basis functions of the source shape, and this transformation result should be consistent with the expression of the refined functional mapping matrix in the Laplacian space. The first residual matrix quantifies the deviation between this theoretical equivalence relation and the actual calculation result. The Frobenius norm, as a standard matrix metric, effectively measures the overall magnitude of this deviation.

[0072] Similarly, in the elastic basis function space, the second residual matrix is ​​calculated between the first elastic basis function matrix of the source 3D shape and the bilateral soft pointwise correspondence matrix, the second elastic basis function matrix of the target 3D shape, and the transpose of the corresponding second mapping submatrix in the refined functional mapping matrix. The Frobenius norm of the second residual matrix is ​​then calculated as the second consistency loss. Since the mixed basis function space contains elastic bases that handle non-isotropic deformations, the same consistency constraints must also be applied in the elastic basis subspace to ensure that the point-level correspondence and functional mapping remain synchronized and self-consistent when the model handles severe non-rigid deformations.

[0073] Subsequently, the first consistency loss and the second consistency loss are added together to obtain the mapping consistency loss. This method of calculating and superimposing in separate base spaces ensures that each component in the mixed base space is sufficiently constrained, avoiding the risk that a single base space dominates the optimization direction while ignoring the characteristics of another base space.

[0074] The mapping consistency loss Express it using the following formula: ;in, The first Laplacian function matrix of the source three-dimensional shape; This is a bilateral soft point-by-point correspondence matrix; The second Laplacian function matrix represents the three-dimensional shape of the target. To refine the first mapping submatrix corresponding to the functional mapping matrix; The first elastic basis function matrix of the source three-dimensional shape; The second elastic basis function matrix represents the three-dimensional shape of the target. To refine the corresponding second mapping submatrix in the functional mapping matrix;

[0075] Finally, the reliability calibration contrast learning loss and mapping consistency loss are weighted and summed to obtain the total loss function, which is used to update the parameters of the feature extraction network. The total loss function integrates optimization objectives from three levels: feature discriminative power, sample reliability, and structural consistency. By minimizing this total loss through the backpropagation algorithm, the feature extraction network can not only learn shape features with high discriminative power but also ensure that the generated correspondences are highly consistent in geometric structure and function space representation, thereby achieving high-precision end-to-end 3D shape correspondence training. It should be understood that the specific calculation formulas for the above loss terms are only an exemplary implementation. Other mathematical expressions or norm definitions that can achieve the same function can also be used without departing from the technical concept of this application.

[0076] This embodiment provides a 3D shape mapping method, applicable to inference or practical applications, for quickly establishing an accurate point-by-point mapping relationship between two 3D shapes without manual annotation. Unlike the previous embodiments that focused on learning and updating model parameters, this embodiment focuses on using a converged model for efficient forward inference. The 3D shape mapping method mainly includes the following steps:

[0077] Step S600: Obtain multi-source feature descriptors for the source 3D shape and the target 3D shape to be matched. The multi-source feature descriptors include geometric structural features and two-dimensional semantic features.

[0078] Specifically, during the inference phase, the source and target 3D shapes are typically unseen test data or real-time acquired 3D scan data. The generation logic of the multi-source feature descriptors remains consistent with that of the training phase, simultaneously calculating the hot kernel feature descriptors representing the intrinsic geometric structure and the 2D semantic features representing high-level semantic information, and then concatenating the two. This approach ensures that the distribution of input data matches the distribution of data during model training, thereby guaranteeing the effective generalization performance of the model. For example, for a new animal model pair, the system automatically extracts its HKS geometric features and semantic features based on the pre-trained visual model to construct an initial input vector containing rich discriminative information.

[0079] Step S700: The multi-source feature descriptors are jointly optimized by the feature extraction network in the pre-trained 3D shape correspondence model to obtain the first shape feature of the source 3D shape and the second shape feature of the target 3D shape.

[0080] Among them, the three-dimensional shape correspondence model is trained based on reliability calibration contrastive learning loss and mapping consistency loss. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space.

[0081] Specifically, "pre-training" here means that the feature extraction network has undergone the complete training process described above, its internal parameters are fixed, and it possesses the ability to extract high-quality, structured embedded features from multi-source raw features. During inference, the feature extraction network only performs forward propagation operations, without involving any backpropagation or parameter updates. Due to the introduction of reliability calibration contrastive learning loss during the training phase, the feature space output by the network has extremely high intra-class compactness and inter-class separation, effectively distinguishing symmetrical parts and repetitive structures; at the same time, the mapping consistency loss ensures the alignment of the feature space with the function space, so that the extracted first and second shape features not only have accurate distance measurements in Euclidean space, but also implicitly contain geometric structural information that satisfies functional mapping constraints. This means that even without additional post-processing optimization during the inference phase, the quality of the features themselves is sufficient to support high-precision correspondence establishment.

[0082] Step S800: Calculate feature similarity based on the first shape feature and the second shape feature, and determine the point-to-point correspondence matrix between the source 3D shape and the target 3D shape based on the feature similarity.

[0083] Specifically, after obtaining the optimized dense feature representation, the degree of matching between vertices can be quantified by calculating the cosine similarity or Euclidean distance between feature matrices. As a preferred implementation, the Nearest Neighbor Search algorithm can be used to solve for point-to-point correspondences. That is, for each vertex on the source shape, the nearest vertex in the feature space of the target shape is searched as the corresponding point, and vice versa, with outliers filtered through a bidirectional consistency check. It should be understood that although this embodiment uses Nearest Neighbor Search as an example, in other implementations, other feature similarity-based matching strategies can be adopted according to the trade-off between accuracy and speed, such as optimal transport-based matching or lightweight attention mechanism matching, as long as feature similarity can be used to establish a point-to-point mapping.

[0084] Specifically, the shape correspondence problem can be transformed into solving the following optimization problem: ;in Representing shape To shape The binary matrix formed by pointwise mapping, i.e., if Indicates from shape arrive Pointwise mapping; if Then there is . For all possible binary matrices; For learned shapes Feature matrix For learned shapes The feature matrix;

[0085] The nearest neighbor search is used as an approximate solution to obtain the following result. ;

[0086] Through steps S600 to S800, this embodiment demonstrates how to deploy a trained high-performance model in practical applications. Thanks to the refined regularization and reliability calibration mechanisms in the mixed basis function space during training, the method in this application does not require time-consuming test-time optimization or complex spectral refinement steps like traditional deep functional mapping methods during the inference phase. It only requires a single forward propagation plus a simple nearest neighbor search to obtain high-precision correspondence results. This not only significantly reduces computational overhead and improves processing efficiency but also enables the technology to adapt to application scenarios with high timeliness requirements, such as real-time animation migration and online 3D retrieval, fully demonstrating the advantages of end-to-end learning frameworks in engineering implementation.

[0087] This embodiment provides a three-dimensional shape correspondence system. This system employs a modular architecture design to implement the training method for the three-dimensional shape correspondence model described in the preceding embodiments. Specifically, the three-dimensional shape correspondence system includes a feature acquisition module, a feature extraction module, a mapping construction and refinement module, and a loss calculation module.

[0088] The feature acquisition module is used to acquire multi-source feature descriptors of the source 3D shape and the target 3D shape. These multi-source feature descriptors include geometric structural features and 2D semantic features. Specifically, the feature acquisition module serves as the data entry point for the entire system and is responsible for executing the preprocessing procedures described in Examples 2 and 3. In actual deployment, this module can be configured to call the graphics rendering engine to generate multi-view images and depth / normal maps, and call the spectral decomposition algorithm library to calculate the eigenvalues ​​and eigenvectors of the Laplacian matrix. By concatenating or tensor-fusion the geometric structural features and 2D semantic features in memory, this module provides a standardized input data stream for subsequent processing. It should be understood that the specific implementation of the feature acquisition module is not limited to software programs; it can also be a hardware component integrating a dedicated image processing unit or a geometric computing accelerator, as long as it can perform the multi-source feature acquisition and fusion function.

[0089] The feature extraction module is used to jointly optimize multi-source feature descriptors through a feature extraction network to obtain the first shape feature of the source 3D shape and the second shape feature of the target 3D shape. Specifically, the feature extraction module internally carries the deep learning model as described in step S200 of Embodiment 1. During the training phase, this module receives multi-source feature descriptors from the feature acquisition module, calculates high-dimensional embedding features through forward propagation, and updates the network weights according to the loss gradient during backpropagation. Since 3D shape data usually has a non-Euclidean structure and a large number of vertices, the feature extraction module is preferably deployed on a graphics processing unit (GPU) or tensor processing unit (TPU) with high parallel computing capabilities to accelerate matrix operations of graph convolution or diffusion operators. In addition, this module can also support a dynamic batching mechanism to adapt to inputs with different topological complexities and maximize hardware utilization.

[0090] The mapping construction and refinement module is used to construct a bilateral soft pointwise correspondence matrix based on the first and second shape features; and to iteratively refine the functional mapping matrix using a multi-scale filtering operator in the mixed basis function space to obtain a refined functional mapping matrix. The mixed basis function space consists of the first and second basis functions. Specifically, this module corresponds to the core algorithm logic in Embodiments 3 and 4. It contains two main sub-processing units: a soft correspondence calculation unit and a spectral domain refinement unit. The soft correspondence calculation unit performs a bidirectional Softmax normalization operation to generate an initial probability correspondence matrix; the spectral domain refinement unit alternately performs mapping coefficient solving and multi-scale filtering smoothing operations in the mixed basis space composed of Laplacian and elastic bases. Since the iterative refinement process involves a large number of spectral transformations and matrix multiplications, this module also highly relies on high-performance computing resources. In some implementations, the frequency domain response of the filtering operator can be pre-calculated and cached to reduce the overhead of repeated computation during runtime, thereby improving training efficiency.

[0091] The loss calculation module is used to calculate the reliability calibration contrastive learning loss and mapping consistency loss based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, in order to update the parameters of the feature extraction network. Specifically, the loss calculation module implements the composite loss function logic as described in Example 4. This module receives the output results from the mapping construction and refinement module, calculates the cross-contrast loss, self-contrast loss, and mapping consistency residual under the mixed basis space, and performs a weighted summation according to preset weights to obtain the total loss value. Subsequently, this module uses an automatic differentiation framework to backpropagate the gradient of the total loss value to the feature extraction module, driving the iterative update of the network parameters. To ensure numerical stability, the loss calculation module can also integrate gradient pruning or outlier detection mechanisms to prevent training divergence caused by extreme samples.

[0092] Regarding the hardware deployment and application scenarios of the aforementioned system, this application provides multiple implementation methods to adapt to different business needs. As a preferred implementation, the 3D shape mapping system can be deployed in a high-performance GPU server cluster, suitable for offline training and model development scenarios with large-scale datasets. In this configuration, data transmission between modules can be optimized through high-speed buses (such as NVLink) or distributed communication frameworks (such as NCCL) to eliminate I / O bottlenecks. As another implementation, after model training converges, the feature acquisition module, feature extraction module, and mapping construction and refinement module can be lightweighted and deployed in edge computing devices or embedded terminals, retaining only the forward computation path required for inference. This edge-side deployment scheme can meet the needs of low-latency application scenarios such as real-time animation migration and online 3D retrieval. It should be understood that although this embodiment is described with modular functional division, in actual engineering, each module can run as an independent software process, or it can be encapsulated as a unified dynamic link library or FPGA firmware. Even the functions of some modules can be completed collaboratively by cloud services and local devices, as long as the overall technical solution described in this application can be achieved. Through this flexible system architecture, this application not only protects the core algorithm logic, but also provides full support for the implementation of the technical solution on different hardware platforms.

[0093] This embodiment provides a computer-readable storage medium as the underlying carrier protection form of the technical solution of this application. Specifically, the computer-readable storage medium stores a computer program, which, when executed by a processor, implements the training method for the three-dimensional shape correspondence model as described above, or implements the three-dimensional shape correspondence method as described above.

[0094] In specific implementations, a computer-readable storage medium refers to any tangible or non-transitory medium capable of containing, storing, communicating, propagating, or transmitting program instructions for use by an instruction execution system, system, or device. For example, such a medium may include, but is not limited to, read-only memory (ROM), random access memory (RAM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical discs (such as CD-ROMs and DVDs), magnetic disk storage devices, or other magnetic storage devices. It should be understood that although various specific types of physical media are listed herein, this is merely an illustrative example provided for full disclosure and is not intended to limit the scope of protection of this application. Any device capable of storing data in electronic, magnetic, optical, electromagnetic, infrared, or semiconductor manner and allowing it to be read by a computer system falls within the scope of computer-readable storage media as referred to in this embodiment.

[0095] When the processor loads and executes the computer program from the storage medium, it is essentially reproducing the technical processes described in the foregoing embodiments. For example, when executing the program instructions corresponding to the training method, the processor sequentially calls the code segment for obtaining multi-source feature descriptors, runs the forward and backward propagation code of the feature extraction network, executes the iterative refinement algorithm in the mixed basis function space, and calculates the optimization routine for calculating the reliability calibration contrast learning loss and mapping consistency loss. Similarly, when executing the program instructions corresponding to the three-dimensional shape correspondence method, the processor focuses on using the pre-trained model for feature extraction and nearest neighbor search matching. This process of transforming abstract algorithm logic into specific machine instructions and embedding them in the medium enables the technical solution of this application to possess portability and distributability independent of specific hardware architectures.

[0096] From a technical application perspective, the storage medium provided in this embodiment has a wide range of applicable scenarios. During the research and development phase, it can manifest as training scripts and model weight files stored on the hard drive of a high-performance GPU server cluster; during the product delivery phase, it can be a dynamic link library file encapsulated in a 3D reconstruction software installation package, or firmware burned into the embedded chip of a handheld 3D scanner; during the service provision phase, it can also be an API service package available for remote invocation in cloud object storage. Regardless of the specific deployment form, as long as its core lies in driving the processor to complete the 3D shape correspondence processing logic described in this application through stored program instructions, it falls within the protection scope of this embodiment. In this way, this application not only protects the method flow and system architecture, but also covers the storage medium, which is a key link in the circulation of software products, thereby constructing a more rigorous and comprehensive intellectual property protection system.

[0097] To verify the effectiveness of the training method for the 3D shape correspondence model and the 3D shape correspondence method provided in the aforementioned embodiments, this embodiment combines multiple publicly available benchmark datasets and extreme test scenarios to conduct a comprehensive quantitative evaluation and qualitative analysis of the technical solution. These experimental results intuitively demonstrate the performance improvement brought about by the combination of core technical means such as multi-source feature fusion, reliability calibration contrastive learning, and hybrid basis functional mapping refinement.

[0098] For quantitative evaluation, this embodiment selected several authoritative datasets covering both near-isometric and non-isometric deformations for testing. For near-isometric deformation scenarios, the FAUST, SCAPE, and SHREC'19 datasets were used. The results are shown in Table 1.

[0099] Table 1. Average geodesic error (%) of this invention and other methods on approximately equidistant reference datasets.

[0100] Experimental results show that the average geodesic error of our proposed method is significantly lower than that of existing state-of-the-art methods on these datasets. Specifically, on the FAUST re-gridized test set, the average geodesic error of our proposed method is only 1.5%, which is better than methods such as ContrastiveFMNet (1.6%), HybridFMaps (1.4% but with weak generalization) and ULRSSM (1.6%). On the more challenging SHREC'19 dataset, our proposed method achieves an average geodesic error of 3.2%, significantly outperforming methods such as ContrastiveFMNet (5.8%) and DeepShells (27.4%). This superior performance is mainly attributed to the introduction of multi-source feature descriptors. By fusing the geometric stability of hot-core feature descriptors with the discriminative power of two-dimensional semantic features, the problem of inaccurate matching of single geometric features in structurally ambiguous regions is effectively overcome. At the same time, the Laplaciansky component in the mixed basis function space maintains extremely high structural fidelity under near-equidistant conditions.

[0101] For non-isotropic deformation scenarios, this embodiment evaluates the algorithm on the SMAL and DT4D-H datasets. These datasets contain drastic topological changes and metric distortions, placing extremely high demands on the algorithm's generalization ability. The results are shown in Table 2.

[0102] Table 2. Average geodesic error (%) of the method of this invention and other methods on non-equidistant reference datasets.

[0103] Experimental data show that the proposed method achieves an average geodesic error of 2.8% on the SMAL dataset and 3.7% on the DT4D-H inter-class matching task, both outperforming comparative methods such as ContrastiveFMNet (3.6% and 3.9%, respectively) and HybridFMaps (3.0% and 3.9%, respectively). The advantage of the proposed method is particularly evident when handling large-scale non-rigid deformations across classes. This verifies the crucial role of elastic basis functions in the mixed basis function space, which effectively captures the geometric changes caused by non-equidistant deformations, compensating for the insufficient expressive power of traditional Laplacianski methods in handling such deformations. Furthermore, the self-comparison mechanism in the reliability calibration contrastive learning loss effectively suppresses erroneous matching of symmetrical parts and repeating structures, further improving the accuracy of non-equidistant matching.

[0104] In terms of qualitative analysis, visualizing the results provides a more intuitive understanding of the technical advantages of this application. For example... Figure 2 and Figure 3As shown, the color and texture mapping results on the SHREC'19 and SMAL datasets are presented respectively. A comparison reveals that other methods (such as ULRSSM, MDND, and ContrastiveFMNet) generally exhibit significant texture distortion or misalignment in detailed areas such as limb extremities, tails, or faces. In contrast, the texture mapping generated by the method in this application is the smoothest and most continuous, highly consistent with the ground truth, with almost no visible distortion artifacts observed. Figure 4 As shown, on the DT4D dataset with its severely non-equidistant deformable shape pairs, the method in this application can still establish accurate and smooth point-to-point correspondences, demonstrating its robustness under complex deformations. These visual effects directly confirm that the refined functional mapping matrix, after iterative optimization by multi-scale filtering operators in the mixed basis space, successfully eliminates high-frequency noise and maintains the consistency of the global structure.

[0105] More importantly, the method described in this application demonstrates outstanding robustness under extremely challenging scenarios. For example... Figure 5 As shown, when there are local structural defects in the source or target shape, most existing methods, relying on complete geometric topological information, often suffer from severe correspondence drift or complete failure near the missing region. However, the method in this application, leveraging the contextual completion capability provided by two-dimensional semantic features and the suppression of outliers through a reliability calibration mechanism, is still able to establish a high-precision correspondence in the remaining complete region. Figure 6 As shown, in scenarios where the input shape contains topological noise (such as mesh connection errors, holes, etc.), the corresponding results of other comparison methods exhibit large-scale confusion and distortion (marked by circles in the figure), while the corresponding results of the method in this application remain accurate and are not significantly affected by topological defects. This indicates that by fusing semantic priors and geometric structures, and combining the joint constraints of dual loss functions, this application constructs a three-dimensional shape correspondence technology scheme that is not only highly accurate but also highly resistant to interference, effectively solving the technical pain points mentioned in the background technology, such as single feature representation, insufficient reliability of contrastive learning, and poor robustness in complex scenarios.

[0106] It should be understood that the above experimental data and visualization results are merely exemplary verifications of the technical solution of this application under specific datasets and configurations, and do not constitute a limitation on the scope of protection of this application. In practical applications, based on specific business needs and hardware conditions, hyperparameters such as the number of basis functions, loss weight coefficients, and feature dimensions can be adaptively adjusted. As long as the core technical concepts of multi-source feature fusion, reliability calibration, and hybrid basis refinement disclosed in this application are followed, corresponding beneficial technical effects can be obtained.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A training method for a three-dimensional shape correspondence model, characterized in that, include: Obtain multi-source feature descriptors of source 3D shape and target 3D shape, wherein the multi-source feature descriptors include geometric structural features and 2D semantic features; The multi-source feature descriptors are jointly optimized by a feature extraction network to obtain the first shape feature of the source three-dimensional shape and the second shape feature of the target three-dimensional shape. Based on the first shape feature and the second shape feature, a bilateral soft point-by-point correspondence matrix is ​​constructed; In the mixed basis function space, the functional mapping matrix is ​​iteratively refined by the multi-scale filtering operator to obtain the refined functional mapping matrix. The mixed basis function space is composed of the first basis function and the second basis function. Based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, the reliability calibration contrastive learning loss and mapping consistency loss are calculated to update the parameters of the feature extraction network. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space.

2. The method according to claim 1, characterized in that, The acquisition of multi-source feature descriptors for the source 3D shape and the target 3D shape includes: Based on the Laplacian matrix of the source 3D shape and the target 3D shape, feature decomposition is performed to obtain the heat core feature descriptor as the geometric structure feature; The source 3D shape and the target 3D shape are used to extract features from the multi-view rendered images using a 2D vision basic model. The extracted 2D features are then back-projected onto the 3D surface vertices and aggregated to obtain a semantic descriptor as the 2D semantic feature. The heat core feature descriptor and the semantic descriptor are concatenated to obtain the multi-source feature descriptor.

3. The method according to claim 2, characterized in that, The process involves extracting features from the multi-view rendered images of the source 3D shape and the target 3D shape using a 2D vision foundation model, and then back-projecting the extracted 2D features onto the 3D surface vertices for aggregation to obtain a semantic descriptor, including: The source 3D shape and the target 3D shape are rendered from multiple preset virtual camera positions to obtain a set of 2D images and corresponding depth maps and normal maps; The two-dimensional image set, the depth map, and the normal map are input into a frozen image generation model, and the two-dimensional features are generated under the guidance of category hints. Using the recorded depth information and camera parameters, the two-dimensional features are back-projected onto the three-dimensional surface vertices of the source three-dimensional shape and the target three-dimensional shape; The semantic descriptor is formed by averaging the two-dimensional features of each three-dimensional surface vertex projected from multiple viewpoints.

4. The method according to claim 1, characterized in that, The iterative refinement of the functional mapping matrix using a multi-scale filtering operator in the mixed basis function space to obtain the refined functional mapping matrix includes: The hybrid basis function space is constructed by concatenating the equidistant Laplacian function and the elastic basis function that handles non-equidistant deformation, wherein the first basis function is the Laplacian function and the second basis function is the elastic basis function. The feature similarity matrix is ​​calculated based on the first shape feature and the second shape feature. The rows and columns of the feature similarity matrix are normalized and then multiplied to obtain the bilateral soft point-by-point correspondence matrix. In the mixed basis function space, the initial functional mapping submatrix is ​​calculated based on the hard point-by-point correspondence matrix, and the initial functional mapping submatrix is ​​iteratively updated using the multi-scale filtering operator to obtain the refined functional mapping matrix.

5. The method according to claim 4, characterized in that, In the mixed basis function space, the initial functional mapping submatrix is ​​calculated based on the hard point-wise correspondence matrix, and the initial functional mapping submatrix is ​​iteratively updated using the multi-scale filtering operator to obtain the refined functional mapping matrix, including: Use the bilateral soft point-by-point correspondence matrix as the initial input to the hard point-by-point correspondence matrix; In each iteration, the first functional mapping submatrix in the Laplacian function space and the second functional mapping submatrix in the elastic basis function space are calculated respectively. The first functional mapping submatrix and the second functional mapping submatrix are filtered using the multi-scale filtering operator to obtain the updated first functional mapping submatrix and the updated second functional mapping submatrix. Based on the updated first functional mapping submatrix and the updated second functional mapping submatrix, the hard point-by-point correspondence matrix is ​​recalculated; After reaching the preset number of iterations, the final updated first functional mapping submatrix and the final updated second functional mapping submatrix are combined to output the refined functional mapping matrix.

6. The method according to claim 1, characterized in that, The calculation of the reliability calibration contrastive learning loss based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix includes: The cross-comparison learning loss is calculated by introducing learnable scaling parameters and offsets to calibrate the feature similarity. Based on the calibrated feature similarity, the set of pseudo-positive samples, and the set of negative samples, the cross-comparison loss value is calculated through an activation function to enhance the optimization ability of hard negative samples. Calculate the self-contrast learning loss: Based on the pseudo-negative sample set of the source 3D shape and the target 3D shape, calculate the self-contrast loss value through the activation function to penalize unreliable soft correspondences; The reliability calibration contrastive learning loss is obtained by weighted summing of the cross-contrast learning loss and the self-contrast learning loss.

7. The method according to claim 6, characterized in that, The calculation of reliability calibration contrasts the learning loss and the mapping consistency loss to update the parameters of the feature extraction network, wherein the calculation method of the mapping consistency loss includes: In the Laplacian function space, calculate the first residual matrix between the first Laplacian function matrix of the source 3D shape, the bilateral soft pointwise correspondence matrix, the second Laplacian function matrix of the target 3D shape, and the transpose of the corresponding first mapping submatrix in the refined functional mapping matrix, and calculate the Frobenius norm of the first residual matrix as the first consistency loss. In the elastic basis function space, calculate the second residual matrix between the first elastic basis function matrix of the source 3D shape, the bilateral soft pointwise correspondence matrix, the second elastic basis function matrix of the target 3D shape, and the transpose of the corresponding second mapping submatrix in the refined functional mapping matrix, and calculate the Frobenius norm of the second residual matrix as the second consistency loss. The first consistency loss and the second consistency loss are added together to obtain the mapping consistency loss; The reliability calibration contrast learning loss and the mapping consistency loss are weighted and summed to obtain the total loss function, which is then used to update the parameters of the feature extraction network.

8. A three-dimensional shape correspondence method, characterized in that, include: Obtain multi-source feature descriptors of the source 3D shape and the target 3D shape to be matched, wherein the multi-source feature descriptors include geometric structural features and two-dimensional semantic features; The multi-source feature descriptors are jointly optimized by the feature extraction network in the pre-trained 3D shape correspondence model to obtain the first shape feature of the source 3D shape and the second shape feature of the target 3D shape. The 3D shape correspondence model is trained based on reliability calibration contrastive learning loss and mapping consistency loss. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss. The mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space. Based on the first shape feature and the second shape feature, a feature similarity is calculated, and based on the feature similarity, a point-to-point correspondence matrix between the source 3D shape and the target 3D shape is determined.

9. A three-dimensional shape correspondence system, characterized in that, include: The feature acquisition module is used to acquire multi-source feature descriptors of the source 3D shape and the target 3D shape, wherein the multi-source feature descriptors include geometric structural features and two-dimensional semantic features; The feature extraction module is used to jointly optimize the multi-source feature descriptors through a feature extraction network to obtain the first shape feature of the source three-dimensional shape and the second shape feature of the target three-dimensional shape. The mapping construction and refinement module is used to construct a bilateral soft point-by-point correspondence matrix based on the first shape feature and the second shape feature; Furthermore, in the mixed basis function space, the functional mapping matrix is ​​iteratively refined using a multi-scale filtering operator to obtain a refined functional mapping matrix. The mixed basis function space is composed of a first basis function and a second basis function. The loss calculation module is used to calculate the reliability calibration contrastive learning loss and the mapping consistency loss based on the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix, so as to update the parameters of the feature extraction network. The reliability calibration contrastive learning loss includes cross-contrast learning loss and self-contrast learning loss, and the mapping consistency loss is used to constrain the structural consistency between the bilateral soft pointwise correspondence matrix and the refined functional mapping matrix in the mixed basis function space.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the training method for the three-dimensional shape correspondence model as described in any one of claims 1 to 7, or implements the three-dimensional shape correspondence method as described in claim 8.

Images