Depth non-rigid three-dimensional shape corresponding method, system, equipment and medium

Abstract

The invention discloses a depth non-rigid three-dimensional shape corresponding method, system and device and a medium. The method comprises the steps that a first feature vector matrix and a second feature vector matrix are determined; in the micro-iteration process, based on the current spectral domain mapping matrix, the first eigenvector matrix and the second eigenvector matrix, a symbolized soft mapping operator based on a kernel function and used for describing the corresponding relation between the vertexes is constructed, the symbolized soft mapping operator is re-projected back to the spectral space, and intermediate state spectral mapping estimation is obtained; performing multi-channel filtering processing on the intermediate state spectral mapping estimation to obtain a refined coupling mapping matrix, and taking the refined coupling mapping matrix as a current spectral domain mapping matrix of next iteration until a preset number of iterations is reached to obtain a target coupling mapping matrix; and converting the target coupling mapping matrix into a point-by-point mapping matrix to realize point-by-point correspondence between the source shape and the target shape. According to the invention, the accuracy corresponding to the deep non-rigid three-dimensional shape can be improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to a method, system, device and medium for mapping depth non-rigid three-dimensional shapes. Background Technology

[0002] Establishing correspondences between non-rigid shapes is a crucial and fundamental problem in computer graphics, geometric processing, and computer vision. Its core objective is to establish point-to-point semantic mapping relationships between different geometric entities. This research not only possesses a strong theoretical foundation but also demonstrates significant research value and application prospects in digital twins, character animation, augmented reality, and medical image analysis. Since most deformations of 3D shapes in the real world are approximately equidistant, finding point-by-point approximate equidistant correspondences between models has become a core problem in the field of shape correspondence research.

[0003] Even with the advent of deep learning, 3D shape mapping still faces significant challenges in complex real-world applications. While feature representation capabilities have improved, existing deep learning frameworks exhibit severe static limitations in spectral filtering mechanisms. Mainstream methods often employ predefined fixed filtering strategies or simple feedforward learning modes when constructing function mapping matrices, neglecting the specificity of different geometric features' spectral distributions. This prevents the model from adaptively and dynamically adjusting its frequency response based on specific deformation types. A more critical technical bottleneck lies in the optimization gap between the feature learning phase and the geometric refinement process. To achieve high-precision matching, classical refinement algorithms are typically required for spectral upsampling. For example, the widely used ZoomOut module (an existing algorithm for optimizing the correspondence between 3D shapes) significantly improves alignment accuracy, but its inherent discrete operations are mathematically non-differentiable. This leads to a severe gradient barrier during training, forcing the feature learning module and the refinement phase to disconnect, preventing the filter parameters from dynamically optimizing along the refinement trajectory. Even with the emergence of methods utilizing symbolic computation... Differentiable solutions for memory overhead have been proposed, but these attempts have focused on differentiability of the process and have failed to achieve deep coupling between feature representation and iterative refinement trajectory.

[0004] Due to the lack of real-time feedback from downstream geometric convergence targets, current shape correspondence methods cannot identify which frequency modulation is most conducive to the convergence of subsequent refinement targets, and their accuracy and overall consistency in handling complex non-equidistant deformations are still significantly insufficient. Summary of the Invention

[0005] This application aims to propose a method, system, device, and medium for mapping deep non-rigid three-dimensional shapes, which can improve the accuracy of mapping deep non-rigid three-dimensional shapes and ensure consistency throughout the process.

[0006] In a first aspect, embodiments of this application provide a method for mapping depth-non-rigid three-dimensional shapes, the method comprising: The source shape and target shape in the target scene are represented as triangular meshes with multiple vertices, thus obtaining the triangular meshes of the source shape and the target shape. The source shape and the target shape are non-rigid three-dimensional shapes. Extract the feature vectors from the source shape and the target shape to obtain the first original geometric feature vector of the source shape and the second original geometric feature vector of the target shape; Based on the first original geometric eigenvector and the second original geometric eigenvector, calculate the initial functional mapping matrix, and use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix; Calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape; Perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix; During the differentiable iteration process, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, a kernel-based symbolic soft mapping operator is constructed to describe the correspondence between vertices. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate state spectral mapping estimate. The intermediate state spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix. The target coupling mapping matrix is ​​converted into a point-by-point mapping matrix, and the point-by-point mapping matrix is ​​used to realize the point-by-point correspondence between the source shape and the target shape.

[0007] In some implementations, constructing a kernel-based symbolic soft mapping operator to describe the correspondence between vertices, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, includes: Calculate the similarity between the first eigenvector matrix and the second eigenvector matrix under the current spectral domain mapping matrix; The similarity is calculated as a ratio to the parameter used to adjust the mapping sharpness, and the ratio calculation result is obtained. The ratio calculation result is processed by the Softmax function to construct a kernel-based symbolic soft mapping operator for describing the correspondence between vertices.

[0008] In some implementations, the step of reprojecting the kernel-based symbolic soft mapping operator back into the spectral space to obtain an intermediate-state spectral mapping estimate includes: The symbolic soft mapping operator based on the kernel function is multiplied by the first eigenvector matrix using the associative law to obtain the multiplication result; Perform a pseudo-inverse operation on the second eigenvector matrix to obtain the pseudo-inverse operation result; Multiplying the result of the multiplication with the result of the pseudo-inverse operation yields the intermediate state spectral mapping estimate.

[0009] In some implementations, the step of performing multi-channel filtering on the intermediate state spectral mapping estimate to obtain a refined coupling mapping matrix includes: A filtering function is constructed using a KAN grid, and a multi-channel filtering module is constructed based on the filtering function. The intermediate state spectrum mapping estimate is processed by the multi-channel filtering module to obtain a refined coupling mapping matrix.

[0010] In some implementations, constructing the filtering function using a KAN grid includes: ; in, Represents the filter function. Indicates an incremental index. This represents the number of eigenvalues ​​of the Laplace-Beltrami operator. express spline functions under index, express Index Spline functions corresponding to the Laplace-Beltrami operator. Indicates the first Eigenvalues ​​of the Laplace-Beltrami operator.

[0011] In some embodiments, the step of using the multi-channel filtering module to perform multi-channel filtering on the intermediate state spectral mapping estimate to obtain a refined coupling mapping matrix includes: Perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first diagonal eigenvalue matrix, and perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second diagonal eigenvalue matrix; The first diagonal eigenvalue matrix is ​​input into the multi-channel filtering module to obtain the first module result; The second diagonal eigenvalue matrix is ​​input into the multi-channel filtering module to obtain the second module result; Multiply the results of the first module, the results of the second module, and the intermediate state spectrum mapping estimate together to obtain the refined coupling mapping matrix.

[0012] In some implementations, converting the target coupling mapping matrix into a pointwise mapping matrix includes: The first feature vector matrix is ​​rotated and aligned using the target coupling mapping matrix to obtain the alignment result; Based on the alignment result, a nearest neighbor search is performed in the spectral space to obtain a pointwise mapping matrix, which contains semantic indices from the vertices of the source shape to the vertices of the target shape.

[0013] Secondly, embodiments of this application also provide a depth-non-rigid three-dimensional shape correspondence system, the system comprising: A triangular mesh representation unit is used to represent the source shape and target shape in the target scene as a triangular mesh with multiple vertices, thereby obtaining the triangular mesh of the source shape and the triangular mesh of the target shape, wherein the source shape and the target shape are non-rigid three-dimensional shapes; A feature vector extraction unit is used to extract feature vectors from the source shape and the target shape to obtain a first original geometric feature vector of the source shape and a second original geometric feature vector of the target shape. The first data calculation unit is used to calculate an initial functional mapping matrix based on the first original geometric feature vector and the second original geometric feature vector, and to use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix. The second data calculation unit is used to calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape, respectively, to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape. The eigenvalue decomposition unit is used to perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and to perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix. A differentiable iterative unit is used to construct a kernel-based symbolic soft mapping operator to describe the correspondence between vertices based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix during the differentiable iteration process. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate state spectral mapping estimate. The intermediate state spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix. A shape point-to-point correspondence unit is used to convert the target coupling mapping matrix into a point-to-point mapping matrix, and realize the point-to-point correspondence between the source shape and the target shape through the point-to-point mapping matrix.

[0014] Thirdly, embodiments of this application also provide an electronic device, including at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, the instructions being executed by the at least one control processor to enable the at least one control processor to perform a depth-non-rigid three-dimensional shape correspondence method as described above.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions for causing a computer to execute a depth-nonrigid three-dimensional shape correspondence method as described above.

[0016] Compared with the prior art, this application has the following beneficial effects: This application represents the source and target shapes in the target scene as triangular meshes with multiple vertices, obtaining triangular meshes for the source and target shapes, respectively. The source and target shapes are non-rigid 3D shapes. Feature vectors are extracted from the source and target shapes to obtain a first original geometric feature vector for the source shape and a second original geometric feature vector for the target shape. Based on the first and second original geometric feature vectors, an initial functional mapping matrix is ​​calculated. This initial functional mapping matrix is ​​used as the initial differentiable iterative current spectral domain mapping matrix, providing a solid data foundation for subsequent differentiable iterations. Furthermore, by calculating the Laplacian matrices of the triangular meshes for the source and target shapes, the first Laplacian matrix for the source shape and the second Laplacian matrix for the target shape are obtained. Generalized eigenvalue decomposition is performed on the first Laplacian matrix to determine the first eigenvector matrix, and similarly, generalized eigenvalue decomposition is performed on the second Laplacian matrix to determine the second eigenvector matrix, again providing a solid data foundation for subsequent differentiable iterations. In the differentiable iterative process, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, a kernel-based symbolic soft mapping operator is constructed to describe the correspondence between vertices. This kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate spectral mapping estimate. This intermediate spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. This refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration, until a preset number of iterations is reached to obtain the target coupling mapping matrix. This fully differentiable iterative refinement loop solves the memory overflow problem faced by traditional methods when processing high-precision meshes and improves matching accuracy under non-equidistant deformation, ensuring consistency throughout the process. Finally, the target coupling mapping matrix is ​​converted into a point-by-point mapping matrix. This point-by-point mapping matrix realizes the point-by-point correspondence between the source and target shapes. Establishing a point-by-point mapping matrix using a high-precision target coupling mapping matrix improves the accuracy of corresponding depth-independent non-rigid 3D shapes. Attached Figure Description

[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is a flowchart illustrating an embodiment of the depth-non-rigid three-dimensional shape correspondence method provided in this application; Figure 2 This is a schematic diagram of the process of fusing differential iteration for deep non-rigid three-dimensional shape correspondence in the preferred embodiment of the deep non-rigid three-dimensional shape correspondence method provided in this application. Figure 3 This is a schematic diagram of the structure of an embodiment of the depth-non-rigid three-dimensional shape correspondence system provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the electronic device provided in this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0019] In the description of this application, the use of terms such as "first," "second," etc., is for the purpose of distinguishing technical features only and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0020] In the description of this application, it should be understood that the orientation descriptions, such as up, down, etc., are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0021] In the description of this application, it should be noted that, unless otherwise explicitly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0022] To address the shortcomings in related technologies, such as the inability to identify which frequency modulation is most conducive to the convergence of subsequent target refinement, and the significant deficiencies in accuracy and overall consistency when dealing with complex non-equidistant deformations, this application proposes a method, system, device, and medium for deep non-rigid three-dimensional shape correspondence.

[0023] Reference Figure 1 This application provides a schematic flowchart of a method for mapping non-rigid three-dimensional shapes at depth. This method is applied to electronic devices, such as servers or mobile terminals. Figure 1 As shown, the method for corresponding non-rigid three-dimensional shapes at depth may include the following steps S101 to S107.

[0024] Step S101: Represent the source shape and target shape in the target scene as triangular meshes with multiple vertices to obtain the triangular meshes of the source shape and the target shape. The source shape and target shape are non-rigid three-dimensional shapes.

[0025] Specifically, under discrete settings, the input three-dimensional shape (i.e., source shape) and three-dimensional shape (i.e., target shape) are respectively represented as having and A triangular mesh with 10 vertices.

[0026] The aforementioned target scenarios can be practical applications such as digital twins and character animation, medical image analysis and anatomical structure matching, and augmented reality and 3D reconstruction.

[0027] Step S102: Extract the feature vectors from the source shape and the target shape to obtain the first original geometric feature vector of the source shape and the second original geometric feature vector of the target shape.

[0028] Specifically, feature vectors are extracted from the source shape and the target shape using a feature extraction network to obtain a first original geometric feature vector of the source shape and a second original geometric feature vector of the target shape. The feature extraction network can be the FMNet network model, or other neural network models known to those skilled in the art; this embodiment does not impose specific limitations or descriptions on this.

[0029] Step S103: Calculate the initial functional mapping matrix based on the first and second original geometric eigenvectors, and use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix.

[0030] Specifically, based on the first original geometric feature vector Second original geometric feature vector ,pass and The coefficient matrix is ​​calculated. and Ideal Any coefficient matrix can be used Transform into ,Right now: When the initial coefficient matrix is ​​known and Then, it can be done through the formula Inverse solution The least squares method can be used to solve this problem.

[0031] Step S104: Calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape.

[0032] Specifically, the discrete Laplace-Beltrami operators for the triangular meshes of the source shape and the target shape are calculated separately to obtain the first Laplace matrix. Second Laplace matrix , and The following formula is used to calculate: ; in, The diagonal matrix is ​​formed by using the local area weights of each vertex of the corresponding shape (which can be either the source shape or the target shape) as its diagonal elements. This is the cotangent weight matrix corresponding to the shape.

[0033] Step S105: Perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix.

[0034] Specifically, generalized eigenvalue decomposition is performed on the first and second Laplacian matrices respectively, and the eigenvalues ​​are extracted from the first and second Laplacian matrices. The first eigenvector matrix is ​​constructed from the eigenvalues. Second eigenvector matrix .

[0035] Step S106: During the differentiable iteration process, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, a kernel-based symbolic soft mapping operator is constructed to describe the correspondence between vertices. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain the intermediate spectral mapping estimate. The intermediate spectral mapping estimate is then subjected to multi-channel filtering to obtain the refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until the preset number of iterations is reached to obtain the target coupling mapping matrix.

[0036] In this embodiment, the similarity between the first and second eigenvector matrices under the current spectral domain mapping matrix is ​​calculated. The similarity is then compared to a parameter used to adjust the mapping sharpness, yielding a ratio calculation result. This ratio result is processed using the Softmax function to construct a kernel-based symbolic soft mapping operator that describes the correspondence between vertices. The kernel-based symbolic soft mapping operator is then multiplied by the first eigenvector matrix using the associative law, yielding a multiplication result. The second eigenvector matrix is ​​then pseudo-inverseed, yielding a pseudo-inverse result. Finally, the multiplication result and the pseudo-inverse result are multiplied together to obtain the intermediate-state spectral mapping estimate.

[0037] A filtering function is constructed using a KAN grid, and a multi-channel filtering module is built based on this function. The multi-channel filtering module is then used to perform multi-channel filtering on the intermediate state spectral mapping estimate, resulting in a refined coupling mapping matrix. Specifically: The filtering function is constructed using a KAN grid, including: ; in, Represents the filter function. Indicates an incremental index. This represents the number of eigenvalues ​​of the Laplace-Beltrami operator. express spline functions under index, express Index Spline functions corresponding to the Laplace-Beltrami operator. Indicates the first Eigenvalues ​​of the Laplace-Beltrami operator.

[0038] The first Laplacian matrix is ​​subjected to generalized eigenvalue decomposition to determine the first diagonal eigenvalue matrix, and the second Laplacian matrix is ​​subjected to generalized eigenvalue decomposition to determine the second diagonal eigenvalue matrix. The first diagonal eigenvalue matrix is ​​input into a multi-channel filtering module to obtain the first module result; the second diagonal eigenvalue matrix is ​​input into a multi-channel filtering module to obtain the second module result; the first module result, the second module result, and the intermediate state spectral mapping estimate are multiplied together to obtain the refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix.

[0039] This embodiment transforms the original non-differentiable geometric refinement process into a mathematical module capable of propagating gradients, and utilizes a neural network with strong fitting capabilities to dynamically adjust the spectral signal. Specifically, it learns the nonlinear response of the spectrum through the Kolmogorov-Arnold Network (KAN) neural network architecture, replacing the traditional polynomial filtering scheme, thereby capturing deformation features more accurately. Simultaneously, it utilizes symbolic computation techniques to construct a coupling layer with low memory consumption and full-process differentiability. This mechanism allows errors generated by spatial domain alignment to be directly fed back to the front-end filter, enabling the filter to automatically adjust its parameters based on the alignment effect, eliminating the gap between feature learning and geometric refinement. Specifically: 1. Iterative recovery of symbolic soft mapping operators based on kernel functions ( ).

[0040] Using the current spectral domain mapping matrix The alignment information is transformed from the abstract spectral domain to the spatial geometric domain. During the initial iteration, the current spectral mapping matrix... The initial functional mapping matrix By calculating the eigenvector matrices of the source shape and the target shape respectively... The similarity under transformation is calculated, and a kernel-based symbolic soft mapping operator is constructed using the Softmax function to describe the soft correspondence between points (which can be vertices on a triangular mesh) and other points. : ; in, The parameter for adjusting the mapping sharpness can be a preset value and can be changed according to the actual situation. This embodiment does not impose specific limitations on this. These are the first and second eigenvector matrices, respectively, with T denoteing the transpose. This kernel-based symbolic soft mapping operator acts as a geometric bridge connecting the spectral and spatial domains, capturing pointwise probability alignment information and remaining continuously differentiable throughout the process, ensuring smooth gradient propagation.

[0041] 2. Iterative update of spectral domain projection ( ).

[0042] The kernel-based symbolic soft mapping operator obtained in the above steps is reprojected back into the linear spectral space to obtain the intermediate state spectral mapping estimate for the current iteration step. : ; In this step, the matrix multiplication is reorganized using symbolic computation logic, that is, by directly calculating the symbolic soft mapping operator based on the kernel function. Acting on the eigenvector matrix The projection is completed using the results. This mechanism does not require explicit instantiation. A dense matrix of this size keeps the memory complexity at a certain level. This level enables end-to-end training of high-resolution mesh models. This represents the pseudo-inverse operation that takes area weight into account.

[0043] 3. Spectral domain iterative coupling based on filter modulation ( ).

[0044] Unlike traditional pure geometric refinement, this embodiment introduces adaptive filtering intervention after projection reconstruction. The intermediate state spectrum obtained from the projection is then mapped and estimated. The signal is fed into a multi-channel filtering module, where the learned filter is used to perform frequency modulation, thereby suppressing spectral noise introduced by spatial projection. ; This step, by refining the spectral mapping matrix, forces the preservation of the geometric smoothness and bijectivity of the mapping. The refined coupling mapping matrix... This will serve as the input benchmark for the next iteration. (That is, the current spectral domain mapping matrix for the next iteration). Through the above iterative process, this embodiment finally obtains a coupled mapping matrix that integrates geometric consistency and spectral domain smoothness. (i.e., the target coupling mapping matrix) This indicates the result of the first module. This indicates the result of the second module.

[0045] Step S107: Convert the target coupling mapping matrix into a point-by-point mapping matrix to achieve point-by-point correspondence between the source shape and the target shape.

[0046] In this embodiment, the first feature vector matrix is ​​rotated and aligned using the target coupling mapping matrix to obtain an alignment result. Based on the alignment result, a nearest neighbor search is performed in the spectral space to obtain a pointwise mapping matrix, which contains semantic indices from the vertices of the source shape to the vertices of the target shape. The pointwise mapping matrix is ​​then used to achieve a pointwise correspondence between the source shape and the target shape.

[0047] This embodiment establishes a point-by-point mapping matrix using a high-precision target coupling mapping matrix, which improves the accuracy of corresponding to depth-independent non-rigid 3D shapes. Specifically: The results obtained using this embodiment The spectral basis (i.e., the first eigenvector matrix) of the source shape is rotated and aligned, and then a nearest neighbor search is performed in the spectral embedding space to obtain an accurate semantic index (i.e., a pointwise mapping matrix) from the vertices of the source shape to the vertices of the target shape. . use The point-by-point mapping matrix calculated from the established high-precision correspondence This allows the vertex displacement vectors or bone skinning weights on the source shape to be directly passed to the target shape, thereby achieving depth-non-rigid 3D shape correspondence.

[0048] To facilitate understanding by those skilled in the art, a set of preferred embodiments is provided below: This embodiment addresses the disconnect between the filtering, learning, and alignment processes in existing non-rigid 3D shape mapping techniques by designing a fully differentiable adaptive matching method. The core idea of ​​this method is to transform the previously non-differentiable geometric refinement process into a mathematical module capable of transmitting gradients, and to utilize a neural network with strong fitting capabilities to dynamically adjust the spectral signal. Specifically, this embodiment first uses a Kolmogorov-Arnold Network (KAN) neural network architecture to learn the nonlinear response of the spectrum, replacing the traditional polynomial filtering scheme, thereby capturing deformation features more accurately. Simultaneously, a low-memory-consumption and fully differentiable coupling layer is constructed using symbolic computation techniques. This mechanism allows errors generated by spatial domain alignment to be directly fed back to the front-end filter, enabling the filter to automatically adjust its parameters based on the alignment effect, eliminating the gap between feature learning and geometric refinement. (Refer to...) Figure 2 The method for mapping depth-nonrigid three-dimensional shapes using fusion differential iteration in this embodiment specifically includes the following: 1. Introduction to relevant technologies.

[0049] (1) Definition and mathematical expression of functional mapping.

[0050] 3D shape mapping aims to establish point-to-point semantic mapping relationships between different geometric entities, which is the foundation for tasks such as motion capture transfer and anatomical structure matching. Given two non-rigid shapes (including the original shape and the target shape), denoted as manifolds respectively. and The corresponding vertex sizes are respectively and The space of square-integrable functions defined on the surface of a manifold is denoted as follows: and .make This represents a pointwise mapping between manifolds, from which linear functional mapping operators can be induced. Its satisfaction , This refers to the composition of functions. For numerical computation, the method in this embodiment selects the eigenfunctions of the Laplace-Beltrammian operator of the manifold as the truncated orthogonal basis, which are expressed as follows: and The number of cutoffs is In this basis space, the functional mapping operator is expressed in matrix form. The conversion relationship between coefficients is satisfied: ; Among them, parameters To define in the source manifold The scalar function on the surface is used to truncate the orthogonal basis (i.e., the eigenvector matrix). The spectral coefficient vector below represents the component weights of the function in the source shape spectral space; parameters To define on the target manifold The corresponding scalar function of the surface is in the truncated orthogonal basis (i.e., the eigenvector matrix). The spectral coefficient vector is used to characterize the component weights of the function in the target shape spectral space.

[0051] In discretized representation, if the pointwise mapping is derived from the permutation matrix... The projection formula of the functional mapping matrix is ​​described as follows: : ; in, This represents a pseudo-inverse operation considering area weights. The core of this mechanism lies in utilizing the low-frequency stability of spectral basis functions to simplify dense point matching into a compact coefficient matrix optimization, thereby greatly improving the efficiency of handling non-rigid deformations.

[0052] (2) Functional mapping matrix The calculation.

[0053] Functional mapping matrix The geometric properties of the unknown mapping are fully encoded, and its solution is key to the function mapping framework. In existing deep learning paradigms, feature extractors (such as the FMNet network model) are used to obtain intrinsic feature descriptors of 3D shapes. and These descriptors are projected onto the spectral space to obtain the coefficient matrix. and ,in The source manifold coefficient matrix is ​​represented by the source manifold characteristic descriptor. In the corresponding truncated orthogonal basis (i.e., eigenvector matrix) The geometric features obtained by projection onto the upper part of the spectrum are used to characterize the source shape in spectral space. The target manifold coefficient matrix is ​​represented by the target manifold feature descriptor. In the corresponding truncated orthogonal basis Obtained by projection, it is used to characterize the geometric features of the target shape in spectral space. The matrix is ​​typically estimated by solving the following optimization problem: ; in, The functional mapping matrix estimated by the feature extractor is used as a transformation operator to describe the initial spectral alignment relationship between the two shapes. This indicates adjusting the weighting parameters. Let Frobenius norm be the square of a matrix, defined as the sum of the squares of the absolute values ​​of all its elements. and manifolds eigenvalue diagonal matrices and manifolds The eigenvalue diagonal matrix, whose diagonal elements reflect the intrinsic geometric spectrum distribution of the surface shape. However, research has found that existing deep learning frameworks have serious limitations in spectral filtering mechanisms, failing to dynamically suppress noise bands according to task requirements, resulting in a bottleneck in mapping accuracy when dealing with non-equidistant deformations.

[0054] (3) The problem of point-to-point restoration and refinement.

[0055] Original point-to-point correspondence It can be recovered from matrix C. Regarding the shape... Points on Its corresponding point It can be obtained by searching for nearest neighbors in the spectral embedding space: ; in, Represents the shape of the source To the target shape The point-to-point mapping relationship, Represents the shape of the target At the apex The eigenvector matrix at that location, Represents the shape of the source At the apex The eigenvector matrix at that location, express Norm.

[0056] To achieve high-precision matching, refinement algorithms such as ZoomOut are typically used. However, because these refinement algorithms are mathematically discrete and non-differentiable, they result in severe gradient barriers during the training phase. This prevents the feature learning module from perceiving the geometric constraints during the refinement process, leading to significant optimization gaps.

[0057] 2. The method of this embodiment.

[0058] This embodiment constructs a closed-loop optimization pipeline that integrates high-order adaptive filtering and differential refinement, achieving a deep fusion of adaptive spectral filtering and axiomatic iterative refinement through a fully differentiable design. This method is primarily applied to processing non-rigid deformation 3D geometric mesh data in computer graphics, computer vision, and medical image analysis. Specifically, the technical solution involved in this embodiment can be optimized for the following practical application scenarios: (1) Digital twin and character animation: used to handle motion capture data transfer between different human body scanning models (such as FAUST, SCAPE dataset, etc.). By establishing point-to-point semantic mapping, the automatic redirection of actions from the source model to the target model is realized.

[0059] (2) Medical image analysis and anatomical structure matching: For the three-dimensional scan image data of organs or bones of different patients, a high-precision anatomical structure correspondence is established to assist doctors in lesion comparison, organ segmentation and personalized surgical plan planning.

[0060] (3) Augmented reality and 3D reconstruction: Real-time registration and tracking of non-isotropic deformable objects (such as human bodies or various animal models wearing different clothes) in dynamic scenes to ensure accurate alignment between virtual information and real geometric entities.

[0061] 2.1 Mechanism for constructing high-order adaptive spectral filters based on KAN network architecture.

[0062] The performance of a functional mapping framework is highly dependent on its ability to modulate spectral signals. Current adaptive filtering methods mostly employ predefined polynomial basis functions (such as Jacobian polynomials) for linear fitting, with the general form being: , Indicates the weighting coefficient. Indicates the number of basis functions. The basis functions are represented by these. However, such linear activation combinations have limited upper limits of expressive power when faced with highly nonlinear frequency responses caused by extreme non-isometric deformations, making it difficult to accurately capture intrinsic geometric differences.

[0063] Therefore, this embodiment aims to explore the use of a neural network architecture with strong fitting properties to construct a parameterized spectral filter. Specifically, this embodiment introduces a KAN network architecture. Unlike traditional neural networks that use fixed activation functions on nodes, this embodiment utilizes KAN's ability to learn nonlinear spline functions on computational edges to achieve adaptive fitting of the Laplace-Belthemi operator eigenvalues. The mathematical expression of its filtering function is as follows: ; in, Represents the filter function. It is an additive index. This represents the dimension of the input feature components, i.e., the number of Laplace-Beltrami operator eigenvalues ​​involved in the spectral modulation calculation. This represents a learnable spline function defined on the computational edges of the output layer of a KAN network. It is used to perform high-order nonlinear integration of low-level features, thereby characterizing complex spectral response patterns. Indicates the input number of the first... The Laplace-Beltrami operator eigenvalues ​​are used as the basic frequency signals for the filter input, and are adaptively fitted through each computational edge of the network. For learnable spline functions defined on computational edges (i.e. Index The spline functions corresponding to the Laplace-Beltrami operators are used to characterize complex spectral response patterns.

[0064] After obtaining the modulation coefficients, the modulation process is implemented using a multi-channel filtering module (i.e., a high-order adaptive spectral filter based on the KAN architecture) to preserve the operator, thereby enhancing the model's ability to perceive salient features. ; in, This represents the filtered functional mapping matrix. This represents the filter function for the s-th filter channel. This represents the number of filter channels. This method estimates the intermediate state spectrum mapping. Through the high-order fitting capability of the KAN network, it establishes a spectral modulation mechanism with intrinsic geometry awareness, providing a high-performance representation foundation for subsequent closed-loop optimization.

[0065] 2.2 Differentiable iteration based on differential symbolic operations.

[0066] To address the isolated feature learning and geometric refinement processes in existing 3D shape correspondence frameworks, and the optimization gradient discontinuity problem caused by the non-differentiability of traditional projection operators, this embodiment constructs a fully differentiable differential spectrum coupling module. The core of this module lies in transforming the originally discrete post-processing refinement logic into a continuously differentiable process within the neural network through differentiable iterative loops, ensuring that geometric alignment errors can be accurately backpropagated to the parameter space of the front-end filter via the refinement trajectory.

[0067] (1) Discrete setting and feature pre-calculation.

[0068] Under discrete settings, the input three-dimensional shape (i.e., the source shape, also the original graphic) and (That is, the target shape, which is also the target graphic) are respectively represented as having and A triangular mesh with 1 vertex. The method in this embodiment first calculates its discrete Laplace-Beltramm operator to obtain the first Laplace matrix. Second Laplace matrix , and The following formula is used to calculate: ; in, The diagonal matrix is ​​formed by using the local area weights of each vertex of the corresponding shape (which can be either the source shape or the target shape) as its diagonal elements. This is the cotangent weight matrix corresponding to the shape. Generalized eigenvalue decomposition is performed on the first and second Laplacian matrices respectively to extract the frontier weights. The first eigenvector matrix is ​​constructed from the eigenvalues. Second eigenvector matrix and the corresponding first diagonal eigenvalue matrix Second diagonal eigenvalue matrix .

[0069] (2) High-efficiency symbolic operation logic for video memory.

[0070] To solve the problem of instantiating the probability correspondence matrix in a high-resolution grid environment Caused To address the memory explosion problem, this embodiment proposes to employ the Diffzo technique (i.e., a differentiable zoomout module). This Diffzo technique is well-known to those skilled in the art, and will not be described in detail here. When performing conversions between the spatial and spectral domains, the method in this embodiment does not store complex dense matrices. Instead, it stores symbolic logic (i.e., symbolic computational logic rules used to perform matrix-vector multiplication), and uses this symbolic logic to perform matrix-vector multiplication in real time within the computation graph. This mechanism maintains memory complexity at [value missing]. This level enables end-to-end iterative training of large-scale mesh models on ordinary hardware devices.

[0071] (3) Differentiable iterative refinement process.

[0072] This embodiment constructs a fully differentiable iterative refinement loop, aiming to solve the memory overflow problem faced by traditional methods when processing high-precision meshes and improve matching accuracy under non-equidistant deformation. This loop abandons the storage-intensive discrete permutation matrix P and instead employs an implicit computational logic based on a kernel-function-based symbolic soft mapping operator T. Process Execution Each iteration of the loop, in each iteration... Each contains the following three strictly ordered and tightly coupled differentiable components: 1) Iterative recovery of symbolic soft mapping operators based on kernel functions ( ).

[0073] Using the current spectral domain mapping matrix The alignment information is transformed from the abstract spectral domain to the spatial geometric domain. During the initial iteration, the current spectral mapping matrix... The initial functional mapping matrix By calculating the eigenvector matrices of the source shape and the target shape respectively... The similarity under transformation is calculated, and a kernel-based symbolic soft mapping operator is constructed using the Softmax function to describe the soft correspondence between points (which can be vertices on a triangular mesh) and other points. : ; in, The parameter for adjusting the mapping sharpness can be a preset value and can be changed according to the actual situation. This embodiment does not impose specific limitations on this. These are the first and second eigenvector matrices, respectively, with T denoteing the transpose. This kernel-based symbolic soft mapping operator acts as a geometric bridge connecting the spectral and spatial domains, capturing pointwise probability alignment information and remaining continuously differentiable throughout the process, ensuring smooth gradient propagation.

[0074] 2) Iterative update of spectral domain projection ( ).

[0075] The kernel-based symbolic soft mapping operator obtained in the above steps is reprojected back into the linear spectral space to obtain the intermediate state spectral mapping estimate for the current iteration step. : ; In this step, the matrix multiplication is reorganized using symbolic computation logic, that is, by directly calculating the symbolic soft mapping operator based on the kernel function. Acting on the eigenvector matrix The projection is completed using the results. This mechanism does not require explicit instantiation. A dense matrix of this size keeps the memory complexity at a certain level. This level enables end-to-end training of high-resolution mesh models. This represents the pseudo-inverse operation that takes area weight into account.

[0076] 3) Spectral domain iterative coupling based on filter modulation ( ).

[0077] Unlike traditional pure geometric refinement, this embodiment introduces adaptive filtering intervention after projection reconstruction. The intermediate state spectrum obtained from the projection is then mapped and estimated. The signal is fed into a multi-channel filtering module, where the learned filter is used to perform frequency modulation, thereby suppressing spectral noise introduced by spatial projection. ; This step, by refining the spectral mapping matrix, forces the preservation of the geometric smoothness and bijectivity of the mapping. The refined coupling mapping matrix... This will serve as the input benchmark for the next iteration. (That is, the current spectral domain mapping matrix for the next iteration). Through the above iterative process, this embodiment finally obtains a coupled mapping matrix that integrates geometric consistency and spectral domain smoothness. (i.e., the target coupling mapping matrix) This indicates the result of the first module. This indicates the result of the second module.

[0078] Before proceeding with any application scenarios, the coupling mapping matrix in the spectral domain needs to be... Pointwise mapping matrix converted to the spatial domain The implementation mechanism is as follows: utilizing the output of this embodiment... The spectral basis (i.e., the first eigenvector matrix) of the source shape is rotated and aligned, and then a nearest neighbor search is performed in the spectral embedding space to obtain an accurate semantic index (i.e., a pointwise mapping matrix) from the vertices of the source shape to the vertices of the target shape. .

[0079] Technical advantages: due to After frequency modulation by the KAN filter and closed-loop refinement with differentiable iterative refinement, the recovered point-to-point model exhibits extremely high robustness when dealing with non-equidistant deformations (such as large-amplitude human body movements, clothing obstruction, etc.).

[0080] Typical application scenarios and problem-solving methods of this embodiment include: Digital twins and virtual character animation (motion transfer) are used in film and television production or metaverse scenarios to capture high-quality motion (source shape) of live actors. Real-time migration to a virtual character (target shape) with completely different proportions and body size. In medical imaging-assisted diagnosis (anatomical matching), doctors need to compare organ or bone scan data (such as CT / MRI reconstructed models) from different patients to identify lesions or make personalized surgical plans. The point-by-point mapping matrix calculated from the established high-precision correspondence It can directly transfer vertex displacement vectors or bone skinning weights on the source shape to the target shape, thereby achieving depth-non-rigid 3D shape correspondence.

[0081] 2.3 Algorithm Flow and Parameter Settings.

[0082] The computational process for fusing adaptive filtering and differential coupling proposed in this embodiment is summarized as follows: Pre-calculation process: Given an input triangular mesh and The number of vertices are respectively and .

[0083] Step 1: Calculate the discrete Laplacian matrix of the triangular mesh. and .

[0084] Step 2: Apply the Laplace matrix and Perform generalized eigenvalue decomposition, let the matrix and Each contains the previous The eigenvector matrix of each feature; the diagonal matrix and Includes the corresponding feature values. General settings. .

[0085] Step 3: Extract the source shape using a feature extraction network. With target shape Their respective original geometric feature vectors and Then, based on the original geometric feature vector and Calculate the initial functional mapping matrix The feature extraction network can be FMNet or a feature extraction network known to those skilled in the art; this embodiment does not impose any specific limitations on it.

[0086] Step 4: Enter the differential spectrum coupling iteration loop and set the maximum number of iterations. In each iteration In the middle, perform the following closed-loop refinement operations: Transmission operator recovery ( Based on the current spectral mapping matrix During the initial iteration, the current spectral mapping matrix for Using the Softmax operator to compute kernel-based symbolic soft mapping operators in the spatial domain This enables the probabilistic recovery of spectral domain alignment information to the spatial domain.

[0087] Implicit spectral domain projection ( ): Utilizing symbolic computation logic, the symbolic soft mapping operator based on kernel functions is... Directly project back into the spectral space. This process avoids errors due to the associative law of matrix multiplication. Explicit instantiation of the scaling matrix, outputting an intermediate state spectral mapping estimate. .

[0088] Spectral domain coupling modulation ( ): Will The data is fed into a multi-channel filtering module, where a learnable frequency response function is used to denoise and smooth the mapping matrix, outputting the refined coupled mapping generated in this iteration. And use it as the input benchmark for the next iteration. .

[0089] The iterative optimization calculation process includes: Input: Source shape With target shape (The number of vertices are respectively) and ); spectral base cutoff number (Default 100); Maximum number of iterations (Default 5); Feature extraction network Multi-channel filter .

[0090] Output: Target coupling mapping matrix Point-by-point correspondence (i.e., point-by-point mapping matrix) .

[0091] Phase 1: Pre-calculation and initialization. The eigenvector matrix and diagonal eigenvalue matrix of the source shape are calculated respectively. ; Calculate the eigenvector matrix and diagonal eigenvalue matrix of the target shape respectively. The original geometric feature vector is extracted using a feature extraction network. The initial functional mapping matrix can be calculated from the geometric eigenvectors (including the first and second original geometric eigenvectors). Solver represents the process of calculating the initial functional mapping matrix.

[0092] Phase Two: Differential Spectrum Coupling Iterative Loop, including: (1) Symbolic soft mapping operator recovery based on kernel function ( The implicit computational feature similarity matrix is ​​calculated as follows: ,in And generate a symbolic soft mapping operator based on kernel functions. (Not explicitly stored).

[0093] (2) Implicit spectral domain projection ( ), using the associative law to project back to the spectral domain as .

[0094] (3) Spectral domain coupling modulation (filtering), using a KAN filter for noise reduction. , This is used to update the status for the next round.

[0095] Phase 3: Post-processing includes: The target coupling mapping matrix is ​​converted into the final point-to-point index. Final output and .

[0096] Compared with the prior art, the method of this embodiment has the following advantages: (1) The filter architecture is more compact and has excellent convergence performance.

[0097] Unlike traditional methods that rely on complex high-dimensional descriptor alignment or cumbersome additional regularization constraints, this embodiment utilizes a KAN network architecture with strong nonlinear fitting capabilities to construct the spectral filter. This design fits the spectral response using spline functions, making the frequency modulation process simpler, more efficient, and more expressive. Thanks to this compact parameterized representation, this embodiment exhibits excellent convergence characteristics, requiring only 2 to 3 iterations for the model to stabilize in actual training. Experimental observations show that the average measurement error decreases rapidly with increasing iteration count, reaching an extremely low level.

[0098] (2) The matching quality and computational efficiency are significantly superior.

[0099] This embodiment of the method significantly optimizes computational overhead while ensuring matching accuracy. By introducing a differential spectrum coupling module based on symbolic computation, the memory complexity of spatial domain to spectral domain conversion is successfully reduced from the traditional... Reduce to Level. Test results on standard datasets such as FAUST and SCAPE show that the cumulative corresponding measurement error curve of the method in this embodiment is always above that of existing similar methods. See Table 1 for reference. The representative function mapping methods such as ULRSSM, DiffZO, DeepFAFM and MDNM in Table 1 are existing technical methods known to those skilled in the art, and will not be specifically described in this embodiment.

[0100] Table 1. Performance comparison between the method in this embodiment and the representative function mapping method.

[0101] (3) It has strong geometric robustness and error feedback mechanism.

[0102] This embodiment employs a fully differentiable design, enabling the method to automatically identify and suppress topological noise through an iterative process when handling challenging standard or non-equidistant datasets. Because spatial alignment errors are accurately fed back to the front-end feature extraction and filtering modules, this embodiment's method can automatically optimize for local geometric deficiencies, exhibiting higher matching accuracy and robustness than existing static methods.

[0103] Reference Figure 3 This application also provides a depth-non-rigid three-dimensional shape mapping system, which includes: The triangular mesh representation unit 301 is used to represent the source shape and target shape in the target scene as triangular meshes with multiple vertices, thereby obtaining the triangular meshes of the source shape and the target shape. The source shape and the target shape are non-rigid three-dimensional shapes. The feature vector extraction unit 302 is used to extract feature vectors from the source shape and the target shape to obtain the first original geometric feature vector of the source shape and the second original geometric feature vector of the target shape. The first data calculation unit 303 is used to calculate the initial functional mapping matrix based on the first original geometric eigenvector and the second original geometric eigenvector, and use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix. The second data calculation unit 304 is used to calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape, so as to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape. Eigenvalue decomposition unit 305 is used to perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and to perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix. Differentiable iterative unit 306 is used to construct a kernel-based symbolic soft mapping operator to describe the correspondence between vertices based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix during the differentiable iteration process. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate spectral mapping estimate. The intermediate spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix. The shape point-to-point correspondence unit 307 is used to convert the target coupling mapping matrix into a point-to-point mapping matrix, and realize the point-to-point correspondence between the source shape and the target shape through the point-to-point mapping matrix.

[0104] It should be noted that since the depth non-rigid three-dimensional shape correspondence system in this embodiment is based on the same inventive concept as the depth non-rigid three-dimensional shape correspondence method described above, the corresponding content in the method embodiment is also applicable to this system embodiment, and will not be described in detail here.

[0105] Reference Figure 4 This application also provides an electronic device, which includes: At least one memory; At least one processor; At least one program; The program is stored in memory, and the processor executes at least one program to implement the depth non-rigid three-dimensional shape correspondence method described above in this disclosure.

[0106] This electronic device can be any smart terminal, including mobile phones, tablets, personal digital assistants (PDAs), and in-vehicle computers.

[0107] The electronic devices according to embodiments of this application will now be described in detail.

[0108] The processor 1600 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this disclosure. The memory 1700 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 1700 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1700 and is called and executed by the processor 1600 to execute the depth-non-rigid three-dimensional shape correspondence method of the embodiments of this disclosure.

[0109] The input / output interface 1800 is used to implement information input and output. The communication interface 1900 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 2000 transmits information between various components of the device (e.g., processor 1600, memory 1700, input / output interface 1800, and communication interface 1900); The processor 1600, memory 1700, input / output interface 1800 and communication interface 1900 are connected to each other within the device via bus 2000.

[0110] This disclosure also provides a storage medium, which is a computer-readable storage medium storing computer-executable instructions for causing a computer to perform the above-described depth-non-rigid three-dimensional shape correspondence method.

[0111] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The embodiments described in this disclosure are for the purpose of more clearly illustrating the technical solutions of this disclosure and do not constitute a limitation on the technical solutions provided by this disclosure. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by this disclosure are also applicable to similar technical problems.

[0113] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this disclosure, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0114] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0115] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0116] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0117] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0119] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0120] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0121] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The embodiments of this application have been described in detail above with reference to the accompanying drawings, but this application is not limited to the above embodiments. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of this application.

[0122] The embodiments of this application have been described in detail above with reference to the accompanying drawings. However, this application is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of this application.

Claims

1. A method for mapping depth-nonrigid three-dimensional shapes, characterized in that, The method includes: The source shape and target shape in the target scene are represented as triangular meshes with multiple vertices, thus obtaining the triangular meshes of the source shape and the target shape. The source shape and the target shape are non-rigid three-dimensional shapes. Extract the feature vectors from the source shape and the target shape to obtain the first original geometric feature vector of the source shape and the second original geometric feature vector of the target shape; Based on the first original geometric eigenvector and the second original geometric eigenvector, calculate the initial functional mapping matrix, and use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix; Calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape; Perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix; During the differentiable iteration process, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, a kernel-based symbolic soft mapping operator is constructed to describe the correspondence between vertices. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate state spectral mapping estimate. The intermediate state spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix. The target coupling mapping matrix is ​​converted into a point-by-point mapping matrix, and the point-by-point mapping matrix is ​​used to realize the point-by-point correspondence between the source shape and the target shape.

2. The depth-non-rigid three-dimensional shape correspondence method according to claim 1, characterized in that, The step of constructing a kernel-based symbolic soft mapping operator to describe the correspondence between vertices, based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix, includes: Calculate the similarity between the first eigenvector matrix and the second eigenvector matrix under the current spectral domain mapping matrix; The similarity is calculated as a ratio to the parameter used to adjust the mapping sharpness, and the ratio calculation result is obtained. The ratio calculation result is processed by the Softmax function to construct a kernel-based symbolic soft mapping operator for describing the correspondence between vertices.

3. The method for corresponding non-rigid three-dimensional shapes at depth according to claim 1, characterized in that, The step of reprojecting the kernel-function-based symbolic soft mapping operator back into the spectral space to obtain the intermediate state spectral mapping estimate includes: The symbolic soft mapping operator based on the kernel function is multiplied by the first eigenvector matrix using the associative law to obtain the multiplication result; Perform a pseudo-inverse operation on the second eigenvector matrix to obtain the pseudo-inverse operation result; Multiplying the result of the multiplication with the result of the pseudo-inverse operation yields the intermediate state spectral mapping estimate.

4. The depth-non-rigid three-dimensional shape correspondence method according to claim 1, characterized in that, The process of performing multi-channel filtering on the intermediate state spectral mapping estimate to obtain the refined coupling mapping matrix includes: A filtering function is constructed using a KAN grid, and a multi-channel filtering module is constructed based on the filtering function. The intermediate state spectrum mapping estimate is processed by the multi-channel filtering module to obtain a refined coupling mapping matrix.

5. The depth-non-rigid three-dimensional shape correspondence method according to claim 4, characterized in that, The process of constructing the filtering function using a KAN grid includes: ； in, Represents the filter function. Indicates an incremental index. This represents the number of eigenvalues ​​of the Laplace-Beltrami operator. express spline functions under index, express Index Spline functions corresponding to the Laplace-Beltrami operator. Indicates the first Eigenvalues ​​of the Laplace-Beltrami operator.

6. The depth-non-rigid three-dimensional shape correspondence method according to claim 4, characterized in that, The step of using the multi-channel filtering module to perform multi-channel filtering on the intermediate state spectral mapping estimate to obtain a refined coupling mapping matrix includes: Perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first diagonal eigenvalue matrix, and perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second diagonal eigenvalue matrix; The first diagonal eigenvalue matrix is ​​input into the multi-channel filtering module to obtain the first module result; The second diagonal eigenvalue matrix is ​​input into the multi-channel filtering module to obtain the second module result; Multiply the results of the first module, the results of the second module, and the intermediate state spectrum mapping estimate together to obtain the refined coupling mapping matrix.

7. The method for corresponding depth-non-rigid three-dimensional shapes according to claim 1, characterized in that, The step of converting the target coupling mapping matrix into a pointwise mapping matrix includes: The first feature vector matrix is ​​rotated and aligned using the target coupling mapping matrix to obtain the alignment result; Based on the alignment result, a nearest neighbor search is performed in the spectral space to obtain a pointwise mapping matrix, which contains semantic indices from the vertices of the source shape to the vertices of the target shape.

8. A depth-non-rigid three-dimensional shape correspondence system, characterized in that, The system includes: A triangular mesh representation unit is used to represent the source shape and target shape in the target scene as a triangular mesh with multiple vertices, thereby obtaining the triangular mesh of the source shape and the triangular mesh of the target shape, wherein the source shape and the target shape are non-rigid three-dimensional shapes; A feature vector extraction unit is used to extract feature vectors from the source shape and the target shape to obtain a first original geometric feature vector of the source shape and a second original geometric feature vector of the target shape. The first data calculation unit is used to calculate an initial functional mapping matrix based on the first original geometric feature vector and the second original geometric feature vector, and to use the initial functional mapping matrix as the initial differentiable iterable current spectral domain mapping matrix. The second data calculation unit is used to calculate the Laplacian matrix of the triangular mesh of the source shape and the triangular mesh of the target shape, respectively, to obtain the first Laplacian matrix of the source shape and the second Laplacian matrix of the target shape. The eigenvalue decomposition unit is used to perform generalized eigenvalue decomposition on the first Laplacian matrix to determine the first eigenvector matrix, and to perform generalized eigenvalue decomposition on the second Laplacian matrix to determine the second eigenvector matrix. A differentiable iterative unit is used to construct a kernel-based symbolic soft mapping operator to describe the correspondence between vertices based on the current spectral domain mapping matrix, the first eigenvector matrix, and the second eigenvector matrix during the differentiable iteration process. The kernel-based symbolic soft mapping operator is reprojected back into the spectral space to obtain an intermediate state spectral mapping estimate. The intermediate state spectral mapping estimate is then subjected to multi-channel filtering to obtain a refined coupling mapping matrix. The refined coupling mapping matrix is ​​used as the current spectral domain mapping matrix for the next iteration until a preset number of iterations is reached to obtain the target coupling mapping matrix. A shape point-to-point correspondence unit is used to convert the target coupling mapping matrix into a point-to-point mapping matrix, and realize the point-to-point correspondence between the source shape and the target shape through the point-to-point mapping matrix.

9. An electronic device, characterized in that, It includes at least one control processor and a memory for communicatively connecting to the at least one control processor; the memory stores instructions executable by the at least one control processor, which, when executed by the at least one control processor, enable the at least one control processor to perform the depth non-rigid three-dimensional shape correspondence method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions for causing a computer to perform the depth-nonrigid three-dimensional shape correspondence method as described in any one of claims 1 to 7.

Images