Non-rigid Incomplete Shape Correspondence Method Based on Maximal Stable Regions

Abstract

The present invention discloses a non-rigid incomplete shape correspondence method and application based on the maximally stable regions, which relates to the field of three-dimensional shape analysis. The method includes: S1: Extracting two non-rigid shape models from a non-rigid three-dimensional model library; S2: Based on diffusion geometry, respectively performing feature detection on the two non-rigid shape models in S1 to discover feature points or feature regions on the shapes, and then describing each feature to assign it a vector describing local information; S3: Pre-defining a component tree and setting an edge descriptor, and then calculating the maximally stable regions based on the edge descriptor, and defining the calculation results as two sets X and Y; S4: Using a prediction function to perform region correspondence, landmark point correspondence, and dense correspondence on each region in turn. The technical solution of the present invention can solve the problem of inaccurate correspondence between an overall shape with non-rigid changes and an incomplete shape.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional shape analysis, and particularly to a non-rigid incomplete shape correspondence method based on the maximally stable extremal regions. Background Art

[0002] The basic premise for the large-scale application and development of three-dimensional models is to perform shape analysis and processing on them with the aid of a computer. Shape correspondence mainly studies finding meaningful relationships or mappings between shape elements from a given set of shapes. A typical problem in the field of shape correspondence is the calculation of the correspondence relationship between three-dimensional shapes with non-rigid changes. In this field, the more challenging problem is non-rigid incomplete shape correspondence. Due to the characteristics of non-rigid incomplete shapes themselves, differences in acquisition methods, and the influence of noise, they not only have non-rigid deformations but also include other changes such as topological changes and scaling transformations. These changes increase the difficulty of analyzing non-rigid incomplete shapes and also pose higher requirements for the accuracy of non-rigid incomplete shape correspondence algorithms. Previous methods for analyzing non-rigid global shapes mostly relied on Euclidean distance and geodesic distance metrics.

[0003] Based on theories such as diffusion distance, heat kernel signature descriptor, and heat kernel mapping in diffusion geometry, the present invention conducts research on fundamental problems in three-dimensional non-rigid incomplete shape correspondence to solve the correspondence problem between global shapes with non-rigid changes and incomplete shapes, and applies the research results to the design of an online virtual fitting system. Summary of the Invention

[0004] The present invention aims to provide a non-rigid incomplete shape correspondence method based on the maximally stable extremal regions to solve the problem of inaccurate correspondence between global shapes with non-rigid changes and incomplete shapes.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A non-rigid incomplete shape correspondence method based on the maximally stable extremal regions, the method comprising:

[0007] S1: Extract two non-rigid shape models from a non-rigid three-dimensional model library, one of which has a global shape and the other has only a partial shape;

[0008] S2: Based on diffusion geometry, perform feature detection on the two non-rigid shape models in S1 respectively to discover feature points or feature regions on the shapes, and then describe each feature, assign it a vector describing local information, and name it a feature descriptor;

[0009] S3: Pre - define the component tree and set the edge descriptor. Then, calculate the maximally stable regions based on the edge descriptor and define the calculation results as two sets X and Y.

[0010] S4: Use the prediction function to perform region correspondence, landmark point correspondence, and dense correspondence for each region in turn.

[0011] Preferably, the steps for setting the edge descriptor in S3 are as follows:

[0012] S3 - 1: Represent the smooth manifolds of the two model surfaces with a discretized undirected graph X=(V, E).

[0013] S3 - 2: Define weights on the edge set of the undirected graph to form an edge - weighted graph (X, f).

[0014] S3 - 3: Set the weight function f as the edge descriptor.

[0015] Preferably, the edge descriptor represents the heat kernel distance or diffusion distance between two endpoints on the edge.

[0016] Preferably, the prerequisite for region correspondence in S4 is that the local areas of X and Y are similar, so as to transform the region correspondence problem into a problem of solving the minimized energy function.

[0017] Preferably, in S4, an error function can be used to perform landmark point correspondence for each region to form sparse correspondence, and then use the correspondence relationship of the sparse correspondence to spread the correspondence to other points to obtain a dense correspondence relationship.

[0018] The principle and beneficial effects of this technical solution:

[0019] (1) The non - rigid incomplete shape correspondence algorithm based on the theory of diffusion geometry in the present invention is used to solve the correspondence problem between three - dimensional incomplete shapes with non - rigid deformation and topological changes. It introduces diffusion mapping and diffusion distance as data parameterization and measurement methods, overcoming the deficiencies of Euclidean distance and geodesic distance in dealing with such problems.

[0020] (2) The present invention proposes a method for calculating the maximally stable regions of non - rigid incomplete shapes and a dense correspondence method based on the prediction function. On the basis of the diffusion geometry theory, with the help of the construction of the edge descriptor, the maximally stable regions on the non - rigid shape are calculated, realizing block - by - block and region - by - region shape sparse correspondence and reducing the corresponding search space. Using the landmark points in the maximally stable regions as source points, the correspondence relationship is spread to the entire shape through the prediction function.

[0021] (3) The present invention intends to establish an evaluation standard for non-rigid 3D model correspondence algorithms. With the help of the publicly available test database of non-rigid shape correspondence algorithms, a reasonable evaluation strategy for correspondence results is designed to evaluate the accuracy of existing correspondence algorithms. And the correspondence algorithms are applied to test in environments such as virtual fitting. Description of the Drawings

[0022] Figure 1 It is the overall technical flow chart provided by the embodiment of the present invention;

[0023] Figure 2 It is the complete and incomplete human body model diagrams provided by the embodiment of the present invention;

[0024] Figure 3 It is the comparison diagram of geodesic distance and diffusion distance on the cat shape provided by the embodiment of the present invention;

[0025] Figure 4 It is the example diagram of stable regions on non-rigid shapes provided by the embodiment of the present invention;

[0026] Figure 5 It is the analysis model of Laplace eigenfunctions provided by the embodiment of the present invention. Detailed Embodiment

[0027] The present invention will be further described in detail below in conjunction with the drawings and embodiments:

[0028] As Figure 1 shown, a non-rigid incomplete shape correspondence method based on the maximum stable region describes the features of a given complete shape model and partial shape model according to diffusion geometry, performs feature description and maximum stable region calculation, and then performs region, landmark point and dense correspondence on them to solve the problem of inaccurate correspondence between the overall shape with non-rigid changes and the incomplete shape. The method includes:

[0029] S1: As Figure 1 and Figure 2 shown, two non-rigid shape models are extracted from the non-rigid 3D model library, one of which has an overall shape and the other has only a partial shape.

[0030] S2: Based on diffusion geometry, feature detection is respectively performed on the two non-rigid shape models in S1 to find the feature points or feature regions on the shapes, and then each feature is described, a vector describing local information is assigned to it, and it is named a feature descriptor;

[0031] First, the heat diffusion process on non-rigid 3D shapes is studied as the theoretical basis. Assume that (M, g) is a manifold with boundary n-dimensional compact Riemannian manifold. Given the initial condition of the heat diffusion equation, that is, the heat distribution u(x, 0) = u0(x) on the manifold M at time t = 0, where x is the local coordinate on M. The process of heat diffusion is a process of smoothing this initial state and will eventually reach a stable state.

[0032] The physical meaning of the heat kernel is the value of the heat transferred from point p to point q within time t, that is, for all p ∈ M, it satisfies H t f(p) = ∫ M k t (p, q)f(q)dq. The diffusion distance between two points p and q on the manifold M can be defined as:

[0033]

[0034] The diffusion distance represents the connectivity rate between points on the shape. This metric is related to the amount of random motion connecting two points. If there is a large amount of random motion between two points at time t or before time t, then the diffusion distance between these two points is large. Using diffusion theory as the definition basis for the metric on non-rigid shapes is to improve the robustness of the algorithm to shape topological changes and non-rigid changes. Among them, its topological invariance can be seen from Figure 3 as shown in the figure, where (a) is the geodesic distance topological map on the cat shape, and (b) is the diffusion distance topological map. In Figure 3 (a), topological noise is added to the three-dimensional shape of the cat at the positions of its two hind legs, and the source points of both the geodesic distance and the diffusion distance are set at the left hind foot position of the cat. The isometric lines diffused from the source point cover the entire shape in sequence, but there are significant differences in the colors of the isometric lines of the two distances at the position of the right hind leg of the cat.

[0035] S3: As Figure 4 shown, pre-define the component tree and set the edge descriptor, and then calculate the maximum stable region based on the edge descriptor, and define the calculation result as two sets X and Y;

[0036] The setting steps of the edge descriptor in the above S3 are as follows:

[0037] S3-1: Represent the smooth manifolds on the surfaces of the two models with a discretized undirected graph X = (V, E);

[0038] A three-dimensional model represented by a triangular mesh surface can be regarded as a two-dimensional smooth Riemannian manifold M. Generally, this manifold has a boundary and has a standard metric dμ. In the process of discretization, the representation and solution of the Laplace - Beltrami operator on the manifold involve many key points in the representation of three-dimensional model analysis quantities, such as the calculation of the eigenfunction basis on the three-dimensional model, etc.

[0039] The discrete representation method of the manifold here can be regarded as a three-dimensional grid model composed of a node set V = {v1,..., v n} and an edge set E = {(i, j)} connecting these points, where 1 ≤ i, j ≤ n.

[0040] A function defined on this manifold can be represented by an n-dimensional vector f = (f(v1),..., f(v n ))). For the discretization of the above Laplacian operator, it can be calculated according to the classical cotangent formula shown below (as Figure 5 shown):

[0041]

[0042] A better way to determine the bases {φ i} and {ψ j} on two shapes is to select them according to their Laplace eigenfunctions. This method can generate a basis as an approximation of the original mapping after truncating the first k coefficients. In the actual calculation process, only the first k Laplace eigenfunction bases are intercepted to form the function spaces Φ n×k and Ψ n×k .

[0043] S3-2: Define weights on the edge set of the undirected graph to form an edge-weighted graph (X, f);

[0044] S3-3: Set the weight function f as an edge descriptor, and the edge descriptor represents the

[0045]

[0046] heat kernel distance or diffusion distance between the two endpoints on the edge:

[0047] And define the subgraph set induced by the edge set E λ = {e ∈ E: f(e) ≤ λ} with all weights less than λ ≥ 0 as a λ cross section.

[0048] For any component C l in X, since C l is a component in the λ cross section and l is taken as its upper weight value, so l ≤ λ. In this way, a component tree {(l, C l )} composed of a set of components is obtained, and its root node is (λ, X). The instability of the component C l is defined as that is, when the area A(C l ) changes more with l, its instability is greater. If the value of S(l) has a local minimum, then the component C lIt is a maximum stable component, corresponding to a maximum stable region on the non-rigid shape.

[0049] S4: Use the prediction function to perform region correspondence, landmark point correspondence, and dense correspondence on each region in turn.

[0050] The correspondence process of non-rigid incomplete shapes can be divided into three steps: region correspondence, landmark point correspondence, and dense correspondence. Given two non-rigid three-dimensional shapes X and Y, and the calculation results of the maximum stable regions Y j and X i The prerequisite for correspondence is that their local areas A are similar, and the entire region correspondence problem is transformed into a problem of solving the minimum energy function.

[0051] After determining the region correspondence results, it is necessary to determine the point set correspondence relationship within each region and establish a preliminary correspondence relationship with the help of the error function. To avoid an overly large search space when calculating the error function, the landmark points within the region are corresponded first. The definition form of the error function needs to reflect two parts of the relationship: the pairwise points between shapes and the relationship between points. Assume that the source region is S, the corresponding target region is T, and their correspondence relationship is represented by the mapping Φ: S → T, and its general form is:

[0052]

[0053] So far, the correspondence of the most stable regions between non-rigid incomplete shapes has been completed, and the correspondence relationship of the landmark points within the regions has also been determined. To obtain the complete correspondence relationship between non-rigid incomplete shapes and other shapes, it is necessary to spread the correspondence to other points with the help of the existing correspondence relationship of landmark points. Through the above processing process, the correspondence of landmark points can be spread to the entire shape to obtain the dense correspondence relationship between two non-rigid shapes.

[0054] The above are only embodiments of the present invention. Specific technical solutions and / or common knowledge such as characteristics well known in the art are not described in detail here. It should be noted that for those skilled in the art, without departing from the technical solution of the present invention, several deformations and improvements can be made, which should also be regarded as the protection scope of the present invention, and these will not affect the implementation effect of the present invention and the practicality of the patent. The protection scope required by this application should be based on the content of its claims, and the specific implementation manners described in the specification can be used to explain the content of the claims.

Claims

1. A non-rigid incomplete shape correspondence method based on the maximum stable region, characterized in that: The method includes: S1: Extract two non-rigid shape models from a non-rigid 3D model library, where one has an overall shape and the other has only a partial shape; S2: Based on diffusion geometry, assume that $(M, g)$ is an $n$-dimensional compact Riemannian manifold with boundary and $k$ t represents the value of heat diffusion from point $p$ to point $q$ on the manifold within time $t$. Its physical meaning is the amount of heat transfer between two points during the heat diffusion process. The theoretical definition of the diffusion distance between two points $p$ and $q$ on the manifold $M$ is: Perform feature detection on the two non-rigid shape models in S1 respectively to find feature points or feature regions on the shapes, then describe each feature, assign it a vector describing local information, and name it a feature descriptor; S3: Predetermine a component tree and set an edge descriptor, then calculate the maximally stable regions based on the edge descriptor, and define the calculation results as two sets X and Y, specifically: S3-1: Represent the smooth manifolds on the surfaces of the two models with a discretized undirected graph X = (V, E), where V is the node set and E is the edge set; S3-2: Define weights on the edge set of the undirected graph to form an edge-weighted graph (X, f), where f is the weight function; S3-3: Set the weight function f as the edge descriptor, and the edge descriptor represents the heat kernel distance or diffusion distance between two endpoints v1 and v2 on the edge, and its calculation form is: Define the set of subgraphs induced by the set of edges \(E'\) whose weights are all less than or equal to \(\lambda\geq0\) as a \(\lambda\)-cross-section, where \(E' = \{e\in E: f(e)\leq\lambda\}\). λ That is, \(E'=\{e\in E: f(e)\leq\lambda\}\), and the set of subgraphs induced by \(E'\) is defined as a \(\lambda\)-cross-section. For any component C in X l , since C l is a component in the λ cross-section, and taking l as its upper bound weight value, so l ≤ λ, thus obtaining a component tree {(l, C l )} composed of a set of components, whose root node is (λ, X), and the instability of component C l is defined as where A(·) is the area function, A is the area of the root node. If the value of S(l) has a local minimum, then the component C l at this time is a maximum stable component, corresponding to a maximum stable region on the non-rigid shape; S4: Use a prediction function to perform region correspondence, landmark point correspondence, and dense correspondence on each region in turn.

2. The non-rigid incomplete shape correspondence method based on the maximum stable region according to claim 1, characterized in that: The prerequisite for region correspondence in S4 is that the local areas of X and Y are similar, so that the region correspondence problem can be transformed into a problem of solving a minimized energy function.

3. The non-rigid incomplete shape correspondence method based on the maximum stable region according to claim 1, characterized in that: In S4, use the prediction function to perform landmark point correspondence on each region to form sparse correspondence, and then use the correspondence relationship of the sparse correspondence to spread the correspondence to other points to obtain a dense correspondence relationship.

Images