Movie and television game animation point cloud three-dimensional curved surface reconstruction hole repairing method

Abstract

According to the movie and television game animation point cloud three-dimensional curved surface reconstruction hole repairing method, an optimized Poisson curved surface reconstruction method is established, noise reduction processing is carried out on wrong point cloud data, abnormal points are removed, if data density is sparse, bicubic spline interpolation is adopted to improve the point cloud density, and the point cloud reconstruction efficiency is improved. Establishing a transfer low-square optimization method to accurately calculate point cloud normal vectors, adjusting the directions of the point cloud normal vectors to be consistent, improving the calculation precision of the normal vectors, and redirecting the normal vectors to ensure the consistency of the point cloud normal vectors; aiming at the fact that holes influencing a data model exist in a triangular mesh model, a triangular mesh optimization, adjustment and repair method based on a radial basis model is adopted, finally, good continuity can be kept between a repair result and a triangular mesh curved surface around the holes, good geometric consistency is kept between a repaired triangular mesh and a triangular mesh on the surface of an original model, and the repair accuracy is improved. The method is high in curved surface reconstruction speed, high in quality and suitable for being applied to multiple industries such as movie and television game cartoons.

Description

Technical Field

[0001] The present application relates to a method for repairing holes in point cloud surface reconstruction, and in particular to a method for repairing holes in point cloud three-dimensional surface reconstruction for movies, games and animations, belonging to the technical field of point cloud surface reconstruction and repair. Background Art

[0002] With the development of big data, computer vision, virtual reality, artificial intelligence, surveying and mapping and other disciplines, point cloud 3D model reconstruction technology has become a research hotspot. While point cloud 3D model reconstruction technology is developing, the demand for 3D models in fields such as ancient building protection, reverse engineering, military, game scenes, and film and television animation is also increasing.

[0003] 3D point cloud data can construct a 3D model with complete details, so as to reconstruct a realistic scene. The application of 3D laser scanning technology to measure the target object can obtain the true shape of the object, so that people can also have an immersive experience in the virtual reality scene. Various computer games use 3D point cloud modeling technology to allow people to experience a nearly realistic scene in the game world. In addition, the role of 3D cities in urban planning and design, urban traffic route planning, etc. is becoming more and more important. The 3D reconstruction of urban scenes focuses on basic geographical information such as landforms, vegetation, water systems, buildings, transportation, special features, etc. within the city. In terms of cultural relics protection, 3D laser scanning technology is used to scan cultural relics and construct 3D models for digital management. The 3D model of cultural relics can not only help to establish archives for archaeological sites and cultural relics, but also use the 3D model of cultural relics to repair and restore the damaged parts of cultural relics in the protection and repair of cultural relics.

[0004] 3D laser scanning technology is developing faster and faster. At present, people in various fields have higher and higher requirements for the comprehensiveness and detail integrity of the target 3D model. In order to obtain models that meet the quality requirements of various projects, how to effectively process the "massive" disordered point cloud data obtained, select the appropriate point cloud surface reconstruction method, and how to improve the retention of point cloud modeling details and comprehensive point cloud surface models have become the focus of research.

[0005] According to the different point cloud surface reconstruction methods, point cloud data surface reconstruction methods can be divided into: parametric surface fitting, hidden surface reconstruction, and polygonal mesh reconstruction based on geometric structure. (1) Parametric surface reconstruction. Since weight factors need to be introduced in NURBS surfaces, if the weight factors are set unreasonably, the parameterization effect of the surface will be poor. In severe cases, the surface structure may be destroyed. (2) Hidden surface reconstruction. The surface reconstruction method based on Poisson function and the surface reconstruction method based on radial basis model are both overall hidden surface reconstruction methods. Both methods consider the overall point cloud data at one time, and then use mathematical functions to process it to establish the object surface. The surfaces established by these two methods have the advantages of high smoothness and good continuity, but have the disadvantages of low computational efficiency and long running time.

[0006] The above three surface reconstruction methods cannot take into account the problem of point cloud holes and the complete capture of local details of the reconstructed surface at the same time. In order to obtain high-precision and high-quality three-dimensional model products, it is necessary to select reasonable modeling methods for point cloud data of different properties. The Poisson function modeling method can consider all point cloud data as a whole and has good surface smoothness, but the traditional Poisson function method will incorrectly connect some blurred areas, resulting in poor quality of triangular facets. In addition, in the process of acquiring three-dimensional point cloud data, due to the occlusion of obstacles, scanning angles, pre-processing of original point cloud data, etc., point cloud data is lost, which will cause holes in the reconstructed surface.

[0007] The existing point cloud surface reconstruction method and point cloud reconstruction model hole repair, the problems to be solved and the key technical difficulties of this application include:

[0008] (1) The existing triangular mesh and surface function surface reconstruction methods cannot simultaneously take into account the problems of point cloud holes and capturing local details of the reconstructed surface. In order to obtain high-quality modeled products, it is urgent to use a variety of modeling methods for different data to perform surface modeling, and compare and analyze the applicability and advantages and disadvantages of different modeling methods. The Poisson surface reconstruction method has good reconstructed surface smoothness, uniform distribution of triangulated network vertices, and consistent triangle sizes, but it is easy to cause incorrect triangulated network connections. Greedy triangulated network surface reconstruction is prone to produce holes in the curves, serious loss of details, and the worst surface smoothness. The existing three-dimensional laser scanning technology cannot meet the requirements of the comprehensiveness and detail integrity of the target three-dimensional model, cannot effectively process massive disordered point cloud data, lacks a suitable point cloud surface reconstruction method, lacks point cloud modeling detail retention and a comprehensive point cloud surface model.

[0009] (2) The existing parametric surface reconstruction NURBS surface needs to introduce weight factors. If the weight factors are not set reasonably, the parameterization effect of the surface will be poor. In severe cases, the surface structure may be destroyed. The existing surface reconstruction methods based on Poisson function and radial basis model are both overall hidden surface reconstruction methods, which have the disadvantages of low computational efficiency and long running time. It is urgent to develop an optimized Poisson modeling method to address the defects of the traditional Poisson modeling method, introduce median-bilateral filtering and bicubic spline interpolation in the point cloud pre-processing stage, use the transfer low-square optimization method to improve the accuracy when obtaining the normal vector, redirect the normal vector to improve the consistency of the normal vector, and improve the traditional Poisson modeling method.

[0010] (3) The existing surface reconstruction methods cannot simultaneously take into account the problem of point cloud holes and the complete capture of local details of the reconstructed surface. The traditional Poisson function method will incorrectly connect some blurred areas, resulting in poor quality of triangular facets. In addition, in the process of acquiring three-dimensional point cloud data, the point cloud data is lost due to reasons such as occlusion by obstacles, scanning angles, and pre-processing of the original point cloud data, which will cause holes in the reconstructed surface. When reconstructing a scene with missing data in a three-dimensional laser point cloud, there will always be some holes on the surface of the surface model due to various reasons, so the hole part of the laser point cloud three-dimensional model needs to be repaired. The existing technology cannot repair the uncoordinated triangles in the mesh, cannot solve the problem of repairing hole areas with large curvature changes, and cannot guarantee the consistency and coordination between the triangulated network of the repaired area and the triangulated network of the original surface. Summary of the invention

[0011] The present application establishes an optimized Poisson surface reconstruction method, performs noise reduction processing on erroneous point cloud data, removes abnormal points, and uses bicubic spline interpolation to increase the point cloud density if data density is sparse. A transfer low-square optimization method is established to accurately calculate the point cloud normal vector and adjust the direction of the point cloud normal vector to be consistent, thereby improving the accuracy of normal vector calculation, and redirecting the normal vector to ensure the consistency of the point cloud normal vector. In view of the holes in the triangular mesh model that affect the data model, a triangular mesh optimization adjustment and repair method based on the radial basis model is adopted, so that the repair result can finally maintain good continuity with the triangular mesh surface around the hole, and the repaired triangular mesh maintains good geometric consistency with the triangular mesh of the original model surface. When constructing the hidden surface, the triangle vertex position information and the normal vector information of each vertex around the hole are used, and the repaired surface takes into account the geometric shape and surface change trend of the hole area. The obtained surface can well reflect the state of the original surface, and the surface reconstruction speed is fast and the quality is high, which is suitable for applications in multiple industries such as film, television, games and animation.

[0012] In order to achieve the above technical effects, the technical solutions adopted in this application are as follows:

[0013] The hole repair method for point cloud 3D surface reconstruction of film, television, game and animation is as follows: first, an optimized Poisson surface reconstruction method is established to reduce noise for erroneous point cloud data and remove abnormal points. If data density is sparse, bicubic spline interpolation is used to increase point cloud density, and a transfer low-square optimization method is established to accurately calculate point cloud normal vectors and adjust the direction of point cloud normal vectors to be consistent; second, for some holes in the triangular mesh model that affect the data model, a triangular mesh optimization adjustment and repair method based on the radial basis model is used, so that the repair result can maintain good continuity with the triangular mesh surface around the hole, and the repaired triangular mesh maintains good geometric consistency with the triangular mesh of the original model surface. When constructing hidden surfaces, the triangular vertex position information around the hole and the normal vector information of each vertex are used. The repaired surface takes into account the geometric shape and surface change trend of the hole area, and the obtained surface can well reflect the state of the original surface;

[0014] 1) Optimized Poisson surface 3D point cloud reconstruction method: In the traditional Poisson modeling method, the point cloud filtering method and the normal vector calculation and adjustment method are integrated. The erroneous point cloud data is deleted by combining the median filter and the bilateral filter on the point cloud. In view of the disadvantage of inconsistent normal vector directions, the transfer low-square optimization method is used to calculate the point cloud normal vector to improve the normal vector calculation accuracy, and the normal vector is redirected to ensure the consistency of the point cloud normal vector;

[0015] 2) Hole repair method for three-dimensional model: including hole surface recognition and preliminary filling method of hole area. The radial basis model is used to select reasonable interpolation constraint points and additional constraint points for the preliminary filled hole area to establish hidden surfaces to repair the hole area. The method of exchanging the common edges of adjacent triangles is incorporated to adjust the triangular mesh repaired by the radial basis model. The triangular mesh adjustment method is used to subdivide larger triangles and narrow triangles, and finally a repair mesh that is consistent with the triangular mesh of the original model is formed.

[0016] Preferably, the calculation and adjustment of the point cloud normal vector: based on the transfer low-square optimization method, it is improved in two aspects: first, a quick fitting function is established, which only requires two parameters: the basis function b(x) and the factor function a(x); second, a dense dominant weight model is adopted, based on the transfer window, it is determined that the value at point x is only affected by other nodes in the node neighborhood, and the nodes outside the neighborhood have no effect on the value at x. The area that affects point x is the dominant domain of point x. During the fitting process of the transfer low-square optimization method, the weight function ω(x) acts on the smoothness of the fitting surface. The weight function ω(x) is defined on the dominant domain of x. In addition, the order of the basis function b(x) has a great effect on the accuracy of surface fitting. The transfer low-square optimization method is used instead of the PCA method to calculate the normal vector of point cloud data, thereby improving the accuracy of normal vector calculation.

[0017] Preferably, the point cloud normal vector is calculated by transferring low-square optimization: using distance as the weight, firstly the approximate function factor is obtained by distance weighted calculation, then the sum of squares of the differences of given factors is calculated, and finally the size of the sum of squares is used to measure the fitting entropy, which incorporates the density dominance of the weighted function and has the characteristics of local factor analysis;

[0018] For a region to be fitted, a local sub-region of the region is selected and the transfer function of the function F(x) to be fitted is defined as It is expressed by formula 1

[0019]

[0020] Where m is the number of basis functions, b j (x) is the basis function, a j (x) is a factor, and the basis function b is set j (x) is a polynomial form, D is a parameter, then the linear basis, quadratic basis, and cubic basis of the basis function are expressed as follows:

[0021] First degree polynomial · linear basis:

[0022]

[0023] Quadratic polynomial · Quadratic basis:

[0024]

[0025] Cubic basis:

[0026]

[0027] Based on formula 1, the weighted low-square optimization function is used to calculate the quadratic form shown in formula 5:

[0028]

[0029] In formula 5, n is the number of nodes near x, w(xx i ) represents the node x i Weight function, node x i The weight function is densely dominated, smooth, and continuous, and for a node x, the weight function inside the densely dominated domain satisfies w i =w(xx i )>0, the weight function w outside the boundary i =0,f i is x i Node value, formula 5 for factor a j (x) is derived.

[0030] Preferably, the densely dominated weight model: in the transfer low-square optimization method, the weight function has dense dominance. When fitting a large number of discrete point data, the weight function performs local processing on the data points. For the point x to be fitted, only the nodes inside the dominance domain of the point x are effective. The densely dominated weight function w(x i ) not only satisfies the condition of being greater than or equal to 0, but also decreases with increasing distance;

[0031] Parameter settings of weight function: size of dominance domain, shape factor β of dominance domain, under uniform nodes, S max =scale×C1, scale is the proportional factor with a value greater than 1, C1 represents the distance between nodes. In the calculation process, the dominance domain of any point to be fitted needs to meet the condition, that is, their dominance domain contains a sufficient number of nodes; the shape factor β reflects the role of each node in the dominance domain. The increase of β leads to w(xx i ) decays faster and the radius of the dominating domain becomes smaller. When β increases, the nodes far away from the fitting point have a smaller effect on the calculation, while the nodes close to the fitting point have a greater effect on the calculation.

[0032] Transfer low-square optimization for local surface fitting Based on the dense dominated weight model, in the process of local surface fitting using transfer low-square optimization, after the size of the local dense dominated domain is fixed, the effect of other nodes participating in local fitting on the nodes to be fitted is calculated. The first step is to calculate the local dense dominated domain of all nodes. The dominated domain is in the form of a circle. If there are N nodes, the calculation process of constructing a circular dense dominated domain for these nodes is as follows:

[0033] Step 1: Set the radius of each circular dense domain of all nodes in N nodes to h i =0;

[0034] Step 2: For each node, first establish a coordinate system with the origin of the coordinate system as the current node, then calculate the distance between the points in each quadrant of the coordinate system and the node, and obtain the point with the shortest distance from the node in each quadrant, and set the distance value between these four points and the node as h. i1 、h i2 、h i3 、h i4 , and finally determine the radius h of the densely dominated domain i , that is, h i =max{h i1 、h i2 、h i3 、h i4}, ensuring that the densely-dominated domain has suitable nodes in each quadrant;

[0035] Step 3: Execute step 2 for all nodes;

[0036] Step 4: According to the data to be processed, select the action factor β and set the radius h of each node. i Multiply by β and use the action factor β to increase the radius of the dense domain to make the fitting more accurate. The value of β is 1.2 to 2.5.

[0037] Preferably, normal vector redirection: Assume that the point cloud data has sufficient density and is evenly distributed without noise, the sampling surface is smooth everywhere, and the normal vectors of two adjacent points are close to parallel. Define the normal vectors of two adjacent points p i and p j The normal vectors are n i and n j , when constructing an undirected graph for a point cloud, 1-|n i n j | is the cost of the edge. At this time, if p i and p j If the normal vector is adjacent, the corresponding edge will have a smaller cost. Then determine the initial direction of the normal vector. First, select the normal vector direction of a point in the point cloud data as the accurate normal vector direction. Then use this point as the propagation source point to propagate the normal vector direction of the propagation source point along the minimum spanning tree of the undirected graph. The propagation of the normal vector direction is carried out along the minimum spanning tree of the traversal graph. The first point to be redirected is the adjacent point with the closest normal vector to the current propagation point. Assume that n i The normal vector of is redirected after the normal vector propagation, n j is the next node to be traversed in the propagation path. If n i gn j <0, then n j Flip, otherwise remain unchanged, select a point with the largest coordinate component as the propagation source point, point p i The x-coordinate component of the point is the largest, and the normal vector of this point is selected to point in the positive direction of the x-axis.

[0038] Preferably, the hole area is identified: the triangular mesh of the surface of the three-dimensional model is a closed mesh. If there is no hole in a certain area of ​​the model surface, each edge of the triangle in this area will have an adjacent triangle sharing this edge, that is, this edge has two adjacent triangles. If there is a hole in the area, the edge constituting the boundary of the hole has only one adjacent triangle, that is, if a surface is a non-closed triangular mesh surface, the edge constituting this surface will only have one adjacent triangle.

[0039] The edge with only one adjacent triangle is the edge participating in the construction of the boundary of the hole area, and this edge is defined as the boundary edge; the edge with two adjacent triangles belongs to the closed triangle mesh area, and this edge is defined as the internal edge; the first and last endpoints of the boundary edge are connected together to form a closed space polygon, and the polygon is defined as the hole boundary; when realizing the recognition and storage of the hole boundary, an ordered linked list corresponding to each hole is established, and the arrangement of the ordered linked list is the connection order of the boundary edges constituting the hole, and a one-dimensional array is established, and the linked list head of each hole boundary is saved in this one-dimensional array. When repairing the hole, you only need to query this array to find the magic hole boundary;

[0040] Repair the single connected holes, and improve the single hole repair method for other types of holes;

[0041] The specific process of boundary pre-processing is as follows:

[0042] (1) Calculate the average length L of all boundary edges that constitute the hole boundary K ;

[0043] (2) Select an edge on the hole boundary as the starting edge, and then traverse the boundary edges that constitute the hole boundary from the starting edge. If an edge P i P j The length L ij >kgL K In the case of , k is the proportional factor, and the k value is selected as 2, then the edge P is calculated i P j The midpoint P new , delete edge P in the hole boundary list i P j , add a new edge P i P new and P new P j , delete the triangle △PP in the mesh model i P j , increase the triangle △P i PP new and △PP new P j , and at the same time update the topological relationship between points, faces and edges in the data structure.

[0044] (3) If new edges and triangles appear in step (2), continue to repeat step (2); if no new edges and triangles appear in step (2), exit the loop and end the pre-processing process of the hole boundary.

[0045] Preferably, the initial filling of the hole area: the hole filling is performed by using a method based on a feature surface, and the spatial polygon is converted to a two-dimensional plane for triangulation processing, which reduces the computational complexity. In addition, the geometric shape of the newly added filling triangles is uniform, and the distribution of the triangles is close to the original triangular mesh, so that the filled triangular mesh has good coordination with the original triangular mesh. The specific steps are as follows:

[0046] Step 1, determine the hole polygon feature surface: first establish a local coordinate system, the origin of the feature surface is the geometric center of all vertices on the hole boundary, then use the principal component analysis method to obtain the normal vector of the feature surface, complete the establishment of the local coordinate system and the feature surface according to the origin and normal vector of the feature surface, project the hole boundary polygon along the feature surface normal vector onto the feature surface to form a feature polygon, and finally convert the feature polygon to the local coordinate system;

[0047] Step 2, construct a new triangle: search for the inner angle of the feature polygon, and determine whether it is necessary to add new points to construct a new triangle based on whether the inner angle is greater than 90 degrees. If the inner angle is less than 90 degrees, no new points need to be added, and the polygon is updated directly. If the inner angle is greater than 90 degrees, new points need to be added according to the established rules, and the feature polygon is updated; then the above steps are repeated until the newly added triangle fills the entire feature polygon;

[0048] Step 3: Check the legality of the newly added triangles: Check whether the newly added points are within the hole boundary feature plane and whether the newly added triangle mesh is self-intersecting with the hole boundary polygon and the triangle patch in the triangle patch mesh, and delete the incorrect filling points.

[0049] Preferably, a radial basis based hidden surface model: given R 3 n scattered points in space {c1, c2, L, c n}, each point has a constraint value {h1, h2, L, h n}, construct a function f(r) so that every scattered point satisfies f(c i )=h i , a hidden surface equation f(r i )=0. When constructing the hidden surface, some scattered points exist on the generated surface. These points passed by the hidden surface are interpolation constraint points, and other points not passed by are defined as additional constraint points. The hidden surface established based on the radial basis model is expressed by equation 6:

[0050]

[0051] In the formula, r represents any data point on the generated surface, c j The scattered points of the equation are sampling points, w jRepresents the weight corresponding to each scattered point of the equation. P(r) is a polynomial form, and its expression is P(r)=p+p1x+p2y+p3z, φ(rc j ) is a radial basis model, solving the weights and multi-phase equations, satisfying the interpolation condition of equation 7 and the orthogonal condition of equation 8:

[0052]

[0053] Where i = 1, 2, L, n, let φ ij =φ(c i -c j ), through equations 7 and 8, we get a semi-positive definite matrix, and there is a unique set of solutions [w1 w1 L w1 p0 p1 p2 p3]. Substituting the result into equation 7, we get the hidden surface equation. When the radial basis model is used to construct the surface equation, the sampling points obtained are the data points around the hole boundary. The amount of data is small, so Gaussian elimination method is directly used.

[0054] Preferably, the collection of interpolation constraint points and hidden equations are established: the use of radial basis model to establish the hidden surface equation of the hole area requires sufficient scattered point data as interpolation constraint points and additional constraint points. When establishing the hole surface equation, the interpolation constraint points are located on the hole surface, that is, the equation f(c i )=0, when establishing a hole surface equation, only interpolation constraint points cannot describe it, and additional constraint points are required. The additional constraint points are located outside the hole surface or inside the hole surface, that is, the equation f(c i )>0or f(c i )<0, for a hole polygon, the selection of interpolation constraint points of the hole surface is not limited to each vertex on the hole boundary edge, but also selects vertices in the triangular mesh around the hole boundary. Moreover, only using the triangular mesh vertices of the hole boundary cannot meet the solution conditions and cannot fully describe the surface. Starting from the hole boundary as the starting layer, it continues to extend outward, and collects the vertices of the triangular meshes of several layers adjacent to the hole boundary as interpolation constraint points;

[0055] The specific process of collecting interpolation constraint points is as follows:

[0056] Step 1: Create a one-dimensional array M to store the vertices of each edge of the hole boundary E;

[0057] Step 2: Search all vertices in array M to determine whether the vertices of the adjacent triangles of the current point are in array M; add the points that are not in the array to the one-dimensional array M. After traversing once, the first layer of triangulated vertices outside the hole boundary are stored in the M array;

[0058] Step 3: Loop through step 2, continuously update the outer triangulated mesh vertices stored in array M according to the set number of layers, and finally obtain a certain number of interpolation constraint points; the number of layers of triangles outside the hole boundary of the interpolation constraint points is set according to the size of the hole, and the number of layers is set to 4;

[0059] After obtaining enough interpolation constraint points, additional constraint points that meet the solution conditions are collected. The collection of additional constraint points is completed using interpolation constraint points. First, the normal vector of the interpolation constraint point is calculated. The selected interpolation constraint points are all vertices of the triangles in the original surface triangular mesh. The normal vector of the interpolation constraint point can be calculated by weighted average estimation of the normal vectors of its adjacent triangular meshes. Then, a new point is marked at a fixed length position in the direction of the normal vector of each interpolation constraint point. The marked new point is the additional constraint point of the previous point, and these additional points are located on the equidistant surface of the hole surface, and these points can reflect the curvature change trend and geometric shape of the surface. The additional constraint points are located on the equidistant surface of the hole surface, they have the same constraint value, and the fixed length is selected as a unit length. These points satisfy the equation f(c i )=1;

[0060] After obtaining the constraints that meet the requirements, substitute the coordinate data of all the interpolation constraint points and additional constraint points into a (2×n+4)×(2×n+4) matrix, where n represents the number of interpolation constraint points. The final hole surface equation is obtained by calculating the weights corresponding to all the scattered points and the multi-phase factors [w1 w1 L w1 p0 p1 p2 p3].

[0061] Preferably, the triangulated network optimization of the hole filling result: the optimization adjustment method adopted is divided into two aspects, firstly, the triangles with too large area are subdivided, and then the narrow and long triangle pieces formed in the subdivision process are adjusted by the common edge transformation method. The specific steps are as follows:

[0062] Step 1: Define the triangulated network formed by the newly added triangles in the hole area after the radial basis model is filled as the initial mesh surface, and calculate the average value S of the side lengths of all triangles in the initial mesh surface K ﹔

[0063] Step 2: Search each triangle patch △p in the triangle mesh i p j p k , calculate the average side length S of the triangle △K If S △K >kgS K , then the shape of the triangle is judged to be too large and inconsistent with the surrounding triangles, and the triangle needs to be subdivided; find the centroid p of the triangle c , p c With pi 、p j 、p k Connect to form a new triangle △p c p j p k , △p c p j p i , △p c p i p k And delete the triangle △p i p j p k , where k is the proportional factor, and the value of k is 1.5;

[0064] Step 3: Traverse each edge in the new mesh, extract the two adjacent triangles formed by this edge, calculate the circumscribed circle of the two adjacent triangles, and determine whether the vertices of the two triangles are both inside the circumscribed circle of the other triangle; if they are inside, the common edges of this group of triangles need to be adjusted, if not, no adjustment is required;

[0065] Step 4: Repeat steps 2 and 3 until step 2 no longer subdivides the triangles in the triangulated network, and the adjustment is completed.

[0066] Compared with the prior art, the innovations and advantages of this application are:

[0067] (1) This application establishes an optimized Poisson surface reconstruction method, performs noise reduction processing on erroneous point cloud data, removes abnormal points, and uses bicubic spline interpolation to increase the point cloud density if data density is sparse. A transfer low-square optimization method is established to accurately calculate the point cloud normal vector and adjust the direction of the point cloud normal vector to be consistent, thereby improving the accuracy of normal vector calculation, and redirecting the normal vector to ensure the consistency of the point cloud normal vector. In view of the holes in the triangular mesh model that affect the data model, a triangular mesh optimization adjustment and repair method based on the radial basis model is adopted, so that the repair result can maintain good continuity with the triangular mesh surface around the hole, and the repaired triangular mesh maintains good geometric consistency with the triangular mesh of the original model surface. When constructing the hidden surface, the triangle vertex position information and the normal vector information of each vertex around the hole are used. The repaired surface takes into account the geometric shape and surface change trend of the hole area. The obtained surface can well reflect the state of the original surface. The surface reconstruction is fast and of high quality, which is suitable for applications in multiple industries such as film, television, games and animation.

[0068] (2) In view of the shortcomings of traditional Poisson modeling, this application proposes an optimized and improved surface reconstruction method, which uses a combination of median filtering and bilateral filtering to delete erroneous points on the point cloud. In view of the shortcomings of the PCA method in obtaining the inconsistency of the normal vector direction, the transfer low-square optimization method is used to obtain the point cloud normal vector to improve the accuracy of the normal vector calculation and redirect the normal vector to ensure the consistency of the point cloud normal vector. Experiments have shown that the traditional Poisson modeling method will incorrectly connect some blurred hole areas, resulting in blurring or loss of detail features. The optimized Poisson modeling method proposed in this application improves the detail integrity of the Poisson modeling method, effectively reduces the connection of erroneous triangles, and has good surface smoothness. The effectiveness and feasibility of the proposed optimized and improved Poisson surface reconstruction method are confirmed, and the spatial geometric relationship of the point cloud data is calculated more accurately, reducing the degree of errors in topological connections. The reconstructed surface has good continuity and smoothness, and the details are well preserved, which improves the accuracy of Poisson function modeling.

[0069] (3) There are often some holes in the triangular mesh model that affect the data model, making the model unable to meet the needs of various fields. This application adopts a triangular mesh optimization adjustment and repair method based on the radial basis model, which ultimately enables the repair result to maintain good continuity with the triangular mesh surface around the hole, and the repaired triangular mesh maintains good geometric consistency with the original model surface triangular mesh, and the repaired surface has a good smooth effect. The method adopted in this application makes full use of the triangular vertex position information around the hole and the normal vector information of each vertex when constructing the hidden surface. The repaired surface can well take into account the geometric shape and surface change trend of the hole area, so the obtained surface can well reflect the state of the original surface. The hole positions of the experimental data of this application are diverse. The experimental results show that the method of this application can obtain a good repair effect on the surface with drastic curvature changes. The optimized and adjusted triangles are more uniform than the unadjusted triangular mesh. It effectively handles the problem of large triangles and narrow triangles in the hole repair mesh, maintains the coordination and consistency of the triangular mesh of the entire model, and ensures the application ability and stability of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram for determining the node dominance domain of the dense dominance model.

[0071] Figure 2 It is a schematic diagram of the action nodes of the point x to be fitted.

[0072] Figure 3 This is a schematic diagram of the overall comparison before and after the happy normal vector is adjusted.

[0073] Figure 4 This is a comparison diagram of the experimental Lion using the combined filter processing of this application.

[0074] Figure 5 It is a schematic diagram comparing the changes in the amount of point cloud data before and after filtering.

[0075] Figure 6 This is a diagram of the hole boundary pre-processing process.

[0076] Figure 7 It is a schematic diagram of the newly added method of adjusting triangle mesh vertices to the surface.

[0077] Figure 8 This is a comparison chart of the hole optimization, adjustment and repair of the Blade model.

[0078] Fig. 9 This is the hole optimization, adjustment and repair result diagram of the Lion model.

[0079] Fig.10 This is the result of hole optimization, adjustment and restoration of Xia Jianbai’s statue model. DETAILED DESCRIPTION

[0080] Below, in conjunction with the accompanying drawings, the technical solution of the method for repairing holes in three-dimensional surface reconstruction of point clouds for film, television, games and animation provided by the present application is further described so that technicians in the field can better understand the present application and implement it.

[0081] With the development of big data, computer vision, virtual reality, artificial intelligence, surveying and mapping and other disciplines, point cloud 3D model reconstruction technology has become a research hotspot. As point cloud 3D model reconstruction technology develops, the demand for 3D models in fields such as ancient building protection, reverse engineering, military, game scenes, and film and television animation is also increasing. Based on the above background, this application will be developed from two perspectives: point cloud surface reconstruction method and 3D model hole repair:

[0082] (1) Aiming at the shortcomings of traditional Poisson modeling, an optimized and improved Poisson surface reconstruction method is established: the point cloud filtering method and the normal vector calculation and adjustment method are integrated into the traditional Poisson modeling method, and the erroneous point cloud data is deleted by using a combined filtering method of median filtering and bilateral filtering on the point cloud. In view of the shortcoming of the PCA method that the normal vector direction is inconsistent, the transfer low-square optimization method is used to calculate the point cloud normal vector to improve the accuracy of the normal vector calculation, and the normal vector is redirected to ensure the consistency of the point cloud normal vector.

[0083] (2) Hole repair method for three-dimensional model: including hole surface recognition and preliminary filling method of hole area. The radial basis model is used to select reasonable interpolation constraint points and additional constraint points for the preliminary filled hole area to establish hidden surfaces to repair the hole area. The method of exchanging the common edges of adjacent triangles is integrated to adjust the triangular mesh after the radial basis model repair. The triangular mesh adjustment method can subdivide larger triangles and narrow triangles, and finally form a repair mesh that is consistent with the triangular mesh of the original model.

[0084] 1. Optimized Poisson surface 3D point cloud reconstruction method

[0085] Poisson modeling has the advantages of both overall surface fitting and local surface fitting, and considers laser point cloud data as a whole. The Poisson modeling method is suitable for point cloud data that does not require point cloud data segmentation and splicing. However, due to problems such as the environment, equipment, and calculation methods in the actual measurement process, various noises and redundant data exist in the measurement results, resulting in inaccurate data. At the same time, the traditional Poisson function modeling method will lose the details of the local reconstructed surface.

[0086] This application improves the framework for point cloud reconstruction based on Poisson surfaces: the input point cloud data is firstly subjected to three-dimensional outlier removal, and then the normal vectors of the newly formed sample points are accurately calculated, and the normal vectors are redirected to make the directions of the normal vectors consistent, and finally the surface optimization and reconstruction model is performed based on the Poisson distribution function method.

[0087] (I) Resampling of point cloud

[0088] The three-dimensional model surface obtained by Poisson function surface reconstruction is a triangulated network composed of many small and uniform triangular surfaces. These triangulated networks can well describe the surface shape of the measured object. Because there are various noises in the acquired point cloud data, these noises lead to incorrect triangular facet connections, resulting in many incorrect triangular facets in the surface triangulation network. For point cloud data containing multiple noises, it is unreasonable to use a single filter for processing. Therefore, for multiple noises, a combination of multiple filters is used to reduce or eliminate the noise in the point cloud data. The use of multiple filter combinations can effectively delete some inaccurate point cloud data in the collected point cloud data, improve the accuracy of point cloud data, reduce the number of point cloud data, and improve modeling efficiency. Specifically, median filtering is used to reduce the effect of noise on point cloud feature extraction, and at the same time remove a large amount of noise in point cloud data. Bilateral filtering is used to process noise in a small range to maintain the details and surface smoothness of the original point cloud data.

[0089] After noise processing, the point cloud data will become sparse. Sparse point cloud data has a great impact on the quality of modeling. In order to improve the quality of point cloud data and make the reconstructed surface more delicate, the interpolation method is used to fit the point cloud data. Especially sparse point clouds may cause regional blur, so it is necessary to increase the density of point clouds.

[0090] Using bicubic spline interpolation, the point cloud data is segmented according to the coordinate range of the sampling points to form several sub-rectangular grids. The basis spline expression of the cubic spline function about its corresponding segmentation on the x and y coordinate axes in the coordinate system where the sampling points are located is calculated, and the factors are solved using the first-order partial derivatives of the rectangular boundary nodes and the second-order partial derivatives of the vertices. Then, the expression of the interpolation function is derived, and the interpolation result is obtained according to the interpolation points and the bicubic interpolation equation.

[0091] (II) Calculation and adjustment of point cloud normal vector

[0092] In Poisson function surface reconstruction, the consistency and accuracy of the normal vector of the point cloud data are directly related to the quality of the surface reconstruction. Traditional Poisson modeling uses the PCA method to estimate the normal vector. Since local surface fitting requires the size of the specified area k, the accuracy of the normal vector obtained by the PCA method is related to the noise level in the point cloud. In addition, the direction of the normal vector obtained by local fitting is uncertain, and the direction of the normal vector is not consistent.

[0093] The present application is based on the transfer low-square optimization method and has two improvements: first, a quick fitting function is established, which only requires two parameters: the basis function b(x) and the factor function a(x); second, a dense dominant weight model is adopted, based on the transfer window, it is determined that the value at point x is only affected by other nodes in the node neighborhood, and the nodes outside the neighborhood have no effect on the value at x. The area that has an effect on point x is the dominant domain of point x. During the fitting process of the transfer low-square optimization method, the weight function ω(x) acts on the smoothness of the fitting surface. The weight function ω(x) is defined on the dominant domain of x. In addition, the order of the basis function b(x) has a great effect on the accuracy of surface fitting. The transfer low-square optimization method is used to replace the PCA method to calculate the normal vector of point cloud data, thereby improving the accuracy of normal vector calculation.

[0094] 1. Transfer low-square optimization to calculate point cloud normal vector

[0095] Using distance as weight, firstly, the approximate function factor is obtained by distance weighted calculation, then the sum of squares of the differences of given factors is calculated, and finally the size of the sum of squares is used to measure the fitting entropy, which incorporates the density dominance of the weighted function and has the characteristics of local factor analysis;

[0096] For a region to be fitted, a local sub-region of the region is selected and the transfer function of the function F(x) to be fitted is defined as It is expressed by formula 1

[0097]

[0098] Where m is the number of basis functions, b j (x) is the basis function, a j (x) is a factor, and the basis function b is set j (x) is a polynomial form, D is a parameter, then the linear basis, quadratic basis, and cubic basis of the basis function are expressed as follows:

[0099] First degree polynomial · linear basis:

[0100]

[0101] Quadratic polynomial · Quadratic basis:

[0102]

[0103] Cubic basis:

[0104]

[0105] Based on formula 1, the weighted low-square optimization function is used to calculate the quadratic form shown in formula 5:

[0106]

[0107] In formula 5, n is the number of nodes near x, w(xx i ) represents the node x i Weight function, node x i The weight function is densely dominated, smooth, and continuous, and for a node x, the weight function inside the densely dominated domain satisfies w i =w(xx i )>0, the weight function w outside the boundary i =0,f i is x i Node value, formula 5 for factor a j (x) is derived.

[0108] 2. Secret ownership model

[0109] In the transfer low-square optimization method, the weight function has dense dominance. When fitting a large number of discrete point data, the weight function performs local processing on the data points. The setting of the weight function is the key to the transfer low-square optimization method when performing surface fitting. For the point x to be fitted, only the nodes inside the dominance domain of the point x are effective. The densely dominance weight function w(x i ) not only satisfies the condition of being greater than or equal to 0, but also decreases with increasing distance.

[0110] Parameter settings of weight function: size of dominance domain, shape factor β of dominance domain, under uniform nodes, S max =scale×C1, scale is the proportional factor with a value greater than 1, C1 represents the distance between nodes. In the calculation process, the dominance domain of any point to be fitted needs to meet the condition, that is, their dominance domain contains a sufficient number of nodes; the shape factor β reflects the role of each node in the dominance domain. The increase of β leads to w(xx i ) decays faster and the radius of the dominating domain becomes smaller. When β increases, the nodes far away from the fitting point have a smaller effect on the calculation, while the nodes close to the fitting point have a greater effect on the calculation.

[0111] Transfer low-square optimization for local surface fitting Based on the dense dominated weight model, in the process of local surface fitting using transfer low-square optimization, after the size of the local dense dominated domain is fixed, the effect of other nodes participating in local fitting on the nodes to be fitted is calculated. The first step is to calculate the local dense dominated domain of all nodes. The dominated domain is in the form of a circle. If there are N nodes, the calculation process of constructing a circular dense dominated domain for these nodes is as follows:

[0112] Step 1: Set the radius of each circular dense domain of all nodes in N nodes to h i =0;

[0113] Step 2: For each node, first establish a coordinate system with the origin of the coordinate system as the current node, then calculate the distance between the points in each quadrant of the coordinate system and the node, and obtain the point with the shortest distance from the node in each quadrant, and set the distance value between these four points and the node as h. i1 、h i2 、h i3 、h i4 , and finally determine the radius h of the densely dominated domain i , that is, h i =max{h i1 、h i2 、h i3 、h i4}, ensuring that the densely dominated domain has suitable nodes in each quadrant, such as Figure 1 (a) shown.

[0114] Step 3: Execute step 2 for all nodes;

[0115] Step 4: According to the data to be processed, select the action factor β and set the radius h of each node. i Multiply by β, and use the action factor β to increase the radius of the dense domain to make the fitting more accurate. The value of β is 1.2 to 2.5, such as Figure 1 (b)

[0116] In the process of surface fitting using the transfer low-square optimization method, the weight function's dominant domain is determined. At this time, the dominant domain of the node selection must be able to completely cover the entire area. In order to ensure the reversibility of the A matrix and the accuracy of surface fitting, each point to be fitted requires a certain number of nodes to participate in the calculation. When transferring the low-square optimization surface fitting, at least three active nodes are required, and the three points cannot be on the same line.

[0117] In the fitting region, there are cases where the dominant domains at several nodes intersect. Figure 1 As shown in the figure, the point x to be fitted is located in the dominant domain of any node among the nodes x1, x2, x3, x4, and x5. For the point x to be fitted, x1, x2, x3, x4, and x5 all have an effect on it. x1, x2, x3, x4, and x5 are taken as the action nodes of point x. When performing local fitting, the effects of points x1, x2, x3, x4, and x5 on point x are calculated. Figure 2 In the figure, x is outside the dominating domain of x6 (dashed circle), the effect of x on x6 is 0, and point x6 is not considered when fitting point x.

[0118] 3. Normal vector redirection

[0119] The normal vector obtained by the transfer least squares method is ambiguous. The normal vector may point to any direction of the desired line, so the final direction of the normal vector cannot be determined. Due to the above reasons, it is necessary to redirect the calculated point cloud normal vector. Assuming that the point cloud data has sufficient density and is evenly distributed without noise, the sampling surface is smooth everywhere, and the normal vectors of two adjacent points are close to parallel, define the two adjacent points p i and p j The normal vectors are n i and n j , when constructing an undirected graph for a point cloud, 1-|n i n j | is the cost of the edge. At this time, if p i and p j If the normal vector is adjacent, the corresponding edge will have a smaller cost. Then determine the initial direction of the normal vector. First, select the normal vector direction of a point in the point cloud data as the accurate normal vector direction. Then use this point as the propagation source point to propagate the normal vector direction of the propagation source point along the minimum spanning tree of the undirected graph. The propagation of the normal vector direction is carried out along the minimum spanning tree of the traversal graph. The first point to be redirected is the adjacent point with the closest normal vector to the current propagation point. Assume that n i The normal vector of is redirected after the normal vector propagation, n j is the next node to be traversed in the propagation path. If n i gn j <0, then n jFlip, otherwise remain unchanged, select a point with the largest coordinate component as the propagation source point, point p i The x-coordinate component of the point is the largest, and the normal vector of this point is selected to point in the positive direction of the x-axis.

[0120] (III) Experimental analysis

[0121] The relevant parameters of the Poisson function surface reconstruction method of the present application are set as follows: the octree depth is 8, the ISO isosurface extraction method depth is 8, and the Gauss-Seidel iterative method depth for solving the linear equation system is 8. Since the happy point cloud data does not contain noise points, the difference between the processing of point cloud data by the method of the present application and the traditional Poisson modeling method lies in the calculation and adjustment of the point cloud normal vector.

[0122] Figure 3 (The bright white color in the figure shows the direction of the normal vector) is the overall comparison of the happy data before and after the direction adjustment. It can be seen that the improved normal vector is more accurate in describing the local area.

[0123] The second set of experimental data is Lion, which is scanned by RIEGL-VZ400 ground laser scanner and contains a lot of noise. The combined filter of this application is used to process and remove the redundant and noisy data, and the results are as follows Figure 4 The amount of point cloud data after filtering changes as shown in Figure 5 shown.

[0124] Depend on Figure 4 It can be seen that the quality of Lion data is greatly improved after point cloud resampling. Many noise points in the original data are removed in the lion's abdomen, tail and head. After resampling, the normal vectors are calculated and adjusted for the obtained data. According to the normal vector graph calculated by the method of the present application, it can be concluded that the calculated normal vectors are very consistent and have high accuracy. This has greatly improved the problem of normal vector inconsistency caused by using PCA to calculate point cloud normal vectors in the traditional Poisson modeling method.

[0125] Based on the above data sets, the surface reconstructed by the optimized Poisson method of this application is smoother and the model details are well preserved, which proves that the method of this application is effective.

[0126] 2. Hole repair of point cloud surface 3D model

[0127] Laser scanners are affected by various other factors when collecting data, such as the target object being blocked, the object having high reflectivity, and the scanning environment being poor. These factors can cause noise, data redundancy, data missing, and other problems in the point cloud data. Ultimately, in the process of surface reconstruction using point cloud data, these data problems will cause some holes on the surface of the reconstructed model, and these holes will bring many inconveniences to the practical application of three-dimensional surface models. For example, many shape analysis software require that the mesh of the input model has high quality, and these holes on the surface of the model will cause the software to be unable to analyze or produce erroneous analysis results. This application solves hole repair based on the radial basis model and optimizes and adjusts the repaired mesh.

[0128] (I) Identify the hole area

[0129] The triangular mesh of the 3D model surface is a closed mesh. If there is no hole in a certain area of ​​the model surface, each edge of the triangle in this area will have an adjacent triangle that shares this edge, that is, this edge has two adjacent triangles. If there is a hole in the area, the edge that constitutes the boundary of the hole has only one adjacent triangle, that is, if a surface is a non-closed triangular mesh surface, the edge that constitutes this surface will only have one adjacent triangle.

[0130] An edge with only one adjacent triangle is an edge that participates in constructing the boundary of the hole area, and this edge is defined as a boundary edge; an edge with two adjacent triangles belongs to a closed triangular mesh area, and this edge is defined as an internal edge; the first and last endpoints of the boundary edge are connected together to form a closed spatial polygon, and the polygon is defined as the hole boundary; when realizing the recognition and storage of hole boundaries, an ordered linked list corresponding to each hole is established, and the arrangement of the ordered linked list is the connection order of the boundary edges that constitute the hole. A one-dimensional array is established, and the head of the linked list of each hole boundary is saved in this one-dimensional array. When repairing the hole, you only need to query this array to find the magical hole boundary.

[0131] Repair the single connected holes, and improve the single hole repair method for other types of holes;

[0132] In actual laser scanning measurement, the amount of point cloud data obtained in the hole area is relatively small. These few points form many narrow and long undesirable triangular pieces in the surface reconstruction process. These narrow and long triangular pieces cause the sides of the triangles to be too long or too short. The boundary edges of the hole boundaries formed by these edges have inconsistent lengths. When the holes are filled, this inconsistent boundary edge will affect the shape of the newly added triangles. Therefore, it is necessary to pre-process the hole boundaries before filling the holes. The process of boundary pre-processing is as follows: Figure 6 As shown in the figure, the specific process of boundary pre-processing is as follows:

[0133] (1) Calculate the average length L of all boundary edges that constitute the hole boundary K ;

[0134] (2) Select an edge on the hole boundary as the starting edge, and then traverse the boundary edges that constitute the hole boundary from the starting edge. If an edge P i P j The length L ij >kgL K In the case of , k is the proportional factor, and the k value is selected as 2, then the edge P is calculated i P j The midpoint P new , delete edge P in the hole boundary list i P j , add a new edge P i P new and P new P j , delete the triangle △PP in the mesh model i P j , increase the triangle △P i PP new and △PP new P j , and at the same time update the topological relationship between points, faces and edges in the data structure.

[0135] (3) If new edges and triangles appear in step (2), continue to repeat step (2); if no new edges and triangles appear in step (2), exit the loop and end the pre-processing process of the hole boundary.

[0136] (II) Initial filling of hole area

[0137] The hole area cannot be filled by direct division. It is very difficult and computationally complex to directly process the hole space polygons. Therefore, a method based on feature surface is used to fill the holes. The space polygons are converted to a two-dimensional plane for triangulation processing to reduce the computational complexity. In addition, the geometric shape of the newly added filling triangles is uniform, and the distribution of the triangles is close to the original triangular mesh, so that the filled triangular mesh has good coordination with the original triangular mesh. The specific steps are as follows:

[0138] Step 1, determine the hole polygon feature surface: first establish a local coordinate system, the origin of the feature surface is the geometric center of all vertices on the hole boundary, then use the principal component analysis method to obtain the normal vector of the feature surface, complete the establishment of the local coordinate system and the feature surface according to the origin and normal vector of the feature surface, project the hole boundary polygon along the feature surface normal vector onto the feature surface to form a feature polygon, and finally convert the feature polygon to the local coordinate system;

[0139] Step 2, construct a new triangle: search for the inner angle of the feature polygon, and determine whether it is necessary to add new points to construct a new triangle based on whether the inner angle is greater than 90 degrees. If the inner angle is less than 90 degrees, no new points need to be added, and the polygon is updated directly. If the inner angle is greater than 90 degrees, new points need to be added according to the established rules, and the feature polygon is updated; then the above steps are repeated until the newly added triangle fills the entire feature polygon;

[0140] Step 3: Check the legality of the newly added triangles: Check whether the newly added points are within the hole boundary feature plane and whether the newly added triangle mesh is self-intersecting with the hole boundary polygon and the triangle patch in the triangle patch mesh, and delete the incorrect filling points.

[0141] (III) Establishing a hidden model of hole surfaces

[0142] 1. Hidden surface model based on radial basis

[0143] Given R 3 n scattered points in space {c1, c2, L, c n}, each point has a constraint value {h1, h2, L, h n}, construct a function f(r) so that every scattered point satisfies f(c i )=h i , a hidden surface equation f(r i )=0. When constructing the hidden surface, some scattered points exist on the generated surface. These points passed by the hidden surface are interpolation constraint points, and other points not passed by are defined as additional constraint points. The hidden surface is established based on the radial basis model using the equation

[0144] Formula 6 shows:

[0145]

[0146] In the formula, r represents any data point on the generated surface, c j The scattered points of the equation are sampling points, w j Represents the weight corresponding to each scattered point of the equation. P(r) is a polynomial form, and its expression is P(r)=p+p1x+p2y+p3z, φ(rc j ) is a radial basis model, solving the weights and multi-phase equations, satisfying the interpolation condition of equation 7 and the orthogonal condition of equation 8:

[0147]

[0148] Where i = 1, 2, L, n, let φ ij =φ(c i -c j), through equations 7 and 8, we get a semi-positive definite matrix, and there is a unique set of solutions [w1 w1 L w1 p0 p1 p2 p3]. Substituting the result into equation 7, we get the hidden surface equation. When the radial basis model is used to construct the surface equation, the sampling points obtained are the data points around the hole boundary. The amount of data is small, so Gaussian elimination method is used directly.

[0149] 2. Establish the collection and hidden equations of interpolation constraint points

[0150] After filling the holes, the newly added triangular mesh is located on the characteristic surface of the hole polygon. However, for holes that appear in areas where the surface curvature changes dramatically, the established characteristic surface will be quite different from the original shape of the object in the hole area. Such filling results are not realistic. Therefore, the filled triangular mesh needs to be adjusted in the area where the curvature changes dramatically. The hidden surface is established based on the radial basis model to adjust the vertices of the filled triangular mesh.

[0151] The radial basis model is used to establish the hidden surface equation of the hole area. It requires sufficient scattered point data as interpolation constraint points and additional constraint points. When establishing the hole surface equation, the interpolation constraint points are located on the hole surface, that is, the equation f(c i )=0, when establishing a hole surface equation, only interpolation constraint points cannot describe it, and additional constraint points are required. The additional constraint points are located outside the hole surface or inside the hole surface, that is, the equation f(c i )>0or f(c i )<0, for a hole polygon, the selection of interpolation constraint points of the hole surface is not limited to each vertex on the hole boundary edge, but also selects vertices in the triangular mesh around the hole boundary. Moreover, only using the triangular mesh vertices of the hole boundary cannot meet the solution conditions and cannot fully describe the surface. Starting from the hole boundary as the starting layer, it continues to extend outward, and collects the vertices of the triangular meshes of several layers adjacent to the hole boundary as interpolation constraint points;

[0152] The specific process of collecting interpolation constraint points is as follows:

[0153] Step 1: Create a one-dimensional array M to store the vertices of each edge of the hole boundary E;

[0154] Step 2: Search all vertices in array M to determine whether the vertices of the adjacent triangles of the current point are in array M; add the points that are not in the array to the one-dimensional array M. After traversing once, the first layer of triangulated vertices outside the hole boundary are stored in the M array;

[0155] Step 3: Loop through step 2, continuously update the outer triangulated mesh vertices stored in array M according to the set number of layers, and finally obtain a certain number of interpolation constraint points; the number of layers of triangles outside the hole boundary of the interpolation constraint points is set according to the size of the hole, and the number of layers is set to 4;

[0156] After obtaining enough interpolation constraint points, additional constraint points that meet the solution conditions are collected. The collection of additional constraint points is completed using interpolation constraint points. First, the normal vector of the interpolation constraint point is calculated. The selected interpolation constraint points are all vertices of the triangles in the original surface triangular mesh. The normal vector of the interpolation constraint point can be calculated by weighted average estimation of the normal vectors of its adjacent triangular meshes. Then, a new point is marked at a fixed length position in the direction of the normal vector of each interpolation constraint point. The marked new point is the additional constraint point of the previous point, and these additional points are located on the equidistant surface of the hole surface, and these points can reflect the curvature change trend and geometric shape of the surface. The additional constraint points are located on the equidistant surface of the hole surface, they have the same constraint value, and the fixed length is selected as a unit length. These points satisfy the equation f(c i )=1;

[0157] After obtaining the constraints that meet the requirements, substitute the coordinate data of all the interpolation constraint points and additional constraint points into a (2×n+4)×(2×n+4) matrix, where n represents the number of interpolation constraint points. The final hole surface equation is obtained by calculating the weights corresponding to all the scattered points and the multi-phase factors [w1 w1 L w1 p0 p1 p2 p3].

[0158] (IV) Add triangle mesh vertices to the surface

[0159] After the hole surface equation is established, the newly added triangular mesh is adjusted to the fitted hole surface so that the newly added triangular mesh fits closely to the surface. The mathematical expression of the adjustment of the triangular mesh vertices is shown in Equation 9:

[0160]

[0161] In the formula, represents the gradient of the surface, is a regular function; γ is the learning rate, which is a small constant greater than 0, R0 is the iteration starting point, and R1 is the calculated new iteration point. During the iterative calculation process, when the norm of the new iteration point and the iteration starting point is less than the set critical value, the iteration ends. The implementation process of the method of adjusting the new triangle mesh vertex to the surface is as follows: Figure 7 shown.

[0162] (V) Triangulation optimization of hole filling results

[0163] The triangular mesh formed by filling the holes has a relatively uniform distribution, and the geometric structure and triangular mesh density are coordinated with the triangular mesh around the holes. However, after the newly added triangular mesh vertices are adjusted along the gradient direction of the surface, the originally optimized triangular mesh may be unevenly distributed, especially for places where the curvature of the hole surface changes greatly, some narrow triangular facets will appear due to stretching deformation. These stretched and deformed triangular facets will affect the coordination and consistency of the vertices and edge lengths in the filled triangular mesh, thereby affecting the surface smoothness of the filled area. Therefore, it is necessary to optimize and adjust the filling results.

[0164] The optimization adjustment method used in this application is divided into two aspects. First, the triangles with too large areas are subdivided, and then the narrow and long triangles formed in the subdivision process are adjusted by the common edge transformation method. The specific steps are as follows:

[0165] Step 1: Define the triangulated network formed by the newly added triangles in the hole area after the radial basis model is filled as the initial mesh surface, and calculate the average value S of the side lengths of all triangles in the initial mesh surface K ﹔

[0166] Step 2: Search each triangle patch △p in the triangle mesh i p j p k , calculate the average side length S of the triangle △K If S △K >kgS K , then the shape of the triangle is judged to be too large and inconsistent with the surrounding triangles, and the triangle needs to be subdivided; find the centroid p of the triangle c , p c With p i 、p j 、p k Connect to form a new triangle △p c p j p k , △p c p j p i , △p c p i p k And delete the triangle △p i p j p k , where k is the proportional factor, and the value of k is 1.5;

[0167] Step 3: Traverse each edge in the new mesh, extract the two adjacent triangles formed by this edge, calculate the circumscribed circle of the two adjacent triangles, and determine whether the vertices of the two triangles are both inside the circumscribed circle of the other triangle; if they are inside, the common edge adjustment method of this group of triangles needs to be adjusted, if not, no adjustment is required.

[0168] Step 4: Repeat steps 2 and 3 until step 2 no longer subdivides the triangles in the triangulated network, and the adjustment is completed.

[0169] (VI) Experimental analysis

[0170] Figure 8 This is a comparison chart of hole optimization, adjustment and repair of the Blade model. Figure 8 (b) The repaired hole has good continuity with the surrounding surface, and the repaired area has good smoothness. Hole B is in an area with a large curvature change. For hole B with a large curvature, the repair method of the present application based on the radial basis model for local hidden surface interpolation can also ensure the smoothness and continuity of the surface, and achieves good results. According to the optimization and adjustment of holes A and B, the triangles in the repaired triangular mesh are more evenly distributed and consistent with the surrounding triangles. Fig. 9 This is the hole optimization adjustment and repair result diagram of the Lion model. Fig. 9 (b) It can be seen that the smoothness and continuity of the surface can be guaranteed for both holes, and a good repair effect is achieved. Fig. 9 The boxed area in (c) is the optimized adjusted and repaired triangular mesh, and the adjusted triangular meshes in the two areas are evenly distributed. The experimental results show that the hole repair method of the present application can well handle holes in areas with large curvature changes.

[0171] The method of the present application is used to repair the holes on the experimental model. The number of triangles added for each set of experimental data is very small, which means that the method has a small amount of calculation. At the same time, the number of triangles added by the optimized adjustment and repair results reflects that the triangles of the triangular mesh after adjustment are more uniform and consistent.

[0172] In order to verify the applicability of the algorithm, the RIEGL-VZ400 terrestrial laser scanner was used to scan and obtain point cloud data. After preprocessing the point cloud, the point cloud data of the Xia Jianbai statue was obtained.

[0173] Fig.10 The hole optimization and repair results of Xia Jianbai's statue model. Fig.10 (b) It can be seen that the smoothness and continuity of the surface can be guaranteed for both holes A and B, and a good repair effect is achieved. Fig.10 (d) and Fig.10The boxed area in (e) is the triangular mesh after optimization, adjustment and repair. The triangular meshes in the two areas after adjustment are evenly distributed, which is consistent with the triangles around the hole, and also confirms the practicality of the hole repair method of the present application.

Claims

1. A hole repair method for point cloud 3D surface reconstruction in movies, games and animations, characterized in that: First, an optimized Poisson surface reconstruction method is established to reduce the noise of erroneous point cloud data and remove abnormal points. If the data density is sparse, bicubic spline interpolation is used to increase the point cloud density. A transfer low-square optimization method is established to accurately calculate the point cloud normal vector and adjust the direction of the point cloud normal vector to be consistent. Second, in view of the holes in the triangular mesh model that affect the data model, a triangular mesh optimization adjustment and repair method based on the radial basis model is adopted, so that the repair result can maintain good continuity with the triangular mesh surface around the hole. The repaired triangular mesh maintains good geometric consistency with the triangular mesh of the original model surface. When constructing the hidden surface, the triangle vertex position information around the hole and the normal vector information of each vertex are used. The repaired surface takes into account the geometric shape and surface change trend of the hole area, and the obtained surface can well reflect the state of the original surface. 1) Optimized Poisson surface 3D point cloud reconstruction method: In the traditional Poisson modeling method, the point cloud filtering method and the normal vector calculation and adjustment method are integrated. The erroneous point cloud data is deleted by combining the median filter and the bilateral filter on the point cloud. In view of the disadvantage of inconsistent normal vector directions, the transfer low-square optimization method is used to calculate the point cloud normal vector to improve the normal vector calculation accuracy, and the normal vector is redirected to ensure the consistency of the point cloud normal vector; 2) Hole repair method for three-dimensional model: including hole surface recognition and preliminary filling method of hole area. The radial basis model is used to select reasonable interpolation constraint points and additional constraint points for the preliminary filled hole area to establish hidden surfaces to repair the hole area. The method of exchanging the common edges of adjacent triangles is incorporated to adjust the triangular mesh repaired by the radial basis model. The triangular mesh adjustment method is used to subdivide larger triangles and narrow triangles, and finally a repair mesh that is consistent with the triangular mesh of the original model is formed.

2. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1 is characterized in that: Calculation and adjustment of point cloud normal vector: Based on the transfer low-square optimization method, improvements are made in two aspects: first, a quick fitting function is established, which only requires two parameters: the basis function b(x) and the factor function a(x); second, a dense dominant weight model is adopted. Based on the transfer window, it is determined that the value at point x is only affected by other nodes in the node neighborhood. Nodes outside the neighborhood have no effect on the value at x. The area that affects point x is the dominant domain of point x. During the fitting process of the transfer low-square optimization method, the weight function ω(x) acts on the smoothness of the fitting surface. The weight function ω(x) is defined on the dominant domain of x. In addition, the order of the basis function b(x) has a great effect on the accuracy of surface fitting. The transfer low-square optimization method is used instead of the PCA method to calculate the normal vector of point cloud data to improve the accuracy of normal vector calculation.

3. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 2 is characterized in that: Transfer low-square optimization to calculate the point cloud normal vector: using distance as the weight, firstly obtain the approximate function factor through distance weighted calculation, then calculate the sum of squares of the difference of the given factor, and finally use the size of the sum of squares to measure the fitting entropy, incorporating the density dominance of the weighted function, and having the characteristics of local factor analysis; For a region to be fitted, a local sub-region of the region is selected and the transfer function of the function F(x) to be fitted is defined as It is expressed by formula 1 Where m is the number of basis functions, b j (x) is the basis function, a j (x) is a factor, and the basis function b is set j (x) is a polynomial form, D is a parameter, then the linear basis, quadratic basis, and cubic basis of the basis function are expressed as follows: First degree polynomial · linear basis: Quadratic polynomial · Quadratic basis: Cubic basis: Based on formula 1, the weighted low-square optimization function is used to calculate the quadratic form shown in formula 5: In formula 5, n is the number of nodes near x, w(xx i ) represents the node x i Weight function, node x i The weight function is densely dominated, smooth, and continuous, and for a node x, the weight function inside the densely dominated domain satisfies w i =w(xx i )>0, the weight function w outside the boundary i =0,f i is x i Node value, formula 5 for factor a j (x) is derived.

4. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 3 is characterized in that: Densely dominated weight model: In the transfer low-square optimization method, the weight function has dense dominance. When fitting a large number of discrete point data, the weight function performs local processing on the data points. For the point x to be fitted, only the nodes inside the dominance domain of point x are effective. The densely dominated weight function w(x i ) not only satisfies the condition of being greater than or equal to 0, but also decreases with increasing distance; Parameter settings of weight function: size of dominance domain, shape factor β of dominance domain, under uniform nodes, S max =scale×C1, scale is the proportional factor with a value greater than 1, C1 represents the distance between nodes. In the calculation process, the dominance domain of any point to be fitted needs to meet the condition, that is, their dominance domain contains a sufficient number of nodes; the shape factor β reflects the role of each node in the dominance domain. The increase of β leads to w(xx i ) decays faster and the radius of the dominating domain becomes smaller. When β increases, the nodes far away from the fitting point have a smaller effect on the calculation, while the nodes close to the fitting point have a greater effect on the calculation. Transfer low-square optimization for local surface fitting Based on the dense dominated weight model, in the process of local surface fitting using transfer low-square optimization, after the size of the local dense dominated domain is fixed, the effect of other nodes participating in local fitting on the nodes to be fitted is calculated. The first step is to calculate the local dense dominated domain of all nodes. The dominated domain is in the form of a circle. If there are N nodes, the calculation process of constructing a circular dense dominated domain for these nodes is as follows: Step 1: Set the radius of each circular dense domain of all nodes in N nodes to h i =0; Step 2: For each node, first establish a coordinate system with the origin of the coordinate system as the current node, then calculate the distance between the points in each quadrant of the coordinate system and the node, and obtain the point with the shortest distance from the node in each quadrant, and set the distance value between these four points and the node as h. i1 、h i2 、h i3 、h i4 , and finally determine the radius h of the densely dominated domain i , that is, h i =max{h i1 、h i2 、h i3 、h i4 }, ensuring that the densely-dominated domain has suitable nodes in each quadrant; Step 3: Execute step 2 for all nodes; Step 4: According to the data to be processed, select the action factor β and set the radius h of each node. i Multiply by β and use the action factor β to increase the radius of the dense domain to make the fitting more accurate. The value of β is 1.2 to 2.

5.

5. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 4 is characterized in that: Normal vector redirection: Assume that the point cloud data has sufficient density and is evenly distributed without noise, the sampling surface is smooth everywhere, and the normal vectors of two adjacent points are close to parallel. Define the normal vectors of two adjacent points p i and p j The normal vectors are n i and n j , when constructing an undirected graph for a point cloud, 1-|n i n j | is the cost of the edge. At this time, if p i and p j If the normal vector is adjacent, the corresponding edge will have a smaller cost. Then determine the initial direction of the normal vector. First, select the normal vector direction of a point in the point cloud data as the accurate normal vector direction. Then use this point as the propagation source point to propagate the normal vector direction of the propagation source point along the minimum spanning tree of the undirected graph. The propagation of the normal vector direction is carried out along the minimum spanning tree of the traversal graph. The first point to be redirected is the adjacent point with the closest normal vector to the current propagation point. Assume that n i The normal vector of is redirected after the normal vector propagation, n j is the next node to be traversed in the propagation path. If n i gn j <0, then n j Flip, otherwise remain unchanged, select a point with the largest coordinate component as the propagation source point, point p i The x-coordinate component of the point is the largest, and the normal vector of this point is selected to point in the positive direction of the x-axis.

6. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1 is characterized in that: Identify hole areas: The triangular mesh of the 3D model surface is a closed mesh. If there is no hole in a certain area on the model surface, each edge of the triangle in this area will have an adjacent triangle that shares this edge, that is, this edge has two adjacent triangles. If there is a hole in the area, the edge that constitutes the hole boundary has only one adjacent triangle, that is, if a surface is a non-closed triangular mesh surface, the edge that constitutes this surface will only have one adjacent triangle; The edge with only one adjacent triangle is the edge participating in the construction of the boundary of the hole area, and this edge is defined as the boundary edge; the edge with two adjacent triangles belongs to the closed triangle mesh area, and this edge is defined as the internal edge; the first and last endpoints of the boundary edge are connected together to form a closed space polygon, and the polygon is defined as the hole boundary; when realizing the recognition and storage of the hole boundary, an ordered linked list corresponding to each hole is established, and the arrangement of the ordered linked list is the connection order of the boundary edges constituting the hole, and a one-dimensional array is established, and the linked list head of each hole boundary is saved in this one-dimensional array. When repairing the hole, you only need to query this array to find the magic hole boundary; Repair the single connected holes, and improve the single hole repair method for other types of holes; The specific process of boundary pre-processing is as follows: (1) Calculate the average length L of all boundary edges that constitute the hole boundary K ; (2) Select an edge on the hole boundary as the starting edge, and then traverse the boundary edges that constitute the hole boundary from the starting edge. If an edge P i P j Length L ij >kgL K In the case of , k is the proportional factor, and the k value is selected as 2, then the edge P is calculated i P j The midpoint P new , delete edge P in the hole boundary list i P j , add a new edge P i P new and P new P j , delete the triangle △PP in the mesh model i P j , increase the triangle △P i PP new and △PP new P j , and update the topological relationship between points, faces and edges in the data structure at the same time; (3) If new edges and triangles appear in step (2), continue to repeat step (2); if no new edges and triangles appear in step (2), exit the loop and end the pre-processing process of the hole boundary.

7. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1 is characterized in that: Initial filling of hole areas: The method based on feature surfaces is used to fill holes. The spatial polygons are converted to a two-dimensional plane for triangulation processing to reduce the computational complexity. In addition, the geometric shapes of the newly added filling triangles are uniform, and the distribution of the triangles is close to the original triangular mesh, so that the filled triangular mesh has good coordination with the original triangular mesh. The specific steps are as follows: Step 1, determine the hole polygon feature surface: first establish a local coordinate system, the origin of the feature surface is the geometric center of all vertices on the hole boundary, then use the principal component analysis method to obtain the normal vector of the feature surface, complete the establishment of the local coordinate system and the feature surface according to the origin and normal vector of the feature surface, project the hole boundary polygon along the feature surface normal vector onto the feature surface to form a feature polygon, and finally convert the feature polygon to the local coordinate system; Step 2, construct a new triangle: search for the inner angle of the feature polygon, and determine whether it is necessary to add new points to construct a new triangle based on whether the inner angle is greater than 90 degrees. If the inner angle is less than 90 degrees, no new points need to be added, and the polygon is updated directly. If the inner angle is greater than 90 degrees, new points need to be added according to the established rules, and the feature polygon is updated. Then repeat the above steps until the newly added triangles fill the entire feature polygon; Step 3: Check the legality of the newly added triangles: Check whether the newly added points are within the hole boundary feature plane and whether the newly added triangle mesh is self-intersecting with the hole boundary polygon and the triangle patch in the triangle patch mesh, and delete the incorrect filling points.

8. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1 is characterized in that: Radial basis based hidden surface model: Given R 3 n scattered points in space {c1, c2, L, c n }, each point has a constraint value {h1, h2, L, h n }, construct a function f(r) so that every scattered point satisfies f(c i )=h i , a hidden surface equation f(r i )=0. When constructing the hidden surface, some scattered points exist on the generated surface. These points passed by the hidden surface are interpolation constraint points, and other points not passed by are defined as additional constraint points. The hidden surface established based on the radial basis model is expressed by equation 6: In the formula, r represents any data point on the generated surface, c j The scattered points of the equation are sampling points, w j Represents the weight corresponding to each scattered point of the equation. P(r) is a polynomial form, and its expression is P(r)=p+p1x+p2y+p3z, φ(rc j ) is a radial basis model, solving the weights and multi-phase equations, satisfying the interpolation condition of equation 7 and the orthogonal condition of equation 8: Where i = 1, 2, L, n, let φ ij =φ(x i -x j ), through equations 7 and 8, we get a semi-positive definite matrix, and there is a unique set of solutions [w1 w1 L w1 p0 p1 p2 p3]. Substituting the result into equation 7, we get the hidden surface equation. When the radial basis model is used to construct the surface equation, the sampling points obtained are the data points around the hole boundary. The amount of data is small, so Gaussian elimination method is used directly.

9. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1, characterized in that: Establishing the collection of interpolation constraint points and hidden equations: Using the radial basis model to establish the hidden surface equation of the hole area requires sufficient scattered point data as interpolation constraint points and additional constraint points. When establishing the hole surface equation, the interpolation constraint points are located on the hole surface, that is, the equation f(c i )=0, when establishing a hole surface equation, only interpolation constraint points cannot describe it, and additional constraint points are required. The additional constraint points are located outside the hole surface or inside the hole surface, that is, the equation f(c i )>0or f(c i )<0, for a hole polygon, the selection of interpolation constraint points of the hole surface is not limited to each vertex on the hole boundary edge, but also selects vertices in the triangular mesh around the hole boundary. Moreover, only using the triangular mesh vertices of the hole boundary cannot meet the solution conditions and cannot fully describe the surface. Starting from the hole boundary as the starting layer, it continues to extend outward, and collects the vertices of the triangular meshes of several layers adjacent to the hole boundary as interpolation constraint points; The specific process of collecting interpolation constraint points is as follows: Step 1: Create a one-dimensional array M to store the vertices of each edge of the hole boundary E; Step 2: Search all vertices in array M to determine whether the vertices of the adjacent triangles of the current point are all in array M; The points with negative judgment results are added to the one-dimensional array M. After traversing once, the first layer of triangulated vertices outside the hole boundary are stored in the M array; Step 3: Loop through step 2, continuously update the outer triangulated mesh vertices stored in array M according to the set number of layers, and finally obtain a certain number of interpolation constraint points; the number of layers of triangles outside the hole boundary of the interpolation constraint points is set according to the size of the hole, and the number of layers is set to 4; After obtaining enough interpolation constraint points, additional constraint points that meet the solution conditions are collected. The collection of additional constraint points is completed using interpolation constraint points. First, the normal vector of the interpolation constraint point is calculated. The selected interpolation constraint points are all vertices of the triangles in the original surface triangular mesh. The normal vector of the interpolation constraint point can be calculated by weighted average estimation of the normal vectors of its adjacent triangular meshes. Then, a new point is marked at a fixed length position in the direction of the normal vector of each interpolation constraint point. The marked new point is the additional constraint point of the previous point, and these additional points are located on the equidistant surface of the hole surface, and these points can reflect the curvature change trend and geometric shape of the surface. The additional constraint points are located on the equidistant surface of the hole surface, they have the same constraint value, and the fixed length is selected as a unit length. These points satisfy the equation f(c i )=1; After obtaining the constraints that meet the requirements, substitute the coordinate data of all the interpolation constraint points and additional constraint points into a (2×n+4)×(2×n+4) matrix, where n represents the number of interpolation constraint points. The final hole surface equation is obtained by calculating the weights corresponding to all the scattered points and the multi-phase factors [w1 w1 L w1 p0 p1 p2 p3].

10. The hole repair method for 3D surface reconstruction of point cloud for movies, games and animations according to claim 1, characterized in that: Triangulation optimization of hole filling results: The optimization adjustment method used is divided into two aspects. First, the triangles with too large areas are subdivided, and then the narrow triangles formed in the subdivision process are adjusted by the common edge transformation method. The specific steps are as follows: Step 1: Define the triangulated network formed by the newly added triangles in the hole area after the radial basis model is filled as the initial mesh surface, and calculate the average value S of the side lengths of all triangles in the initial mesh surface K ﹔ Step 2: Search each triangle patch △p in the triangle mesh i p j p k , calculate the average side length S of the triangle △K If S △K >kgS K , then the shape of the triangle is judged to be too large and inconsistent with the surrounding triangles, and the triangle needs to be subdivided; find the centroid p of the triangle c , p c With p i 、p j 、p k Connect to form a new triangle △p c p j p k , △p c p j p i , △p c p i p k And delete the triangle △p i p j p k , where k is the proportional factor, and the value of k is 1.5; Step 3: Traverse each edge in the new mesh, extract the two adjacent triangles formed by this edge, calculate the circumscribed circle of the two adjacent triangles, and determine whether the vertices of the two triangles are both inside the circumscribed circle of the other triangle; if they are inside, the common edges of this group of triangles need to be adjusted, if not, no adjustment is required; Step 4: Repeat steps 2 and 3 until step 2 no longer subdivides the triangles in the triangulated network, and the adjustment is completed.

Images