Self-adaptive density semi-automatic hole repairing method and device

Abstract

The invention discloses a density-adaptive semi-automatic hole repairing method and device in the technical field of three-dimensional models. The repairing method comprises the following steps: adding a to-be-processed model into a rendering scene and analyzing, and storing vertexes into a point set V; extracting a front surface patch located at the edge of the cavity from the user frame selection area, extracting a boundary edge in the front surface patch, extracting coordinate information of a corresponding boundary point according to the set V, and storing the coordinate information in a set M; modeling the points in the set M to obtain a curved surface model, performing point cloud interpolation according to the curved surface model to obtain a plurality of interpolation points, and storing the interpolation points in a set I; screening out interpolation points in the cavity from the set I, and storing the interpolation points into a set Ifinal; combining the set Ifinal and the set M as an input point cloud for surface reconstruction, and obtaining a triangular patch set NewTris of the cavity area; and combining the set NewTris with the original model. According to the repairing method, the repairing precision can be ensured on the premise of reducing manual intervention, and the repairing efficiency is improved.

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional models, and in particular to a semi-automatic cavity repairing method and device with adaptive density. Background Art

[0002] Point cloud is a digital representation of the physical world. It can accurately reflect the spatial form of objects and is widely used in biomedicine, autonomous driving, virtual reality, archaeological research, robot mapping and positioning. Point clouds are usually displayed in a discrete form, and there is no spatial index relationship between points. In order to better depict real objects, it is often necessary to reconstruct the surface of the point cloud. Surface reconstruction is a technology that uses discrete point clouds to construct continuous surfaces. At present, there are various algorithms for surface reconstruction, which can be roughly divided into two categories: interpolation and approximation. The model constructed by the interpolation method is input through the point cloud. Common algorithms include greedy triangulation and forward algorithm. In contrast, the approximation method uses the input point cloud to construct a surface to approximate itself. The vertices on the model do not necessarily come from the input point cloud. The corresponding algorithms include Poisson distribution. Due to the limitations of the sensor itself or the influence of the external environment, there are holes in the model after surface reconstruction. In order to better restore the true appearance of the object, the holes on the model need to be repaired.

[0003] The existing methods for repairing holes in three-dimensional models can be divided into two categories: fully automatic and semi-automatic. The biggest difference between fully automatic and semi-automatic methods is reflected in the acquisition of hole positions. For fully automatic methods: first, according to artificial agreements, all hole boundaries that meet the conditions are found from the input model. Secondly, surface fitting or other methods are used to automatically repair the identified holes. However, due to the different types of holes, the automatic recognition method may result in misidentification or omissions, and the accuracy of the subsequent repair process is difficult to guarantee. When errors occur, the parameters need to be adjusted and recalculated, and the processing efficiency will also be reduced. For semi-automatic methods, most of them use human eye recognition to directly fill in points at the holes, and then complete the surface reconstruction work based on the filling results. However, if manual filling is used for reconstruction, the efficiency is low and the labor cost is high.

[0004] With the development of science and technology, methods for repairing holes in three-dimensional models continue to emerge, but ensuring the accuracy of repairs while reducing human intervention remains a problem to be solved. Summary of the invention

[0005] The present application provides a semi-automatic cavity repair method and device with adaptive density, thereby ensuring repair accuracy and improving repair efficiency while reducing manual intervention.

[0006] The embodiment of the present application provides a semi-automatic hole repair method with adaptive density, which realizes human-computer interaction based on a third-party open source rendering engine visualization toolkit (VTK), including the following steps:

[0007] S1: Model input:

[0008] Add the model to be processed to the rendering scene and parse it; the parsing is to store the vertices in the model into a point set V and the triangles into a triangle set T;

[0009] S2: Boundary point extraction:

[0010] Obtaining the empty boundary points within the user's lasso selection range includes the following sub-steps:

[0011] S2.1: Extract the front face patches located at the edge of the hole from the user-selected area;

[0012] S2.2: extracting the boundary edges in the front face patch, and according to the set V, extracting the coordinate information of the corresponding boundary points and storing them in the set M;

[0013] S3: Surface fitting and interpolation:

[0014] S3.1: Modeling the points in the set M to obtain a surface model;

[0015] S3.2: performing point cloud interpolation according to the surface model to obtain a plurality of interpolation points and storing them in set I;

[0016] S4: Interpolation point screening: Interpolation points inside the hole are screened out from the set I and stored in the set I final ;

[0017] S5: Surface reconstruction: The set I final Merge with the set M as the input point cloud for surface reconstruction, and obtain the hole area triangle patch set New_Tris;

[0018] S6: Model Fusion:

[0019] The hole area triangle patch set New_Tris and the original model to be processed are merged into a whole.

[0020] Based on the above embodiments, the present application can be further improved as follows:

[0021] In one embodiment of the present application, the data structure of the set V includes an int type, which is used to record the triangle number to which each point belongs, and 3 double types, which are used to record the three-dimensional coordinate information of each point; the data structure of the set T includes 3 int types, which are used to record the index values ​​of the three vertices of the current triangle face in the set V.

[0022] In one embodiment of the present application, in step S2.1, the steps of extracting the front face patch located at the edge of the hole from the framed area are as follows:

[0023] S2.1.1: Starting from the viewpoint, only the patches that intersect with the ray emitted from the viewpoint for the first time are retained;

[0024] S2.1.2: For the patches retained in S2.1.1, sort their vertices according to the Euclidean distance from the viewpoint;

[0025] S2.1.3: For the point cloud after sorting in S2.1.2, filter the non-frontal faces according to the spatial distribution characteristics of the point cloud.

[0026] In one embodiment of the present application, in step S3.1, a quadratic polynomial is used to model the points in the set M, and the parameter model of the quadratic polynomial is shown in formula (1):

[0027] z estimate =a0*x 2 +a1*y 2 +a2*x*y+a3*x+a4*y+a5 (1);

[0028] In formula (1), a0, a1, a2, a3, a4, and a5 are the parameters to be determined, x and y represent the plane coordinates of the input point, and Z estimate Represents the estimated value of the point in the Z direction.

[0029] In one embodiment of the present application, in step S3.2, the point cloud interpolation step is as follows:

[0030] According to the spatial range of the set M, three-dimensional gridding is performed according to formula (3); the center of gravity of the grid is substituted into the surface model equation to obtain Z insert , by judging Z insert -Z central Whether it is less than the set threshold value is determined to determine whether to retain the current interpolation point. The retained interpolation point is stored in set I. Formula (3) is as follows:

[0031]

[0032] In formula (3), R, C, and H represent the number of grids divided in the X, Y, and Z directions respectively; Mminx 、M maxx 、M miny 、M maxy 、M minz 、M maxz They represent the minimum and maximum values ​​in the set M in the X, Y, and Z directions respectively, and step represents the length of the grid side.

[0033] In one embodiment of the present application, in step S4, the step of screening out the interpolation points inside the hole is as follows:

[0034] S4.1: Calculate the centroid of the set M and record it as P, and find the triangle facet from the set C that is closest to P in terms of Euclidean distance, and record it as close_plane;

[0035] S4.2: Project the set of M points onto the close_plane and record it as M pro , calculate M pro The center of gravity is denoted as P pro , M pro Sort in order of front to back;

[0036] S4.3: Extract the point cloud inside the cavity using the ray method.

[0037] In one embodiment of the present application, in step S4.3, extracting the point cloud inside the cavity is specifically as follows:

[0038] S4.3.1: Project the set I onto close_plane and record it as I pro ; According to formula (5), the sorted M pro ,I pro Rotate to a plane parallel to XOY and record it as M pro (xoy),I pro (xoy); Formula (5) is as follows:

[0039]

[0040] In formula (5), V axis , θ represent the rotation axis and rotation angle required for rotation respectively. Represents the normal vector of the XOY plane (0, 0, 1);

[0041] S4.3.2: Traverse M in sequence pro (xoy) constructs the edge of a polygon by constructing two points on it;

[0042] S4.3.3: Select interpolation point: Use the ray method to determine I according to the constructed polygon pro(xoy) which points are inside the hole, and find the interpolation points inside the hole in the set I through the index relationship, and store the screening results in I final .

[0043] In one of the embodiments of the present application, the hole repairing method further includes step S7: the user confirms whether to save the repaired model. If not, the hole area triangle patch set New_Tris is deleted, and the process returns to step S2 to repair the model again.

[0044] The embodiment of the present application further provides a semi-automatic cavity repair device with adaptive density, according to the above-mentioned semi-automatic cavity repair method, comprising:

[0045] A model input module is used to add the model to be processed to the rendering scene and parse it; the parsing is to store the vertices in the model into a point set V and the triangles into a triangle set T;

[0046] The boundary point extraction module is used to obtain the empty boundary points within the user's lasso selection range, extract the coordinate information of the corresponding boundary points and store them in the set M;

[0047] A surface fitting and interpolation module is used to model the points in the set M to obtain a surface model; perform point cloud interpolation according to the surface model to obtain a plurality of interpolation points and store them in the set I;

[0048] An interpolation point screening module is used to screen out interpolation points inside the hole from the set I and store them in the set I final ;

[0049] A surface reconstruction module is used to transform the set I final Merge with the set M as the input point cloud for surface reconstruction, and obtain the triangle face set New_Tris in the hole area;

[0050] The model fusion module is used to merge the hole area triangle patch set New_Tris and the model to be processed into a whole.

[0051] An embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the semi-automatic hole repair method in any of the aforementioned embodiments when executing the program. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0053] Figure 1 This is a flowchart of the steps of a semi-automatic cavity repair method with adaptive density in an embodiment of the present application;

[0054] Figure 2 This is a schematic diagram of distinguishing the front and non-front panels in the embodiment of the present application;

[0055] Figure 3 This is a schematic diagram of the ray method in the embodiment of this application;

[0056] Figure 4 This is a schematic diagram of the boundary point extraction results in the embodiment of this application;

[0057] Figure 5 This is a schematic diagram of the interpolation point extraction results in the embodiment of the present application;

[0058] Figure 6 This is a schematic diagram of the patch repair results in the embodiment of the present application;

[0059] Figure 7 This is a schematic diagram of the model fusion results in the embodiment of this application;

[0060] Figure 8 This is a structural block diagram of a semi-automatic cavity repair device with adaptive density in an embodiment of the present application;

[0061] Fig. 9 This is a structural block diagram of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0062] The present invention is further explained below in conjunction with specific implementation methods. It should be understood that these implementation methods are only used to illustrate the present invention and are not used to limit the scope of the present invention. After reading the present invention, various equivalent forms of modifications to the present invention by those skilled in the art all fall within the scope defined by the claims attached to this application.

[0063] In the description of the present invention, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in the present invention and the features of different embodiments or examples without contradiction.

[0064] Example:

[0065] like Figure 1 As shown, a semi-automatic hole repair method with adaptive density is implemented based on a third-party open source rendering engine visualization toolkit (VTK) to achieve human-computer interaction, including the following steps:

[0066] S1: Model input:

[0067] The model to be processed is added to the rendering scene and parsed; the parsing here is to store the vertices in the model to be processed into the point set V and the triangles into the triangle set T.

[0068] Among them, the data structure of the point set V contains an int type to record the triangle number to which the point belongs, and 3 double types to record the three-dimensional coordinate information of the point; the data structure of the set T contains 3 int types to record the index values ​​of the three vertices of the current triangle face in the set V.

[0069] S2: Boundary point extraction:

[0070] Obtaining the empty boundary points within the user's lasso selection range includes the following sub-steps:

[0071] S2.1: Extract the target triangular facets located at the edge of the hole from the selected area. The target triangular facets are the front facets connected to the edge of the hole. When directly selecting faces using the lasso method in VTK, the faces inside and on the back may be included. This part of information is redundant. In order to extract only the front facets connected to the edge of the hole, the implementation steps are as follows:

[0072] S2.1.1: Starting from the viewpoint, only the patches that intersect with the ray emitted from the viewpoint for the first time are retained;

[0073] S2.1.2: For the patches retained in S2.1.1, sort their vertices according to the Euclidean distance from the viewpoint;

[0074] S2.1.3: For the point cloud after sorting in S2.1.2, filter the non-frontal faces according to the spatial distribution characteristics of the point cloud.

[0075] The schematic diagram of filtering non-frontal faces based on the spatial distribution characteristics of the point cloud is as follows Figure 2 shown.

[0076] Figure 2 In the figure, View is the current viewpoint position, the arrow direction indicates the current observation direction; F1, F2, and F3 respectively represent the sets of triangles located on the front, middle, and back sides within the selection range; the blue sphere represents the Euclidean distance of the triangle vertex from the viewpoint, in meters.

[0077] After following the above steps, the faces that meet the conditions within the selected range are stored in set C.

[0078] S2.2: Extract the boundary edges of the target triangle patch, combine the aforementioned point set V, extract the coordinate information of the boundary points and store them in the set M;

[0079] Boundary edges are determined as follows: if two triangles are connected, then they must have a common edge between them. Conversely, boundary edges do not have a shared edge.

[0080] According to the above judgment rules, the boundary edges in set C in step S2.1 are extracted and stored in set E. The coordinate information of the corresponding boundary points is extracted from set E and set V in step S1 and stored in set M.

[0081] S3: Surface fitting and interpolation:

[0082] S3.1: Quadratic polynomial fitting: Use a quadratic polynomial model to model the points in the set M and obtain a surface model;

[0083] This step is used to determine the surface equation of the local area. The hole area is not always flat, and directly using plane fitting for modeling may deviate from the actual situation. In this regard, in order to make the repaired model smoother, the present invention uses a quadratic polynomial model to model the points in the local area. The quadratic polynomial parameter model is defined as formula (1).

[0084] z estimate =a0*x 2 +a1*y 2 +a2*x*y+a3*x+a4*y+a5 (1);

[0085] In formula (1), a0, a1, a2, a3, a4, and a5 are the parameters to be determined, x and y represent the plane coordinates of the input point, and Z estimate Represents the estimated value of the point in the Z direction.

[0086] Calculate the difference between the observed value and the estimated value and solve it using the least squares method. The least squares algorithm is shown in formula (2):

[0087]

[0088] In formula (2), z measure Indicates the observation value of the current point in the Z direction.

[0089] Specifically, the following steps are also included: constructing a kd_tree for set M, extracting the nearest 10 points for each point in set M by using a nearest neighbor search and storing them in set N, and performing a deduplication operation on set N after the search is completed. The surface model and average point cloud density of set N are calculated using the least squares algorithm. The average point cloud density is used as a threshold to determine the subsequent point cloud interpolation interval, and to achieve adaptive point cloud density in the hole area.

[0090] S3.2: Point cloud interpolation: Perform point cloud interpolation based on the surface model calculated in step S1 to obtain multiple interpolation points and store them in set I; the specific steps are:

[0091] First, according to the spatial range of the set M, a three-dimensional grid is formed according to formula (3); secondly, the center of gravity of the grid is substituted into the surface equation to obtain Z insert , by judging Z insert -Z central Is it less than the set threshold, so as to decide whether to keep the current interpolation point, where Z insert is the Z value of the point calculated by the surface equation; Z central is the Z value of the specified grid center point. After the above processing, the interpolation points that meet the screening conditions are stored in set I.

[0092]

[0093] In formula (3), R, C, and H represent the number of grids divided in the X, Y, and Z directions respectively; int represents the rounding operation; M minx 、M maxx 、M miny 、M maxy 、M minz 、M maxz They represent the minimum and maximum values ​​in the X, Y, and Z directions in the set M respectively. Step represents the length of the grid side in m.

[0094] S4: Interpolation point screening: Screen the interpolation points inside the hole from set I and store them in set I final ;

[0095] The interpolation points obtained in step S3 are not only distributed inside the hole. In order to facilitate the fusion of subsequent models, the interpolation points outside the hole need to be deleted here. The specific sub-steps include:

[0096] S4.1: Calculate the projection face. Calculate the centroid (average value of coordinates) of the set M and record it as P. Find the triangle face with the closest Euclidean distance to P from the set C in step S2 according to the Euclidean distance and record it as close_plane.

[0097] S4.2: Point cloud sorting. Project the points of the set M onto the close_plane and record them as Mpro , calculate M pro The center of gravity is denoted as P pro , M pro Sort in the order of front to back. The sorting process is as follows:

[0098] S4.2.1: From M pro Take any point in the middle and denote it as P1. According to formula (4), calculate P1 and P pro The plane equation of .

[0099]

[0100] In formula (4) represents the normal vector of the plane to be found, Respectively represent P1 to P pro The vector and the normal vector of close_plane; P1[x], P1[y], P1[z] represent the x, y, and z values ​​of P1 respectively. Respectively The components in the x, y, and z directions, D represents the distance from the origin to the plane;

[0101] S4.2.2: Sort by the plane equations calculated in S4.2.1. pro Divide into two parts, called part1 and part2. For part1, take out one point and connect P pro Calculate its The angles between them are stored in set A. Sort A from small to large. If the angles are consistent, sort them according to the angle between the point and P. pro The distance between A and B is sorted; for part 2, the angle is calculated in the same way and stored in set B. The difference is that B is sorted from large to small. Combining A and B can complete M pro of order.

[0102] S4.3: Ray method to extract the point cloud inside the hole. The ray method is a method to determine whether a point is inside or outside a polygon. Figure 3 As shown, the principle of the ray method is to distinguish the target point by judging the parity of the number of intersections between the ray and the polygon in the X-axis direction.

[0103] The specific steps of this part are as follows.

[0104] S4.3.1: Point cloud conversion: First, project the set I onto the close_plane and record it as I pro ; Secondly, according to formula (5), the sorted M pro ,I pro Rotate to a plane parallel to XOY and record it as M pro (xoy),Ipro (xoy).

[0105]

[0106] In formula (5), V axis , θ represent the rotation axis and rotation angle required for rotation respectively. represents the normal vector of the XOY plane (0, 0, 1), Keep the same as in step S4.2.

[0107] S4.3.2: Construct polygon: traverse M in sequence pro (xoy) Two points on the polygon form the edge.

[0108] S4.3.3: Select interpolation points: The polygon constructed according to S4.3.2 is judged by the ray method described above. pro Which points in (xoy) are inside the hole, and find the interpolation points in I that are inside the hole through the index relationship. Store the screening results in I final .

[0109] S5: Surface reconstruction: Set I final Merge with the set M as the input point cloud and send it to the CGAL forward surface reconstruction pipeline. After the algorithm is executed, the triangle patch set (New_Tris) of the hole area can be obtained.

[0110] There are numerous surface reconstruction algorithms, some of which use point cloud normal information, and some use isosurface fitting. Considering factors such as execution efficiency and integrity, the present invention selects the forward algorithm in the open source library CGAL for construction. The specific implementation process is: replace step 4 with I final Merge it with M in step 2 as the input point cloud and send it to the CGAL forward normal surface reconstruction pipeline. After the algorithm is executed, the triangle patch set New_Tris in the hole area can be obtained.

[0111] S6: Model Fusion:

[0112] The hole area triangle patch set New_Tris is merged with the original model into a whole.

[0113] S7: The user confirms whether to save the repaired model: manually observe the repair model effect. If not satisfied, do not save and delete the repaired face (the triangular face of the hole area in S5). If satisfied, save the repaired face; and return to step S2 to select and repair other holes. Add human-computer interaction function based on VTK to save and delete the user's current operation, improving the flexibility of processing.

[0114] The following is a preferred embodiment of the semi-automatic cavity repair method of the present invention:

[0115] In this embodiment, a model containing a hole is selected, and the point density is about 36 (points) / m 2 , add it to the VTK rendering scene and parse it. The boundary points obtained after processing in the same way as the model boundary points extracted in step S2 are as follows Figure 4 shown. Figure 4 The white grid in the middle represents the input model, and the red dots represent the extracted boundary points.

[0116] Construct a kd_tree for the set M obtained in S2. For each point in the set M, use the nearest neighbor search to extract the 10 nearest points and store them in the set N. After the search is completed, perform a deduplication operation on the set N. Calculate the surface model and average point cloud density of the set N according to the least squares algorithm introduced in step S3. Use the average point cloud density as the threshold to determine the interpolation interval. After calculation, the interpolation interval in the data is about 0.1167m. According to the interpolation point screening rules in step S4, the interpolation points finally retained are as follows Figure 5 shown. Figure 5 The red points represent the hole boundary points, and the blue points represent the interpolated points after filtering.

[0117] Using the surface reconstruction algorithm introduced in step S5, the patch information obtained is as follows: Figure 6 shown. Figure 6 The medium white color represents the patched model, and the blue dots represent the points at the boundary of the model.

[0118] from Figure 4 , Figure 6 It can be seen from the figure that the boundary points of the holes in the original model are consistent with the boundary points on the patched model. The two are merged, and the final result is as follows: Figure 7 shown. Figure 7 In the figure, red represents the patched model and white represents the original input model.

[0119] from Figure 7 It can be seen that the hole repair algorithm can complete the hole repair and seamless fusion with the original model at the same time.

[0120] The technical solutions in the above embodiments of the present application have at least the following technical effects or advantages:

[0121] 1. This hole repair method uses the spatial distribution characteristics of the point cloud to filter out possible non-frontal faces during the selection stage to avoid the risk of incorrect repair; it combines the polygon cutting algorithm to accurately extract the interpolated point cloud within the hole range to facilitate subsequent fusion between models; it ensures repair accuracy and improves repair efficiency while reducing manual intervention.

[0122] 2. By adopting this cavity repair method, the cavity can be repaired and seamlessly integrated with the original model at the same time, which improves the repair effect and effectively improves the repair efficiency.

[0123] 3. This hole repair method adds human-computer interaction function based on VTK, so as to save and delete the user's current operations and improve the processing flexibility.

[0124] Figure 8 The structural block diagram of a semi-automatic cavity repair device with adaptive density provided by an embodiment of the present invention adopts the above-mentioned semi-automatic cavity repair method, and the device includes:

[0125] A model input module is used to add the model to be processed to the rendering scene and parse it; the parsing is to store the vertices in the model into a point set V and the triangles into a triangle set T;

[0126] The boundary point extraction module is used to obtain the empty boundary points within the user's lasso selection range, extract the coordinate information of the corresponding boundary points and store them in the set M;

[0127] A surface fitting and interpolation module is used to model the points in the set M to obtain a surface model; perform point cloud interpolation according to the surface model to obtain a plurality of interpolation points and store them in the set I;

[0128] An interpolation point screening module is used to screen out interpolation points inside the hole from the set I and store them in the set I final ;

[0129] A surface reconstruction module is used to transform the set I final Merge with the set M as the input point cloud for surface reconstruction, and obtain the triangle face set New_Tris in the hole area;

[0130] A model fusion module, used to merge the hole area triangle patch set New_Tris and the model to be processed into a whole;

[0131] The confirmation module is used for the user to confirm whether the repaired model should be saved. If not, the triangle patch set New_Tris in the hole area will be deleted and the model will be returned for re-repair.

[0132] Figure 8 The functions of each module in the embodiment correspond to the contents in the corresponding method embodiment and will not be repeated here.

[0133] Fig. 9The structural schematic diagram of an electronic device 90 provided in an embodiment of the present invention is shown, and the electronic device 90 includes at least one processor 901 (such as a CPU), at least one input and output interface 904, a memory 902, and at least one communication bus 903, which is used to realize the connection and communication between these components. At least one processor 901 is used to execute computer instructions stored in the memory 902, so that the at least one processor 901 can execute any embodiment of the aforementioned semi-automatic hole repair method. The memory 902 is a non-transitory memory, which may include a volatile memory, such as a high-speed random access memory (RAM: Random Access Memory), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. The communication connection with at least one other device or unit is realized through at least one input and output interface 904 (which may be a wired or wireless communication interface).

[0134] In some implementations, the memory 902 stores a program 9021, and the processor 901 executes the program 9021 to execute the content of any of the aforementioned table partitioning method embodiments.

[0135] The electronic device may exist in a variety of forms, including but not limited to:

[0136] (1) Mobile communication equipment: This type of equipment is characterized by having mobile communication functions and its main purpose is to provide voice and data communications. This type of terminal includes: smart phones, multimedia phones, functional phones, and low-end phones.

[0137] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, etc.

[0138] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players, handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0139] (4) Specific server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0140] (5) Other electronic devices with data interaction functions.

[0141] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A semi-automatic cavity repair method with adaptive density, characterized in that: The following steps are involved: S1: adding the model to be processed to the rendering scene and parsing it; the parsing is storing the vertices in the model into a point set V and the triangles into a triangle set T; S2: Obtain the boundary points of the holes within the user's lasso selection range, which specifically includes the following sub-steps: S2.1: Extract the front face patches located at the edge of the hole from the user-selected area; S2.2: extracting the boundary edges in the front face patch, and according to the set V, extracting the coordinate information of the corresponding boundary points and storing them in the set M; S3: Surface fitting and interpolation, including: S3.1: Modeling the points in the set M to obtain a surface model; S3.2: performing point cloud interpolation according to the surface model to obtain a plurality of interpolation points and storing them in set I; S4: Filter out interpolation points inside the hole from the set I and store them in the set I final ; S5: The set I final Merge with the set M as the input point cloud for surface reconstruction, and obtain the hole area triangle patch set New_Tris; S6: Merge the hole area triangle patch set New_Tris with the original model to be processed.

2. The semi-automatic cavity repair method according to claim 1, characterized in that: The data structure of the set V contains an int type, which is used to record the triangle number to which each point belongs, and 3 double types, which are used to record the three-dimensional coordinate information of each point; the data structure of the set T contains 3 int types, which are used to record the index values ​​of the three vertices of the current triangle face in the set V.

3. The semi-automatic cavity repair method according to claim 1, characterized in that: In step S2.1, the steps of extracting the front face patch located at the edge of the hole from the user-selected area are as follows: S2.1.1: Starting from the viewpoint, only the patches that intersect with the ray emitted from the viewpoint for the first time are retained; S2.1.2: For the patches retained in S2.1.1, sort their vertices according to the Euclidean distance from the viewpoint; S2.1.3: For the point cloud after sorting in S2.1.2, select the frontal face patches according to the spatial distribution characteristics of the point cloud.

4. The semi-automatic cavity repair method according to claim 1, characterized in that: In step S3.1, a quadratic polynomial is used to model the points in the set M, and the parameter model of the quadratic polynomial is shown in formula (1): <h2 style=";text-align:left;direction:ltr">z<h2 style=";text-align:left;direction:ltr"> estimate <h2 style=";text-align:left;direction:ltr"> =a0*x<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +a1*y<h2 style=";text-align:left;direction:ltr"> 2 <h2 style=";text-align:left;direction:ltr"> +a2*x*y+a3*x+a4*y+a5 (1); In formula (1), a0, a1, a2, a3, a4, and a5 are the parameters to be determined, x and y represent the plane coordinates of the input point, and Z estimate Represents the estimated value of the point in the Z direction.

5. The semi-automatic cavity repair method according to claim 4, characterized in that: In step S3.2, the point cloud interpolation steps are as follows: According to the spatial range of the set M, three-dimensional gridding is performed according to formula (3); the center of gravity of the grid is substituted into the surface model equation to obtain Z insert , by judging Z insert -Z central Whether it is less than the set threshold value is determined to determine whether to retain the current interpolation point. The retained interpolation point is stored in set I. Formula (3) is as follows: In formula (3), R, C, and H represent the number of grids divided in the X, Y, and Z directions respectively; M minx 、M maxx 、M miny 、M maxy 、M minz 、M maxz They represent the minimum and maximum values ​​in the set M in the X, Y, and Z directions respectively, and step represents the length of the grid side.

6. The semi-automatic cavity repair method according to claim 1, characterized in that: In step S4, the step of screening out the interpolation points inside the hole is as follows: S4.1: Calculate the centroid of the set M and record it as P, and find the triangle facet from the set C that is closest to P in terms of Euclidean distance, and record it as close_plane; S4.2: Project the set of M points onto the close_plane and record it as M pro , calculate M pro The center of gravity is denoted as P pro , M pro Sort in order of front to back; S4.3: Extract the point cloud inside the cavity using the ray method.

7. The semi-automatic cavity repair method according to claim 6, characterized in that: In step S4.3, extracting the point cloud inside the cavity is specifically as follows: S4.3.1: Project the set I onto close_plane and record it as I pro ; According to formula (5), the sorted M pro ,I pro Rotate to a plane parallel to XOY and record it as M pro (xoy),I pro (xoy); Formula (5) is as follows: In formula (5), V axis , θ represent the rotation axis and rotation angle required for rotation respectively. represents the normal vector of the XOY plane (0, 0, 1), represents the normal vector of the plane to be found; S4.3.2: Traverse M in sequence pro (xoy) constructs the edge of a polygon by constructing two points on it; S4.3.3: Select interpolation point: Use the ray method to determine I according to the constructed polygon pro (xoy) which points are inside the hole, and find the interpolation points inside the hole in the set I through the index relationship, and store the screening results in I final .

8. The semi-automatic cavity repair method according to claim 1, characterized in that: The hole repair method further includes step S7: the user confirms whether to save the repair model. If not, the hole area triangle patch set New_Tris is deleted, and the process returns to step S2 to repair the model again.

9. A semi-automatic cavity repair device with adaptive density, characterized in that: The semi-automatic cavity repair method according to any one of claims 1 to 8 comprises: A model input module is used to add the model to be processed to the rendering scene and parse it; the parsing is to store the vertices in the model into a point set V and the triangles into a triangle set T; The boundary point extraction module is used to obtain the empty boundary points within the user's lasso selection range, extract the coordinate information of the corresponding boundary points and store them in the set M; A surface fitting and interpolation module is used to model the points in the set M to obtain a surface model; perform point cloud interpolation according to the surface model to obtain a plurality of interpolation points and store them in the set I; An interpolation point screening module is used to screen out interpolation points inside the hole from the set I and store them in the set I final ; A surface reconstruction module is used to transform the set I final Merge with the set M as the input point cloud for surface reconstruction, and obtain the triangle face set New_Tris in the hole area; The model fusion module is used to merge the hole area triangle patch set New_Tris and the model to be processed into a whole.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the semi-automatic hole repairing method according to any one of claims 1 to 8 are implemented.

Images