3D reconstruction of the structure of real scenes using open surfaces.

Abstract

To provide a computer-implemented method for 3D reconstruction of a structure of a real scene.SOLUTION: A method provides a first open triangulated surface that is a set of triangles of a tetrahedral meshing of a 3D point cloud. The 3D point cloud represents at least a part of a structure. The method further determines a second open triangulated surface. The second open triangulated surface represents a skin of a portion of the structure. The method also explores open triangulated surface candidates each being a set of triangles of the tetrahedral meshing and penalizes a high rank of the open triangulated surface candidates according to a lexicographic order. The lexicographic order orders a first open triangulated surface candidate having first triangles which are ordered according to a decreasing rank of a triangle order, relative to a second open triangulated surface candidate having second triangles which are ordered according to the decreasing rank of the triangle order.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of computer programs and systems, and more particularly to methods, systems and programs for 3D reconstruction of structures in real scenes. [Background technology]

[0002] The market offers numerous systems and programs for designing, engineering, and manufacturing objects. CAD is an acronym for Computer-Aided Design, which refers to software solutions for designing objects, for example. CAE is an acronym for Computer-Aided Engineering, which refers to software solutions for simulating the physical behavior of future products, for example. CAM is an acronym for Computer-Aided Manufacturing, which refers to software solutions for defining manufacturing processes and operations, for example. In such computer-aided design systems, graphical user interfaces play a key role in the efficiency of the technology. These technologies can be incorporated into product lifecycle management (PLM) systems. PLM is a business strategy that helps companies share product data, apply common processes, and leverage corporate knowledge to develop products from concept to life across the extended enterprise. Dassault Systèmes' PLM solutions (under the trademarks CATIA, ENOVIA and DELMIA) provide an Engineering Hub that organizes product engineering knowledge, a Manufacturing Hub that manages manufacturing engineering knowledge, and an Enterprise Hub that enables enterprise integration and connection to the Engineering and Manufacturing Hubs. The combined system provides an open object model that links products, processes and resources to enable dynamic, knowledge-based product creation and decision support that drives the optimization of product definition, manufacturing preparation, production and service. Summary of the Invention [Problem to be solved by the invention]

[0003] In these and other contexts, 3D reconstruction of the structure of real scenes is gaining widespread importance.

[0004] Existing methods for 3D reconstruction of the structure of real scenes have several drawbacks.

[0005] In this regard, there is a need for improved methods for reconstructing the structure of real scenes in 3D. [Means for solving the problem]

[0006] Accordingly, a computer-implemented method for reconstructing a structure of a real scene in 3D is provided. The method includes providing a first open triangulation surface. The first open triangulation surface is a set of triangles of a tetrahedral mesh of a 3D point cloud. The 3D point cloud represents at least a portion of the structure. The method further includes determining a second open triangulation surface. The second open triangulation surface represents the skin of the portion of the structure. The determining step includes searching for open triangulation surface candidates, each of which is a set of triangles of the tetrahedral mesh. The determining step includes penalizing open triangulation surface candidates that are higher ranked in a lexicographical order. The lexicographical order is based on triangle order. The lexicographical order orders first open triangulation surface candidates having first triangles ordered according to descending triangle order relative to second open triangulation surface candidates having second triangles ordered according to descending triangle order. The triangle order penalizes triangle size. The determined second open triangulation surface violates the labeling consistency of the tetrahedral mesh with two given labels. A triangle respects the labeling consistency if it belongs to the first open triangulation surface and separates two tetrahedrons with different labels, or if it does not belong to the first open triangulation surface and separates two tetrahedrons with the same label.

[0007] The method may include one or more of the following: A tetrahedral mesh has the following properties: The tetrahedrons of the tetrahedral mesh join to form the convex hull of the points of the point cloud; · The intersection of any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron; The triangle ordering penalizes high values ​​of the radius of the minimum enclosing circle for each triangle; The triangle ordering further penalizes smaller values ​​of the circumscribing circle radius for the first and second triangles with the same minimum enclosing circle; A tetrahedral mesh is a regular triangulation; Searching and giving penalties: Visiting triangles of a tetrahedral mesh according to descending triangle order; · It involves discarding visited triangles that are higher ranked in the triangle ordering as long as labeling consistency is maintained; Discarding involves, for each visited triangle, if the triangle violates labeling consistency, either giving the triangle a reward or adjusting the labeling to respect consistency; The destruction is further done for each triangle visited: · Rewarding a triangle if it separates two tetrahedrons in the same connected component and violates labeling consistency; If a triangle separates two tetrahedra in different connected components: When a triangle violates labeling consistency, we merge the connected components by switching the label of one of the connected components. ··Including merging connected components when triangles respect labeling consistency; Providing a first open triangular surface includes: · Providing one or more loops in the tetrahedral mesh; · determining an open triangulated surface within a tetrahedral mesh having one or more loops as a boundary; The structure includes at least one corner structure; The 3D point cloud is created by photogrammetry, laser scanning, LIDAR measurements, RGB D measurements, and / or medical or industrial tomography, and / or; The 3D point cloud represents a portion of a closed shape and / or the ratio of the number of points in the 3D point cloud to the computer memory size is greater than a predetermined threshold.

[0008] Additionally, a computer program comprising instructions for carrying out the method is provided.

[0009] Further provided is a computer-readable storage medium having the computer program recorded thereon.

[0010] Further provided is a computer comprising a processor coupled to a memory having the computer program stored thereon. [Brief explanation of the drawings]

[0011] Embodiments of the invention will now be described by way of non-limiting examples and with reference to the accompanying drawings, in which: [Figure 1] 1 shows a flowchart of an example of the method. [Figure 2] 1 illustrates the method. [Figure 3] The method is illustrated below. [Figure 4] The method is illustrated below. [Figure 5] The method is illustrated below. [Figure 6] The method is illustrated below. [Figure 7] The method is illustrated below. [Figure 8] The method is illustrated below. [Figure 9] The method is illustrated below. [Figure 10] The method is illustrated below. [Figure 11] The method is illustrated below. [Figure 12] The method is illustrated below. [Figure 13] The method is illustrated below. [Figure 14] The method is illustrated below. [Figure 15] The method is illustrated below. [Figure 16] The method is illustrated below. [Figure 17] The method is illustrated below. [Figure 18] The method is illustrated below. [Figure 19] The method is illustrated below. [Figure 20] The method is illustrated below. [Figure 21] The method is illustrated below. [Figure 22] The method is illustrated below. [Figure 23] The method is illustrated below. [Figure 24] The method is illustrated below. [Figure 25] The method is illustrated below. [Figure 26] The method is illustrated below. [Figure 27] The method is illustrated below. [Figure 28] The method is illustrated below. [Figure 29] The method is illustrated below. [Figure 30] The method is illustrated below. [Figure 31] The method is illustrated below. [Figure 32] The method is illustrated below. [Figure 33] The method is illustrated below. [Figure 34] The method is illustrated below. [Figure 35] The method is illustrated below. [Figure 36] The method is illustrated below. [Figure 37] The method is illustrated below. [Figure 38] The method is illustrated below. [Figure 39] The method is illustrated below. [Figure 40] The method is illustrated below. [Figure 41] The method is illustrated below. [Figure 42]The method is illustrated below. [Figure 43] 1 shows an example of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0012] Referring to the flowchart of FIG. 1, a computer-implemented method for 3D reconstruction of a structure of a real scene is proposed. The method includes steps S10-S40 of providing a first open triangulation surface. The first open triangulation surface is a set of triangles of a tetrahedral mesh of a 3D point cloud. The 3D point cloud represents at least a portion of the structure. The method further includes step S50 of determining a second open triangulation surface. The second open triangulation surface represents the skin of the portion of the structure. In the determining step S50, candidate open triangulation surfaces are searched, each of which is a set of triangles of a tetrahedral mesh. In the determining step S50, a penalty is assigned to candidate open triangulation surfaces that are higher in lexicographic order. The lexicographic order is based on the triangle order. The lexicographic order orders candidate first open triangulation surfaces with first triangles ordered according to descending triangle order relative to candidate second open triangulation surfaces with second triangles ordered according to descending triangle order. The triangle order penalizes the triangle size. The determined second open triangulation surface violates the labeling consistency of the tetrahedral mesh with two given labels. A triangle respects the labeling consistency if it belongs to the first open triangulation surface and separates two tetrahedrons with different labels, or if it does not belong to the first open triangulation surface and separates two tetrahedrons with the same label.

[0013] This constitutes an improved method for reconstructing the structure of real scenes in 3D.

[0014] In particular, the method allows for determining an open triangulation surface (i.e., a second open triangulation surface) that represents the skin of a portion of a structure of a real scene. The method thus performs a reconstruction of a portion of a 3D structure of a real scene. Furthermore, the method performs a reconstruction of a portion of a 3D structure of a real scene without requiring assumptions about the 3D point cloud and / or smoothness of the structure. Therefore, the method is robust, e.g., able to handle a variety of real scene structures. The method may also process irregular point clouds that represent irregular structures, such as corner structures. The method is also ergonomically friendly, since it requires little (e.g., no) manual adjustment of parameters. In particular, the search and penalization may be performed automatically.

[0015] The method determines a second open triangulation surface representing the skin of a portion of the structure by searching for candidate open triangulation surfaces and rewarding (e.g., selecting) candidates that result in the determined open triangulation surface. The method rewards / selects candidates based on two constraints. The first constraint is that the candidate must not have too high a rank in the lexicographic order. Otherwise, it is penalized (i.e., discarded) by the determining step S50. In other words, the candidate must minimize or tend to minimize the lexicographic order among the candidates. The second constraint is that the selected candidate must violate labeling consistency (i.e., all triangles of the candidate must violate labeling consistency). In other words, the method finds a candidate open triangulation surface (i.e., a set of triangles of a tetrahedral mesh) that violates labeling consistency but is least penalized by its lexicographic rank among the candidates. In other words, the method determines open triangulation surfaces that violate labeling consistency but minimize, or at least tend to minimize, the lexicographic order between open triangulation surfaces that violate labeling consistency. In other words, the method tends to minimize the lexicographic order between a set of open triangulation surfaces that violate labeling consistency.

[0016] Here, the constraint that the determined second open triangulation surface must violate labeling consistency is a continuation of the constraint that the determined second open triangulation surface must have the same boundary as the first open triangulation surface. In other words, by determining an open triangulation surface that violates labeling consistency, the method determines an open triangulation surface that has the same boundary as the first open triangulation surface. By determining an open triangulation surface that violates labeling consistency and that minimizes, or at least tends to minimize, the lexicographic order, the method actually determines an open triangulation surface that has the same boundary as the first open triangulation surface and minimizes, or at least tends to minimize, the lexicographic order between open triangulation surfaces that have the same boundary as the first open triangulation surface.

[0017] Considering that, as will be explained later, labeling consistency is defined by the constraint that a triangle respects labeling consistency if it belongs to the first open triangulation surface and separates two tetrahedrons with different labels, or if it does not belong to the first open triangulation surface and separates two tetrahedrons with the same label, the constraint that the determined second open triangulation surface must have the same boundary as the first open surface is actually captured by the constraint that the determined second open triangulation surface must violate labeling consistency, i.e., all of the triangles must violate labeling consistency.

[0018] As described further below, a high lexicographical ranking for a triangulation surface captures the fact that the surface contains large triangles (e.g., large and / or flat triangles) and / or has an excessive amount of such triangles. These triangles represent regions of the 3D point cloud characterized by noise, which may consist of outlier points that may result from, for example, measurement errors of the sensor that acquired the point cloud. That is, as described below, an open triangulation surface that is high lexicographically ranking does not accurately represent the skin of a portion of a structure because the skin must be meshed in a dense portion of the point cloud. Conversely, an open triangulation surface that is low ranked accurately represents the skin. Therefore, based on a first open triangulation surface, the method identifies a second open triangulation surface that has the same boundary but that better represents the skin of the portion of the structure than the first open triangulation surface.

[0019] The present method can be used to 3D reconstruct a real scene that is partially represented by a 3D point cloud. Specifically, physical constraints of the real scene, such as occlusion and / or optical properties, can result in the 3D point cloud only partially representing the structure of the real scene, i.e., representing at least a portion of the structure. The physical constraints can lead to partial and incomplete 3D capture, such as having holes as features. That is, the 3D point cloud may have holes as features where points representing a portion of the structure should be. In other words, the 3D point cloud may represent only a portion of a closed shape, i.e., a portion of the closed shape, rather than the entire closed shape. For example, when a 3D point cloud represents a building, physical constraints can result in only a 3D point cloud that partially represents the building's shape, preventing the complete shape of the building from being captured in 3D as a 3D point cloud. However, even in such cases, the present method enables accurate and hole-free 3D reconstruction. For example, the method may provide a first open triangulation surface that is bounded by a boundary of a portion of the structure represented by the 3D point cloud. Because this is a partial 3D acquisition, this boundary may not correspond to the real-world outer boundary of the structure. That is, the boundary may be the border of a portion of the structure represented by the 3D point cloud. The method may provide any such open triangulation surface as the first open triangulation surface.

[0020] For example, a user may define / mark a boundary as one or more (e.g., one) loops within a tetrahedral mesh. A loop is a set of edges and vertices, where each vertex belongs to an even number of edges. The method may also infer a first open triangulation surface from the one or more loops. This may be done in any manner, including any manner that results in a first open triangulation surface that does not accurately represent the skin, i.e., that is too highly ranked in the lexicographical order. Next, the determining step S50 finds another open triangulation surface (i.e., a second open triangulation surface) that has the same boundary as the first open triangulation surface but tends to minimize the lexicographical order, thereby forming a higher-quality skin of the portion of the structure. In other words, based on a user-provided definition of the boundary of the portion of the structure, the method may determine an open triangulation surface that has the defined portion boundary as its boundary and tends to minimize the lexicographical order, thereby forming an accurate skin of the portion of the structure.

[0021] The method may additionally or alternatively be used to determine the skin of large 3D point clouds, i.e., point clouds featuring a large number of points, such as when the ratio between the number of points in the 3D point cloud and the computer's memory size is greater than a predetermined threshold. This may be the case, for example, when the point cloud represents a large terrain. The size of such point clouds (in terms of number of points) may make it impossible, or at least complicated, for a standard or inexpensive computer to calculate a triangulation surface representing the complete skin of the point cloud within an acceptable time. However, the method may enable processing of such point clouds by considering their partitioning. Specifically, the method may provide a first open triangulation surface that represents only the skin of a boundary portion of a structure represented in the 3D point cloud. This first open triangulation surface may be inferred from one or more computer-provided loops, e.g., each loop bounding such a boundary portion. For example, if the 3D point cloud represents a terrain, the first open triangulation surface may represent the skin of a section of the bounded terrain. The first open triangulation surface may inaccurately represent the skin because it is too high in lexicographic order. The method then determines a second open triangulation surface that has the same boundary as the first open triangulation surface but a lower lexicographic order, thereby accurately representing the skin of the boundary portion. The method may be performed for each of a plurality of boundary portions that divide the structure.

[0022] In other words, the method considers a large structure that can be divided by boundary portions, such as a terrain divided into sections, and determines the respective skins of each of the portions. This avoids determining the entire skin of the structure in a single process, which would require a significant amount of time and / or be infeasible on a standard or inexpensive computer. The method may also define the boundaries of the boundary portions as watertight (e.g., if they provide a corresponding first open triangulation surface or loop). This allows the skin of the structure to be parallelized, i.e., the determined skins of each portion can be processed independently of each other. For example, the method can recalculate one of the skins upon a change in the corresponding portion of the structure without recalculating the other skins. In other words, the method can independently calculate and process the accurate skins of the portions that divide a large structure, such as a terrain section, thereby not only obtaining the accurate skin of the structure, or at least one or more portions of the structure, but also executing within an acceptable time using the resources of a standard or inexpensive computer. The division of a structure into parts, such as the division of a terrain into sections, may be determined by a user or may be determined automatically, for example based on the consideration that for each part the points of the 3D point cloud representing that part must fit into computer memory.

[0023] The method is further described with reference to the flow chart of FIG.

[0024] A real scene refers to a portion of the real world, which may be or include an object or an arrangement of objects. A structure of a real scene refers to a tangible (e.g., tangible), closed physical shape of the real scene, such as a closed layout of materials in the real scene. Thus, a portion of the structure is a part of this closed shape. For example, the real scene may be a building such as a cathedral. In this case, a portion may be a portion of the building, such as the cathedral without a roof. As another example, a real scene is a terrain. A portion of the terrain may be a part of the terrain, such as a parcel / plot. The skin of a portion is the envelope or envelope of that portion, i.e., the material of the structure that forms the contact between the opaque material within the structure and the air and / or material outside. A portion may have a boundary, and the skin does not necessarily form a complete boundary between the outside of the structure and the inside of the structure, but may form only a portion of such a boundary. For example, the skin of a portion of the terrain may represent the boundary between the ground and the air that the terrain forms, but only for the parcel / plot that it bounds. Yet another example is the skin of a portion of a building, such as a cathedral, which may form a partial interface between the building walls and the air (which may be outside air and / or air within the building).

[0025] The method contemplates a tetrahedral meshing of a 3D point cloud representing at least a portion of the structure of the real scene, e.g., only a portion of the structure. The method may include a step S10 of providing the 3D point cloud, e.g., before providing the first open triangulation surface. Alternatively, the method may directly provide the first open triangulation surface and / or tetrahedral mesh, e.g., by retrieving the first open triangulation surface and / or tetrahedral mesh from a memory where they are stored after their calculation.

[0026] A 3D point cloud representing at least a portion of a structure of a real scene, as used herein, is a data structure in which each point represents a respective geometric entity located in at least a portion of the structure of the real scene. Each geometric entity represents a respective position of the structure within the scene (in other words, a respective portion and / or layout of the materials that make up the structure). The collection (i.e., combination or juxtaposition) of geometric entities collectively represents at least a portion of the structure. As used herein, any 3D point cloud may be composed of more than 100,000, 1,000,000, or 10,000,000 points. In particular, the present method can process more than 10,000,000 points on a standard or inexpensive computer (e.g., with 16 gigabytes of memory). It should be understood that a 3D point cloud may include points that do not represent respective geometric entities located in the structure. These points may form or be part of regions of low point density, as described below. Such regions may, for example, represent parts of the scene that do not belong to the structure. The point cloud may also contain outlier points due to measurement errors and therefore may not represent the geometric reality of the structure.

[0027] The real scene may be an architectural scene, such as a building seen from the outside. Here, a structure may be a closed tangible shape formed by the building structure (walls, roof, etc.). The skin of a part of the structure may in this case consist of the surface of the building structure, e.g. the wall and / or roof surfaces facing the outside and / or inside of the building. The real scene may also be any other civil engineering scene including any civil engineering structure (e.g. a tunnel, such as a mining tunnel). The scene may for example be a scan of a factory, i.e. a scan of the factory interior.

[0028] Alternatively, the real scene may be a terrain scene: the structure may be the terrain itself, and the skin of a portion of the terrain may be the ground of a patch of terrain, with the boundary of the patch as its boundary.

[0029] Alternatively, the real scene may be a mechanical part or an organic tissue, in which case the skin of the part is, for example, part of the outer envelope of the mechanical part or the organic tissue. The mechanical parts may be any of the following: land vehicle parts (including, for example, automobile and light truck equipment, racing cars, motorcycles, trucks and motor equipment, trucks and buses, trains, etc.), air vehicle parts (including, for example, airframe equipment, aerospace equipment, propulsion equipment, defense products, aviation equipment, space equipment), marine vehicle parts (including, for example, naval equipment, commercial vessels, offshore equipment, yachts and workboats, marine equipment), general mechanical parts (including, for example, industrial manufacturing machinery, heavy machinery or equipment, installation equipment, industrial machinery products, metal fabrication products, tire manufacturing products), electric mechanical or electronic parts (including, for example, consumer electronics appliances, security and / or control and / or measurement products, computing and communications equipment, semiconductors, medical devices and equipment), consumer goods (including furniture, home and garden products, leisure goods, fashion products, durable goods retailer products, textile retailer products), packaging (including, for example, food and beverage and tobacco, beauty and personal care, household goods packaging). Additionally or alternatively, the mechanical part may be any of the following: a formed part (i.e., a part produced by a forming manufacturing process), a machined part (i.e., a part produced by a machining manufacturing process), a drilled part (i.e., a part produced by a drilling manufacturing process), a turned part (i.e., a part produced by a turning manufacturing process), a forged part (i.e., a part produced by a forging manufacturing process), a stamped part (i.e., a part produced by a stamping manufacturing process), and / or a folded part (i.e., a part produced by a folding manufacturing process).

[0030] In an embodiment, the structure includes at least one corner structure. A corner structure refers to a portion of a structure that has or substantially has the shape of a corner. A corner of a structure may be a portion of the structure that corresponds to a material layout that forms an edge or a sharp angle of a shape. The corner structure may be a corner of a building, a corner of a terrain patch, a corner of a tunnel, a corner formed by a room wall, a corner of a mining tunnel, a corner or sharp angle in the layout of materials that make up a machine part, or a corner or sharp angle in the layout of tissue that makes up an organic tissue. Therefore, the method is robust for handling corner structures.

[0031] In an embodiment, the 3D point cloud is created by photogrammetry, laser scanning, RGBD measurements, and / or medical or industrial tomography.

[0032] In this method, point clouds representing the structure of an architectural / building scene may be obtained from RGBD measurements or photogrammetry, or may be formed from structure-from-motion analysis. Point clouds representing the structure of mechanical parts may be created from industrial tomography or laser scanning. Point clouds representing the structure of a mining scene (or other similar civil engineering scene) may be created from laser scanning of, for example, the interior of a mine. Point clouds representing the structure of organic tissue may be created from medical tomography. Point clouds representing terrain may be created from LIDAR measurements.

[0033] As used herein, any 3D point cloud may be determined from physical measurements in a real scene. The step S10 of providing a 3D point cloud may include, in particular, providing one or more physical sensors, each configured to acquire a respective physical signal, and operating the one or more physical sensors in the real scene to acquire the one or more physical signals (i.e., scanning the real scene with each sensor). The step S10 of providing a 3D point cloud may include automatically determining the 3D point cloud based on measurements according to any known technique. The one or more sensors may consist of multiple (e.g., RGB and / or image or video) cameras, and the determination may consist of structure-from-motion analysis. Alternatively or additionally, the one or more sensors may include one or more depth sensors (e.g., on an RGB-D camera). Alternatively or additionally, the one or more sensors may include a laser (e.g., LIDAR) or an ultrasonic emitter / receiver. Alternatively or additionally, the one or more sensors may include one or more tomographic sensors. The tomographic sensor may be a medical or industrial tomographic sensor.

[0034] Alternatively, the step S10 of providing a 3D point cloud may include visiting a database in which the 3D point cloud is stored and retrieving the 3D point cloud from the database. Although the providing step S10 may not actually include taking measurements and determining a 3D point cloud, the 3D point cloud may be obtained by operating one or more physical sensors and determining the 3D point cloud based on measurements taken by the one or more sensors, as described above, and then stored in a database. In either case, the step S10 of providing a 3D point cloud may be performed by a user. The step S10 of providing a 3D point cloud may further include displaying the 3D point cloud on a display (e.g., a GUI) of a computer executing the method.

[0035] The provided 3D point cloud may include one or more regions with different point densities. This means that the point density within the point cloud may vary from region to region of the point cloud. The 3D point cloud may include holes / openings resulting from physical constraints, such as occlusion or optical properties, when 3D acquisition of the 3D point cloud was performed. Holes may represent regions that should contain sufficient point density but do not due to physical constraints. Thus, the 3D point cloud may represent only a portion of a closed shape (i.e., structure) that would represent the entire closed shape if there were no physical constraints.

[0036] The skin of a structure or part thereof may correspond to a region of the point cloud with the highest point density, which, for example, separates at least a portion of the exterior from at least a portion of the interior. This region may be open, i.e., the region of the point cloud with the highest point density may form an open shape with a boundary. In this case, the interior and / or exterior may be regions characterized by a lower point density. In an example, "highest density" means that the point density of the open shape is higher than the ambient noise resulting from the measurement. The 3D point cloud may further feature outliers, which are points in the 3D point cloud that are in regions of low point density and are due to measurement errors. In other words, outliers do not represent geometric entities in the real scene but correspond to numerical and / or measurement artifacts. However, the method handles these outliers, making the method robust to outliers and density fluctuations.

[0037] The 3D point cloud may include a large number of points. For example, the ratio of the number of points in the 3D point cloud to the computer memory size may be greater than a predetermined threshold. For example, the predetermined threshold may be greater than 10 million points for a memory size of 16 gigabytes (i.e., substantially the size of standard or inexpensive computer memory), e.g., 10 million or more points for a memory size of 16 gigabytes.

[0038] Here, we briefly describe the concept of 3D reconstruction of a structure represented by a 3D point cloud. 3D reconstruction of a structure generally refers to the act of calculating / determining a surface connecting the points of the 3D point cloud. A surface may be said to "represent the skin of a portion of a structure" because it connects the points of the 3D point cloud to an open surface that represents the transition (e.g., boundary) between at least a portion of the structure's exterior and at least a portion of the structure's interior. Thus, a surface represents a portion of the structure's outer envelope, or in other words, the skin of a portion of the structure. For example, as mentioned above, the skin may correspond to an open shape with a high point density in the 3D point cloud. 3D reconstruction of a structure, in this case, may refer to determining / calculating a surface connecting the points of the open shape to form the open surface envelope of the structure.

[0039] As mentioned above, the method manipulates a tetrahedral meshing of a 3D point cloud representing at least a portion of the structure. The method may include a step S20 of calculating the tetrahedral mesh. Alternatively, the method may directly provide the first open triangulation surface and / or the tetrahedral mesh, for example, by retrieving the first open triangulation surface and / or the tetrahedral mesh from a stored memory and further calculating the first open triangulation surface and / or the tetrahedral mesh. The step S20 of calculating the tetrahedral mesh may be performed in any known manner.

[0040] In an example, the tetrahedral mesh has the following properties: the tetrahedrons of the tetrahedral mesh join to form a convex hull of the points of the 3D point cloud; and the intersection of a first tetrahedron of the tetrahedral mesh with a second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron. This improves the quality of the mesh and, consequently, the quality of the determined open triangulation surface.

[0041] In the example, the tetrahedral mesh is a regular triangulation. This further improves the quality of the mesh and the quality of the determined open triangulation surface. As is known per se in the field of computational geometry, a regular triangulation of a 3D point cloud is a triangulation guided by the shadows of the faces of a polyhedron of one dimension higher (i.e., four dimensions). A regular triangulation is simply the dual of a Laguerre-Voronoi diagram; see, for example, "Algorithmic geometry, Jean-Danel Boissonnat and Mariette Yvinec, Cambridge University Press New York, NY, USA," incorporated herein by reference. The step S20 of computing the tetrahedral mesh may include computing a regular triangulation of the 3D point cloud as a tetrahedral mesh by any known method, and may include: Lift the points of the point cloud, i.e. add a coordinate to each point, so that the set of points is embedded in a dimensional space equal to the original space (i.e. the 3D space to which the 3D point cloud belongs) containing the set of points plus one; Compute the convex hull of the set of lifted points; triangulating the convex hull in a known manner such that the resulting tetrahedral mesh has the following properties: the tetrahedrons of the tetrahedral mesh join to form a convex hull of the set of lifted points; the intersection of a first tetrahedron of the tetrahedral mesh and a second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron; and Project the underside of this triangulation of the convex hull back into the original space.

[0042] In the example, the regular triangulation is a Delaunay triangulation. In this case, points are lifted onto a paraboloid, and the lifting is parameterized by the vertical difference between the lifted points and the paraboloid. This further improves the quality of the mesh and the determined open triangulation surface. Furthermore, the method performs 3D reconstruction by going beyond simply computing a Delaunay triangulation. While the method certainly benefits from the quality of a Delaunay tetrahedral mesh, it goes beyond Delaunay triangulation by determining, in step S50, which of all candidate open triangulation surfaces formed by the triangular faces of the Delaunay triangulation actually represents the skin. In other words, the method is able to select the open triangulation surface formed by the triangular faces of the Delaunay tetrahedral mesh that best represents the skin. Therefore, the method can utilize a Delaunay tetrahedral convex hull mesh of the 3D point cloud, a known concept that is described in detail, for example, in the reference "F. Cazals, J. Giesen, Delaunay triangulation based surface reconstruction, Effective computational geometry for curves and surfaces, pages 231-276, 2006," which is incorporated by reference. A Delaunay tetrahedral mesh object is a collection of tetrahedrons with the following properties: The topology is a collection of adjacent tetrahedrons such that any triangle is completely shared by two regions. The first sharing situation is a triangle shared by two adjacent tetrahedrons. The second sharing situation is a triangle shared by a tetrahedron and an outer region without a boundary. A triangle shared by a tetrahedron and an outer region is called a boundary triangle. A geometric property of a Delaunay mesh is that for any tetrahedron, the sphere defined by its four points (called the circumsphere) does not contain any other point of the point cloud.

[0043] Providing a first open triangulated surface will now be described.

[0044] A first open triangulation surface is a set of triangles of a tetrahedral mesh that form an open triangulation surface with a boundary. More generally, in this disclosure, any open triangulation surface is a triangulation surface of a tetrahedral mesh that has a boundary. The first open triangulation surface may be any open triangulation surface that represents the skin of a portion of a structure and therefore may be one of the candidates in the determining step. However, the first open triangulation surface may be highly ranked in the lexicographic order, or at least so highly ranked that it does not minimize or tend to minimize the lexicographic order. As previously mentioned, the determining step S50 finds a second open triangulation surface that has the same boundary as the first open triangulation surface but minimizes or at least tends to minimize the lexicographic order. This results in a quality of the determined open triangulation surface that is better than the quality of the first open triangulation surface, i.e. the determined surface better represents the skin and does not contain any, or at least fewer, large and / or flat triangles that, for example, indicate low-density areas and / or outliers in the point cloud.

[0045] The first open triangulation surface may be provided directly; for example, as described above, the method does not calculate / determine the first open triangulation surface. Alternatively, providing the first open triangulation surface may include calculating / determining the first open triangulation surface. Calculating / determining the first open triangulation surface may include applying any method to find an open triangulation surface within the tetrahedral mesh that at least roughly represents the skin of a portion of the structure. In other words, providing the first open triangulation surface may include finding any open triangulation surface within the tetrahedral mesh that at least roughly represents the skin, even if the first open triangulation surface is highly ranked in lexicographic order. Next, the determining step S50 finds an open triangulation surface that minimizes or tends to minimize the lexicographic order among such open triangulation surfaces. In other words, providing the first open triangulation surface may be performed by any method that finds an open triangulation surface that at least roughly / inaccurately represents the skin.

[0046] In an example, providing the first open triangulation surface includes step S30 of providing one or more loops in the tetrahedral mesh and step S40 of determining an open triangulation surface in the tetrahedral mesh having the one or more loops as a boundary. As used herein, a loop is a set of edges and vertices of the tetrahedral mesh such that each vertex in the loop belongs to an even number of edges of the loop. The one or more loops form a boundary of a portion of the structure. As previously mentioned, the portion of the structure may be a portion of the structure represented by a 3D point cloud, or a subportion of this portion, in which case the one or more loops form a boundary of this portion.

[0047] Due to partial acquisition of the 3D point cloud, the boundary may partially coincide with the boundary of the actual shape of the structure. Alternatively or additionally, due to partial acquisition of the 3D point cloud, the boundary may coincide with the boundary of one or more holes in the 3D point cloud. For example, one loop may correspond, for example, in part, to the actual shape of the boundary of the real-world structure. Additionally or alternatively, one loop may correspond to the boundary of a hole in the 3D point cloud.

[0048] Additionally or alternatively, the portion of the structure may correspond to a portion of the 3D point cloud resulting from the division, for example as described above, the boundary of this portion being provided as a loop in S30.

[0049] Step S30 of providing one or more loops may be performed by a user, for example, through graphical user interaction. For example, to provide a particular loop of the one or more loops, the user may click or touch points and edges of the tetrahedral mesh to form the loop. Alternatively or additionally, to provide the particular loop, the user may perform a series of clicks or touches on the tetrahedral mesh or point cloud, where each click or touch defines a point. The method may then automatically infer edge and vertex loops from the user-selected series of points, which may include oversampling the user-selected points and / or finding edge and vertex loops that fit the user-selected points. Additionally or alternatively, to provide the particular loop, the user may draw a closed line on the tetrahedral mesh or point cloud, where the line defines (e.g., roughly) the shape of the boundary. The method may then automatically infer edge and vertex loops from the user-defined line, which may include sampling the user-defined line and / or finding edge and vertex loops that fit the user-defined line.

[0050] Alternatively, step S30 of providing one or more loops may be performed automatically. For example, providing one or more loops may be based on computer memory considerations. For example, step S30 of providing one or more loops may include dividing the 3D point cloud or tetrahedral mesh into bounded regions / portions such that each region / portion has a number of 3D point cloud points that fit into a given computer memory (e.g., 16 gigabytes for a standard or inexpensive computer). One or more loops may then each correspond to the boundary of each portion. The resulting division of the tetrahedral mesh may be watertight, i.e., the separated portions / regions coincide along their common boundaries.

[0051] Next, we will describe an implementation of step S30 that automatically provides one or more loops. This implementation is particularly applicable when there are a large number of points in the 3D point cloud, e.g., more than 10 million points for a standard or inexpensive computer with 16 gigabytes of memory. This implementation subdivides the point cloud / tetrahedral mesh into regions whose number of points fits into the computer memory, and ensures that the separated mesh regions coincide along their common boundaries, making the resulting division / mesh regions watertight. This implementation works as follows: The step S20 of computing the tetrahedral mesh includes computing a 3D Delaunay triangulation of the 3D point cloud using a known out-of-core method, which may include any method capable of performing a 3D Delaunay triangulation per region, such as the method disclosed in "Lo, SH "Parallel Delaunay triangulation in three dimensions," Computer Methods in Applied Mechanics and Engineering 237 (2012): 88-106," which is incorporated herein by reference.

[0052] The step S30 of providing one or more loops includes extracting a silhouette of the entire 3D Delaunay triangulation when viewed from above, where the 3D Delaunay triangulation is a convex 3D object whose silhouette is a single loop.

[0053] The providing step S30 further includes repeating the following until each loop encloses as many points as will fit into computer memory: ·Define vertical planes; Extract the tetrahedrons of the 3D Delaunay triangulation that cut this plane; and Define the lexicographically smallest path in the extracted tetrahedron, which divides the loop into two smaller loops.

[0054] The step S40 of determining an open triangulation surface of the tetrahedral mesh, having one or more loops as boundary, may be performed by any suitable method or any suitable technique. The determined (S40) open triangulation surface is a first open triangulation surface. In an embodiment, the determining step S40 may implement Algorithm 2, which will be described later. Instead of Algorithm 2, a suitable combination of other geometric algorithms may also be implemented.

[0055] The step S50 of determining a second open triangulated surface representing the skin of a portion of the structure will now be further described.

[0056] The determining step S50 searches for candidate open triangulation surfaces. Each candidate is a set of triangles of the tetrahedral mesh that form an open triangulation surface in the tetrahedral mesh. The determining penalizes candidates that rank highly in the lexicographic order and searches for candidates that violate labeling consistency. That is, the determining performs a selection among the candidate open triangulation surfaces and selects one candidate that minimizes, or at least tends to minimize, the lexicographic order and violates labeling consistency.

[0057] The selection between the candidates for open triangulation surfaces involves lexicographical ordering and triangle ordering, as explained below.

[0058] The lexicographic order orders the first open triangulation surface candidates relative to the second open triangulation surface candidates, i.e., it orders the first open triangulation surface candidates and the second open triangulation surface candidates. The concept of lexicographic order is known per se. The first open triangulation surface candidate has the first triangle of the tetrahedral mesh (e.g., is composed of the first triangle), and the second open triangulation surface candidate has the second triangle of the tetrahedral mesh (e.g., is composed of the second triangle). Both the first triangle and the second triangle are ordered according to the descending order of the triangles.

[0059] Ordering the first open triangulation surface candidate relative to the second open triangulation surface candidate according to lexicographic order may particularly include evaluating the first triangle order and the second triangle order in descending triangle order, since the lexicographic order is based on the triangle order. The ordering may further include establishing a rank of the first open triangulation surface candidate relative to the second open triangulation surface candidate in lexicographic order based on the evaluated ordering of both the first triangles and the second triangles. In other words, the ordering may include establishing a relative ranking between the first open triangulation surface candidate and the second open triangulation surface candidate in lexicographic order by comparing the ordered first and second triangles.

[0060] We now describe an example of ordering a first set of potential open triangulation surfaces relative to a second set of potential open triangulation surfaces according to lexicographic order: Let A be the set of first triangles and B be the set of second triangles.

number

number

[0061] This ordering may result in the set of all open triangulation surface candidates being ordered lexicographically, i.e., each open triangulation surface candidate has its own rank in the lexicographic order. In the determining step S50, high-ranking open triangulation surface candidates are penalized. For example, the determining step S50 may be performed such that high-ranking open triangulation surface candidates are discarded and low-ranking open triangulation surface candidates are not discarded.

[0062] Triangle ordering penalizes triangle size. The concept of penalizing triangle size is now described. Let T and T′ be two triangles. Penalizing triangle size may include, for each triangle T and T′, evaluating a triangle size measure and giving a triangle with a larger measure a higher ranking in the triangle ordering. The triangle size measure may also be referred to as a triangle weight or triangle weight. The size measure may include, for example, a first real-valued function that contributes to the triangle size measure, a second real-valued function that contributes differently to the triangle size measure, and / or a pair consisting of a first function and a second function. Penalizing triangle size may include, for example, ordering the two evaluated size measures relative to each other according to any order for ordering two real numbers or two pairs of real numbers relative to each other, as known per se in the art. That is, if T is larger (i.e., larger in size) than T′, then the size measure of T is larger than the size measure of T′ (e.g., according to the order of the measures). Now, if the pair (T, T') is ordered according to the descending order of the triangles, then T comes before T' in this ordering, as described above for the first and second triangles.

[0063]

number

[0064] When ordering two candidate open triangulation surfaces as described above, the candidate with the triangle that is most penalized by the triangle ordering (e.g., the largest) of the two is given a higher lexicographical ranking. As described above, such triangles may represent regions of the 3D point cloud (e.g., portions of a mesh) with low point density, such as those composed of outliers. By penalizing the higher rankings in the lexicographical ordering and discarding candidate open triangulation surfaces with such triangles, it is possible to find an open triangulation surface that accurately represents the skin of a portion of a structure and is less susceptible to density variations and outliers. This results in a robust and accurate method.

[0065] In an example, the method ensures that the triangles of the determined open triangulation surface (e.g., all of the triangles, or a significant portion of the triangles) are as small as possible (i.e., small in size), including triangles that are not particularly elongated in shape, e.g., triangles with a relatively small area and a height-to-base ratio close to 1. Indeed, when comparing two triangles, the order of the triangles does not necessarily penalize the flattest triangle of the two, but if lexicographical ordering is based on the triangle ordering, flat triangles may be avoided as much as possible. In practice, lexicographical ordering compares chains of triangles (each of which forms an open triangulation surface) rather than single pairs of triangles, and attempts to reward (i.e., give a lower rank to) chains that are composed of the smallest possible triangles. Comparing chains of triangles rather than single pairs of triangles and penalizing chains with large triangles (e.g., a large number of) may actually result in the determined open triangulation surface being composed of (e.g., generally) small, not too flat triangles. Such triangles represent the skin. This results in a robust and accurate method.

[0066] In the example, the triangle ordering penalizes, for each triangle, the minimum enclosing circle, i.e., the high value of the radius of the triangle's minimum enclosing circle; The minimum enclosing circle of a triangle is the circle with the smallest radius of all the circles that enclose the points of the triangle. The radius of the minimum enclosing circle of a triangle contributes to measuring the size of the triangle, and penalizing high values ​​of the radius of the minimum enclosing circle contributes to penalizing the size of the triangle. In other words, the radius of the minimum enclosing circle is a real-valued function that contributes to measuring the size of the triangle, and may be part of the measurement of the size of the triangle, as described above. Furthermore, penalizing high values ​​of the radius of the minimum enclosing circle is a particularly efficient and robust way of penalizing the size of a triangle. For example, let T and T' be two triangles, and R B (T) and R B Let (T') be the radius of each of its smallest encompassing circles. In this case, in these examples, the inequality R B (T) <R BBy (T') we capture that T' is larger in size than T. Penalizing size in this case may be giving T' a higher rank in the triangle order than T. Ordering T and T' relative to each other according to descending triangle order means that T' comes before T in this case.

[0067] In the example, the triangle ordering further penalizes smaller values ​​of the circumscribing circle, i.e., the radius of the circumscribing circle of a triangle (either the first or the second), for a first triangle and a second triangle that have the same minimum enclosing circle.

[0068] The circumscribing circle of a triangle is the circle that passes through the points of the triangle. The radius of the circumscribing circle of a triangle contributes to measuring the size of the triangle, and penalizing small values ​​of the circumscribing circle radius contributes to penalizing the size of the triangle, especially when the smallest enclosing circle is the same, as described below. In other words, the radius of the circumscribing circle is a real-valued function that contributes to measuring the size of the triangle, and may be part of the measurement of the size of the triangle, as described above.

[0069] TIFF0007727487000004.tif113170

[0070] As mentioned above, in the example, the size of each triangle T is measured by a triangle size measure, which is a pair of real-valued functions consisting of the radius of the smallest encompassing circle of T and the inverse of the radius of the circumscribing circle of T. The measures are:

number

number

number

number

number

number

[0071] The triangle ordering will be explained in more detail with reference to Figures 2 to 5, which illustrate the triangle ordering.

[0072] The order of triangles illustrated in these figures is:

number

[0073] Figure 2 illustrates triangle T along with its encompassing and circumscribing circles. Since it is an obtuse triangle,

number

[0074] If a triangle is characterized by three aligned or coincident points, then the circumcircle is not defined. This degenerate situation is beyond the scope of this disclosure, since point groups are general in nature. Let two triangles be T and T', and their relative ordering be

number

number

[0075] This ordering is actually a lexicographic order defined by pairs of numbers.

[0076]

number

[0077]

number

number

[0078] The determined open triangulation surface violates the labeling consistency of the tetrahedral mesh with two predetermined labels. Therefore, the step S50 of determining an open triangulation surface selects, from among the candidates, an open surface that violates the labeling consistency. Since the step S50 of determining also penalizes high rank in the lexicographic order, the step S50 of determining selects, from among the candidates that violate the labeling consistency, an open surface that minimizes, or at least tends to minimize, the lexicographic order.

[0079] Labeling is further explained below.

[0080] Labeling involves labeling a tetrahedral mesh with two predefined labels. That is, a label is assigned to each tetrahedron of the tetrahedral mesh. Labels can also be assigned to the outer regions, i.e., regions not meshed by the tetrahedral mesh. Labels can be colors, for example, so that each tetrahedron has a specific color. Other kinds of labels can also be used.

[0081] The determining step S50 may include assigning labels. For example, the determining step S50 may include assigning the same label to all tetrahedrons before searching for candidates. The labeling may then be used as a variable in the search and penalization. That is, when determining the second open triangulation surface that is lexicographically smallest among the open triangulation surfaces that contradict the labeling (S50), the determining step S50 may include iteratively modifying the labels of the tetrahedrons. The determining step S50 may iteratively attempt to ensure labeling consistency, for example, by visiting triangles of the tetrahedral mesh and iteratively changing the labels of the tetrahedrons to adjust the labeling to maintain consistency. In this case, the determining step S50 may iteratively detect each triangle for which this adjustment cannot be performed and identify / reward such triangle as being part of the second open triangulation surface to be determined.

[0082] Each triangle of a tetrahedron may be associated with a term that defines whether the triangle belongs to the first open triangulation surface. This term may define how the triangle respects or violates labeling. A triangle defined as belonging to the first open triangulation surface respects labeling consistency only if it separates two tetrahedrons with different labels. A triangle defined as not belonging to the first open triangulation surface respects labeling consistency only if it separates two tetrahedrons with the same label. The determining step S50 may include assigning such a term to each triangle. Unlike labeling, this term of the triangle is not a variable in the determining step S50; i.e., the determination does not change the term. In other words, the term itself provides its constraint.

[0083] In an example, an item may be a permutation. A permutation is either an identity or a switch (also called a "swap"). When each triangle in the first open triangulation surface is assigned a switch / swap, this indicates that the triangle respects labeling consistency only if it separates two tetrahedrons with different labels (i.e., the triangle switches labeling). When each triangle in the first open triangulation surface is assigned an identity, this indicates that the triangle respects labeling consistency only if it separates two tetrahedrons with the same label (i.e., the triangle keeps its label).

[0084] The determined second open triangulation surface violates labeling consistency, meaning that each triangle of the open triangulation surface does not respect labeling consistency. The requirement that the determined second open triangulation surface violates labeling consistency ensures that the second open triangulation surface has the same boundary as the first open triangulation surface, as will be further explained below. Since the determining step S50 searches for candidates for open triangulation surfaces that meet this requirement, determining step S50 selects a candidate open triangulation surface that meets this requirement. Such a candidate exists because the first open triangulation surface already meets the requirement by switching its triangles. Next, determining step S50 finds another candidate that meets the requirement, with the additional requirement that the candidate must be lexicographically smallest.

[0085] Here, an example of the determining step S50 will be described.

[0086] In these examples, before searching and penalizing, the determining step S50 may include assigning a label to each tetrahedron of the tetrahedron mesh, e.g., assigning the same label to each tetrahedron. Before searching and penalizing, the determining step S50 may also include assigning a permutation to each triangle of the tetrahedron mesh, e.g., assigning a switch permutation to each triangle of the first open surface and an identity permutation to the remaining triangles.

[0087] The searching and penalizing may include visiting triangles of the tetrahedral mesh in descending triangle order, and then discarding visited triangles that are higher in the triangle order as long as labeling consistency can be maintained.

[0088] The visitation of a triangle involves a search for a candidate triangulation surface. Each time a triangle is visited, the triangle is either discarded or kept. Searching for a candidate triangulation surface means corresponding to a specific way in which the triangles of the tetrahedral mesh are visited and kept or discarded. The visitation visits the triangles according to the descending triangle order. That is, the triangle of the tetrahedral mesh with the highest rank in the triangle order, i.e., the triangle with the largest size among the triangles of the mesh, is visited first. Then, the triangle with the second highest rank is visited, and so on, with the triangle with the lowest rank being visited last. Each time a triangle is visited, the triangle is either kept, i.e., selected as belonging to the determined second open triangulation surface, or discarded, i.e., not becoming part of the determined second open triangulation surface.

[0089] The penalization discards visited triangles with a high triangle order ranking. This prevents the determined second open triangulation surface from featuring large and / or flat triangles, or at least prevents the determined second open triangulation surface from featuring too many large and / or flat triangles. However, these high-ranked triangles are discarded only if labeling consistency can be maintained when discarding the triangles. This makes it possible to discard these high-ranked triangles when it is certain that a labeling-inconsistent second open triangulation surface can still be determined without using the high-ranked triangles but by using lower-ranked triangles. As a result, a searched triangle is rewarded, i.e., determined to be part of the determined second open triangulation surface, only if labeling consistency cannot be maintained due to the presence of that triangle. Because triangles are searched in descending triangle order, this prevents large and / or flat triangles from belonging to the determined second open triangulation surface as much as possible. These triangles are explored first / early and may maintain labeling consistency without necessarily rewarding these triangles even if they violate labeling consistency when explored.

[0090] Pruning may include, for each visited triangle, evaluating whether the triangle violates or respects labeling consistency. Then, for each visited triangle, pruning may include either rewarding the triangle if the triangle violates labeling consistency or adjusting the labeling to respect consistency. Rewarding a triangle means adding the triangle to the determined second open triangulation surface, i.e., the triangle ultimately belongs to the determined second open triangulation surface after exploration and penalization. Pruning in this manner avoids systematically rewarding triangles that violate labeling consistency. If a visited triangle violates labeling consistency, the label may be adapted to respect consistency so that the triangle does not need to be rewarded. This may be especially true for large or flat triangles that are explored first / early. If adjusting the labeling does not re-establish labeling consistency, the visited triangle that violates consistency is necessarily rewarded. This is especially true for triangles that are explored last / late and have decreased size or flatness.

[0091] Discarding may involve, for each triangle visited, for example after assessing whether the triangle violates or respects labeling consistency: Reward a triangle if it separates two tetrahedra in the same connected component and violates labeling consistency; If a triangle separates two tetrahedra in different connected components: If the triangle violates labeling consistency, switch the label of one of the connected components (i.e., all the tetrahedrons of the connected component) and merge the connected components (i.e., the two connected components thereby form a single connected component), · If the triangles respect the labeling consistency, merge the connected components (i.e., merge two connected components to form a single connected component).

[0092] A connected component is a set of triangles shared with a tetrahedron, and for each pair of tetrahedra in the set, there exists a path between adjacent tetrahedra in the set (i.e., two tetrahedra sharing a triangle), connecting the pair of tetrahedra, where each triangle in the path respects labeling consistency. Performing discarding in this manner allows our method to reward triangles that violate labeling consistency only if a triangle separates tetrahedra in the same connected component. This allows us to reward triangles that violate labeling consistency only if switching the labeling of the connected component separated by the triangle cannot re-establish labeling consistency. This allows lower-ranked triangles to be explored last / later, so if they are explored and violate labeling consistency, we can reward the lower-ranked triangle without the possibility of performing a switch to re-establish consistency.

[0093] Any suitable embodiment of the above-described search and penalty assignment method may be used. For example, a dual graph of a tetrahedral mesh may be used. The concept of a dual graph of a tetrahedral mesh is well known. A dual graph is a graph with nodes and arcs connecting the nodes, where each node represents a tetrahedron of the tetrahedral mesh. If the tetrahedrons represented by two nodes share a triangle, each arc connects the two nodes, and the shared triangle is represented by an arc. A new node is added to represent the outer region, and each triangle facing the outer region is represented by an edge in the dual graph. In particular, triangles may be visited as follows: associate a weight according to the triangle order of the corresponding triangle with each arc of the dual graph, and then visit the arcs of the relative graph in descending order of triangle weight, starting with the arc connecting the triangle to the outer region.

[0094] Next, the implementation of the method will be described.

[0095] This implementation makes it possible to determine an open triangulated surface that forms the skin of a portion of a structure in a real scene, at least in part, represented by a 3D point cloud, given one or more loops as part of its boundary. The 3D point cloud may contain a very large number of points, may be incomplete (e.g., have one or more holes), and / or may be prone to errors. This implementation is still successful. The 3D point cloud may be characterized by density variations, for example, that may be very large. This implementation is still successful. It may also be characterized by outliers, which are sparse collections of spurious points randomly spread in space. This implementation is still successful. Furthermore, the actual shape of the structure and / or the smoothness of the real scene may be unknown. This implementation is still successful.

[0096] In other words, this implementation computes a second open triangulated surface from such a point cloud in a reasonable computation time, regardless of density variations. Hence, this implementation is robust and does not involve parameter tuning.

[0097] FIG. 6 shows a flowchart of an implementation of the method. As shown in FIG. 6, this implementation includes step S10 of providing a 3D point cloud. The 3D point cloud is provided by a user. This implementation further includes step S20 of calculating a 3D Delaunay tetrahedral mesh of the 3D point cloud. Because the point cloud contains three-dimensional points, the resulting Delaunay mesh is composed of adjacent tetrahedrons, each of which is surrounded by four triangles. This implementation further includes defining a lexicographic order, which includes providing (e.g., calculating) triangle weights for the triangles of the Delaunay tetrahedral mesh and providing a dual graph of the tetrahedral mesh. The concept of a dual graph for a Delaunay triangulation is known per se and has already been described above. This graph captures the adjacency relationships of tetrahedra through triangle sharing. This implementation further includes ordering the arcs of the dual graph, each of which represents a triangle of the Delaunay mesh, by triangle weights. Thus, each node of the dual graph represents a tetrahedron in the mesh, each arc of the dual graph represents a triangle in the mesh, and each arc of the dual graph is labeled with the rank of the corresponding triangle in lexicographical order. In particular, this implementation searches the arcs of the dual graph in descending order when performing the determining step S50, as described below.

[0098] Continuing with reference to the flowchart of FIG. 6, this implementation further includes providing a predetermined boundary for a first open triangulation surface. The first open surface may be provided directly by a user. Alternatively, providing the first open surface may include step S30 of providing one or more loops that define the predetermined boundary. Step S30 of providing one or more loops may be performed by a user or automatically, as described above. Given one or more loops, the implementation may then include step S40 of determining a first open surface composed of a set of triangles in the Delaunay tetrahedral mesh, the boundary of which coincides with the provided boundary.

[0099] Continuing with reference to the flowchart of FIG. 6, given the arcs of an ordered dual graph of a Delaunay triangulation and a first open triangulation surface, implementations include identifying dual arcs of the first open triangulation surface that represent triangles of the first open triangulation surface. Next, the implementations include a reduction algorithm that performs step S50 to determine a second open triangulation surface that represents the skin of a portion of the structure based on the dual graph and its ordered arcs and the identified dual arcs of the first open triangulation surface, thereby obtaining a reconstructed open surface that represents the skin. The first open triangulation surface is constrained only by its boundary and generally does not pass through the most point-dense regions. However, when provided as input to the reduction algorithm, the algorithm determines the optimal surface (i.e., with respect to minimizing lexicographic order and passing through the most point-dense regions) with the same boundary (S50). The reduction algorithm discards arcs of the dual graph so that the remaining dual arcs define an open triangulation surface that is the reconstructed second open triangulation surface that represents the skin.

[0100] This implementation is particularly applicable to 3D reconstruction applications that require the creation of open surfaces, i.e., surfaces with boundaries. In this case, given a point cloud, this implementation provides, either at user command or automatically, a boundary consisting of one or more polygonal loops (i.e., closed polylines) along with the vertices in the point cloud. This implementation can create an open mesh whose vertices belong to the point cloud and whose boundary coincides with the specified loop. This implementation makes it possible to mesh open surfaces that represent terrain from LIDAR data, among other things, and its ability to manage boundaries allows for the enforcement of common boundaries between adjacent terrain regions. Applications of this implementation are described in more detail below.

[0101] Before describing in detail the algorithms involved in the implementation, a brief outline of the implementation will be provided.

[0102] In the implementation, cycles are operated on. A 1-cycle of a 3-dimensional Delaunay complex is the union of loops, i.e., the set of edges where each vertex belongs to an even number of edges. A 0-cycle of a 2-dimensional Delaunay complex is an even number of vertices. Figures 7-9 show the implementation of the method for finding the lexicographically minimal chain at a given boundary in a 2-dimensional Delaunay complex, i.e., one dimension less. In this 2D diagram, the total ordering of edges involves comparing the lengths of the edges. Figure 7 shows loop A provided in S30. Loop A consists of four vertices and is shown in Figure 7 with a large circle. Figure 8 shows step S40 of determining an open surface with loop A as its boundary. This step S40 involves calculating the chain Γ that bounds the provided set A, i.e., has A as its boundary, i.e., in algebraic notation, ∂Γ = A. In Figure 8, the chain Γ is shown with a dotted line. An algorithm for performing this determining step S40 is discussed further below, although as mentioned above, alternative suitable algorithms, for example, a suitable combination of alternative geometric algorithms, may also be used. Figure 9 shows the determining step S50 and involves assuming the chain Γ resulting from the first step and calculating the lexicographically smallest chain Γ that has the same bounds (and is in fact a homology) as Γ, i.e., in algebraic notation, ∂Γ = ∂Γ = A. The chain Γ is shown in Figure 9 by a dotted polyline. An algorithm for performing this determining step S50 is discussed further below.

[0103] Here, the concept of this implementation will be explained.

[0104] simplicial complex

number

number

number

number

[0105] Single Chain Let K be a simplicial complex of at least d dimension. The concept of a chain can be defined in terms of coefficients in any ring, but in this application the definition is given for the field

number

number

number

number

number

number

[0106] <boundary operator> d alone

number

[0107]

number

number

number

[0108] <lexicographical order> d simplex of K, σ1< σ n Let there be a total order on d Let (K) be the order. From this order, a lexicographical total order of d-chains is defined as follows:

number

number

[0109] Problem: Lexicographically Optimal Chaining of Given Bounds (LOCGB) In this implementation, the following problems are anticipated:

[0110] Problem 1: Totally ordered d-simples and d-chains

number

[0111]

number

number

number

[0112] In problem 2, ∂ d A set such that Γ0=A

number

[0113] <d-pseudomanifold> A d-pseudomanifold is a pure d-dimensional simplicial complex in which each (d - 1)-face has exactly two d-dimensional cofaces.

[0114] <dual graph> The dual graph of a d-pseudomanifold M is a graph whose vertices correspond one-to-one with the d-simplices of M and whose edges correspond one-to-one with the (d - 1)-simplices of M. An edge e connects the two vertices v1 and v2 of the graph only when the edge e corresponds to a (d - 1)-face having the cofaces corresponding to the vertices v1 and v2.

[0115] <strongly connected d-pseudomanifold> A strongly connected d-pseudomanifold is a d-pseudomanifold whose dual graph is connected.

[0116] <dual boundary operator> Regard the graph G as a 1-dimensional simplicial complex, and the dual boundary operator

[0117] [Mathematics] and, in the standard basis of the simplex, the boundary operator

[0118] [Mathematics] To avoid introducing unnecessary formal definitions, the coboundary operator is defined for chains, since for finite simplicial complexes the canonical inner product defines a natural bijection of chains and cochains.

[0119] Finding 1 For a given graph G = (ν, ε), let ν and ε be its set of vertices and edges, respectively. Let M be a given (d+1)-quasi-manifold. d-chain

number

number

[0120]

number

number

number

number

[0121]

number

number

[0122] An algorithm for solving Problem 1 and performing the determining step S50 as described above is now described. In this discussion, let M be a simplicial complex, envisaged as a strongly connected (d+1)-pseudomanifold whose d-homology is trivial. In this implementation, M is the simplicial complex obtained by completing a 3D Delaunay complex, i.e., by tetrahedral meshing.

number

[0123] Problem Reformulation 1 As in the general case, in practice, ∂ d We find a solution to Problem 2 by first determining the d-chain Γ0 such that Γ0 = A (corresponding to step S40), and then solving Problem 1 (corresponding to step S50). M is assumed to have trivial d-homology, which can be expressed as all d-cycles being boundary.

[0124]

number

number

[0125]

number

[0126] Problem 1.2: Totally ordered sets of ε and edges

number

[0127]

number

number

number

number

number

[0128] Here, sum

number

number

[0129] <Labeling> In this implementation, labeling refers to coloring, but other similar labeling may be used. The labeling has two predetermined labels, which are two elements, alphabets {b, r} meaning "blue" and "red" respectively, and a set of permutations σ{b, r} = {ident, swap}, defined as ident(b) = b, ident(r) = r, swap(b) = r, swap(r) = b.

[0130] <Enlarged graph> The augmented graph is a triple (ν,ε,π), where (ν,ε) is a (possibly disconnected) graph,

number

number

[0131] <Label assignment> A label assignment is a map that associates a label (i.e., a color) with each node in the graph.

number

[0132] <Satisfactory assignment> We say that an augmented graph (ν,ε,π) approves a satisfactory assignment if there exists an assignment L that is consistent with π, i.e., for each edge

number

number

[0133] In this case, the graph can be said to respect labeling consistency.

[0134] Finding 2 L is a satisfactory allocation,

number

number

number

[0135] <Valid cut> Consider an expanded graph (ν,ε,π),

number

number

[0136] To solve problem 1.2, another problem can be considered.

[0137] Problem 1.3: Given an augmented graph (ν,ε,π) and a total ordering on ε, find the lexicographically smallest valid cut Γmin in (ν,ε,π).

number

[0138] Lemma 1: The solution to Problem 1.2 is the augmented graph (ν,ε,π Γ0 ) is the solution to problem 1.3:

[0139]

number

number

number

number

number

[0140] Consider the following assignment:

[0141]

number

number

number

number

number

number

number

number

number

number

number

[0142] Second, Γ

number

number

number

[0143] L

number

number

number

number

number

[0144] set

number

[0145]

number

number

number

[0146]

number

number

number

[0147]

number

number

number

[0148]

number

number

number

[0149] In summary, e is

number

number

[0150] Gamma min and Γ′ min are the solutions to Problem 1.2 and Problem 1.3, respectively.

number

number

[0151] Also, Γ min +Γ′ min exactly vertices of B

number

number

number

[0152] This proves the lemma.

[0153] <Finding 3> The augmented graph (ν,ε,π) approves a satisfactory assignment, e1,...,e n If is a closed path in (ν,ε):

number

[0154] Lemma 2: Let (ν,ε,π) be an augmented graph with ε where π is a total order <.

number

number

number

[0155] Then,

number

number

[0156] Furthermore, L

number

number

number

number

number

number

[0157] In case a), L is again

number

[0158] In case b),

number

number

[0159] where:

number

number

number

[0160] Proof of Lemma 2

[0161] Gamma min be the lexicographically smallest valid cut of (ν,ε′,π), and

number

number

number

number

number

number

number

[0162]

number

number

number

number

[0163]

number

number

number

number

number

[0164] If v1 and v2 are in the same connected component of (ν,ε′), then

number

number

number

number

number

number

number

[0165] This means that v1 and v2

number

number

number

[0166] where v1 and v2 are

number

number

number

number

number

number

number

[0167] v1 and v2

number

number

number

[0168]

number

number

number

number

[0169] This proves Lemma 2.

[0170] We now describe the reduction algorithm that implements step S50 of determining. The algorithm is presented in the general context of problems 1, 2, and 1.1-1.3 considered above. The application of the Delaunay tetrahedron mesh computed in S20 to the dual graph G is by associating a swap permutation to each edge belonging to the first open triangulation surface in the dual graph, and an identity permutation to each other edge.

[0171] The reduction algorithm is described in the following pseudocode:

[0172] Algorithm 1: Lexicographically least significant cut input:

number

[0173] The algorithm takes as input a set of edges sorted in descending lexicographical order. A SwapCC operation on a representative r of connected components involves swapping the labels of all its elements, as per Observation 2 or Lemma 2. Algorithm 2 visits edges in descending order, thereby completing Γ in sublinear time. min It becomes possible to establish the

number

[0174] Proposal 1: Algorithm 1 computes the solution to Problem 1.3 for a given augmented graph (ν,ε,π). Assuming the input edge set ε is classified, the algorithm has time complexity O(nα(n)), where n is the cardinality of ε and α is the inverse of the Ackermann function.

[0175] Proof of Proposition 1: MakeSet, FindSet, and UnionSet are standard operations on disjoint set data structures, while a naive implementation of SwapCC requires a basic operation on each element of the connected component. However, by modifying the disjoint set data structure, it can be implemented in near-constant time.

[0176] One possibility is to implement connected components by pairs of base connected components, using a map from the base connected components to pairs containing one "blue" and one "red". A modified MakeSet operation then creates pairs with one vertex in the blue component and one vertex in the empty red component. A modified FindSet operation applies the standard FindSet operation and uses a map to return the pair it contains. Between two paired connected components, a modified UnionSet operation applies the standard UnionSet operation to each "blue" and "red" base connected component and pairs the results into a new paired connected component. SwapCC then involves only swapping the "blue" and "red" base connected components in the paired connected components.

[0177] This implementation of the operation is convenient in practice because it uses the standard disjoint sets data structure as a black box. However, mapping from a base connected component to the pairs that contain it can add a factor of log(n) to the computational complexity. The nα(n) complexity of the disjoint sets data structure can be maintained if we are willing to adapt the implementation. In brief: without changing the computational complexity, we can add an attribute {ident,swap} to each "parent" relation at the center of the disjoint sets data structure. This attribute indicates whether the node and its parent have the same or different colors.

[0178] MakeSet creates a connected component with one element, where it is its parent and the attribute associated with this relationship is identity. This attribute means that the color of the vertex for which it is the representative is "blue". The FindSet operation must then construct a permutation along the path including the last relationship from the representative to itself, which defines the representative's color relative to "blue": applying this construction to the color "blue" determines the node's actual color. UnionSet associates one of its parent representatives with the attribute needed for each color, either identity or swap. SwapCC only swaps the representative's color, i.e., it applies the swap to the attribute associated with the relationship as its parent.

[0179] This proves Proposition 1.

[0180] Algorithm 1 will now be further described with reference to Figures 10 to 32, which show an example of Algorithm 1 executed on the dual graph of a Delaunay triangulation. In each figure, the algorithm is shown along with a diagram showing the current step of the algorithm on the dual graph, with the current step circled in the appropriate place on the diagram.

[0181] Figure 10 shows a graphical representation of a Delaunay triangulation. The edges represent triangles, and the points connected by the edges represent tetrahedra.

number

[0182] Figures 11 and 12 show the dual graph of the triangulation shown in Figure 10. In Figure 11, the dual graph is superimposed on the triangulation, while in Figure 12, only the dual graph is shown. The edges of the dual graph are labeled with the same numbers as the corresponding triangles in Figure 10.

[0183] Edges 7, 5, and 2, which represent triangles in the first open triangulation surface, are highlighted with bold lines in Figure 13. This representation is omitted in subsequent figures. Edges 7, 5, and 2 are assigned the switch / swap permutation. The other edges are assigned the identity permutation.

[0184] Figure 14 shows the first step of the algorithm, where all vertices of the dual graph are assigned the same label "b". Another label, "r", is not yet assigned to any vertex. These vertices are represented by large dots in Figure 14. The solution Γmin, which finally contains all dual edges of the triangles of the determined second open triangulation surface, is initialized as empty.

[0185] Figure 15 shows the next steps in the algorithm: Edge

number

[0186] Figures 16 and 17 show the next step in the algorithm: visit edge 7. Edge 7 is identified as violating labeling consistency because it separates two tetrahedra with the same label while being assigned a swap permutation. Furthermore, edge 7 separates two tetrahedra that belong to two different connected components. Therefore, as shown in Figure 17, the labels of one of the two connected components are swapped, and then the two connected components are merged. As a result of the swap, the vertices are labeled with the label "r" and are indicated by small gray dots in Figure 17 and the following figures. Edge 7 is located in Γ min is not added to the list, i.e. edge 7 is discarded.

[0187] Figures 18 and 19 show the next step of the algorithm: visit edge 6. Edge 6 separates two tetrahedra with different labels, while being assigned an identity permutation, and is therefore identified as violating labeling consistency. Furthermore, edge 6 separates two tetrahedra that belong to two different connected components. Therefore, the labels of one of the two connected components are swapped, and then the two connected components are merged, as shown in Figure 19. Edge 6 is located in Γ min is not added to the list, i.e. edge 6 is discarded.

[0188] Figures 20 and 21 show the next step of the algorithm: visit edge 5. Edge 5 is identified as violating labeling consistency because it separates two tetrahedra with the same label while being assigned a swap permutation. Furthermore, edge 5 separates two tetrahedra that belong to two different connected components. Therefore, the labels of one of the two connected components are swapped, and then the two connected components are merged, as shown in Figure 21. Edge 5 is located in Γ min is not added to the list, i.e. edge 5 is discarded.

[0189] Figures 22 and 23 show the next step of the algorithm: visit edge 4. Edge 4 separates two tetrahedra with different labels, while it is assigned an identity permutation and is therefore identified as violating labeling consistency. Furthermore, edge 4 separates two tetrahedra that belong to the same connected component. Thus, as shown in Figure 23, edge 4 is a tetrahedron that is part of the solution Γ. min (represented by cutting edge 4 in Figure 23). In other words, edge 4 is given a reward.

[0190] Figure 24 shows the next step in the algorithm: visiting edge 3. Edge 3 is identified as not violating labeling consistency because it separates two tetrahedra with the same label while being assigned an identity permutation. Therefore, as shown in Figure 24, edge 3 is in the solution Γ min Edge 3 is not added to the tetrahedrons that are already in the same connected component.

[0191] Figures 25 and 26 show the next step in the algorithm: visit edge 2. Edge 2 is identified as violating labeling consistency because it separates two tetrahedra with the same label while being assigned a swap permutation. Furthermore, edge 2 separates two tetrahedra that belong to the same connected component. Thus, as shown in Figure 26, edge 2 is located in the solution Γ min (represented by cutting edge 2 in Figure 26). In other words, edge 2 is given a reward.

[0192] Figures 27 and 28 show the next step of the algorithm: visit edge 1. Edge 1 separates two tetrahedra with different labels, while it is assigned an identity permutation and is therefore identified as violating labeling consistency. Furthermore, edge 1 separates two tetrahedra that belong to the same connected component. Thus, as shown in Figure 28, edge 1 is located in the solution Γ min (represented by cutting edge 1 in Figure 28). In other words, edge 1 is given a reward.

[0193] Figure 29 shows the end of the algorithm, where all edges have been visited. Figure 30 shows the output, including rewarded edges 4, 2, and 1. Figures 31 and 32 show the corresponding output triangles 4, 2, and 1. The output triangles are shown as thick black lines in Figures 31 and 32 and form the determined second open triangulation surface.

[0194] We now describe an algorithm for performing step S40 of determining a first open triangulated surface having one or more loops as its boundary. The algorithm:

number

number

[0195] Before describing this algorithm further, the concept of a Delaunay complex will be explained in more detail.

[0196] <Dlaunay complex>

[0197] A (finite) set of points

number

number

[0198] The task the algorithm performs can be reformulated as follows: Consider a set A of edges of a Delaunay complex such that each vertex of the complex is shared by an even number of edges in A, e.g., 0 or 2; this example occurs in many practical applications. The algorithm searches for a set Γ0 of triangles such that every edge in A is shared by an odd (e.g., 1) triangle in Γ0, and every edge not in A is shared by an even (e.g., 0 or 2) triangle in Γ0. Saying that a Delaunay complex has trivial 1-homology precisely means that there exists such a Γ0 for whatever A is chosen. Figure 8 shows the solution Γ given a loop A in two dimensions.

[0199] Sublinks of vertices of a 3D Delaunay complex Links of simplicial complexes K and τ

number

number

number

number

number

number

[0200] FIG. 33 shows a link 332 of a vertex 330 and a lower link 334 of the vertex 330, with the axis indicating the z coordinate. If Del(V) is three-dimensional, then link lk of vertex a (Del(V)) (a) and lower link llk (Del(V)) (a) is a two-dimensional simplicial complex. Each tetrahedron in Del(V) containing a is lk (Del(V)) (a) creates a triangle, and each triangle of Del(V) containing a is lk (Del(V)) (a) creates an edge, and each edge of Del(V) containing a is lk (Del(V)) (a) creates a vertex. These triangles or edges have the z coordinates of all their vertices z a llk if less than (Del(V)) It belongs to (a).

[0201] FIG. 34 shows the lower link LL of vertex a.

[0202] As is known per se, a topological space, in particular a simplicial complex, is said to be contractible if it has a homotopy class of one point. In particular, a contractible simplicial complex K is connected, i.e., every pair of vertices in K is connected by a path of edges in K. The algorithm implementing S40 incorporates the fact that in a Delaunay complex, the sublinks of a vertex are empty or contractible. In particular, the following lemma holds:

[0203] Lemma 3: Let Del(V) be

number

number

[0204]

number

[0205] <Proof of Lemma 3> The coordinate z of a is z a Delaunay triangulation and llk Del(V) By the definition of (a), llk Del(V) There is a one-to-one correspondence between the vertices in (a) and the (possibly unbounded) facets that contribute to the boundary of the Voronoi cell in a.

[0206] By definition, the z-coordinate v z A vertex v with a Voronoi cell has a common boundary with the Voronoi cell of a, and v z <z a A vertex is in the lower link of a if and only if a has minimum z in V. Thus, the Voronoi cell of a is contained in the half-space containing a and bounded by the plane bisector of a and v. Thus, such a vertex exists if and only if no perpendicular ray starting from a and going to negative z is contained in the Voronoi cell of a. We see that a perpendicular ray starting from a and going to negative z is contained in the Voronoi cell of a if and only if a has minimum z in V, thus proving the first statement. The subenvelope of a's Voronoi cell is the union of facets that are dual to a vertex of a's sublinks. If this subenvelope is nonempty, its projection onto the horizontal plane is homeomorphic to a 2-dimensional convex polytope. Hence, this subenvelope is contractible. Since the sublinks of v are the pulse body of the set of facets of the Voronoi cell's subenvelope, the second statement of the lemma follows from the pulse body theorem.

[0207] This proves Lemma 3.

[0208] We now describe an algorithm for computing 2 chains for a given boundary of a 3-dimensional Delaunay complex that implements S40.

[0209] The algorithm is described in the following pseudocode: Algorithm 2: Finding a set of triangles with a given boundary Input: Delaunay complex Del(V) and the set A0 of edges of Del(V) with zero boundary Output: The set of triangles in Del(V) such that ∂Γ0 = A0 [Table 2] That is, given a 2-cycle A0, Algorithm 2 computes the 2-chain Γ0 such that ∂Γ0 = A0. In practical implementations, we operate on sets rather than vectors, so the chain or cycle of zeros is a set-theoretic representation rather than an algebraic "0".

number

number

number

[0210] <Principle and accuracy of Algorithm 2> Next, the principle and correctness of Algorithm 2 will be explained with reference to Figures 35 to 37, which explain the algorithm. Γ0 is initialized as the empty set, and A is initialized as A0. If A is empty, the algorithm returns. Otherwise, the procedure GetHighestVertex(A) returns a, the highest vertex of A with the maximum z coordinate. Since A is a cycle, its bounds are zero by definition. As shown in Figure 35, an even number of edges in A are those that connect a to the descendant links llk of a in Del(V) returned by the procedure GetAdjacentVertices(a,A). (Del(V)) (a) The set of vertices V a and connect. a is the highest vertex of V, so V a The vertices of are below a and connected to a by an edge. a is the lower link llk (Del(V)) A is a subset of the vertices in (a). This set is nonempty and has cardinality because A is a cycle. The procedure GetLowerLink(a, Del(V)) returns a skeleton LL of the lower links of a in Del(V), i.e., a graph whose vertices and edges are the vertices and edges of the lower links, as shown in Figure 35.

[0211] Then, the main step of Algorithm 2 is to divide this even edge into V a llk that connects pairs of vertices in (Del(V)) (a) Path E = GetEdgesConnecting(V a ,LL) (see Figure 36) to V a vertices of Γ0, and for each edge e of these paths,

number

[0212] Figure 37 shows the updated A cycle after the main step. After each step of the algorithm, the following properties hold:

number

number

number

[0213]

number

[0214] After each step, the z-coordinate of the highest vertex of A decreases, so

number

[0215] <Procedure GetHighestVertex(A)> This procedure needs to maintain an ordered map over the evolving set of edge vertices of A. Del(V) For a size of n, each vertex in Del(V) can only be inserted once, so the total cost of all calls to the procedure is O(n log n).

[0216] <Procedure GetLowerLink(a,Del(V))> The procedure GetLowerLink(a,Del(V)) returns one skeleton LL of the lower link of a in Del(V), i.e., a graph whose vertices and edges are the vertices and edges of the lower link, as shown in Figure 35. Since no two vertices in Del(V) have the same z coordinate, we can define the following total order on the edges of Del(V): Each edge in Del(V) can be represented by an ordered pair of vertices (v1,v2) such that z(v1)>z(v2). Then, we can define the following total order on the edges of Del(V):

number

[0217] In other words,

number

[0218] Similarly, each triangle in Del(V) is a triple of three ordered vertices (v1) > z(v2) > z(v3). 1、 v 2、 v3), and the following total ordering can be defined for the triangles in Del(V):

number

[0219] Again, sorting all triangles along this order (or, in other words, inserting them into an ordered set data structure) costs 0log(n), where n is the size of Del(V). As mentioned before, the descendant link llk of vertex a (Del(V)) The set of edges in (a) has a one-to-one correspondence with the set of ordered triples whose first vertex is a. These triples of the form (a,.,.) are consecutive in the ordered set of triples.

[0220] It can be observed that this construction implies that the set of all vertices of all lower links has a one-to-one correspondence with the edges of Del(V), and the set of all edges of all lower links has a one-to-one correspondence with the triangles of Del(V), and thus the upper bound on the total size of the 1-skeleton of all lower links is the size n of Del(V).

[0221] Thus, after a preprocessing step that generates an ordered set of edges and triangles at cost log(n), each call to the procedure GetLowerLink searches for an entry in the ordered set of edges and triangles at cost log(n), resulting in llk (Del(V)) The vertices and edges in (a) are contiguous in their respective ordered sets, each with an upper bound on size n. The total cost of GetLowerLink is Olog(n).

[0222] Procedure GetAdjacentVertices (a,A) Given A as a set of edges, i.e., a set of vertex pairs, it is possible to associate a set of adjacent vertices to each vertex in time linear in the size of A. Along the way, the algorithm updates A, so maintaining these adjacencies has a cost of at most Olog(n), since each edge can only be inserted once. GetAdjacentVertices(a,A) returns the set of adjacent vertices for each vertex.

number

[0223] Procedure GetEdgesConnecting(v a ,LL) This procedure requires the V a The number of vertices in must be even and the graph LL must be connected.

[0224] Let m denote the size of the lower link LL. One possibility is to compute a path for each pair of vertices, for example by Dijkstra's algorithm in O(mlogm) or by a recursive search in O(m) (it is not necessary to minimize the paths), and then add all these paths (modulo 2). However, even if this solution is likely to be efficient in practice, it does not yield the optimal worst-case complexity. Indeed, in the particular case where the lower link of the highest vertex a0 includes all other vertices (m=n-1), and A connects a0 to 2k vertices in LL, it is necessary to compute k paths in LL with cost km, so that k=m / 2=(n-1) / 2, where n is a quadratic cost.

[0225] Here, we explain the O(m) algorithm for GetEdgesConnecting, which maintains the overall O(nlogn) complexity of Algorithm 2.

[0226] In fact, the procedure GetEdgesConnecting is similar to the overall algorithm with one dimension less: 0 cycles V a Specify ∂E=V a However, there is no immediate recursion here. To apply a similar algorithm, we first need to define an ordering of vertices such that all descendant links of a vertex are contractible except for the empty one.

[0227] To obtain this ordering, we first consider the spanning tree ST of LL. LL This is possible because the graph is connected. LL Since is connected, E is ST LL You can find it in ST LL The computation of can be performed in O(m) time using a recursive search of LL, assigning a parent field to each vertex. To do so, one can choose an arbitrary vertex as the root, mark it as "visited," assign "invalid" to its parent field, and store it on the stack. Next, if the stack is not empty, one pops the top vertex v from the stack, assigns the "parent" field to v for all of its neighbors not marked as visited, marks them as visited, and stores them on the stack. Along the same process, one can assign an integer rank to each vertex, while maintaining O(m) complexity, such that the root has O rank and each non-root vertex has a higher rank than its parent. For example, one can increment a counter each time a vertex is marked as "visited," and assign the counter value as the vertex's rank.

[0228] In that case, we have a similar situation to Lemma 3, where the height z is replaced by the vertex rank. In fact, each non-root vertex is in the ST LL has a single vertex as a subordinate link of ST LL The descendant link of the root is empty. ST LL Starting from m′, another algorithm, Algorithm 3, is presented. <mはT=ST LLis the number of vertices in the tree. Algorithm 3 finds the set of edges for a particular boundary in the tree.

[0229] Algorithm 3 is explained in the following pseudocode: Algorithm 3: Find the set of edges for a given boundary in the tree structure Input: T as a tree structure, V0 as an even set of vertices of T represented by a matrix of memberships Output: A set E of edges in T such that ∂E=V0 [Table 3] Algorithm 3 is similar to Algorithm 2, but with one less dimension. Here, the procedure GetHighestVertex(V) is not needed.

[0230] Indeed, the required highest vertex belonging to V is given by a for loop iterating from the highest-ranked vertex m′−1 to the vertex of rank 1, where height z is replaced by the rank of the vertex. At each step, V contains an even number of vertices, Vertex(rank) is the highest vertex of V, and this highest vertex cannot be Vertex(0), so the for loop terminates at rank 1.

[0231] If an array of size m contains all the vertices indexed by rank, each call to the procedure Vertex(rank) has cost O(1). If the set of vertices to evolve, V, is represented by a Boolean array of size m, whose entry k indicates the membership of the vertex of rank k to v, then the membership decision

number

number

[0232] Lemma 4: Given a Delaunay complex Del(V) of size n and A0 a cycle (the set of edges with zero boundaries) of Del(V), Algorithm 2 computes in O(n log n) time the Γ0,2-chain (the set of triangles) of Del(V) such that ∂Γ0 = A0.

[0233] As mentioned above, Algorithms 1 and 2 may implement steps S40 and S50 of the method, where the lexicographic order is based on the total triangle order, i.e., the triangle order, as will be explained in more detail next.

[0234] Simplex ordering Let the radius of the smallest enclosing sphere of the 2-simple σ and the radius of the circumscribed circle be R, respectively. B (σ), R C (σ). A total order can be defined for 2-simplices as follows:

[0235]

number

number

[0236] Proposition 2: N≧3

number

number

number

[0237]

number

[0238] We will now describe an example application of the method in which the above implementations are used.

[0239] A first application of the method in which the above implementations are used will now be described with reference to Figures 38-41.

[0240] In this first application, it is possible to mesh a partial acquisition, i.e., determine an open triangulated surface representing the skin of a portion of the structure of a real scene, where the real scene is partially represented by a point cloud resulting from the partial acquisition (e.g., featuring holes). This first application arises in contexts where physical constraints of real-world scenes (e.g., surface occlusions and optical properties) can lead to partial and incomplete 3D acquisitions. In these contexts, providing an accurate, hole-free mesh reconstruction can be challenging. By taking as input one or more 3D paths (i.e., loops) that define the boundaries of the surface to be reconstructed, the method of the present invention guarantees a hole-free mesh and provides high-quality reconstructions, even for noisy acquisitions. In this first application, as described above, one or more loops are defined by the user.

[0241] Figures 38 to 40 show an example of the first application. In the example shown in these figures, the real-world scene is a building scene including a cathedral structure. The 3D point cloud partially represents the cathedral structure. It should be understood that the cathedral is an example, and the first application also applies to other architectural scenes. The 3D point cloud shown in Figure 38 contains 943K points.

[0242] As shown in Figure 38, the user interactively selects a list of ordered points that roughly define the boundary of the desired surface. The list of user-selected points is represented by large circular dots in Figure 38. The boundary can be oversampled so that all edges on the boundary are present in a 3D triangulation of the points. Figure 39 shows the oversampled boundary from the list of user-selected points. The method then efficiently meshes the points under the imposed boundary, i.e., determines an open triangulated surface that forms the skin of a portion of the cathedral (i.e., the boundary between the cathedral's walls and the air outside and inside the walls). Figure 40 shows the determined skin, obtained in 18 seconds.

[0243] FIG. 41 shows two other examples of open triangulated surfaces determined in step S50 by the method under the imposed boundaries.

[0244] A second application of the method, in which the above implementation is used, is described with reference to FIG. 42. This second application occurs in the context of meshing large scenes and parallelizing and updating the mesh of large scenes. Modern acquisition techniques can generate a large number of accurate measurements. As the acquisition boundaries become larger, the storage capacity (RAM or disk) required to simultaneously hold all measurements can become a limiting factor when meshing these large scenes. For example, standard or inexpensive computers are limited by 16 gigabytes of memory, which may be too small for point clouds of more than 10M points.

[0245] The present method avoids these problems by using spatial tiling to divide the acquisition into smaller chunks. Furthermore, the present method ensures that two meshes on adjacent tiles have a common junction. In fact, the present method allows specifying cycles that bound the resulting open triangulated surface. In other words, one or more loops provided in step S30 (e.g., by a user, as described above, or automatically) allow the scene to be divided into smaller chunks with common junctions that are watertight.

[0246] Furthermore, local mesh edits can be performed efficiently by defining the boundaries of the region affected by the change and recalculating the open triangulation surface within the affected region, without the need to recalculate the rest of the open triangulation surface, and the junctions with the affected region correspond exactly to the rest of the open triangulation surface.

[0247] A second application is illustrated in Figure 42, which shows a large point cloud representing a terrain that has been meshed into two parts thanks to the definition of a boundary (i.e., a loop). The loops are represented by circular dots in Figure 42. That is, an open triangulated surface is determined in step S50 for each region / section of the terrain that is defined to have a given loop as its boundary. Imposing a boundary by defining a loop creates an intersection between the two meshes, as shown in Figure 42.

[0248] The method is computer-implemented, meaning that the steps (or substantially all steps) of the method are performed by at least one computer or any system. Thus, the steps of the method are performed by a computer, possibly fully automatically or semi-automatically. In an example, triggering of at least some steps of the method may be performed via user-computer interaction. The level of user-computer interaction required depends on the expected level of automation and may be balanced against the need to implement user wishes. In an example, this level may be user-defined and / or predefined.

[0249] A typical example of a computer implementation of the method is executing the method on a system adapted for this purpose. The system may include a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program including instructions for carrying out the method. The memory may also store a database. The memory is hardware adapted for such storage, possibly including several physically distinct parts (e.g., one for the program and one for the database).

[0250] FIG. 43 shows an example of a system, where the system is a client computer system, for example a user's workstation.

[0251] The client computer in this example includes a central processing unit (CPU) 1010 connected to an internal communication BUS 1000 and a random access memory (RAM) 1070 also connected to the BUS. The client computer further includes a graphics processing unit (GPU) 1110 associated with a video random access memory 1100 connected to the BUS. The video RAM 1100 is also known in the art as a frame buffer. A mass storage controller 1020 manages access to mass storage devices such as a hard drive 1030. Mass memory devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks 1040. Any of the foregoing may be supplemented by or incorporated by specially designed application-specific integrated circuits (ASICs). A network adapter 1050 manages access to a network 1060. The client computer may also include a haptic device 1090, such as a cursor control device, keyboard, etc. A cursor control device is used in the client computer to allow a user to selectively position a cursor at any desired location on the display 1080. Furthermore, the cursor control device allows a user to select various commands and input control signals. The cursor control device includes several signal generating devices for inputting control signals to the system. Typically, the cursor control device may be a mouse, and the buttons on the mouse are used to generate the signals. Alternatively or additionally, the client computer system may include a pressure-sensitive pad and / or a pressure-sensitive screen.

[0252] A computer program may include computer-executable instructions, and the instructions may include means for causing the system to perform the method. The program may be recordable on any data storage medium, including the system's memory. The program may be implemented, for example, in digital electronic circuitry, or computer hardware, firmware, software, or a combination thereof. The program may be implemented as an apparatus, such as an article of manufacture tangibly embodied in a machine-readable storage device for execution by a programmable processor. The method steps may be performed by a programmable processor that executes a program of instructions, operates on input data, and generates output to perform the functions of the method. Thus, the processor is programmable and may be coupled to receive data and instructions from and transmit data and instructions to a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high-level procedural or object-oriented programming language, and may be implemented in assembly or machine language as appropriate. In either case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. Applying the program to a system in any case provides instructions for performing the method.

Claims

1. 1. A computer-implemented method for reconstructing a structure of a real scene in 3D, comprising: providing a first open triangulation surface (S10-S40), the first open triangulation surface being a set of triangles of a tetrahedral mesh of a 3D point cloud representing at least a portion of a structure; a step (S50) of determining a second open triangulation surface representing the skin of a portion of the structure, said determining step (S50) searching for candidate open triangulation surfaces, each being a set of triangles of the tetrahedral mesh, and penalizing the top-ranked candidate open triangulation surfaces according to a lexicographical order, said lexicographical order being based on the triangle order, wherein a first candidate open triangulation surface having first triangles ordered according to a descending triangle order is ordered against a second candidate open triangulation surface having second triangles ordered according to a descending triangle order, said triangle order penalizing triangle size; and the determined second open triangulation surface violates the consistency of labeling of the tetrahedral mesh by two predetermined labels, wherein a triangle respects the consistency of labeling if it belongs to the first open triangulation surface and separates two tetrahedrons with different labels, or if it does not belong to the first open triangulation surface and separates two tetrahedrons with the same label. A method comprising:

2. The tetrahedral mesh has the following properties: the tetrahedrons of the tetrahedral mesh connect to form a convex hull of the points of the point cloud; an intersection of any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron; 2. The method of claim 1, comprising:

3. The method of claim 1 or 2, wherein the triangle ordering penalizes, for each triangle, high values of the radius of the smallest enclosing circle.

4. The method of claim 3 , wherein the triangle ordering further penalizes small values of circumscribing circle radius for the first triangle and the second triangle having the same minimum enclosing circle.

5. The method of claim 1 , wherein the tetrahedral mesh is a regular triangulation.

6. The search and penalty assignment may be: visiting the triangles of the tetrahedral mesh according to descending triangle order; 6. The method of claim 1, comprising: discarding visited triangles that are higher in the triangle order as long as labeling consistency is maintained.

7. 7. The method of claim 6, wherein the discarding comprises, for each visited triangle, if the triangle violates labeling consistency, rewarding the triangle or adjusting the labeling to respect consistency.

8. For each triangle visited, discard: Rewarding a triangle if it separates two tetrahedra in the same connected component and violates labeling consistency; If a triangle separates two tetrahedra in different connected components: When a triangle violates labeling consistency, we merge the connected components by switching the label of one of the connected components. The method of claim 7, further comprising: merging connected components when triangles respect labeling consistency.

9. The step of providing a first open triangulated surface comprises: providing one or more loops in a tetrahedral mesh (S30); The method according to any one of claims 1 to 8, comprising the step (S40) of determining an open triangulated surface within the tetrahedral mesh having said one or more loops as boundary.

10. The method of claim 1 , wherein the structure comprises at least one corner structure.

11. 11. The method according to claim 1, wherein the 3D point cloud is created by photogrammetry, laser scanning, LIDAR measurements, RGBD measurements, and / or medical or industrial tomography.

12. 12. The method of claim 1, wherein the 3D point cloud represents a portion of a closed shape and / or the ratio of the number of points in the 3D point cloud to the computer memory size is greater than a predetermined threshold.

13. A computer program comprising instructions for carrying out the method according to any one of claims 1 to 12.

14. A computer-readable storage medium having the computer program according to claim 13 recorded thereon.

15. 14. A computer comprising a processor coupled to a memory having the computer program of claim 13 recorded thereon.

Images