3d reconstruction of the structure of a real scene

Abstract

The invention relates in particular to a computer-implemented method for 3D reconstruction of a structure of a real scene. The method comprises providing a 3D point cloud representing the structure. The method further comprises determining a closed triangulated surface of a skin representing the structure. The determining explores candidate closed triangulated surfaces, each of the candidate closed triangulated surfaces at least meshing a respective portion of the 3D point cloud. The determining penalizes high ranks of the candidate closed triangulated surfaces according to a lexicographic order. The lexicographic order is based on a triangle order. The lexicographic order orders a first candidate closed triangulated surface having a first triangle with respect to a second candidate closed triangulated surface having a second triangle, the first triangle ordered according to a decreasing rank of the triangle order, the second triangle ordered according to a decreasing rank of the triangle order. The triangle order penalizes triangle size. This constitutes an improved method for 3D reconstruction of a structure of a real scene.

Description

Technical Field

[0001] This invention relates to the field of computer programs and systems, and more particularly to methods, systems, and programs for 3D reconstruction of the structure of a real scene. Background Technology

[0002] The market offers numerous systems and programs for the design, engineering, and manufacturing of objects. CAD stands for Computer-Aided Design, and it refers to software solutions for designing objects. CAE stands for Computer-Aided Engineering, and it refers to software solutions for simulating the physical behavior of future products. CAM stands for Computer-Aided Manufacturing, and it refers to software solutions for defining manufacturing processes and operations. In such computer-aided design systems, graphical user interfaces play a crucial role in technical efficiency. These technologies may be embedded in Product Lifecycle Management (PLM) systems. PLM refers to a business strategy that helps companies share product data, apply common processes, and leverage enterprise knowledge for product development across extended enterprise concepts, from concept to the end of the product lifecycle. PLM solutions offered by Dassault Systèmes (traded as CATIA, ENOVIA, and DELMIA) provide an engineering center for organizing product engineering knowledge, a manufacturing center for managing manufacturing engineering knowledge, and an enterprise center that enables the enterprise to integrate and connect with the engineering and manufacturing centers. The entire system provides an open object model that links products, processes, and resources to enable dynamic, knowledge-based product creation and decision support, driving optimized product definition, manufacturing preparation, production, and service.

[0003] In this context and other contexts, 3D reconstruction of real-world scene structures is becoming increasingly important.

[0004] Some existing 3D reconstruction methods are based on smoothness assumptions, which may not apply to certain contexts. Notably, these methods are inefficient in scenes characterized by corner structures (e.g., the corners of buildings).

[0005] Other methods fit implicit functions to point clouds, but their accuracy is not satisfactory.

[0006] Other methods are sensitive to density variations within point clouds, making them less robust. For example, such methods may not be robust to noise and outliers, and may fail to construct watertight models for some point clouds that do not meet appropriate density requirements.

[0007] Other methods may involve fine-tuning parameters such as light consistency, surface visibility, or regularity. Therefore, these methods may require some manual modifications for specific examples of the scene, making them less general and / or less ergonomic.

[0008] In this context, there is still a need for an improved method for 3D reconstruction of structures in real-world scenes. Summary of the Invention

[0009] Therefore, a computer-implemented method for 3D reconstruction of structures in a real-world scene is provided. The method includes providing a 3D point cloud representing the structure. The method also includes determining closed triangulation surfaces representing the skin of the structure. This determination explores candidate closed triangulation surfaces, each of which meshes at least a corresponding portion of the 3D point cloud. The determination penalizes high-ranking candidate closed triangulation surfaces according to lexicographical order. The lexicographical order is based on triangle order. First candidate closed triangulation surfaces having a first triangle are sorted relative to second candidate closed triangulation surfaces having a second triangle, the first triangle being sorted according to a descending rank of the triangle order, and the second triangle being sorted according to a descending rank of the triangle order. The triangle order penalizes the size of the triangles.

[0010] This constitutes an improved method for 3D reconstruction of structures in real-world scenes.

[0011] The method may include one or more of the following:

[0012] - Each candidate closed triangulation surface includes a corresponding set of triangular faces, and each triangular face is a triangular face of a corresponding tetrahedron divided by the tetrahedral mesh of the 3D point cloud;

[0013] -The tetrahedral meshing exhibits the following characteristics:

[0014] The union of the tetrahedrons in the tetrahedral mesh forms the convex hull of the points in the point cloud; and

[0015] The intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex, an edge, or a face of the first tetrahedron.

[0016] - For the corresponding triangle, the triangle sequence penalizes the highest value of the radius of the smallest enclosing circle;

[0017] - For a first triangle and a second triangle with the same minimum enclosing circle, the triangle order also penalizes the smaller value of the radius of the circumcircle;

[0018] - Each candidate closed triangulation surface includes a corresponding set of triangular faces. Each triangular face is a triangular face of a corresponding tetrahedron divided by a tetrahedral mesh of the 3D point cloud. The tetrahedral mesh division is a regular triangulation.

[0019] -The exploration and the punishment include:

[0020] Access the triangles of the tetrahedral mesh according to the descending order of the triangle sequence; and

[0021] Discard the visited triangle with the highest ranking in the triangle order;

[0022] - Each tetrahedron in the tetrahedral mesh defines a corresponding region, and the determination further includes:

[0023] The outer tetrahedron and the inner tetrahedron of the tetrahedral mesh are declared, wherein the outer tetrahedron and the inner tetrahedron represent the outer region and the inner region of the structure, respectively.

[0024] and:

[0025] The access to the triangles includes: for each accessed triangle, evaluating whether the accessed triangle separates the interior region from the exterior region; and

[0026] The discarding of the triangle includes: for each visited triangle that does not separate the inner region from the outer region, the triangle separates the two regions, suppresses the triangle, and merges the two regions;

[0027] - Each visited triangle that separates the internal region from the external region is preserved, and the preserved triangles form the defined closed triangulation surface;

[0028] - The determination includes providing a region within the 3D point cloud representing the internal location of the structure, all of which surround the candidate closed triangulation surfaces;

[0029] - The structure includes at least one corner structure; and / or

[0030] - The 3D point cloud is derived from photogrammetry, laser scanning, RGB-D measurement and / or medical or industrial tomography, and / or the real scene is an architectural scene, a building interior scene, a mining scene, organic tissue or mechanical parts.

[0031] A computer program is also provided, which includes instructions for performing the method.

[0032] A computer-readable storage medium having the computer program thereon also is provided.

[0033] A computer is also provided, which includes a processor coupled to a memory and a graphical user interface, wherein the computer program is recorded in the memory. Attached Figure Description

[0034] Embodiments of the invention will now be described by way of non-limiting example and with reference to the accompanying drawings, wherein:

[0035] - Figure 1 A flowchart illustrating an example of the method is shown;

[0036] - Figure 2 An example of the system is shown;

[0037] - Figures 3A-3B The method is shown in 9A-9B; and

[0038] - Figures 10 to 39 The method is shown. Detailed Implementation

[0039] refer to Figure 1 The flowchart presents a computer-implemented method for 3D reconstruction of structures in real-world scenes. The method includes providing a 3D point cloud representing the structure (S10). The method further includes determining, S30, closed triangulation surfaces representing the skin of the structure. Determination S30 explores candidate closed triangulation surfaces, each candidate closed triangulation surface meshing at least a corresponding portion of the 3D point cloud. Determination S30 penalizes high-ranking candidate closed triangulation surfaces according to lexicographical order. The lexicographical order is based on triangle order. The lexicographical order sorts first candidate closed triangulation surfaces with a first triangle relative to second candidate closed triangulation surfaces with a second triangle, the first triangle being sorted according to descending rank in the triangle order, and the second triangle being sorted according to descending rank in the triangle order. The triangle order penalizes the size of the triangles.

[0040] This constitutes an improved method for 3D reconstruction of structures in real-world scenes.

[0041] In particular, this method allows for the 3D reconstruction of the structure of a real-world scene. Furthermore, it does this without requiring smoothness assumptions on the provided S10 point cloud and / or structure, making it robust and capable of handling diverse real-world scene structures. In fact, for example, the method is able to perform 3D reconstruction of structures from a provided irregular 3D point cloud, which includes, for example, portions representing irregular structures, such as corner structures. Moreover, the method does this with minimal (e.g., none) manual parameter tuning, making it ergonomic. It is noteworthy that, as discussed further below, exploration and penalty can be performed automatically in any example of this method.

[0042] Furthermore, Determination S30 penalizes high rankings of candidate closed triangulation surfaces according to lexicographical order (i.e., penalizes the height of the ranking). The lexicographical order is based on the triangle order relative to the second candidate closed triangulation surface, ranking the first candidate closed triangulation surface. The triangles of each candidate closed triangulation surface are ranked according to the descending ranking of the triangle order, and the triangle order penalizes the size of the triangles. This is equivalent to Determination S30 selecting from the candidate closed triangulation surfaces and penalizing the presence and / or large number of large triangles within the candidate closed triangulation surfaces. Large triangles (e.g., triangles with extensively stretched shapes) can represent regions characterized by outliers and / or low-density points. In other words, large triangles represent regions of the point cloud that do not represent any part of the structure's skin, such as regions including outliers and / or regions corresponding to the interior or exterior of the structure. Therefore, by employing lexicographical order and triangle order in this way, Determination S30 seeks the surface among the candidate closed triangulation surfaces that best represents the skin of the structure. In particular, this method effectively handles the presence of outliers and / or noise and / or low-density regions.

[0043] The method is computer-implemented. This means that the steps (or essentially all steps) of the method are executed by at least one computer or any similar system. Therefore, the execution of the steps by a computer may be fully automatic or semi-automatic. In the example, at least some steps of the method may be triggered through user-computer interaction. The required level of user-computer interaction may depend on the anticipated level of automation and be balanced with the need to fulfill the user's intentions. In the example, this level may be user-defined and / or predefined.

[0044] A typical example of a computer implementation of the method is to execute it using a system suitable for this purpose. This system may include a processor coupled to memory and a graphical user interface (GUI) on which a computer program containing instructions for executing the method is stored. The memory may also store a database. The memory is any hardware suitable for such storage and may include several physically distinct parts (e.g., one for the program and possibly one for the database).

[0045] Figure 2 An example of a system is shown, where the system is a client computer system, such as a user's workstation.

[0046] The client computer in this example includes a central processing unit (CPU) 1010 connected to an internal communication bus 1000 and random access memory (RAM) 1070 also connected to the bus. The client computer is also provided with a graphics processing unit (GPU) 1110 associated with video random access memory 1100 connected to the bus. The video RAM 1100 is also referred to in the art as a frame buffer. A mass storage device controller 1020 manages access to a mass storage device (e.g., a hard disk drive 1030). Mass storage devices suitable for tangibly representing computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM disks 1040. Any of the above can be supplemented or incorporated into a specially designed ASIC (Application-Specific Integrated Circuit). A network adapter 1050 manages access to a network 1060. The client computer may also include a haptic device 1090, such as a cursor control device, a keyboard, etc. A cursor control device is used on the client computer to allow the user to selectively position the cursor at any desired location on the monitor 1080. Furthermore, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes multiple signal generating devices for inputting control signals to the system. Typically, the cursor control device can be a mouse, with buttons used to generate signals. Alternatively or additionally, the client computer system may include a sensitive pad and / or a sensitive screen.

[0047] The computer program may include computer-executable instructions, including units for causing the system to perform the method. The program may be recordable on any data storage medium, including the system's memory. The program may be implemented, for example, as digital electronic circuitry or as computer hardware, firmware, software, or a combination thereof. The program may be implemented as an apparatus, such as a product tangibly embodied in a machine-readable storage device for execution by a programmable processor. The method steps can be performed by a programmable processor executing the instruction program to perform the function of the method by manipulating input data and generating output. Therefore, the processor may be programmable and coupled to receive data and instructions from the data storage system, at least one input device, and at least one output device, and to send data and instructions to the data storage system, at least one input device, and at least one output device. The application may be implemented in a high-level procedural or object-oriented programming language, or assembly or machine language, if desired. In any case, the language may be a compiled language or an interpreted language. The program may be a complete installation or update program. In any case, the application of the program on the system results in instructions for performing the method.

[0048] This system can be a CAD system, which typically manipulates modeled objects. In the context of this disclosure, any 3D point cloud representing the structure of a real-world scene and / or any 3D meshing or triangulation of that point cloud or at least a portion thereof can be a modeling object of a CAD system. In other words, the method can typically manipulate modeled objects.

[0049] A modeling object is any object defined by data stored, for example, in a database. By extension, the term "modeling object" identifies the data itself. Depending on the type of system, modeling objects can be defined by different kinds of data. This system can indeed be a CAD system. In this system, modeling objects are defined by corresponding data. Therefore, one might speak of CAD objects.

[0050] The term "CAD system" also implies any system suitable for designing modeling objects based on their graphical representations, such as CATIA. In this context, the data defining the modeling object includes the data that allows the modeling object to be represented. CAD systems can provide a representation of CAD modeling objects, for example, using edges or lines (and in some cases, faces or surfaces). Lines, edges, or surfaces can be represented in various ways, such as non-uniform rational B-splines (NURBS). Specifically, CAD files contain specifications from which geometry can be generated, which in turn allows the generation of representations. The specifications of the modeling object can be stored in a single CAD file or multiple CAD files. The typical size of a file representing a modeling object in a CAD system is in the range of one megabyte per part. And modeling objects can often be assemblies of thousands of parts.

[0051] In the context of CAD, modeling objects can typically be 3D modeling objects, such as representations of products, like parts or assemblies of parts, or possibly assemblies of products. The term "3D modeling object" refers to any object modeled from data that allows for its 3D representation. 3D representation allows viewing parts from all angles. For example, when represented in 3D, 3D modeling objects can be manipulated and rotated around any axis of their model or around any axis on the screen on which the representation is displayed. Specifically, this does not include 2D icons that are not 3D modeled. The display of 3D representations facilitates design (i.e., increases the statistical speed at which designers complete their tasks). Since product design is part of the manufacturing process, it can accelerate manufacturing processes in industry.

[0052] 3D modeling objects can represent the geometry of products to be manufactured in the real world (e.g., industrial) after their virtual design is completed using, for example, CAD software solutions or CAD systems. Examples include (e.g., mechanical) parts or assemblies of parts (or equivalent assemblies of parts, since from a methodological perspective, an assembly of parts can be viewed as the parts themselves, or the method can be applied independently to each part of the assembly) or more generally, any rigid body assembly (e.g., a moving mechanism). CAD software solutions allow for product design in a wide variety of unrestricted industrial sectors, including: aerospace, construction, building, consumer goods, high-tech equipment, industrial equipment, transportation, and marine and / or offshore oil / gas production or transportation.

[0053] We will discuss the concept of providing S10 to represent the 3D point cloud of this structure. Before discussing this concept, we will now discuss the data structures involved.

[0054] A real-world scene represents a portion of the real world, which may be or may include objects or arrangements of objects. The structure of a real-world scene is a tangible (e.g., concrete) physically closed shape within the scene, such as a closed arrangement of materials. The skin of a structure is its closed shell (or envelope), i.e., the material of the structure that forms contact between the opaque material inside the structure and the air and / or material outside. Therefore, the skin of a structure separates the interior of the structure (which is the opaque material inside) from the exterior of the structure (which is the air and / or material outside).

[0055] The 3D point cloud representing the structure of a real-world scene in this paper is a data structure in which each point represents a corresponding geometric entity located within the structure of the real-world scene. Each geometric entity represents a corresponding location within the structural scene (in other words, a corresponding portion and / or layout of the material constituting the structure). The aggregation (in other words, the union or juxtaposition) of the geometric entities fully represents the structure. Any 3D point cloud in this paper may include multiple points greater than 100,000, 1,000,000, or 10,000,000. It should be understood that the 3D point cloud may include points that do not represent corresponding geometric entities located within the structure. Those points may form regions with lower point density or may be part of regions with lower point density, as described below. Such regions, for example, represent the external regions of the structure. Some points in the point cloud may also be outliers, i.e., points originating from measurement errors and therefore not representing geometric entities of the structure.

[0056] Real-world scenarios can be architectural scenes such as buildings as seen from the outside. The structure here can be a closed, tangible physical shape formed by the building's exterior walls and foundation. In this case, the structure's skin consists of the surfaces of the exterior walls and the surface separating the building's foundation from the ground. Another example of an architectural scene is a tunnel. The structure here can be formed by the tunnel walls, in which case the structure's skin consists of the surfaces of the tunnel walls.

[0057] Alternatively, the real-world scenario could be an interior scene of a building, such as the interior of an enclosed room. The structure here could consist of the walls of the enclosed room, in which case the skin of the structure would be formed by the surfaces of the walls.

[0058] Alternatively, a real-world scenario could be a mining scenario, such as a mining tunnel. The structure here could consist of the tunnel walls, and the skin could be formed by the surface of those walls.

[0059] Alternatively, the real-world scenario could be organic tissue. The structure here could be the organic tissue itself, in which case the skin is the tissue's outer shell.

[0060] Alternatively, the real-world scenario could be a mechanical part. The structure here could be the part itself, in which case the skin is the part's outer shell.

[0061] Mechanical parts can be one or any possible combination of the following: parts of land vehicles (including, for example, automobile and light truck equipment, racing cars, motorcycles, truck and motor vehicle equipment, trucks and buses, trains), parts of aircraft (including, for example, fuselage equipment, aerospace equipment, propulsion equipment, defense products, aviation equipment, space equipment), parts of naval vehicles (including, for example, naval equipment, commercial ships, marine equipment, yachts and workboats, marine equipment), general mechanical parts (including, for example, industrial manufacturing machinery, heavy mobile machinery or equipment, installed equipment, industrial equipment products, metal products, tire products), electromechanical or electronic parts (including, for example, consumer electronics products, safety and / or control and / or instrumentation products, computing and communication equipment, semiconductors, medical devices and equipment), consumer goods (including, for example, furniture, home and garden products, leisure goods, fashion products, products of hard goods retailers, products of soft goods retailers), and packaging (including, for example, food and beverage and tobacco, beauty and personal care, and household product packaging). Additionally or alternatively, a mechanical part may be one or any possible combination of the following: molded part (i.e., part manufactured by a molding process), machined part (i.e., part manufactured by a machining process), drilled part (i.e., part manufactured by a drilling process), turned part (i.e., part manufactured by a turning process), forged part (i.e., part manufactured by a forging process), stamped part (i.e., part manufactured by a stamping process), and / or folded part (i.e., part manufactured by a folding process).

[0062] In the example, the structure includes at least one corner structure. A corner structure indicates that the structure has or substantially has a corner shape. The corner of the structure can be part of the structure that corresponds to the layout of materials forming the edge or acute angle of the shape. The corner structure can be a corner of a building, a corner of a tunnel, a corner formed by the wall of a room, a corner of a mining tunnel, a corner or acute angle in the layout of materials constituting a mechanical part, or a corner or acute angle in the layout of an organization constituting an organic structure.

[0063] In the example, the 3D point cloud originates from photogrammetry, laser scanning, RGB-D measurement, and / or medical or industrial tomography.

[0064] In the context of this method, point clouds representing the structure of architectural scenes can be derived from RGB-D measurements or photogrammetry, and optionally from structure-from-motion analysis. Point clouds representing the structure of mechanical parts can be derived from industrial tomography or laser scanning. Point clouds representing the structure of excavated scenes can be derived from laser scanning, such as laser scanning of the interior of a mine. Point clouds representing the structure of organic tissues can be derived from medical tomography.

[0065] Any 3D point cloud in this paper can be determined from physical measurements on a real-world scene. The provision of the 3D point cloud, S10, may specifically include: providing one or more physical sensors, each configured to acquire a corresponding physical signal, and acquiring one or more corresponding physical signals by operating the one or more physical sensors on the real-world scene (i.e., scanning the real-world scene with each sensor). The provision S10 may include the automatic determination of the measurement-based 3D point cloud according to any known technology. The one or more sensors may include multiple (e.g., RGB and / or image or video) cameras, and the determination may include motion-based structural analysis. The one or more sensors may alternatively or additionally include one or more depth sensors (e.g., on an RGB-D camera). The one or more sensors may alternatively or additionally include lasers (e.g., lidar) or ultrasonic transmitter-receiver systems. The one or more sensors may alternatively or additionally include one or more tomographic sensors. The tomographic sensors may be medical or industrial tomographic sensors.

[0066] Alternatively or additionally, providing the 3D point cloud S10 may include accessing and retrieving the 3D point cloud from a database storing the 3D point cloud. As mentioned above, the 3D point cloud may have already been acquired by operating one or more physical sensors and determining the 3D point cloud based on measurements obtained by the one or more sensors, and then stored in the database, even if providing S10 may not actually include acquiring the measurement results and determining the 3D point cloud. In any case, providing the 3D point cloud S10 can be performed by user actions. Providing the 3D point cloud in S10 may further include displaying the 3D point cloud on the display (e.g., GUI) of the computer performing the method.

[0067] The provided 3D point cloud may include one or more regions with different point densities. This means that the point density within the point cloud may vary from one region to another. The separation between the exterior and interior of a structure can typically correspond to the region of the point cloud with the highest point density. This region can be closed, meaning the region of the point cloud with the highest point density can form a closed shape that separates the region inside the structure (potentially called the "inner region") from the region outside the structure (potentially called the "outer region"). The closed shape can be called a closed dashed shape (because it is composed of points from the 3D point cloud) or simply a closed shape. The inner and outer regions are typically characterized by lower point densities. When providing a 3D point cloud, the closed shape can be so closed. Alternatively, the closed shape can be characterized by one or more holes, which are portions of the closed shape with lower point densities, for example, due to the technical impossibility of performing measurements on the real-world regions corresponding to these holes and / or due to measurement errors and / or due to measurement differences. In such cases, the provision of the 3D point cloud S10 may include the step of filling these holes with points upon user action or automatically to form a closed shape. Filling these holes can involve adding points to a 3D point cloud until the point density is substantially the same at any part of the closed shape. Adding points can be performed through graphical interaction between the user and the screen of the computer performing the method, for example, by repeatedly clicking on the point cloud to create points. In any case, regardless of whether filling occurs, the separation between the exterior and interior of the structure is represented by a region of the 3D point cloud that is a closed shape with the highest point density that separates the interior of the structure from the exterior, where both the interior and exterior regions have lower point densities. In the example, "highest density" means that the point density in the closed shape is higher than the ambient noise originating from the measurement. The 3D point cloud can be further characterized by outliers, which are points in the 3D point cloud located in regions with low point density and originating from measurement errors. In other words, outliers do not represent the geometric entities of the real scene but correspond to numerical and / or measurement artifacts.

[0068] The concept of 3D reconstruction of structures represented by 3D point clouds will now be briefly discussed. 3D reconstruction of a structure typically involves the action of calculating / determining the surface connecting the points of the 3D point cloud. This surface can be referred to as the "skin representing the structure" because it connects the points of the 3D point cloud to a closed surface that defines the boundaries (e.g., limits) between the regions representing the exterior and interior of the structure. Therefore, this surface represents the closed shell of the structure, in other words, the skin. For example, as previously mentioned, the boundaries can correspond to a high-density closed shape of points within the 3D point cloud. In this case, 3D reconstruction of the structure can involve defining / calculating the surface connecting the points of the closed shape to form a surface shell surrounding the closed shape.

[0069] Now let's discuss an example of 3D point clouds.

[0070] Figure 3A An example of the provided 3D point cloud is shown, which here represents a wall of a portion of a bridge. The scene here is an urban architectural scene, and the provision of the 3D point cloud S10 specifically includes motion-based structural analysis, as previously described. Figure 3B The 3D reconstruction of a structure represented by the point cloud in Figure 3A is shown as an example of the method to be discussed below.

[0071] Figure 4A An example of the provided 3D point cloud is shown, representing an engine in this context. The scene here depicts mechanical parts, and the 3D point cloud is derived from industrial tomography. Figure 4B An example is shown based on the method discussed below. Figure 4A 3D reconstruction of the structure represented by point cloud.

[0072] Figure 5A An example of the provided 3D point cloud is shown, representing the wall of a mine. The scene here is a mining scene, and the 3D point cloud is derived from a laser scan of the mine interior. Figure 5B An example is shown based on the method discussed below. Figure 5A 3D reconstruction of the structure represented by point cloud.

[0073] Figure 6A An example of the provided 3D point cloud is shown, representing the mandible in this case. The scene here is organic tissue, and the 3D point cloud is derived from medical tomography. Figure 6B An example is shown based on the method discussed below. Figure 6A 3D reconstruction of the structure represented by point cloud.

[0074] Figure 7A An example of the provided 3D point cloud is shown, which represents a rocker in this case. The scene here is a mechanical part, and the 3D point cloud is derived from a scan of the mechanical part. Figure 7B An example is shown based on the method discussed below. Figure 7A 3D reconstruction of the structure represented by point cloud.

[0075] Figure 8A An example of the provided 3D point cloud is shown, which here represents the exterior wall of a building. The scene here is an architectural scene, and the 3D point cloud is derived from a correlation process of several images of the scene, and S10 is provided involving motion-based structural analysis, as previously described. Figure 8B An example is shown based on the method discussed below. Figure 8A 3D reconstruction of the structure represented by point cloud.

[0076] Figure 9A An example of the provided 3D point cloud is shown, which here represents the Buddha. The scene here is an organic organization, and the 3D point cloud is derived from tomography. Figure 9B An example is shown based on the method discussed below. Figure 9A 3D reconstruction of the structure represented by point cloud.

[0077] The determination of the closed triangular section surface S30 representing the skin of this structure is now discussed.

[0078] The determined closed triangulation surface represents the skin of the structure. In other words, the closed triangulation surface represents the separation between the regions inside and outside the structure, such as the closed surface shell of the aforementioned closed shape of a 3D point cloud with the highest point density that separates the regions inside and outside the structure. The closed triangulation surface can be a meshed surface, for example, each vertex of a meshed closed triangulation surface can correspond to a point cloud of the 3D point cloud. In other words, the closed triangulation surface can form a surface that meshes at least a portion of the points of the closed shape of the 3D point cloud with the highest point density that separates the regions inside and outside the structure. In other words, the method uses the original points of the provided 3D point cloud to perform surface reconstruction, which makes the method particularly accurate because, as previously stated, the points of the point cloud represent real geometric entities located on the structure.

[0079] S30 identifies candidate closed triangulation surfaces. Each candidate closed triangulation surface has triangles, for example, at least four triangles. Each candidate closed triangulation surface meshes at least a portion of the 3D point cloud. For example, each candidate closed triangulation surface is a mesh that meshes the at least portion, where the vertices of the mesh correspond to points in the at least portion. Notably, in the example, each candidate closed triangulation surface includes a corresponding set of triangular faces, each triangular face being a triangular face of a corresponding tetrahedral mesh of the 3D point cloud. Through "exploration" and "penalty," it should be particularly understood that S30 selects from several closed triangulation surfaces (i.e., candidate closed triangulation surfaces) and selects only one of them (representing the closed triangulation surface of the skin of the structure).

[0080] The selection of candidate closed triangulation surfaces involves, in particular, lexicographical order and triangle order, which will now be discussed.

[0081] The lexicographical order sorts the first candidate closed triangulation surface relative to the second candidate closed triangulation surface, providing a ranking between the first and second candidate closed triangulation surfaces. The concept of lexicographical order is known in itself. The first candidate closed triangulation surface has first triangles (e.g., constituted by them), such as first triangular faces, each of which is a triangular face in the corresponding tetrahedron of a tetrahedral mesh of a 3D point cloud (as previously described), and the second candidate closed triangulation surface has second triangles (e.g., constituted by them), such as second triangular faces, each of which is a triangular face in the corresponding tetrahedron of a tetrahedral mesh of a 3D point cloud (as previously described). Both the first and second triangles are ordered according to a descending ranking of the triangle order.

[0082] Sort the first candidate closed triangular facets relative to the second candidate closed triangular facets according to lexicographical order. Specifically, this may include: since lexicographical order is based on triangle order, evaluating the ranking of the first and second triangles according to a descending lexicographical order. This ranking may further include a ranking based on the evaluation of both the first and second triangles, establishing the ranking by the lexicographical order of the first candidate closed triangular facets relative to the second candidate closed triangular facets. In other words, ranking may include establishing a relative ranking between the first and second candidate closed triangular facets by comparing the sorted first and second triangles in lexicographical order.

[0083] Now we discuss an example of sorting the first candidate closed triangulation surface relative to the second candidate closed triangulation surface according to lexicographical order. For explanation, let A be the set of the first triangles and B be the set of the second triangles. Let C = AΔB = (AUB)\ A∩B, that is, C is the set of triangles that belong to either A or B, but not both. If C is empty, then A = B. In this case, the first and second candidate closed triangulation surfaces have the same lexicographical ranking. If C is not empty, let T be the largest triangle of C according to the triangle order, that is, if the triangles of C are sorted according to the descending rank of the triangle order, then T is the first triangle of C. In other words, T is a triangle of C that is most penalized by the triangle order, for example, it is the largest triangle among the triangles of C, as described below. If T belongs to A, then A is greater than B. In this case, the first candidate closed triangulation surface has a higher rank than the second candidate closed triangulation surface in lexicographical order. Conversely, if T belongs to B, then B is greater than A. In this case, the second candidate closed triangulation surface has a higher rank than the first candidate closed triangulation surface in lexicographical order. It is important to note that if... (Strictly speaking), then A is greater than B.

[0084] This sorting results in the set of all candidate closed triangulation surfaces being ordered lexicographically, meaning each candidate closed triangulation surface has a corresponding rank in lexicographical order. Determining S30 penalizes high-ranking candidate closed triangulation surfaces. In the example, this means that determining S30 allows a candidate closed triangulation surface to be discarded because of its high rank, and one or more candidate closed triangulation surfaces to be rewarded because of their low rank. In any case, determining S30 allows the candidate closed triangulation surface with the lowest lexicographical rank among all ranks of all candidate closed triangulation surfaces to be determined as the closed triangulation surface representing the skin of the structure.

[0085] Triangle ordering penalizes the size of triangles. The concept of penalizing triangle size is now discussed. For explanation, let T and T' be two triangles. Penalizing triangle size may include evaluating the size of each triangle T and T' using a corresponding metric and assigning a higher rank in the triangle ordering to the triangle with the larger metric. The metric of triangle size may also be called the triangle weight or the weight of the triangle. The metric of size may include, for example, a first real-valued function that helps measure the size of the triangle, a second real-valued function that helps measure the size of the triangle in a different way, and / or a pair consisting of the first and second functions. Penalizing triangle size may include ordering the two measures of size relative to each other, for example, according to any order in which two real numbers or pairs of real numbers are ordered relative to each other, as is known in the art itself. In other words, if T is greater than (i.e., has a larger size) T', then the metric of size of T (e.g., according to the order of the metrics) is greater than the metric of size of T'. Now, if this pair (T, T') is to be ordered according to a descending rank in the triangle ordering, as with the first and second triangles discussed earlier, then in that order, T comes before T'. Similarly, if T1, T2, ..., T n This is a sorted list of triangles (i.e., T1 is the first, T2 is the second, and so on), and if the sorting is based on a descending order of triangle size, it means that the measure of size of T1 is greater than the measure of size of T2, which is greater than the measure of size of T3, and so on. In the example, the measure of size of triangle T is a real-valued function pair consisting of the radius of the smallest enclosing circle of T and the reciprocal of the radius of the circumcircle of T. Such examples constitute a robust and efficient way to measure and penalize the size of triangles.

[0086] This is equivalent to saying that when two candidate closed triangulation surfaces are sorted as described above, the higher rank in the lexicographical order gives the candidate closed triangulation surface that has the largest triangle subject to the triangle sorting penalty (e.g., it is the largest triangle). As mentioned earlier, such a triangle can represent a region of a low-density 3D point cloud with points (e.g., it can be part of its mesh), and may contain outliers. By penalizing their high rank in the lexicographical order, discarding candidate closed triangulation surfaces with such triangles allows for obtaining closed triangulation surfaces that accurately represent the structure skin and reduces sensitivity to density variations and / or outliers. This makes the method robust and accurate.

[0087] In the example, this method can result in the triangles (e.g., all or most of them) of the determined closed triangulation surface being small triangles (i.e., having small sizes), including triangles whose shapes are particularly less stretched, such as triangles with relatively small areas and a height-to-base ratio close to 1. In practice, although the triangle order does not necessarily penalize the flattest of the two triangles when comparing them, making the lexicographical order based on the triangle order may (e.g., generally) lead to the avoidance of flat triangles. Indeed, the lexicographical order compares chains of triangles (each forming a closed triangulation surface), and not just pairs of triangles, and attempts to reward (i.e., to give a lower ranking, e.g., the lowest ranking) chains composed of the smallest possible triangles. Comparing chains of triangles, and not just pairs of triangles, and penalizing chains with (e.g., an excessive number) large triangles can actually lead to the determined closed triangulation surface being composed of (e.g., generally) small and non-flat triangles. Such triangles represent the skin of a structure. This makes the method robust and accurate.

[0088] In the example, for the corresponding triangle, the triangle order penalizes the high value of the radius of the smallest enclosing circle (i.e., the smallest enclosing circle of the triangle).

[0089] The minimum enclosing circle of a triangle is the circle with the smallest radius among all the circles that enclose the points of the triangle. The radius of the minimum enclosing circle helps in measuring the size of the triangle, and penalizing the high values ​​of the minimum enclosing circle's radius helps in penalizing the size of the triangle. In other words, the radius of the minimum enclosing circle is a real-valued function that helps in measuring the size of the triangle, and it can be part of a metric for the size of the triangle as previously discussed. Furthermore, penalizing the high values ​​of the minimum enclosing circle's radius is a particularly effective and robust way to penalize the size of the triangle. For example, let T and T' be two triangles, and let R... B (T) and R B (T′) is the radius of their corresponding smallest enclosing circle. Then, in these examples, the inequality R... B (T) < R B (T′) captures the fact that the size of T′ is greater than the size of one of T. In this case, penalizing the size could include giving T′ a higher rank in the triangle order instead of T. Moreover, sorting T and T′ relative to each other according to the descending rank of the triangle order means that in this case T′ comes before T.

[0090] In the example, for the first and second triangles with the same minimum enclosing circle, the triangle order further penalizes the smaller value of the radius of the circumcircle (i.e., the circumcircle of the (first or second) triangle).

[0091] The circumcircle of a triangle is the circle passing through the points of the triangle. The radius of the circumcircle helps in measuring the size of the triangle, and penalizing smaller values ​​of the circumcircle radius helps in penalizing the size of the triangle, especially when the smallest enclosing circles are equal, as described below. In other words, the radius of the circumcircle is a real-valued function that helps in measuring the size of the triangle, and it can be part of a measure of the size of the triangle, as previously stated.

[0092] Penalizing the smaller value of the circumcircle radius when the smallest enclosing circles are equal can improve the robustness of the method. It is worth noting that we can let T and T' be two triangles, and let R... B (T) and R B (T′) is the radius of its corresponding smallest enclosing circle. Assume R... B (T)=R B (T′), for example, this might happen if both T and T′ are obtuse triangles. In other words, it might be possible that evaluating the radius of the corresponding smallest enclosing circle for both T and T′ and penalizing the higher value of the evaluated radius may not be sufficient to establish a relative triangle order between T and T′ in order to penalize the triangle size. In other words, it might be impossible to determine which of T and T′ is the largest so that it can be penalized based solely on the smallest enclosing circle. However, further penalizing the smaller value of the circumcircle radius allows for this determination, and in this respect, improves the robustness of the method. Furthermore, penalizing the smaller value of the circumcircle radius is a particularly effective and robust way to penalize the triangle size. For example, let R C (T) and R C (T′) is the radius of the corresponding circumcircle of T and T′. Then, in these examples, in R... B (T)=R B In the case of (T′), the inequality R C (T)>R C (T′) captures the fact that the area of ​​T′ is greater than the area of ​​one of T, and greater than T. In other words, equation R... B (T)=R B (T′) indicates that the radius of the smallest enclosing circle is insufficient to determine which triangle is the largest of T and T′. However, the inequality R C (T)>R C(T′) represents the fact that the area of ​​T′ is greater than the area of ​​T, which allows us to determine that T′ is greater than T, even though the minimum enclosing circles between the two triangles are equal. In this case, penalizing the size could involve giving T′ a higher rank in the triangle order than T. Furthermore, ordering T and T′ relative to each other according to their descending rank in the triangle order means that in this case, T′ is ordered before T.

[0093] As mentioned earlier, in the examples, the size of each triangle T is measured by a metric for triangle size, which is a real-valued function of the reciprocal of the radius of the smallest enclosing circle of T and the radius of the circumcircle of T. This metric can be expressed by the formula... Given. In this case, the triangle order can be an order of measurements of triangle size, such as an order in which comparisons between the radii of the smallest enclosing circles take precedence, and where comparisons between the reciprocals of the circumcircles' radii only apply if the smallest enclosing circles are equal. For example, let T and T' be two triangles. If R B (T) < R B (T′), then the measure of size T′ in metric order. A measure of size greater than T This is because the comparison between the radii of the smallest enclosing circles takes precedence. When the smallest enclosing circles are equal, i.e., R... B (T)=R B (T′), the comparison between the reciprocals of the circumcircle radius begins to take effect, and more specifically: if Then, the measure of size T' is ordered by the order of measurement. A measure of size greater than T Then, if the metric of T is less than the metric of T' in order of metric size, then T' ranks higher than T in the triangle order. However, if T and T' are ordered relative to each other in descending order of triangle order, then T comes after T'.

[0094] We will now further discuss the candidate closed triangulation surface.

[0095] As previously mentioned, in the example, each candidate closed triangulated surface comprises a corresponding set of triangular faces, each of which is a triangular face of a corresponding tetrahedral mesh of the 3D point cloud. The triangular faces are the previously discussed triangles of the candidate closed triangulated surfaces. In the example, the tetrahedral mesh can be a tetrahedral mesh common to all candidate surfaces (i.e., each candidate closed triangulated surface comprises a corresponding set of triangular faces, each of which is a triangular face of a corresponding tetrahedral mesh of the same and common tetrahedral mesh of the 3D point cloud). The tetrahedral mesh can be calculated / determined once and for all after the point cloud is provided (S10) and before the closed triangulated surface is determined (S30). In other words, the method can include the mesh of the provided 3D point cloud (S20). This simplifies the method and makes it faster. Alternatively, the method can operate based on several tetrahedral meshes.

[0096] In the example, the tetrahedral mesh exhibits the following characteristics: the union of the tetrahedrons in the tetrahedral mesh forms the convex hull of the points in the point cloud, and the intersection between any first tetrahedron in the tetrahedral mesh and any second tetrahedron in the tetrahedral mesh that intersects with the first tetrahedron is a vertex, an edge, or a face of the first tetrahedron. This improves the quality of the mesh and, consequently, the quality of the determined closed triangulation surface.

[0097] In the example, the tetrahedral mesh is a regular triangulation. This further improves the quality of the mesh and thus the quality of the determined closed triangulated surface.

[0098] As is known in computational geometry itself, regular triangulation of a 3D point cloud is caused by the shading of the faces of a polyhedron in a higher dimension (i.e., dimension 4). Regular triangulation is simply the dual of power diagrams; see, for example: Algorithmic geometry, Jean-Danel Boissonnat and Mariette Yvinec, Cambridge University Press, New York, NY, USA. Regular triangulation can be obtained, for example, at mesh S20 of the point cloud, using any conventional triangulation method, in the following way:

[0099] - Lifting the points in a point cloud involves adding coordinates to each point. This embeds the point set into a space whose dimension is equal to the dimension of the original space containing the point set (i.e., the 3D space to which the 3D point cloud belongs) plus one.

[0100] - Calculate the convex hull of the lifted point set;

[0101] - Triangulate the convex hull using any known method, resulting in a tetrahedral mesh with the following properties: the union of the tetrahedrons of the tetrahedral mesh forms the convex hull of the lifted set of points; and the intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects the first tetrahedron is a vertex, an edge, or a face of the first tetrahedron; and

[0102] - Project the lower face of this triangulation of the convex hull onto the original space.

[0103] In the example, the regular triangulation is a Delaunay triangulation. In this case, points are lifted to parabolic surfaces, and the lifting is parameterized by the vertical gap between the lifted points and the parabolic surfaces. This further improves the quality of the mesh generation and thus the quality of the determined closed triangulation surface. Furthermore, the method goes a step further than simply calculating the Delaunay triangulation. While the method does benefit from the quality of the Delaunay tetrahedral mesh, it goes a step further by identifying a candidate closed triangulation surface that truly represents the skin of the structure from all candidate closed triangulation surfaces composed of the triangular faces of the Delaunay triangulation. In other words, the method selects the best representation of the structural skin from the closed triangulation surfaces composed of the triangular faces of the Delaunay tetrahedral mesh.

[0104] In practice, penalizing the high ranking of candidate closed triangulation surfaces formed by triangular faces of regular (e.g., Delaunay) tetrahedral meshes relative to the lexicographical order of triangle order based on the size of the penalized triangles makes the determined closed triangulation surfaces particularly representative of the skin of the structure. It is noteworthy that, in this case, the triangles of the determined closed triangulation surfaces are not too large and / or not very flat; for example, the determined triangles are, conversely, triangles with relatively small areas and a height-to-base ratio close to 1.

[0105] In fact, the closed triangulation surface defined by the triangular faces of a 3D Delaunay tetrahedral mesh is (or at least similar to) a 2D Delaunay triangulation of the portion of a 3D point cloud corresponding to the skin of a structure. 2D Delaunay triangulation has the property that the constitutive value of each triangle in the triangulation maximizes the value of the minimum angle of that triangle. Therefore (e.g., generally) 2D Delaunay triangulation avoids flat triangles. Indeed, and as further discussed in the "Commentary on Algorithms" section below, it has been observed that, given a set of coplanar points, a 2D Delaunay triangulation of that set of coplanar points can be defined as a minimum value according to some standard. In the example, given a 2D point cloud K, and denoted by T as the 2D mesh partition of its convex hull, denoted by Ω, the 2D Delaunay triangulation T of K is... * It is one that minimizes the quantity Q(T,p) defined by the following formula:

[0106]

[0107] For any real number p∈[1,+∞], and where It is a mesh division T Linear interpolation. 2D Delaunay triangulation is a surface that is formed by T... * The triangle forms and surrounds T * Of all the surfaces of the tetrahedron, this surface corresponds to the minimum value of the quantity Q as p approaches infinity. In fact, in the tetrahedron formed by T... * The triangle forms and encloses T * In the tetrahedral surface, the search for the closed triangulation surface corresponding to the minimum value of the quantity Q as p approaches infinity corresponds to (e.g., producing the same result): among the candidate closed triangulation surfaces (including those formed by T) * The triangle forms and surrounds T *The search for the lowest-ranked closed triangulated surface in a lexicographical order based on the triangle order is performed on the tetrahedral surfaces of the 2D Delaunay points. Therefore, for such a set of coplanar points, even considering all possible meshing of triangle pairs (one triangle for any three points), including meshes not solely composed of triangles from 3D Delaunay tetrahedral meshes, the search for closed candidate triangulated surfaces composed of such triangles with the lowest rank in the lexicographical order will only select triangles from 2D Delaunay triangulations. It has been observed that 2D Delaunay triangulation is optimal for many triangulation quality criteria, including the best triangle shape (flatness). It has now been further observed that, in practice, what has just been explained for multiple sets of coplanar points still applies to multiple sets of non-coplanar points, and sorting candidate closed triangulated surfaces meshed with multiple sets of non-coplanar points using the same lexicographical order based on the same triangle order still yields definite closed triangulated surfaces that are or at least similar to 2D Delaunay triangulations, especially closed triangulated surfaces with small and less flat triangles.

[0108] In any case, determining S30 can result in the determined closed triangulation surface being (or at least similar to) a 2D Delaunay triangulation representing the skin of the structure, even if the candidate triangulation surface is not composed of triangles divided by a 3D Delaunay tetrahedral mesh. In particular, the determined closed triangulation surface is composed of (e.g., typically) small and less flat triangles. However, restricting the candidate triangulation surfaces to those composed of triangles divided by the same Delaunay tetrahedral mesh reduces the number of candidate triangulation surfaces, which improves the speed and efficiency of the method.

[0109] Now let's discuss examples of identifying S30.

[0110] In the example, S30 is defined as a region within the 3D point cloud that provides a representation of the internal location of the structure, and the candidate closed triangulation surfaces all surround this region.

[0111] As previously mentioned, the structure corresponds to a closed shape of a high-density (e.g., density higher than ambient noise) 3D point cloud. However, it is possible that two or more such closed shapes with substantially equal densities exist within the point cloud. To distinguish between the two or more that should correspond to the structure, determining S30 includes providing a region within the 3D point cloud representing the internal location of the structure. This internal location is typically completely surrounded (e.g., enclosed) by exactly one of the closed shapes. Therefore, it is required that the region be surrounded by candidate closed triangulation surfaces explored by determining S30. Thus, providing the region allows selection of one of the closed shapes of the 3D point cloud that will truly correspond to the structure, and allows the skinned representation of S30 to be determined. In other words, providing the region makes the method more accurate and robust. Notably, the method may be able to reconstruct the structure simultaneously in several regions of the point cloud characterized by high point density (in addition to the region corresponding to the structure).

[0112] The provision of the region can be performed by the user. For example, a 3D point cloud can be displayed on the monitor (e.g., a GUI) of the computer performing the method, and the user defines the region, for example, through graphical interaction with the monitor. For instance, the user can click on the location of the point cloud to define the region. In this case, exploration and penalty can be performed automatically when the user provides the region.

[0113] Figures 10 to 12 The effect of lexicographical ordering among candidate closed triangulation surfaces is shown. Figure 10 Two closed shapes 100 and 102 within the point cloud are shown, each with a high point density. Shape 102 corresponds to the structure to be reconstructed, which is the mesh of the closed triangulation surface to be determined according to the method. Shape 100 represents another portion of the point cloud, which can be meshed by candidate closed triangulation surfaces that rank higher in lexicographical order than the mesh of closed triangulation surface shape 102. Determination S30 includes providing an “interior” region 104 within the 3D point cloud. This region represents the interior location of the structure corresponding to shape 102. Figure 11 The following illustrates what might happen without lexicographical order. Despite the high ranking of the closed triangulation surface 110 in the lexicographical order, it is still possible to obtain a closed triangulation surface 110 representing (i.e., meshing) shape 100 instead of structure 102. Figure 12The result of determining S30 according to the method of this disclosure is shown, wherein, in particular because of the lexicographical order, the shape 102 that truly represents the structure has been meshed by the closed triangulation surface 120 of the determined S30, which is the lowest ranked of the surfaces 110 and 120 in the lexicographical order. In fact, the holes in shape 100 are larger than those in shape 102. These largest holes will result in the largest edges of shape 100. Moreover, since the lexicographical order is compared, the largest edges penalize shape 100.

[0114] Figures 13 to 15 The effect of lexicographical ordering among candidate closed triangulation surfaces is shown. Figure 13 Two closed shapes 130 and 132 within the point cloud are shown, each with a high point density. Shape 132 corresponds to the structure to be reconstructed, i.e., the closed triangulation surface meshing shape 132 to be determined according to the method. Shape 130 represents another portion of the point cloud, which can be partitioned by candidate closed triangulation surface meshes that rank higher than closed triangulation surface meshing shape 132 in lexicographical order. Determination S30 includes providing an “interior” region 134 within the 3D point cloud. This region represents the interior location of the structure corresponding to shape 132. Figure 14 The following illustrates what might happen without lexicographical order. Despite the high ranking of the closed triangulation surface 140 in the lexicographical order, it is still possible to obtain a closed triangulation surface 140 representing (i.e., meshing) shape 130 instead of structure 132. Figure 15 The result of determining S30 according to the method of this disclosure is shown, wherein, in particular because of the lexicographical order, the shape 132 that truly represents the structure has been meshed by the closed triangulation surface 150 of the determined S30, among surfaces 140 and 150, the closed triangulation surface 150 of the determined S30 is the lowest ranked one in the lexicographical order. In fact, shape 130 has a hole larger than shape 132. This single largest hole will result in a single largest edge in shape 130. Moreover, due to the comparison of the lexicographical order, the single largest edge is sufficient to penalize shape 130.

[0115] We will now discuss other examples for determining S30. In these examples, the candidate closed triangulation surfaces each consist of a corresponding set of triangular faces, each of which is a triangular face of a corresponding tetrahedron from a tetrahedral mesh of a 3D point cloud.

[0116] Exploration and penalty can include accessing triangles in a tetrahedral mesh based on their decreasing ranking in triangle order, and discarding visited triangles with higher rankings in triangle order.

[0117] Visiting triangles in a tetrahedral mesh can include visiting all triangles of the tetrahedral mesh or only a portion of them, for example, iterating until a closed triangulation surface is determined. It should be understood that visiting triangles and discarding some of them yields a closed triangulation surface representing the skin of the structure. In other words, after all the triangles to be visited have been visited, and after all the triangles to be discarded have been discarded, the remaining triangles (those not yet discarded) form a closed triangulation surface representing the skin of the structure. In that respect, discarding more visited triangles, fewer visited triangles, and / or other different visited triangles will yield another closed triangulation surface, which actually corresponds to one of the candidate closed triangulation surfaces. In other words, the corresponding candidate closed triangulation surface corresponds to the corresponding way of discarding triangles. The choice of one of the different ways of discarding triangles by the S30 operation is determined by penalizing the candidate closed triangulation surfaces with the highest rank in lexicographical order; this is equivalent to the operation choosing one of the different candidate closed triangulation surfaces, as previously discussed.

[0118] Triangles in a tetrahedral mesh are accessed in descending order of triangle order. This means that the larger the triangle size in a tetrahedral mesh, the earlier the triangle is accessed, as the triangle order penalizes it. In the example discussed earlier, triangle size was measured by a metric, meaning that the larger the metric, the earlier the triangle is accessed. This allows for the rapid and efficient detection of triangles ordered by triangle order, i.e., the largest triangles, so that they are discarded and prevented from belonging to the set of non-discarded triangles, which correspond to the closed triangulation surface representing the structure. More specifically, the closed triangulation surface representing the structure should consist of relatively small and not too large triangles. As mentioned earlier, this is because the structure represents a closed shape of a 3D point cloud with a high point density. Such triangles all have small rankings in the triangle order because the latter penalizes the size of the triangles. Therefore, by accessing triangles in descending order of triangle order and discarding accessed triangles with high rankings (i.e., discarding them because of their high ranking), this method ensures that larger triangles are discarded and only triangles with satisfactory small sizes are retained to form the closed triangulation surface representing the structure.

[0119] In the examples, each tetrahedron of the tetrahedral mesh defines a corresponding region. In these examples, determining S30 also includes declaring the outer tetrahedron and the inner tetrahedron of the tetrahedral mesh. The outer tetrahedron and the inner tetrahedron represent the outer region and the inner region of the structure, respectively. In these examples, triangle access includes evaluating for each accessed triangle whether the accessed triangle separates the inner region from the outer region. In these examples, for each accessed triangle that does not separate the inner region from the outer region, triangle discarding includes the triangle separating the two regions, thereby suppressing the triangle and merging the two regions.

[0120] The declaration of the outer and inner tetrahedrons can be performed by the user. For example, a 3D point cloud can be displayed on the monitor (e.g., a GUI) of the computer performing the method, and the user can define the outer and inner tetrahedrons through graphical interaction with the monitor. In this case, exploration and penalty can be automatically performed when the user declares the outer and inner tetrahedrons, and thus access and discard can be performed. The inner tetrahedron can, for example, be located within a provided region in the 3D point cloud representing the interior location of the structure. In the example, the inner tetrahedron is located in a region of a 3D mesh with low point density and representing the interior location of the structure. The outer tetrahedron can be a tetrahedron that is not part of the mesh itself, but is added to the tetrahedron mesh as a representation of an outer region (i.e., a region located outside the mesh). However, the inner tetrahedron can be part of the tetrahedron mesh. In other words, declaring an internal tetrahedron can include selecting a tetrahedron for a tetrahedral mesh, for example, by clicking on the tetrahedron's representation and marking it as an internal tetrahedron, while declaring an external tetrahedron can include, for example, adding supplementary and boundary tetrahedrons to the mesh and marking them as external tetrahedrons. In an example that will be discussed further below, a tetrahedral mesh can be represented by its dual graph, where the nodes represent tetrahedrons of the tetrahedral mesh and its arcs represent triangles. In such an example, a declaration can include selecting (e.g., by clicking) a node of the graph to represent an internal triangle, and adding (e.g., by clicking) nodes to the graph to represent external triangles.

[0121] The access to triangles and the discarding of visited triangles with high rankings in the triangle order must ensure that enough triangles are discarded so that the closed triangulation surface formed by the non-discarded triangles truly represents the structure, i.e., including as few triangles as possible with high rankings in the triangle order. However, simultaneously, access and discarding must avoid discarding triangles that should be discarded in the closed triangulation surface representing the skin of the structure, for example, by discarding incorrect triangles and / or too many triangles. The triangles forming the closed triangulation surface representing the skin of the structure do indeed correspond to the triangles of the tetrahedral mesh that separates the interior and exterior of the structure. In addition to discarding high-ranking triangles, evaluating whether triangles separate the interior and exterior regions can further allow for the accurate identification of the triangles that separate the interior and exterior of the structure, preserving them and suppressing other triangles. This improves the robustness and accuracy of the method.

[0122] The declaration, access, and discard will now be discussed in more detail.

[0123] Before any triangle is visited, the outer tetrahedron lies within the exterior of the structure. In other words, the outer tetrahedron represents the outer region, which can correspond to the infinite space surrounding the entire point cloud. The inner tetrahedron represents the inner region corresponding to the portion of the point cloud representing the interior of the structure. Now let T be the first triangle visited. This visit involves evaluating whether T separates the inner and outer regions. If T does not separate the inner and outer regions, then T separates the two tetrahedrons, the first tetrahedron defining the first region and the second tetrahedron defining the second region, wherein at least one of the first and second regions is neither an outer nor an inner region. In this case, discarding involves merging the two regions defined by the two tetrahedrons. If the first (or second) region represents the outer region, then the second (or first) region is merged with the outer region. If the first (or second) region represents the inner region, then the second (or first) region is merged with the inner region. Otherwise, the two regions are simply merged into a new region that is neither an outer nor an inner region. Triangle T is suppressed / discarded in all cases. Conversely, if T does indeed separate the inner region from the outer region, then T is neither discarded nor suppressed, and the regions separated by T are not merged. In this case, T is retained. Then, the second triangle being visited (e.g., denoted as T') is evaluated to see if it separates the inner region (possibly due to a previous merge) from the outer region (also possibly due to a previous merge). Then, as with T, either T is discarded (in which case, the second merge occurs just like the previous merge), or T' is retained. Then the next triangle is visited, and so on. In other words, triangles are iteratively suppressed / discarded, and regions are iteratively merged with each other and with the inner and / or outer regions until only the triangle that separates the inner region representing the interior of the structure from the outer region representing the exterior of the structure remains.

[0124] In the example, each visited triangle that separates the inner and outer regions is preserved, and the preserved triangles form a defined closed triangulation surface.

[0125] Now, an example of the method is discussed. In this example, exploration and penalty involve accessing triangles in a tetrahedral mesh according to their decreasing ranking in triangle order, and discarding accessed triangles with higher rankings in triangle order. Each tetrahedron in the tetrahedral mesh defines a corresponding region. Determining S30 also includes declaring the outer tetrahedron and the inner tetrahedron of the tetrahedral mesh, where the outer tetrahedron and the inner tetrahedron represent the outer and inner regions of the structure, respectively. Accessing a triangle involves evaluating for each accessed triangle whether it separates the inner region from the outer region. Discarding a triangle involves: for each accessed triangle that does not separate the inner region from the outer region, the triangle separates the two regions to suppress the triangle and merge the two regions. Each accessed triangle that separates the inner region from the outer region is retained, and the retained triangles form a defined closed triangulation surface. Figure 3B An example of this method is shown. Figure 3A 3D surface reconstruction of point clouds. Figure 4B An example of this method is shown. Figure 4A 3D surface reconstruction of point clouds. Figure 5B An example of this method is shown. Figure 5A 3D surface reconstruction of point clouds. Figure 6B An example of this method is shown. Figure 6A 3D surface reconstruction of point clouds. Figure 7B An example of this method is shown. Figure 7A 3D surface reconstruction of point clouds. Figure 8B An example of this method is shown. Figure 8A 3D surface reconstruction of point clouds. Figure 9B An example of this method is shown. Figure 9A 3D surface reconstruction of point clouds.

[0126] Now for reference Figures 16 to 39 This section discusses examples of implementations of this method. Figure 16 A flowchart illustrating an example of this implementation is shown.

[0127] refer to Figure 16The flowchart of this implementation example includes providing a 3D point cloud (S10) and meshing the 3D point cloud (S20), as previously discussed. Meshing is a Delaunay triangulation of the provided 3D point cloud. Since the point cloud contains three-dimensional points, the resulting mesh consists of adjacent tetrahedrons, each bounded by four triangles. This implementation example then includes determining the closed triangulated surface (S30), as previously discussed. Determining S30 includes defining a lexicographical order on the triangles. A weight formula is used to set this order. The next step in the determination includes computing the dual graph of the tetrahedral mesh. This graph captures tetrahedral adjacency through triangle sharing. Therefore, each node in the dual graph represents a tetrahedron of the mesh, each arc in the dual graph represents a triangle of the mesh, and each arc in the dual graph is labeled with a corresponding triangle rank in lexicographical order. The arcs in the dual graph are used in descending order. Determining S30 also includes selecting tetrahedrons inside unknown surfaces. As mentioned above, the determination also includes access and discarding performed by applying a reduction algorithm. The algorithm is initialized by selected internal and external nodes and utilizes the dual graph and its ordered arcs. The goal is to discard the arcs of the dual graph in such a way that the remaining biarrives define the closed surface of the triangulation, which is the reconstructed surface.

[0128] In contrast to existing methods, this implementation example involves the local density of the point cloud, which eliminates the need for forced parameter adjustments. In cases where the point cloud is very dense, and for performance purposes, an optional filtering distance parameter can be used to reduce the number of points in the Delaunay triangulation. This implementation example creates a watertight surface that is robust to outliers and density variations within the point cloud. This implementation example also features theoretical complexity equivalent to, and generally better than, graphical cutting methods in practice. A trade-off for this reconstruction method is that the point cloud needs to be closed when sampling open surfaces and specifying interior regions that, in the context of building reconstruction, can be easily inferred from the building's footprint.

[0129] Mesh partitioning and dual graph

[0130] This implementation example utilizes a Delaunay tetrahedral convex hull mesh of a 3D point cloud (see F. Cazals, J. Giesen, Delaunay triangulation based surface reconstruction, Effective computational geometry for curves and surfaces, pages 231-276, 2006). A tetrahedral mesh object is a set of tetrahedra characterized by the following properties: The topology is a set of adjacent tetrahedra such that any triangle is shared by exactly two regions. The first sharing case is a triangle shared by two adjacent tetrahedra. The second sharing case is a triangle shared by a tetrahedron and an unbounded outer region. The triangle shared by a tetrahedron and an outer region is called a boundary triangle. The geometric property of the Delaunay mesh is that for any tetrahedron, the sphere defined by its four points (called the circumsphere) does not include any other points of the cloud.

[0131] The dual graph of a tetrahedral mesh is defined as follows. Nodes in the dual graph are associated with the tetrahedron, and additional nodes are associated with the outer regions. Arcs in the dual graph are associated with triangles. If a triangle associated with an arc is shared by regions associated with nodes, then the arc connects the two nodes.

[0132] Figures 17 to 19 A grid consisting of three tetrahedrons 1, 2, and 3 is shown. Figure 17 The 3D point cloud is shown. Figure 18 The geometry of the convex hull mesh is shown, which includes points, lines, triangles, and tetrahedrons. Figure 20 The corresponding dual graphs are shown. For example, because triangle "a" of the mesh is shared by tetrahedrons 1 and 2, arc "a" of the dual graph connects nodes 1 and 2. Because triangle "b" of the mesh is shared by tetrahedron 2 and outer region 4, arc "b" of the dual graph connects nodes 2 and 4. Because triangle "c" is shared by tetrahedron 2 and outer region 4, arc "c" of the dual graph connects nodes 2 and 4. Because triangle "d" of the mesh is shared by tetrahedrons 1 and 3, arc "d" of the dual graph connects nodes 1 and 3. Because triangle "e" is shared by tetrahedron 3 and outer region 4, arc "e" of the dual graph connects nodes 2 and 4. Triangles "b", "c", and "e" are boundary triangles.

[0133] algorithm

[0134] For clarity, this implementation example is illustrated using a 2D problem. The correspondence with 3D problems is as follows: 2D problems deal with planar point clouds, not 3D point clouds. A tetrahedron in a 3D problem is a triangle in a 2D problem. A 3D problem where triangles form the boundaries of a tetrahedron is a 2D problem where line segments form the boundaries of triangles. Finally, the generally accepted 3D problem of triangulating closed surfaces is a 2D problem of closed polylines.

[0135] Figure 20 The initial point cloud is shown. Visually, it clearly defines an L-shaped closed polyline, and this implementation example is intended to calculate the L-shaped closed polyline. Figure 21 This illustrates the triangulation (or tetrahedral 3D meshing) of the point cloud. For example... Figure 22 As shown, line segments (or 3D triangles) are labeled with their respective rankings based on their weights and descending order. In the context of the 2D problem, the weight is the length of the line segment. The first line segment (1) is the longest line segment, and the last line segment (32) is the shortest line segment. The 3D problem of calculating the weights of triangles is detailed below.

[0136] Figure 23 The diagram shows the dual graph and the point cloud after meshing. Each node (empty circle) in the dual graph is associated with a triangle (or 3D tetrahedron), and each arc in the dual graph is associated with a line segment (or 3D triangle) of the mesh. Note the external node associated with infinite space surrounding the point cloud. This node is reused for initialization purposes. Figure 24 The dual graph is shown, where the arcs are labeled with their respective rank in a sorting. Internal and external nodes are solid circles.

[0137] The reduction algorithm computes the so-called minimum chain. Its form is independent of the problem's dimension (2D vs. 3D). The input to the reduction algorithm is the dual graph D = (V... D E D ), where V D It is a set of nodes, E D This is a list of arcs sorted in descending order of weight. Additionally, two input nodes α1 and α2 are given. The tetrahedron associated with input node α1 (or α2) should be located inside (or outside) the unknown surface, and vice versa. The output of the reduction algorithm is a set of input dual graphs labeled Γ that define the minimum chain. minThe reduction algorithm utilizes a so-called disjoint set data structure (see 4. A Galler, and M.J. Fisher. An improved equivalence algorithm. Communications of the ACM, 7(5):301–303, 1964). This data structure involves three basic functions: MakeSet(·), FindSet(·), and UnionSet(·). The function MakeSet(x) creates a set in the data structure containing the element x, denoted as {x}. The function FindSet(x) returns the only set in the data structure containing the element x. The function UnionSet(X,Y) creates a new set X∪Y in the data structure and removes the input sets X and Y from the data structure. The algorithm can be described by the following pseudocode.

[0138]

[0139]

[0140] After calculating the minimum chain, the corresponding lines (or 3D triangles) of the meshed point cloud are collected, and it is theoretically guaranteed that they define closed polylines (or 3D triangulation surfaces), which are the output objects.

[0141] When run on a 2D example, the reduction algorithm eliminates... Figure 25 The dashed double arc in the middle. For example... Figure 26 As shown, the corresponding line segments (or 3D triangles) of the mesh are also eliminated, which produces the resulting polyline (or 3D triangulated surface), as... Figure 27 As shown.

[0142] Triangle dictionary sorting

[0143] This implementation example uses a triangle sorting strategy (denoted as ≤). ∞ This is based on two concepts: the minimum enclosing circle and the circumcircle. By definition, the minimum enclosing circle is the circle that encloses the points of a triangle, characterized by its minimum radius. By definition, the circumcircle is the circle that passes through the three points of the triangle. Given a triangle T, the radius of its minimum enclosing circle is denoted as R. B (T), and the radius of its circumcircle is denoted as R. C (T).

[0144] Figure 28 The diagram shows triangle T and its enclosing and circumcircles. Because triangles are characterized by obtuse angles, they make R... B (T)≠R C (T). Conversely Figure 29A triangle T with three acute angles is shown. This causes the enclosing circle and the circumcircle to coincide, therefore R... B (T)=R C (T).

[0145] When a triangle is characterized by three aligned or coincident points, the circumcircle is undefined. Because point clouds are inherently general, this degeneracy is outside the scope of this implementation example. Given two triangles T and T′, their relative ordering ≤ ∞ Defined as follows: If R B (T) < R B (T′) or if R B (T)=R B (T′) and R C (T)≥R C (T′), then T≤ ∞ T′. In other words, when the radius of the enclosing circle is R B (T) and R B When (T′) are not equal, they are related; otherwise, the circumcircle radius R C (T) and R C (T′) is relevant. Figure 30 It shows the corresponding R B Two acute triangles ordered by their radii. Figure 31 Two obtuse triangles are shown. They are enclosed by the same minimum radius R of the circle. B (Dashed lines) are the characteristics, therefore they pass through R C Sort by radius.

[0146] This sorting is actually a lexicographical order defined by several numerical values. The n triangles that make up the grid are denoted as T. i , i = 1, ..., n, they are based on pairs (where i = 1, ..., n) is sorted lexicographically. This sorting is very subtle because it combines the shape of the triangles (sharp versus obtuse angles) and their dimensions in a simple formula. It aims to assign lower-ranking triangles to "small and flat" triangles, which are more likely to appear on the unknown, separated surfaces to be identified.

[0147] Alternative sorting

[0148] To compare with the strategies described in the previous section, three alternative weighted strategies are investigated. The first strategy is the radius of the smallest circle enclosing the triangle, as R... B The second strategy is the radius of the circumcircle, as shown by R. CThe third strategy is triangle area. Random selection is performed among several triangles characterized by equal weights. Experiments on industrial data show that the surfaces obtained using alternative weights are very similar to those obtained using lexicographical order. However, the quality is not very good in the following sense: The theory guarantees that the identified surfaces are tightly triangulated surfaces, but some non-manifold edges may appear. Non-manifold edges (relative to manifold edges) of closed surfaces are shared by 2n triangles, while manifold edges are shared by exactly 2 triangles, such as... Figure 32 and Figure 33 As shown.

[0149] Experiments show that, in all test cases, the dictionary order generates the recognized surface with the minimum number of non-manifold edges, as shown in the table below.

[0150] Test cases dictionary order Alternative Strategy 1 Alternative Strategy 2 Alternative Strategy 3 1 187 211 696 2016 2 100 234 6384 5516 3 109 125 1160 2331

[0151] Comments on the algorithm

[0152] Now let's discuss the ideas involved in this algorithm.

[0153] The initial idea was to provide the best understanding of Delaunay triangulation, relative to the traditional geometric understanding. Given a point cloud K, its convex hull mesh is denoted as T, and the convex hull itself is denoted as Ω. The Delaunay mesh T of K... * It is a mesh partition that minimizes the number Q(T,p) defined by the following formula.

[0154]

[0155] For any real number p∈[1,+∞], and where It is a mesh division T Linear interpolation. See especially L. Chen, J.C. Xu, Optimal Delaunay triangulation, Journal of Computational Mathematics, Vol. 22, No. 2, 2004, 299-308. References Figure 34 , mapping The definition is as follows. Given x∈Ω and the triangulation T of Ω, there exist triangles (a,b,c) and nonnegative scalars α,β,γ of T such that x=αa+βb+γc and α+β+γ=1. Therefore, L(T,x)=α‖a‖ 2 +β‖b‖ 2 +γ‖c‖ 2 .

[0156] The optimal representation means that for any p∈[1,+∞[,Q(T)] * ,p)=minT Q(T,p), where the minimum value is taken from all meshes T of the convex hull Ω of K.

[0157] The second idea is a new theoretical result constructed for the purposes of this implementation example, which is an explicit formula for Q(T,p) in the limit p→+∞ when dealing with triangles forming a mesh. The result is exactly the weighted sum lexicographical order of the triangles introduced in the previous section, which advantageously replaces the integral version of Q(T,p).

[0158] Intuitively, the best possible triangulation surface minimizes the weighted average, and this minimum is obtained by using lexicographical order. Now, consider the Delaunay 3D mesh T of a 3D point cloud K and its convex hull Ω. * And T * The selected tetrahedron h. The unknown surface is formed by T including tetrahedron h. * A closed surface composed of triangles. Of course, many such surfaces exist, but identifying a surface Σ is a different matter. * Retained by T * The triangles form the best possible triangulation. In other words, the identified surface Σ * Make T * The triangles that form and enclose all closed surfaces Σ of h minimize Q(Σ,∞). It must be understood that this principle is closely related to the fact that point clouds are denser near the surfaces to be identified.

[0159] Figures 34 to 36 The principle is illustrated with a 2D example: a point cloud K, a convex hull Ω, and its meshing T performed with a selected h. * . Figures 37 to 39 The diagram shows the non-optimal Σ1 and Σ2 surrounding h, as well as the optimal Σ according to the minimum criterion. * .

Claims

1. A computer-implemented method for 3D reconstruction of real-world scene structures, the method comprising: - Provide (S10) a 3D point cloud representing the structure; - Determine (S30) the closed triangulation surface representing the skin of the structure, the determination (S30) explores candidate closed triangulation surfaces, each of the candidate closed triangulation surfaces being meshed at least for the corresponding portion of the 3D point cloud, and the determination (S30) penalizes the high ranking of the candidate closed triangulation surfaces according to lexicographical order, the lexicographical order being based on triangle order, and the lexicographical order sorting a first candidate closed triangulation surface having a first triangle relative to a second candidate closed triangulation surface having a second triangle, the first triangle being sorted according to a descending ranking of the triangle order, the second triangle being sorted according to a descending ranking of the triangle order, the triangle order penalizing triangle size including sorting two evaluated measures of the size relative to each other; -The exploration and penalty of candidate closed triangulation surfaces includes: selecting from candidate closed triangulation surfaces by discarding candidate closed triangulation surfaces with high ranking in the lexicographical order and rewarding candidate closed triangulation surfaces with low ranking in the lexicographical order; -Where, for the corresponding triangle, the triangle sequence penalizes the height value of the smallest enclosing circle radius; -In this context, for a first triangle and a second triangle with the same minimum enclosing circle, the triangle order also penalizes smaller values ​​of the radius of the circumcircle of the triangle.

2. The method according to claim 1, wherein, Each candidate closed triangulation surface includes a set of corresponding triangular faces, and each triangular face is a triangular face of a corresponding tetrahedron divided by the tetrahedral mesh of the 3D point cloud.

3. The method according to claim 2, wherein, The tetrahedral mesh division exhibits the following characteristics: - The union of the tetrahedrons divided by the tetrahedral mesh forms the convex hull of the points in the point cloud; and - The intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex, an edge, or a face of the first tetrahedron.

4. The method according to any one of claims 1 to 3, wherein, Each candidate closed triangulation surface includes a set of corresponding triangular faces. Each triangular face is a triangular face of a corresponding tetrahedron from the tetrahedral mesh of the 3D point cloud. The tetrahedral mesh is a regular triangulation.

5. The method according to claim 4, wherein, The exploration and the punishment include: - Access the triangles of the tetrahedral mesh according to the descending order of the triangle sequence; and - Discard the visited triangle with the highest ranking in the triangle order.

6. The method according to claim 5, wherein, Each tetrahedron in the tetrahedral mesh defines a corresponding region, and the determination (S30) further includes: - Declare the outer tetrahedron and the inner tetrahedron of the tetrahedron mesh, wherein the outer tetrahedron and the inner tetrahedron represent the outer region and the inner region of the structure, respectively; And among them: - The access to the triangles includes: for each accessed triangle, evaluating whether the accessed triangle separates the inner region from the outer region; and - The discarding of the triangle includes: for each visited triangle that does not separate the inner region from the outer region, the triangle separates the two regions, suppresses the triangle, and merges the two regions.

7. The method according to claim 6, wherein, Each visited triangle that separates the internal and external regions is preserved, and the preserved triangles form the defined closed triangulation surface.

8. The method according to any one of claims 1 to 3, wherein, The determination (S30) includes providing a region within the 3D point cloud representing the internal location of the structure, with the candidate closed triangulation surfaces all surrounding the region.

9. The method according to any one of claims 1 to 3, wherein, The structure includes at least one corner structure.

10. The method according to any one of claims 1 to 3, wherein, The 3D point cloud is derived from photogrammetry, laser scanning, RGB-D measurement and / or medical or industrial tomography, and / or the real scene is an architectural scene, a building interior scene, a mining scene, organic tissue or mechanical parts.

11. A computer program comprising instructions for performing the method according to any one of claims 1 to 10.

12. A computer-readable storage medium having a computer program as described in claim 11 recorded thereon.

13. A computer comprising a processor and a display coupled to a memory, wherein the memory has a computer program as claimed in claim 11 stored thereon.

Images