3D reconstruction of the structure of a real scene

The method addresses inefficiencies in existing 3D reconstruction by penalizing triangle size using lexicographic ordering, enabling robust and ergonomic reconstruction of angular structures with reduced sensitivity to noise and outliers.

JP7591352B2Active Publication Date: 2024-11-28DASSAULT SYSTEMES SA
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2020069291
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2019-04-10
Filing Date
2020-04-07
Publication Date
2024-11-28
Estimated Expiration
2040-04-07

AI Technical Summary

Technical Problem

Existing 3D reconstruction methods are inefficient for scenes featuring angular structures, sensitive to noise and outliers, and require manual adjustments, lacking robustness and universality.

Method used

A computer-implemented method for 3D reconstruction that determines a closed triangulation surface by penalizing triangle size using lexicographic ordering based on triangular ordering, effectively handling angular structures and reducing sensitivity to noise and outliers.

Benefits of technology

The method provides robust and ergonomic 3D reconstruction of real-world scenes, particularly angular structures, without requiring smoothness assumptions and minimizing manual adjustments, while maintaining accuracy and efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007591352000033
    Figure 0007591352000033
  • Figure 0007591352000034
    Figure 0007591352000034
  • Figure 0007591352000035
    Figure 0007591352000035
Patent Text Reader

Abstract

To provide a computer implementation method of a three-dimensional reconstruction of a structure of a real scene.SOLUTION: A method comprises: providing a 3D point cloud representing a structure; and determining a closed triangulated split surface representing a skin of the structure. Determining explores candidates of the closed triangulated split surfaces each meshing at least a corresponding part of the 3D point cloud. The determining penalizes a high rank of the candidate of the closed triangulated split surfaces according to a lexicographic order. The lexicographic order is based on triangle order. The lexicographic order orders a first candidate of the closed triangulated surface having first triangles which are ordered according to a decreasing rank of the triangle order, relative to a second candidate of the closed triangulated split surface having second triangles which are ordered according to a decreasing rank of the triangle order. The triangle order penalizes triangle size.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to the field of computer programs and systems, and more particularly to a method, a system and a program for 3D reconstruction of structures of real scenes. [Background technology]

[0002] For the design, engineering and manufacturing of objects, numerous systems and programs are offered on the market. CAD is an acronym for Computer-Aided Design, e.g., related to software solutions for designing objects. CAE is an acronym for Computer-Aided Engineering, e.g., related to software solutions for simulating the physical behavior of a future product. CAM is an acronym for Computer-Aided Manufacturing, e.g., related to software solutions for defining manufacturing processes and operations. In such Computer-Aided Design systems, the graphical user interface plays a key role regarding the efficiency of the technique. These technologies may be incorporated within Product Lifecycle Management (PLM) systems. PLM refers to a business strategy that helps companies share product data, apply common processes and leverage corporate knowledge to help develop products from concept to life, beyond long-term corporate concepts. The PLM solutions offered by Dassault Systèmes (under the trademarks CATIA, ENOVIA and DELMIA) provide an Engineering Hub that organizes product engineering knowledge, a Manufacturing Hub that manages manufacturing engineering knowledge, and an Enterprise Hub that allows connections to both the Enterprise Hub and the Engineering and Manufacturing Hubs. Overall, the system provides an open object model that links products, processes, and resources to enable dynamic knowledge-based product creation and decision support that drives optimized product definition, manufacturing preparation, production, and service.

[0003] In this and other contexts, 3D reconstruction of the structure of real scenes is gaining widespread importance.

[0004] Some existing 3D reconstruction methods are based on a smoothness assumption, which may not be applicable in some contexts. In particular, such methods are not efficient in the context of scenes featuring corner structures, such as building corners.

[0005] Other methods fit implicit functions to the point cloud, which are not accurate enough.

[0006] Other methods are sensitive to density variations within the point cloud, which makes them not robust enough. For example, such methods are not robust to noise and outliers, and may fail to build watertight models for some point clouds that may not meet the appropriate density requirements.

[0007] Other methods may involve fine-tuning of parameters such as photoconsistency, surface visibility or regularity, etc. These methods may therefore require some manual modification for a particular instance of a scene, making the scene less generic and / or less ergonomic.

[0008] In this context, there is still a need for improved methods for 3D reconstruction of structures of real scenes. Summary of the Invention

[0009] Accordingly, the present invention provides a computer-implemented method for 3D reconstruction of a structure of a real scene. The method includes providing a 3D point cloud representing the structure. The method further includes determining a closed triangulation surface representing a skin of the structure. The determining explores closed triangulation surface candidates, each meshing at least a corresponding portion of the 3D point cloud. The determining penalizes high-ranking closed triangulation surface candidates according to a lexicographical order. The lexicographical order is based on a triangle order. The lexicographical order orders first closed triangulation surface candidates having first triangles ordered according to decreasing rank of the triangle order against second closed triangulation surface candidates having second triangles ordered according to decreasing rank of the triangle order. The triangle order penalizes the size of the triangles.

[0010] This constitutes an improved method for 3D reconstruction of real-world structures.

[0011] The method may include one or more of the following. Each of the candidate closed triangulation surfaces comprises a corresponding set of corresponding triangular faces of a corresponding tetrahedron of a tetrahedral mesh of the 3D point cloud. The tetrahedral mesh exhibits the following properties: ··The collection of tetrahedrons in the tetrahedral mesh forms a convex envelope of the points in the point cloud. The intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron. The triangle ordering penalizes high values ​​of the minimum enclosing circle radius for each triangle. The triangle ordering further penalizes smaller values ​​of the circumscribing circle radius for a first triangle and a second triangle that have the same smallest enclosing circle. Each of the candidate closed triangulation surfaces comprises a corresponding set of corresponding triangular faces of corresponding tetrahedrons of a tetrahedral mesh of the 3D point cloud, the tetrahedral mesh being a regular triangulation. Said searching and said penalizing includes: Visiting the triangles of the tetrahedral mesh according to a descending order of said triangle orders. Discarding visited triangles that have a higher position in the triangle order. Corresponding tetrahedrons of said tetrahedral mesh define corresponding regions, said determining step including: ...Involves declaring exterior tetrahedrons of the tetrahedral mesh and interior tetrahedrons of the tetrahedral mesh, where the exterior tetrahedrons and interior tetrahedrons represent regions exterior to the structure and regions interior to the structure, respectively. Visiting the triangles includes, for each visited triangle, evaluating whether the visited triangle separates an interior region from an exterior region. The discarding of the triangle includes, for each visited triangle that does not separate an interior region from an exterior region, the triangle separating the two regions, suppressing the triangle, and merging the two regions. Each visited triangle that separates the interior region from the exterior region is kept, and the kept triangles form a determined closed triangulation surface. The determining step includes providing zones in the 3D point cloud that represent interior locations of the structure, and each of the candidate closed triangulation surfaces encompasses the zone. the structure comprises at least one corner structure; and / or The 3D point cloud originates from photogrammetry, laser scanning, RGB-D measurement and / or medical or industrial tomography, and / or the real-world scene is an architectural scene, a building interior scene, a mining scene, an organic tissue, or a machine part.

[0012] Further provided is a computer program comprising instructions for carrying out the above method.

[0013] Further provided is a computer readable storage medium having the above computer program recorded thereon.

[0014] Further provided is a computer including a processor, coupled to a memory and a display, having the above computer program recorded thereon. [Brief description of the drawings]

[0015] [Figure 1] 1 shows a flow chart of an example method. [Diagram 2] FIG. 1 illustrates an example of a system. [Figure 3A] This method is shown below. [Figure 3B] This method is shown below. [Figure 4A] This method is shown below. [Figure 4B] This method is shown below. [Figure 5A] This method is shown below. [Figure 5B] This method is shown below. [Figure 6A] This method is shown below. [Figure 6B] This method is shown below. [Figure 7A] This method is shown below. [Figure 7B] This method is shown below. [Figure 8A] This method is shown below. [Figure 8B] This method is shown below. [Figure 9A] This method is shown below. [Figure 9B] This method is shown below. [Figure 10] This method is shown below. [Figure 11] This method is shown below. [Figure 12] This method is shown below. [Figure 13] This method is shown below. [Figure 14] This method is shown below. [Figure 15] This method is shown below. [Figure 16] This method is shown below. [Figure 17] This method is shown below. [Figure 18] This method is shown below. [Figure 19] This method is shown below. [Figure 20] This method is shown below. [Figure 21] This method is shown below. [Figure 22] This method is shown below. [Figure 23] This method is shown below. [Figure 24] This method is shown below. [Diagram 25] This method is shown below. [Figure 26] This method is shown below. [Figure 27] This method is shown below. [Figure 28] This method is shown below. [Figure 29] This method is shown below. [Diagram 30] This method is shown below. [Diagram 31] This method is shown below. [Diagram 32] This method is shown below. [Diagram 33] This method is shown below. [Diagram 34] This method is shown below. [Diagram 35] This method is shown below. [Diagram 36] This method is shown below. [Figure 37] This method is shown below. [Figure 38] This method is shown below. [Figure 39] This method is shown below. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] With reference to the flow chart of Fig. 1, a computer-implemented method for 3D reconstruction of a structure of a real scene is proposed. The method includes a step S10 of providing a 3D point cloud representing the structure. The method further includes a step S30 of determining a closed triangulation surface representing the skin of the structure. The determining step S30 explores candidates for closed triangulation surfaces, each meshing at least a corresponding portion of the 3D point cloud. The determining step S30 penalizes high-ranking candidates for closed triangulation surfaces according to a lexicographical order. The lexicographical order is based on a triangle order. The lexicographical order orders first candidates for closed triangulation surfaces with first triangles ordered according to a decreasing rank of the triangle order against candidates for closed triangulation surfaces with second triangles ordered according to a decreasing rank of the triangle order. The triangle order penalizes the size of the triangles.

[0017] This constitutes an improved method for 3D reconstruction of structures of real scenes.

[0018] In particular, the method allows to reconstruct the structure of a real scene in 3D. Moreover, the method does so without requiring any smoothness assumption on the point cloud and / or structure provided in step S10, thereby making it robust and able to handle different real scene structures. In fact, for example, the method can perform 3D reconstruction of the structure from a non-regular provided 3D point cloud, including parts representing non-regular structures, such as corner structures. Also, the method requires a low amount (e.g., no) of manual adjustment of parameters and is ergonomic. In particular, in any example of the method, the searching and penalization may be performed automatically, as will be further discussed below.

[0019] Furthermore, the determining step S30 penalizes high ranks of the closed triangulation surface candidates according to a lexicographical order (i.e. penalizes high ranks). The lexicographical order orders the first closed triangulation surface candidates relative to the second closed triangulation surface candidates based on the triangle order. The triangles of each closed triangulation surface candidate are ordered according to the decreasing rank of the triangle order, which penalizes the size of the triangles. This means that the determining step S30 operates the selection between the closed triangulation surface candidates and penalizes the presence of large triangles and / or the amount of large triangles in the closed triangulation surface candidates. Large triangles, for example triangles with widely elongated shapes, can represent outliers and / or regions characterized by a low density of points. In other words, large triangles represent regions of the point cloud that do not represent any part of the skin of the structure, for example regions containing outliers and / or regions corresponding to the interior or exterior of the structure. Thus, by using the lexicographical order and triangle order in such a way, decision S30 seeks among the candidate closed triangulated surfaces which best represent the skin of the structure as possible. In particular, the method efficiently handles the presence of outliers and / or noise and / or low density regions.

[0020] The method is computer-implemented. This means that the steps of the method (or substantially all steps) are performed by at least one computer or any system. Thus, the steps of the method are performed by a computer, possibly fully automatically or semi-automatically. In an example, the triggering of at least some of the steps of the method may be performed via user / computer interaction. The level of user / computer interaction required depends on the level of automation expected and may be balanced against the need to implement the user's wishes. In an example, this level may be user-defined and / or predefined.

[0021] A typical example of a computer implementation of the method is to execute the method using a system adapted for this purpose. The system may comprise a processor coupled to a memory and a graphical user interface (GUI), the memory having recorded thereon a computer program comprising instructions for executing the method. The memory may also store a database. The memory is any hardware adapted for such storage, and may optionally comprise several physically distinct parts (e.g. a part for the program and possibly a part for the database).

[0022] FIG. 2 shows an example of a system where the system is a client computer system, eg, a user's workstation.

[0023] The client computer in this example comprises a central processing unit (CPU) 1010 connected to an internal communications bus 1000, and a random access memory (RAM) 1070 also connected to the bus. The client computer is further provided with a graphical processing unit (GPU) 1110 associated with a video random access memory 1100 connected to the BUS. The video RAM 1100 is also known in the art as a frame buffer. A mass storage controller 1020 manages access to mass storage devices such as a hard drive 1030. Mass memory devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM disks 1040. Any of the foregoing may be supplemented by or incorporated in specially designed ASICs (application specific integrated circuits). A network adapter 1050 manages access to a network 1060. The client computer may also include a cursor control device, a haptic device 1090 such as a keyboard. A cursor control device is used in the client computer to allow the user to selectively position a cursor at any desired location on the display 1080. In addition, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes a number of signal generators for inputting control signals to the system. Typically, the cursor control device may be a mouse, with the buttons of the mouse being used to generate the signals. Alternatively or additionally, the client computer system may include a sensitive pad and / or a sensitive screen.

[0024] A computer program may include instructions executable by a computer, the instructions including means for causing the device to carry out the method. The program may be recordable on any data storage medium including the memory of the system. The program may be implemented, for example, in digital electronic circuitry, or in computer hardware, firmware, software, or a combination thereof. The program may be implemented as an apparatus, for example an article tangibly embodied in a machine-readable storage device for execution by a programmable processor. The steps of the method may be executed by a programmable processor executing a program of instructions for performing the functions of the method by operating on input data and generating output. Thus, the processor is programmable and may be coupled to receive data and instructions from, and to transmit data and instructions to, the data storage system, at least one input device, and at least one output device. The application program may be implemented in a high level procedural or object oriented programming language, or in assembly or machine language as appropriate. In either case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. Applying the program on the system will in any case provide instructions for carrying out the method.

[0025] The system may generally be a CAD system that manipulates modeled objects. In the context of the present disclosure, any 3D point cloud representing a structure of a real scene and / or any 3D mesh or triangulation of the point cloud or at least a part thereof may be a modeled object of a CAD system. In other words, the method may generally manipulate modeled objects.

[0026] A modeled object is any object that is defined by data stored, for example in a database. By extension, the expression "modeled object" designates the data itself. Depending on the type of system, the modeled subject may be defined by different kinds of data. The system may indeed be a CAD system, in which the modeled object is defined by the corresponding data. We can therefore speak of CAD objects.

[0027] A CAD system further means any system adapted to at least design a modeled object based on a graphical representation of the modeled object, such as CATIA. In this case, the data defining the modeled object includes data enabling the representation of the modeled object. A CAD system may, for example, provide a representation of a CAD modeled object using edges or lines, in some cases with faces or surfaces. The lines, edges, or surfaces may be represented in various ways, for example, with Non-Uniform Rational B-Splines (NURBS). In particular, a CAD file contains specifications from which a geometric shape may be generated, and thus a representation may be generated. The specifications of the modeled object may be stored in a single CAD file or in multiple CAD files. A typical size of a file representing a modeled object in a CAD system is in the range of one megabyte per part. And the modeled object may typically be an assembly of thousands of parts.

[0028] In the context of CAD, a modeled object may typically be, for example, a 3D modeled object that represents a product, such as a part or an assembly of parts, or possibly an assembly of a product. By "3D modeled object" is meant any object that is modeled by data that allows for its 3D representation. The 3D representation allows for viewing the part from all angles. For example, when a 3D modeled object is represented in 3D, it can be manipulated and rotated around any of its axes or around any axis within the screen on which the representation is displayed. This notably excludes 2D icons that are not 3D modeled. The display of 3D representations facilitates design (i.e. increases the speed at which designers statistically accomplish their tasks). This speeds up the manufacturing process in industry, since the design of the product is part of the manufacturing process.

[0029] The 3D modeled object may represent a (e.g., mechanical) part or assembly (or equivalently an assembly of parts, since an assembly of parts can be viewed as a part itself from the perspective of the method) of a geometrical portion of a (e.g., industrial) product that is manufactured in the real world after completion of the virtual design using, for example, a CAD software solution or CAD system, or more generally any rigid assembly (e.g., a moving mechanism). CAD software solutions enable the design of products in a variety of unlimited industrial sectors, including aerospace, architecture, construction, consumer goods, high tech devices, industrial equipment, transportation, marine, and / or offshore oil / gas production or transportation.

[0030] The concept of step S10, which provides a 3D point cloud representing the structure, will now be explained. Prior to the discussion of this concept, the data structures involved will now be discussed.

[0031] A real scene specifies a portion of the real world, which may be or may include an object or an array of objects. A structure in a real scene is a tangible (e.g., concrete) closed physical shape in the real scene, such as a closed layout of materials in the real scene. The skin of a structure is the closed hull (or envelope) of the structure, i.e., the material of the structure forms the contact between the opaque material in the structure's interior and the air and / or material outside the structure. Thus, the skin of a structure separates the interior of the structure, which is the opaque material in the structure's interior, from the exterior of the structure, which is the air and / or material outside the structure.

[0032] A 3D point cloud representing a structure of a real scene is here a data structure in which each point represents a respective geometric entity located within the structure of the real scene. Each geometric entity represents a corresponding position in the structural scene (in other words, a corresponding portion and / or layout of the materials that make up the structure). The collection (i.e., combination or juxtaposition) of the geometric entities represents the entire structure. Any 3D point cloud herein may include a large number of points, more than 100000, 1000000, or 100000000. It should be understood that a 3D point cloud may include points that do not represent a respective geometric entity located within the structure. These points may form or be part of an area of ​​low density of points, as described below. Such an area may represent, for example, an outer area of ​​the structure. Some points of the point cloud may be outliers, i.e. points resulting from measurement errors and therefore not representing a geometric entity of the structure.

[0033] The real scene may be an architectural scene, such as a building seen from the outside. The structure may here be a closed tangible physical shape formed by the exterior walls of the building and the building's foundation. The skin of the structure in this case is made of the surfaces of the exterior walls and the surfaces separating the building's base from the ground. Another example of an architectural scene is a tunnel. The structure may here be made of the tunnel's walls, in which case the skin of the structure is made of the surfaces of the tunnel's walls.

[0034] Alternatively, the real scene may be an interior scene of a building, such as the interior of an enclosed room. The structure may now be made of walls enclosing the room, in which case the skin of the structure is made of the surfaces of the walls.

[0035] Alternatively, the real scene may be a mining scene, such as a mining tunnel, where the structure may be made from the walls of the tunnel and the skin may be made from the surfaces of the walls of the tunnel.

[0036] Alternatively, the real scene may be an organic tissue, and the structure here may be the organic tissue itself, in which case the skin is the outer shell of the tissue.

[0037] Alternatively, the real scene may be a machine part, where the structure may be the part itself, in which case the skin is the outer hull of the part.

[0038] The mechanical parts may be one or any reasonable combination of the following: Parts of land vehicles (including, for example, automobiles and light truck equipment, racing cars, motorcycles, trucks and motor equipment, trucks and buses, trains); Parts of air vehicles (including, for example, airframe equipment, aerospace equipment, propulsion equipment, defense equipment, aircraft equipment, space equipment); Parts of marine vehicles (including, for example, naval equipment, commercial vessels, offshore equipment, yachts and workboats, marine equipment); General mechanical parts (including, for example, industrial manufacturing machinery, heavy moving machinery or equipment, installation equipment, industrial equipment, fabricated metal products, tire manufacturing products); Electrical machinery or electronic parts (including, for example, consumer electronics, security and / or control and / or instrumentation products, computer and communication equipment, semiconductors, medical devices and equipment); Consumer goods (including, for example, furniture, home and garden products, leisure goods, fashion goods, fashion products, hard goods retailer products, soft goods retailer products); Packaging (including, for example, food and beverage and tobacco, beauty and personal care, household goods packaging). Additionally or alternatively, the machined part may be one or any reasonable combination of the following: formed part (i.e., a part produced by a forming manufacturing process), machined part (i.e., a part produced by a machining manufacturing process), punched part (i.e., a part produced by a punching manufacturing process), turned part (i.e., a part produced by a turning manufacturing process), forged part (i.e., a part produced by a forging manufacturing process), pressed part (i.e., a part produced by a pressing manufacturing process), and / or folded part (i.e., a part produced by a folding manufacturing process).

[0039] In an embodiment, the structure includes at least one corner structure. The corner structure refers to a part of the structure that has or substantially has the shape of a corner. The corner of the structure may be a part of the structure that corresponds to the layout of materials forming a shape edge or an acute angle. The corner structure may be a corner of a building, a corner of a tunnel, a corner formed by the walls of a room, a corner of a mining tunnel, a corner or an acute angle in the layout of materials constituting a machine part, or a corner or an acute angle in the layout of tissue constituting an organic tissue.

[0040] In examples, the 3D point cloud originates from photogrammetry, laser scanning, RGB-D measurements, and / or medical or industrial tomography.

[0041] In the context of this method, point clouds representing the structure of architectural scenes may originate from RGB-D or photo measurements and, optionally, from structure-based motion analysis. Point clouds representing the structure of mechanical parts may result from industrial tomography or laser scanning. Point clouds representing the structure of mining scenes may originate from laser scanning of e.g. the interior of a mine. Point clouds representing the structure of organic tissues may originate from medical tomography.

[0042] Any 3D point cloud herein may be determined from physical measurements on a real scene. The step S10 of providing the 3D point cloud may in particular include providing one or more physical sensors, each configured to acquire a corresponding physical signal, and acquiring the one or more corresponding physical signals by manipulating the one or more physical sensors on the real scene (i.e., by scanning the real scene with each sensor). Providing S10 may include an automatic determination of the 3D point cloud based on the measurements according to any known technique. The one or more sensors may comprise multiple (e.g., RGB, and / or image or video) cameras, and the determining may comprise structure from motion analysis. The one or more sensors may alternatively or additionally include one or more depth sensors (e.g., on an RGB-D camera). The one or more sensors may alternatively or additionally include a laser (e.g., lidar) or an ultrasonic emitter-receiver. The one or more sensors may alternatively or additionally include one or more tomography sensors. The tomography sensors may be medical or industrial tomography sensors.

[0043] Alternatively or additionally, the step S10 of providing a 3D point cloud may include accessing a database in which the 3D point cloud is stored and retrieving the 3D point cloud from the database. The 3D point cloud may be obtained by operating one or more physical sensors and determining the 3D point cloud based on measures acquired by the one or more sensors as described above, and then stored in the database, even if the providing step S10 does not actually involve obtaining measures and determining a 3D point cloud. In any case, the step S10 of providing a 3D point cloud may be performed by a user action. The step S10 of providing a 3D point cloud may further include displaying the 3D point cloud on a display (e.g., a GUI) of a computer executing the method.

[0044] The provided 3D point cloud may include one or more regions with different densities of points. This means that the concentration of points in the point cloud may vary from one region of the point cloud to another. The separation between the exterior of the structure and the interior of the structure may typically correspond to the region of the point cloud with the highest density of points. This region may be closed, i.e. the region of the point cloud with the highest density of points may form a closed shape that separates the region inside the structure (possibly called the "interior region") from the region outside the structure (possibly called the "internal region"). The closed shape may be called closed point-like (as made of the points of the 3D point cloud) or may be called the same as a closed shape. The interior and exterior regions are typically regions characterized by a lower density of points. The closed shape may be such that it is closed when the 3D point cloud is provided. Alternatively, the closed shape may be characterized by one or more holes that are part of a closed shape with a lower point density, for example, due to technical impossibility of performing measurements on real-world zones corresponding to these holes, and / or due to measurement errors, and / or due to measurement inconsistencies. In such a case, the step S10 of providing the 3D point cloud may include filling these holes with points to form a closed shape, either in response to a user's action or automatically. Filling these holes may consist of adding points to the 3D point cloud until the density of points is substantially the same in any part of the closed shape. Adding points may be performed by a user graphically interacting with a screen of the computer executing the method, for example by repeatedly clicking on the point cloud to create points. In any case, whether filling occurs or not, the separation between the exterior and interior of the structure is represented by an area of ​​the 3D point cloud that has the highest density of points and is a closed shape of the 3D point cloud that separates areas of the interior of the structure from areas of the exterior of the structure, the interior and exterior of the area having a lower density of points, respectively. In an embodiment, "having the highest density" means that the density of points of the closed shape is higher than the ambient noise resulting from the measurement. The 3D point cloud may further feature outliers, which are points of the 3D point cloud that have a low density of points and are located in areas that originate from measurement errors.In other words, the outliers do not represent geometric entities of the real scene, but correspond to numerical and / or measurement artifacts.

[0045] We now briefly describe the concept of 3D reconstruction of a structure represented by a 3D point cloud. The 3D reconstruction of a structure generally specifies the operation of calculating / determining a surface connecting the points of the 3D point cloud. The surface may be said to "represent the skin of the structure" since it connects the points of the 3D point cloud to a closed surface that represents a boundary (e.g., a boundary) between an area outside the structure and an area inside the structure. The surface therefore represents a closed hull of the structure, in other words, a skin. For example, as mentioned above, a delimiter may correspond to a dense closed shape of points in the 3D point cloud. The 3D reconstruction of a structure may in this case specify the determination / calculation of a surface connecting the points of the closed shape to form a surface hull that envelops the closed shape.

[0046] Next, an example of a 3D point cloud will be described.

[0047] Figure 3A shows an example of a provided 3D point cloud, here representing a wall of a part of a bridge. The scene here is an urban architectural scene, and the step S10 of providing the 3D point cloud notably includes structure from motion analysis, as described above. Figure 3B shows a 3D reconstruction of the structure represented by the point cloud of Figure 3A according to an example of the method described below.

[0048] Fig. 4A shows an example of a provided 3D point cloud, here representing an engine. The scene here is a machine part and the 3D point cloud comes from industrial tomography. Fig. 4B shows a 3D reconstruction of the structure represented by the point cloud of Fig. 4A according to an example of the method described below.

[0049] Figure 5A shows an example of a provided 3D point cloud, here representing the wall of a mine. The scene here is a mining scene, and the 3D point cloud comes from a laser scan of the interior of the mine. Figure 5B shows a 3D reconstruction of the structure represented by the point cloud of Figure 5A according to an example of the method described below.

[0050] Figure 6A shows an example of a provided 3D point cloud, here representing the mandible. The scene here is an organic tissue and the 3D point cloud originates from a medical tomography. Figure 6B shows a 3D reconstruction of the structure represented by the point cloud of Figure 6A according to an example of the method described below.

[0051] Figure 7A shows an example of a provided 3D point cloud, here representing a locker. The scene here is a machine part, and the 3D point cloud comes from a scan of the machine part. Figure 7B shows a 3D reconstruction of the structure represented by the point cloud of Figure 7A according to an example method described below.

[0052] Figure 8A shows an example of a provided 3D point cloud, here representing the exterior walls of a building. The scene here is an architectural scene, the 3D point cloud originates from a correlation process of several images of the scene, and the providing step S10 involves structure from motion analysis, as described above. Figure 8B shows a 3D reconstruction of the structure represented by the point cloud of Figure 8A according to an example of the method described below.

[0053] Figure 9A shows an example of a provided 3D point cloud, here representing a Buddha. The scene here is an organic tissue, and the 3D point cloud results from a tomography. Figure 9B shows a 3D reconstruction of the structure represented by the point cloud of Figure 9A according to an example of the method described below.

[0054] Next, step S30 of determining a closed triangulated surface representing the skin of the structure will be described.

[0055] The determined closed triangulation surface represents the skin of the structure. In other words, the closed triangulation surface represents a separation between regions inside the structure and regions outside the structure, e.g., the closed surface hull of the previously discussed closed shape of the 3D point cloud having the highest point density and a region separating the regions inside the structure from the regions outside the structure. The closed triangulation surface may be a mesh-like surface, e.g., each vertex of the mesh-like closed triangulation surface may correspond to a point cloud of the 3D point cloud. In other words, the closed triangulation surface may mesh at least a portion of the points of the closed shape of the 3D point cloud having the highest density of points, forming a surface separating the regions inside the structure from the regions outside the structure. In other words, the method performs a surface reconstruction using the original points of the provided 3D point cloud, which makes the method particularly accurate since, as mentioned above, the points of the point cloud represent the actual geometric entities located on the structure.

[0056] The determining step S30 searches for candidate closed triangulated surfaces. Each candidate closed triangulation surface has triangles, for example at least four triangles. Each candidate closed triangulation surface meshes at least a portion of the 3D point cloud. For example, each candidate closed triangulation surface is a mesh that meshes at least a portion, and the vertices of the mesh correspond to the points of at least the portion. In particular, in the example, each candidate closed triangulation surface includes a corresponding set of corresponding triangular faces of corresponding tetrahedrons of the tetrahedral mesh of the 3D point cloud. By "searching" and "penalizing", it should be particularly understood that the determining step S30 selects among several closed triangulation surfaces (i.e. candidates closed triangulation surfaces) and selects only one of them (the closed triangulation surface representing the skin of the structure).

[0057] Selection among candidate closed triangulated surfaces specifically includes lexicographic ordering and triangle ordering, which are described below.

[0058] The lexicographic order orders the first closed triangulation surface candidates with respect to the second closed triangulation surface candidates, i.e. provides an ordering between the first closed triangulation surface candidates and the second closed triangulation surface candidates. The concept of lexicographic order is known per se. The first closed triangulation surface candidates have a first triangle (e.g. as discussed above), e.g. a corresponding first triangular face of a corresponding tetrahedron of the tetrahedral mesh of the 3D point cloud, as discussed above, and the second closed triangulation surface candidates have a second triangle (e.g. consisting of a corresponding second triangular face of a corresponding tetrahedron of the tetrahedral mesh of the 3D point cloud, as discussed above). Both the first triangle and the second triangle are ordered according to a decreasing order of the triangle order.

[0059] Ordering the first closed triangulation surface candidates relative to the second closed triangulation surface candidates according to a lexicographical order may include evaluating the first triangle ordering and evaluating the second triangle ordering according to a descending rank of the triangle ordering, particularly since the lexicographical order is based on a triangle order. The ordering may further include establishing a rank in the lexicographical order of the first closed triangulation surface candidates relative to the second closed triangulation surface candidates based on the evaluated ordering of both the first triangles and the second triangles. In other words, the ordering may include establishing a relative ranking between the first closed triangulation surface candidates and the second closed triangulation surface candidates in the lexicographical order by comparing the ordered first and second triangles.

[0060] An example of ordering a first closed triangulation surface candidate relative to a second closed triangulation surface candidate according to lexicographical order will now be described.

number

number

[0061] The ordering may result in the set of all closed triangulation surface candidates being ordered according to lexicographical order, i.e. each closed triangulation surface candidate having a respective rank in the lexicographical order. The determining step S30 penalizes high ranks of the closed triangulated surface candidates. In an example, this means that the determining step S30 may be such that closed triangulation surface candidates are discarded due to their high rank and one or more closed triangulation surface candidates are rewarded due to their low rank. The determining step S30 may result in the closed triangulation surface candidate having the smallest rank in the lexicographical order among all ranks of all closed triangulation surface candidates being determined as the closed triangulation surface representing the skin of the structure.

[0062] The triangle order penalizes the size of the triangles. The concept of penalizing the size of the triangles is discussed here. For the sake of explanation, let T and T' be "two triangles". Penalizing the triangle size may include evaluating, for each of the triangles T and T', a respective measure of the triangle size and attributing a higher rank in the triangle order to the triangle having the larger measure. The measure of the size of the triangle may also be referred to as the triangle weight or triangle weight. The measure of the size may include, for example, a first real-valued function that contributes to the measurement of the triangle size, a second real-valued function that contributes differently to the measurement of the triangle size, and / or a combination of the first and second functions. Penalizing the triangle size may include combining and ordering the two evaluated measures of size together, for example according to any order of combining and ordering two real numbers or two combined real numbers together, as known per se in the art. In other words, if T is larger than T' (i.e. has a larger size), then the measure of the size of T is larger than the measure of the size of T' (e.g. according to the order of the measures). Now, if the pair (T and T') is ordered according to descending triangle order, then, like the first and second triangles above, T comes before T' in this ordering. Similarly,

number

[0063] This means that when ordering two closed triangulation surface candidates as described above, the higher rank in the lexicographical order is due to the triangle that is most penalized by the triangle order, e.g., the one of the two with the largest triangle. As described above, such a triangle may represent (e.g., be part of its mesh) an area of ​​the 3D point cloud with a low density of points, including outliers. Disclosing closed triangulation surface candidates with such triangles by penalizing their high rank in the lexicographical order allows obtaining a closed triangulation surface that accurately represents the skin of the structure, reducing sensitivity to density variations and / or outliers. This makes the method robust and accurate.

[0064] In an example, this method may result in the triangles of the determined closed triangulation surface (e.g., all of them or a significant portion thereof) being small triangles (i.e., having a small size), including triangles that are not very elongated in shape, e.g., triangles that have a relatively small area and a ratio between the triangle height and the triangle base that is close to 1. Indeed, while the triangle order does not necessarily penalize the flattest triangle of the two when comparing two triangles, basing the triangle order on a lexicographical order may, however, result in flat triangles being avoided (e.g., in general). In fact, the lexicographical order compares triangle chains (each of which forms a closed triangulation surface) and tries to reward (i.e., give a low rank, e.g., the lowest rank) chains made of the smallest possible triangles, rather than a single pair of triangles. Comparing triangle chains and penalizing having (e.g., too many) large triangles, rather than a single pair of triangles, may indeed result in the determined closed triangulation surface being made of (e.g., in general) small and not very flat triangles. Such triangles represent the skin of the structure. This makes the method robust and accurate.

[0065] In the example, for each triangle, the triangular order gives a high value of the radius of the minimum enclosing circle, that is, penalizes the minimum enclosing circle of the triangle.

[0066] The minimum enclosing circle of a triangle is the circle with the minimum radius among all the circles that enclose the points of the triangle. The radius of the minimum enclosing circle of a triangle contributes to measuring the size of the triangle, and penalizing a high value of the radius of the minimum enclosing circle contributes to penalizing the size of the triangle. In other words, the radius of the minimum enclosing circle is a real-valued function that contributes to measuring the size of the triangle and, as discussed above, can be part of the measure of the size of the triangle. Further, penalizing a high value of the radius of the minimum enclosing circle is a particularly efficient and robust way to penalize the size of the triangle. For example, let T and T’ be two triangles, and let the radii of their respective minimum enclosing circles be RB(T) and RB(T’). And in these examples, the inequality RB(T) < RB(T’) captures the fact that T’ has a larger size than T. In this case, penalizing the size includes giving a higher rank to T’ than to T in the triangular order. Also, ordering T and T’ in descending order of their ranks in the triangular order means that T’ comes before T.

[0067] In the example, for a first triangle and a second triangle having the same minimum enclosing circle in the triangular order, a small value of the radius of the circumcircle, that is, the circumcircle of the (first or second) triangle, is further penalized.

[0068] The circumcircle of a triangle is a circle passing through the points of the triangle. The radius of the circumcircle of a triangle contributes to measuring the size of the triangle, and penalizing a small value of the radius of the circumcircle contributes to penalizing the size of the triangle, particularly in the case of the equation of the minimum circumcircle, as will be described later. In other words, the radius of the circumcircle is a real-valued function that contributes to measuring the size of the triangle and, as discussed above, can be part of the measure of the size of the triangle.

[0069] Penalizing small values ​​of the circumscribing circle radius in such an example of a minimum enclosing circle equation improves the robustness of the method. In particular, let T and T' be two triangles and RB(T) and RB(T') be the radii of their respective minimum enclosing circles. Assume RB(T)=RB(T'), which may occur, i.e., when both T and T' are obtuse angled triangles. In other words, evaluating the radii of the respective minimum enclosing circles for both T and T' and penalizing high values ​​of the evaluated radii may not be sufficient to establish a relative triangle order between the triangle sizes and penalize the triangle sizes. In other words, it may not be possible to determine which of T and T' is the largest, as penalizing based only on the minimum enclosing circle. However, further penalizing small values ​​of the circumscribing circle radius makes such a determination possible, and in that respect improves the robustness of the method. Moreover, penalizing small values ​​of the circumscribing circle radius is a particularly efficient and robust way of penalizing triangle sizes. For example, let RC(T) and RC(T') be the radii of the circumscribing circles of T and T', respectively. Then, in these examples, if RB(T)=RB(T'), then the inequality RC(T)>RC(T') captures the fact that T' has a larger area than T. In other words, the equality RB(T)=RB(T') expresses the fact that the radius of the smallest enclosing circle is not sufficient to determine whether T or T' is the largest. However, the inequality RC(T)>RC(T') expresses that T' has a larger area than T, which allows one to determine that T' is larger than T, regardless of the smallest enclosing circle between the two triangles. In this case, penalizing size may include giving T' a higher rank in the triangle order than T. Also, ordering T and T' according to descending triangle order means that T' comes before T.

[0070] As discussed above, in the example, the size of each triangle T is measured by a measure of the size of a triangle that is a pair of real-valued functions consisting of the reciprocal of the radius of the minimum enclosing circle of T and the radius of the circumcircle of T. The measure is given by the following formula.

Number

Number

Number

Number

Number

Number

[0071] Candidate closed triangulated surfaces are further discussed below.

[0072] As mentioned above, in the example, each candidate closed triangulation surface comprises a corresponding set of corresponding triangular faces of corresponding tetrahedrons of the tetrahedral mesh of the 3D point cloud. The triangular faces are the triangles discussed above of the candidate closed triangulation surface. The tetrahedral mesh may in the example be a common tetrahedral mesh for all candidate surfaces (i.e. each candidate closed triangulation surface comprises a corresponding set of corresponding triangular faces of corresponding tetrahedrons of the same common tetrahedral mesh of the 3D point cloud). The tetrahedral mesh may be calculated / determined once and for all after the step S10 of providing the point cloud and before the step S30 of determining the closed triangulation surface. In other words, the method may comprise meshing S20 of the provided 3D point cloud. This simplifies the method and makes it faster. Alternatively, the method may operate on the basis of several tetrahedral meshes.

[0073] In the example, the tetrahedral mesh presents the following properties: the union of the tetrahedral mesh tetrahedrons forms a convex envelope of the points of the point cloud, and the intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron, which improves the quality of the meshing and, consequently, the quality of the determined closed triangulation surface.

[0074] In the example, the tetrahedral mesh is a regular triangulation, which further improves the quality of the meshing and therefore the quality of the determined closed triangulated surface.

[0075] In the field of computational geometry, as known per se, a regular triangulation of a 3D point cloud is a triangulation induced by the shadows of the faces of one higher dimensional (i.e. dimension 4) polyhedron. A regular triangulation is simply the dual of a power diagram, see for example Algorithmic geometry, Jean-Danel Boissonnat and Mariette Yvinec, Cambridge University Press New York, NY, USA. As known per se, a regular triangulation can for example be obtained in a meshing S20 of a point cloud, for example using any regular triangulation method, as follows: Lift the points of the point cloud, i.e. add a coordinate to each point. This embeds the set of points into a space whose dimensions are equal to the dimensions of the original space (i.e. the 3D space the 3D point cloud belongs to) that contains the set of points plus 1. Compute the convex envelope of the set of lifted points. Triangulate the convex envelope by any known method: the union of the tetrahedral mesh tetrahedrons forms the convex envelope of the set of lifted points, and the intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of said first tetrahedron, or a face of the first tetrahedron. ·Project the underside of this triangulation of the convex envelope back into the original space.

[0076] In the example, the regular triangulation is a Delaunay triangulation. In such a case, the points are lifted onto a paraboloid, the lifting being parametrized by a vertical gap between the lifted points and the paraboloid. This further improves the quality of the meshing and therefore the quality of the determined closed triangulated surface. Moreover, the method performs a 3D reconstruction by going further than simply computing a Delaunay triangulation. The method indeed benefits from the quality of the Delaunay tetrahedral mesh, but goes beyond it by a step S30 of determining among all candidate closed triangulated surfaces made of triangular faces of the Delaunay triangulation, the one that truly represents the skin of the structure. In other words, the method selects the best representation of the skin of the structure among the closed triangulated surfaces made of triangular faces of the Delaunay tetrahedral mesh.

[0077] Indeed, penalizing the higher ranking of candidates for closed triangulation surfaces made of triangular faces of a regular (e.g. Delaunay) tetrahedral mesh, with respect to a lexicographical order based on triangle ordering that penalizes triangle size, makes the determined closed triangulation surface particularly representative of the skin of the structure. In particular, in such a case, the triangles of the determined proximal triangulation surface are not too large and / or not too flat, for example the determined triangles are, on the contrary, triangles having a relatively small area and a ratio between the triangle height and the triangle base close to 1.

[0078] In practice, the determined closed triangulation surface made of the triangular faces of the 3D Delaunay tetrahedral mesh is (or at least resembles) a 2D Delaunay triangulation of the portion of the 3D point cloud that corresponds to the skin of the structure. The 2D Delaunay triangulation has the property that, for each triangular component of the triangulation, it maximizes the value of the minimum angle of the triangles. Thus, the 2D Delaunay triangulation avoids (e.g., in general) flat triangles. Indeed, as discussed further below in section "Notes on the algorithm", it has been observed that given a set of coplanar points, the 2D Delaunay triangulation of the set of coplanar points can be defined as the minimum of several criteria. As an example, given a 2D point cloud K (where T is a 2D meshing of its convex hull Ω), the 2D Delaunay triangulation T* of K is a triangulation of all real

number

number

number

number

[0079] In either case, even if the candidate triangulation surface is not made of triangles of a 3D Delaunay tetrahedral mesh, determining S30 can result in the determined closed triangulation surface being (or at least similar to) a 2D Delaunay triangulation representing the skin of the structure. In particular, the determined closed triangulation surface will (e.g., typically) be made of small, not too flat triangles. However, limiting the candidate triangulation surfaces to faces made of triangles of the same Delaunay tetrahedral mesh reduces the number of candidate triangulation surfaces, improving the speed and efficiency of the method.

[0080] Here, an example of the determining step S30 will be described.

[0081] In an example, the determining step S30 includes providing zones within the 3D point cloud that represent interior locations of the structure, with each candidate closed triangulation surface encompassing the zone.

[0082] As mentioned above, a structure corresponds to a closed shape of the 3D point cloud that is dense, e.g., a density higher than the ambient noise. However, it may happen that two or more such closed shapes with substantially equal density are present in the point cloud. In order to distinguish between the two or more that correspond to the structure, the determining step S30 includes providing a zone in the 3D point cloud that represents an internal location of the structure. This internal location is typically surrounded (e.g., enclosed) by exactly one of the closed shapes. Therefore, the candidate closed triangulation surface searched for by the determining step S30 is required to encompass the zone. Thus, the provision of the zone makes it possible to select one of the closed shapes of the 3D point cloud that truly corresponds to the structure, in which case a representation of the skin can be determined (S30). In other words, the provision of the zone makes the method more accurate and robust. In particular, the method is able to reconstruct the structure while there are some regions of the point cloud that are characterized by a high density of points in addition to the regions that correspond to the structure.

[0083] The provision of the zone may be performed by a user. For example, the 3D point cloud may be displayed on a display (e.g., a GUI) of the computer executing the method, and the user defines the zone, for example, by graphical interaction with the display. For example, the user may click on a location in the point cloud to define the zone. In such a case, the search and penalty may be performed automatically upon provision of the zone by the user.

[0084] 10-12 show the effect of lexicographical ordering among candidates for closed triangulation surfaces. FIG. 10 shows two closed shapes 100 and 102 in a point cloud, each having a high density of points. Shape 102 corresponds to a structure to be reconstructed, i.e., a closed triangulated surface meshing shape 102 is determined according to the present method. Shape 100 represents another part of the point cloud that can be meshed by a candidate for closed triangulation surface that has a higher rank in the lexicographical ordering than the closed triangulation surface meshing shape 102. The determining step S30 includes providing a zone "interior" 104 in the 3D point cloud. The zone represents an interior location of the structure corresponding to shape 102. FIG. 11 shows what can happen without lexicographical ordering. A closed triangulation surface 110 that represents (i.e., meshes) shape 100 but not structure 102 can be obtained despite the higher rank of closed triangulation surface 110 in the lexicographical ordering. 12 shows the results of determining S30 according to the method of the present disclosure, specifically that due to the lexicographic order, shape 102 that truly represents the structure is meshed by the determined S30 closed triangulated surface 120, which is the one that has the lowest rank in the lexicographic order among faces 110 and 120. In fact, shape 100 has larger holes than shape 102. These largest holes will result in the largest edges in shape 100, and because the lexicographic orders are compared, the largest edges penalize shape 100.

[0085] 13 to 15 show the effect of lexicographic ordering among candidates for closed triangulation surfaces. FIG. 13 shows two closed shapes 130 and 132 in a point cloud, each having a high density of points. Shape 132 corresponds to a structure to be reconstructed, i.e., a closed triangulation surface meshing shape 132 is determined according to the present method. Shape 130 represents another part of the point cloud that can be meshed by a candidate for closed triangulation surface having a higher rank in the lexicographic order than closed triangulation surface meshing shape 132. The determining step S30 includes providing a zone "interior" 134 in the 3D point cloud. The zone represents an interior location of the structure corresponding to shape 132. FIG. 14 shows what can happen without lexicographic ordering. A closed triangulation surface 140 representing (i.e., meshing) shape 130 but not structure 132 can be obtained despite the higher rank of closed triangulation surface 140 in the lexicographic order. 15 shows the result of determination S30 according to the method of the present disclosure, in particular because of the lexicographic order, that shape 132 that truly represents the structure is meshed by the closed triangulated surface 150 determined in step SD30, which is the one that has the lowest rank in the lexicographic order among faces 140 and 150. Effectively, shape 130 has one more hole than shape 132. This single largest hole will be the single largest edge in shape 130. And because the lexicographic orders are compared, the single largest edge is sufficient to penalize shape 130.

[0086] Further examples of the determining step S30 will now be described, in which each candidate closed triangulated surface comprises a corresponding set of corresponding triangular faces of corresponding tetrahedrons of the tetrahedral mesh of the 3D point cloud, as described above.

[0087] The searching and penalizing may include visiting triangles of the tetrahedral mesh according to decreasing rank of the triangle order and discarding visited triangles having a high rank of the triangle order.

[0088] Visiting the triangles of the tetrahedral mesh may consist of visiting all the triangles of the tetrahedral mesh, or only a part of them, for example, iteratively until a closed triangulation surface is determined. It should be understood that visiting the triangles and discarding some of them results in a closed triangulation surface representing the skin of the structure. In other words, after all the triangles to be visited have been visited and after all the triangles to be discarded have been discarded, the remaining triangles (those that are not discarded) form the closed triangulation surface representing the skin of the structure. In that respect, discarding more visited triangles, less visited triangles, and / or other different visited triangles results in another closed triangulation surface, which in fact corresponds to one of the candidates for the closed triangulation surface. In other words, each candidate for the closed triangulation surface corresponds to a respective triangle discarding method. By penalizing the high rank of the candidates for the closed triangulation surface in the lexicographical order, the decision step S30, as described above, operates a selection among different methods of discarding triangles, which in turn operates a selection among different candidates for the closed triangulation surface.

[0089] The triangles of the tetrahedral mesh are visited according to a decreasing rank of the triangle order. This means that the more the size of a triangle of the tetrahedral mesh is penalized by the triangle order, the earlier the triangle is visited. In the above example where the triangle size is measured by a measure, this means that the larger the triangle size measure, the earlier the triangle is visited. This allows to quickly and efficiently detect triangles that are highly ranked in the triangle order, i.e. the largest, and discard them to prevent them from belonging to the set of non-discarded triangles, the latter corresponding to closed triangulation surfaces representing the structure. More specifically, a closed triangulation surface representing the structure should be made of relatively small, not too large, triangles. This is because, as discussed above, this structure represents a closed shape of a 3D point cloud with a high density of points. Such triangles have a small rank in the triangle order, respectively, because of the penalization of the triangle size. Thus, by visiting triangles in decreasing rank of the triangle order and discarding visited triangles with high rank (i.e., discarding triangles because of their high rank), the method ensures that larger triangles are discarded and only triangles that are satisfactorily small are retained to form a closed triangulated surface representing the structure.

[0090] In the examples, each tetrahedron of the tetrahedral mesh defines a respective region. In these examples, the determining step S30 further includes declaring tetrahedrons exterior to the tetrahedral mesh and tetrahedrons interior to the tetrahedral mesh. The exterior and interior tetrahedrons represent exterior and interior regions of the structure, respectively. In these examples, the triangle visiting includes, for each visited triangle, evaluating whether the visited triangle separates the interior region from the exterior region. In these examples, the triangle discarding includes, for each visited triangle that does not separate the interior region from the exterior region, whether the triangle separates the two regions, suppressing the triangle, and merging the two regions.

[0091] The declaration of the external and internal tetrahedrons may be performed by a user. For example, the 3D point cloud may be displayed on a display (e.g., a GUI) of the computer executing the method, and the user may define the external and internal tetrahedrons by graphical interaction with the display. In such a case, the search and the giving of penalties, and therefore the visiting and discarding, may be performed automatically once the user declares the external and internal tetrahedrons. The internal tetrahedrons may for example be located within a provided zone in the 3D point cloud that represents an internal location of the structure. In an example, the internal tetrahedrons are located in a zone of the 3D mesh that has a low density of points and represents an internal location of the structure. The external tetrahedrons may be tetrahedrons that are added to the tetrahedral mesh as a representation of an external region, which is a region that does not belong to the mesh and is located outside the mesh. However, the internal tetrahedrons may themselves belong to the tetrahedral mesh. In other words, declaring an interior tetrahedron may consist of selecting a tetrahedron of the tetrahedral mesh, e.g., by clicking on the representation of the tetrahedron, and labeling it as an interior tetrahedron while declaring an exterior tetrahedron by adding supplementary and boundary tetrahedrons to the mesh, e.g., by clicking on them, and labeling them as exterior tetrahedrons. In an example described further below, a tetrahedral mesh can be represented by its dual graph, whose nodes represent the tetrahedrons of the tetrahedral mesh and whose arcs represent the triangles. In such an example, the declaration consists of selecting (e.g., by clicking on) a node of the graph to represent the interior triangle, and adding (e.g., by clicking on) a node of the graph to represent the exterior triangle.

[0092] The visiting of triangles and the discarding of visited triangles with a high rank in the triangle order must be such that enough triangles are discarded so that the closed triangulation surface formed by the non-discarded triangles truly represents the structure, i.e. contains as few as possible number of triangles with a high rank in the triangle order. At the same time, however, the visiting and discarding must be such that discarding triangles that should be in the closed triangulation surface representing the skin of the structure is avoided, for example by discarding wrong triangles and / or too many triangles. The triangles that form the closed triangulation surface representing the skin of the structure actually correspond to the triangles of the tetrahedral meshing that separate the interior of the structure from the exterior. In addition to discarding high-ranked triangles, by evaluating whether a triangle separates the interior region from the exterior region, it is possible to accurately determine the triangles that separate the interior of the structure from the exterior, to keep them and to suppress the others. This improves the robustness and accuracy of the method.

[0093] The declaration, visitation and destruction are discussed in more detail here.

[0094] Before visiting the triangles, the exterior tetrahedrons are located outside the structure. In other words, the exterior tetrahedrons represent the exterior region, which can correspond to all the infinite space around the point cloud. The interior tetrahedrons represent the interior region, which corresponds to the portion of the point cloud that represents the interior of the structure. Now consider the first visited triangle T. The visitation involves evaluating whether T separates the interior region from the exterior region. If T does not separate the interior and exterior regions, T separates the two tetrahedrons. The first tetrahedron defines a first region and the second tetrahedron defines a second region. Here, it follows that at least one of the first region and the second region is neither exterior nor interior to the region. In such a case, discarding involves merging the two regions defined by the two tetrahedrons. If the first (or second) region represents an exterior region, the second (or first) region is merged with the exterior region. If the first (or second) region represents an interior region, the second (or first) region is merged with the interior region. Otherwise, the two regions are simply merged into a new region that is neither exterior nor interior. In any case, triangle T is suppressed / discarded. Conversely, if T separates an interior region from an exterior region, T is not discarded or suppressed and the regions separated by T are not merged. In this case, T is kept. Then, for example, it is evaluated whether the second triangle visited (let's say T' for example) separates an interior region (possibly resulting from a previous merging) from an exterior region (possibly resulting from a previous merging). Then, like T, T' is also discarded if a second merging occurs, or T' is kept. Then, the next triangle is visited, and so on. In other words, triangles are iteratively suppressed / discarded and regions are iteratively merged with each other and with interior and / or exterior regions until only triangles remain that separate an interior region representing the interior of the structure from an exterior region representing the exterior of the structure.

[0095] In the example, each visited triangle that separates the interior region from the exterior region is retained, and the retained triangles form a determined closed triangulation surface.

[0096] Next, an example of the method is described. In this example, the search and penalization includes visiting the triangles of the tetrahedral mesh according to descending triangle order and discarding the visited triangles with a higher triangle order. Each tetrahedron of the tetrahedral mesh defines a respective region. The determining step S30 further includes declaring the tetrahedrons outside the tetrahedral mesh and the tetrahedrons inside the tetrahedral mesh, the outside tetrahedrons and the inside tetrahedrons representing the exterior region of the structure and the interior region of the structure, respectively. The triangle visiting includes evaluating, for each visited triangle, whether the visited triangle separates the interior region from the exterior region. The triangle discarding includes, for each visited triangle that does not separate the interior region from the exterior region, the triangle separates the two regions, suppressing the triangle, and merging the two regions. Each visited triangle that separates the interior region from the exterior region is kept, and the kept triangles form the determined closed triangulation surface. FIG. 3B shows a 3D surface reconstruction of the point cloud of FIG. 3A according to this example of the method. FIG. 4B shows a 3D surface reconstruction of the point cloud of FIG. 4A according to this example of the method. FIG. 5B shows a 3D surface reconstruction of the point cloud of FIG. 5A according to this example of the method. FIG. 6B shows a 3D surface reconstruction of the point cloud of FIG. 6A according to this example of the method. FIG. 7B shows a 3D surface reconstruction of the point cloud of FIG. 7A according to this example of the method. FIG. 8B shows a 3D surface reconstruction of the point cloud of FIG. 8A according to this example of the method. FIG. 9B shows a 3D surface reconstruction of the point cloud of FIG. 9A according to this example of the method.

[0097] An embodiment of this method will now be described with reference to Figures 16 to 45. Figure 16 is a flow chart illustrating this embodiment.

[0098] Referring to the flow chart of FIG. 16, this embodiment includes the steps of providing a 3D point cloud S10, and a mesh S20 of the 3D point cloud, as discussed above. The mesh is a Delaunay triangulation of the provided 3D point cloud. Since the point cloud contains three-dimensional points, the resulting mesh is made of adjacent tetrahedrons, each tetrahedron being surrounded by four triangles. Next, this embodiment includes the step of determining a closed triangulation surface S30, as previously described. The determining step S30 includes the step of defining a lexicographical order over the triangles. To establish this ordering, a weight formula is used. Next, determining includes computing a dual graph of the tetrahedral mesh. This graph captures adjacent quadrilaterals through triangle sharing. Thus, each node of the dual graph represents a tetrahedron of the mesh, each arc of the dual graph represents a triangle of the mesh, and each arc of the dual graph is labeled with the rank of the corresponding triangle in the lexicographical order. The arcs of the dual graph are used in descending order. The determining step S30 further includes the step of selecting a tetrahedron that is in the interior of the unknown surface. The determining further includes visiting and discarding, which is performed by applying a reduction algorithm, as discussed above. The algorithm is initialized with selected interior and exterior nodes and utilizes the dual graph and its sorted arcs. Its goal is to discard arcs of the dual graph such that the remaining dual arcs define a closed surface divided into triangles, which is the reconstructed surface.

[0099] In contrast to prior art methods, this embodiment includes the local density of the point cloud, which frees it from mandatory tuning parameters. For very dense point clouds, an arbitrary filtering distance parameter can be used to reduce the number of points in the Delaunay triangulation for performance purposes. This embodiment produces watertight surfaces that are robust to outliers and density variations in the point cloud. This embodiment also features a theoretical complexity that is comparable to graph cut methods and often better in practice. The tradeoff of this reconstruction method is the need to close the point cloud when sampling open surfaces and specifying interior regions that can be easily inferred from building footprints in the context of building reconstruction.

[0100] Meshes and dual graphs This implementation uses a Delaunay tetrahedral convex envelope mesh of 3D point clouds (F. Cazals, J. Giesen, Delaunay triangulation based surface reconstruction, Effective computational geometry for curves and surfaces, pages 231-276, 2006.). A tetrahedral mesh object is a collection of tetrahedrons characterized by the following properties: The topology is a set of adjacent tetrahedrons such that any triangle is shared by exactly two regions. The first sharing situation is a triangle shared by two adjacent tetrahedrons. The second sharing situation is a triangle shared by a tetrahedron and an unbounded exterior region. A triangle shared by a tetrahedron and an exterior region is called a boundary triangle. A geometric property of a Delaunay mesh is that for any tetrahedron, the sphere defined by its four points (called the circumscribing sphere) does not contain any other points of the cloud.

[0101] The dual graph of a tetrahedral mesh is defined as follows: the nodes of the dual graph are associated with tetrahedra and the extra nodes are associated with the exterior area. The arcs of the dual graph are associated with triangles. An arc connects two nodes if the triangles associated with the arcs are respectively shared by the areas associated with the nodes.

[0102] 17-19 show a mesh consisting of three tetrahedrons 1, 2, and 3. FIG. 17 shows a 3D point cloud. FIG. 18 shows the geometry of a convex hull mesh including points, lines, triangles, and tetrahedrons. FIG. 20 shows the corresponding dual graph. For example, dual graph arc “a” connects nodes 1 and 2 because mesh triangle “a” is shared with tetrahedrons 1 and 2. Dual graph arc “b” connects nodes 2 and 4 because mesh triangle “b” is shared with tetrahedron 2. Dual graph arc “c” connects nodes 2 and 4. Dual graph arc “d” connects nodes 1 and 3 because mesh triangle “d” is shared with tetrahedrons 1 and 3. Dual graph arc “e” connects nodes 2 and 4 because triangle “e” is shared with tetrahedron 3.

[0103] Algorithm For clarity, this example is shown with a 2D problem. The correspondence with the 3D problem is as follows: the 2D problem deals with planar point clouds as opposed to 3D point clouds. The tetrahedrons in the 3D problem are the triangles in the 2D problem. The triangles enclosing the tetrahedrons in the 3D problem are the line segments enclosing the triangles in the 2D problem. Finally, the recognized closed triangulated surfaces of the 3D problem are the closed polylines in the 2D problem.

[0104] Figure 20 shows the initial point cloud. Visually, this clearly defines the L-shaped proximity polyline that the implementation aims to compute. Figure 21 shows the triangulation of the point cloud (reprojected tetrahedral 3D mesh). As shown in Figure 22, the line segments (or 3D triangles) are labeled with their respective ranks according to their respective weights and descending order. In the context of the 2D problem, the weights are the lengths of the line segments. The first line segment (1) is the longest line segment and the last line segment (32) is the shortest line segment. The weight calculation for triangles in the 3D problem is detailed below.

[0105] Figure 23 shows the dual graph along with the meshed point cloud. Each node of the dual graph (empty circle) is associated with a triangle (each 3D tetrahedron) and each arc of the dual graph is associated with a line segment of the mesh (each 3D triangle). Note that there is an exterior node associated with infinite space all around the point cloud. This node is reused for initialization purposes. Figure 24 shows the dual graph with its arcs labeled with their respective ranks in the ordering. Interior and exterior nodes are filled dots.

[0106] The reduction algorithm computes so-called minimum chains. Its formulation is independent of the dimensionality issue (2D vs. 3D). The input of the reduction algorithm is a dual graph.

number

number

number

number

[0107] After the termination of the minimal chain has been calculated, the corresponding lines (or 3D triangles) of the meshed point cloud are collected and theory guarantees that they define a closed polyline (or 3D triangulated surface), which is the output object.

[0108] When performed on a 2D example, the reduction algorithm eliminates the dotted dual arcs in Figure 25. The corresponding line segments of the mesh (e.g., 3D triangles) are also removed, as illustrated in Figure 26, resulting in the resulting polylines (3D triangulated surfaces) as illustrated in Figure 27.

[0109] Triangle lexicographical ordering The triangle ordering strategy of this embodiment (

number

[0110] Figure 28 shows a triangle T with its encircling and circumscribing circles, such that RB(T)≠RC(T) since the triangle features obtuse angles. Conversely, Figure 29 shows a triangle T with three acute angles, such that the encircling and circumscribing circles are coincident and RB(T)=RC(T).

[0111] If a triangle is characterized by three aligned or coincident points, then the circumscribing circle is not defined. This degenerate situation is outside the scope of this example since point clouds are general in nature. Given two triangles T and T', their relative ordering

number

number

[0112] This ordering is actually a lexicographic ordering defined by two numbers.

number

number

[0113] Alternating Ordering Three alternative weighting strategies were considered for comparison with the one described in the previous section. The first one is the radius of the smallest circle enclosing the triangle, just like RB. The second one is the radius of the circumscribing circle, just like RC. The third one is the triangular area. A random selection is performed among several triangles featuring the same weight. Experiments on industrial data show that the resulting surfaces obtained with alternating weighting are quite similar to those obtained with lexicographical order. Nevertheless, the quality is not as good in the following sense. Although this theory guarantees that the recognized surface is a closely triangulated surface, some non-manifold edges may appear. A non-manifold edge of a closed surface is shared by 2n triangles, in contrast to a manifold edge that is shared by exactly two triangles, as shown by figures 32 and 33.

[0114] Experiments show that in all test cases, lexicographical ordering results in recognized surfaces with the least number of non-manifold edges, as shown in the following table. [Table 1]

[0115] Notes on the algorithm Next, the ideas contained in the algorithm will be explained.

[0116] The first idea is the optimal understanding of Delaunay triangulations, as opposed to the traditional geometric understanding: given a set of points K, T denotes the mesh of its convex hull Ω. The Delaunay mesh T* of K can be any real number

number

number

number

number

number

number

[0117] What is the optimal characterization?

number

number

[0118] The second idea is a new theoretical result constructed for the purpose of this example, which shows that the limit

number

[0119] The intuition is that the best possible triangulated surface minimizes the weights and this minimum is obtained by using the lexicographical order. Now consider a 3D point cloud K, a Delaunay 3D mesh T* of its convex hull Ω, and a selected tetrahedron h of T*. The unknown surface is a closed surface made of triangles T* that contain tetrahedron h. Of course, there are many such surfaces Σ, but the surface to be recognized Σ* preserves the best possible triangulation made of triangles T*. In other words, the recognized surface Σ* minimizes Q(T,p) among all closed surfaces Σ that are made of and enclose triangles T* and tetrahedrons h. It must be understood that this principle is closely related to the fact that the point cloud is denser in the neighborhood of the surface to be recognized.

[0120] Figures 34-36 show the principle with a 2D example: a point cloud K, a convex hull Ω, and its selected mesh T*. Figures 37-39 show non-optimal Σ1 and Σ2 surrounding h, as well as an optimal Σ* according to the minimum criterion.

Claims

1. 1. A computer-implemented method for 3D reconstruction of a structure of a real scene, comprising: - providing a 3D point cloud representing the structure (S10); - determining a closed triangulation surface representing the skin of the structure (S30), searching for candidates for closed triangulation surfaces each meshing at least a corresponding portion of the 3D point cloud, penalizing the top-ranked candidates for closed triangulation surfaces according to a lexicographical order, the lexicographical order being based on a triangle order, the lexicographical order ordering triangles with a first candidate for closed triangulation surface having a first triangle ordered according to a descending triangle order against a second candidate for closed triangulation surface having a second triangle ordered according to a descending triangle order, the triangle order penalizing triangle size; The method includes:

2. Each of the candidate closed triangulation surfaces includes a corresponding set of corresponding triangular faces of a corresponding tetrahedron of a tetrahedral mesh of the 3D point cloud. The method of claim 1.

3. The tetrahedral mesh exhibits the following properties: The set of tetrahedrons in the tetrahedral mesh forms a convex envelope of the points in the point cloud, The intersection between any first tetrahedron of the tetrahedral mesh and any second tetrahedron of the tetrahedral mesh that intersects with the first tetrahedron is a vertex of the first tetrahedron, an edge of the first tetrahedron, or a face of the first tetrahedron. The method of claim 2.

4. The triangle ordering penalizes high values ​​of the minimum enclosing circle radius for each triangle.

4. The method according to any one of claims 1 to 3.

5. The triangle ordering further penalizes the smaller value of the circumscribing circle radius for a first triangle and a second triangle that have the same smallest enclosing circle. The method according to claim 4.

6. Each of the candidate closed triangulation surfaces comprises a corresponding set of corresponding triangular faces of a corresponding tetrahedron of a tetrahedral mesh of the 3D point cloud, the tetrahedral mesh being a regular triangulation.

6. The method according to any one of claims 1 to 5.

7. The searching and the penalizing include: visiting the triangles of the tetrahedral mesh according to a descending order of said triangle orders; discarding visited triangles having a high ranking in the triangle order; The method of claim 6, comprising:

8. Corresponding tetrahedrons of the tetrahedral mesh define corresponding regions, and the determining step (S30) comprises: Declaring exterior tetrahedrons of a tetrahedral mesh and interior tetrahedrons of the tetrahedral mesh, the exterior tetrahedrons and the interior tetrahedrons representing regions exterior to the structure and regions interior to the structure, respectively, where visiting the triangles includes, for each visited triangle, evaluating whether the visited triangle separates an interior region from an exterior region; The discarding of triangles includes, for each visited triangle that does not separate an interior region from an exterior region, where the triangle separates the two regions, suppressing the triangle, and merging the two regions. The method according to claim 7.

9. Each visited triangle that separates the interior region from the exterior region is retained, and the retained triangles form a closed triangulation surface of the determined triangulation surface. The method according to claim 8.

10. The determining step (S30) includes providing zones in the 3D point cloud that represent internal locations of the structure, the candidate closed triangulation surfaces each encompassing one of the zones.

10. The method according to any one of claims 1 to 9.

11. The structure includes at least one corner structure.

11. The method according to any one of claims 1 to 10.

12. The 3D point cloud originates from photogrammetry, laser scanning, RGB-D measurements, and / or medical or industrial tomography, and / or the real-world scene is an architectural scene, a building interior scene, a mining scene, an organic tissue, or a machine part.

12. The method according to any one of claims 1 to 11.

13. A computer program for causing a computer to execute a method according to any one of claims 1 to 12.

14. A computer-readable storage medium having the computer program according to claim 13 recorded thereon.

15. 14. A computer comprising a processor coupled to a memory having a computer program according to claim 13 recorded thereon and a display, the processor being operated by said computer program.

Citation Information

Patent Citations

  • Three-dimensional model generating device and method

    JP2006065472A

  • Shape measuring device and program

    JP2010133752A

  • Polygon model generation device, polygon model generation method, and program

    WO2018179253A1