Parametric model generation based on constrained geometric parameters

Abstract

Parametric model generation based on constrained geometric parameters. The invention relates to a computer-implemented method for generating a parametric model of a physical structure based on point cloud data, the method comprising: obtaining point cloud data comprising geometric information of an environment; obtaining geometric information of a set of predefined geometric shapes related to a plurality of parts; performing a classification on the point cloud data to detect parts in the point cloud data based on the geometric information; assigning a set of points of the point cloud to each detected part; detecting physical connections of the detected parts; generating a parametric model of the physical structure based on the detected physical connections of the detected parts and the geometric information, the method further comprising deriving geometric parameters for each detected part from the geometric information from the set of predefined geometric shapes, the generating of the parametric model of the physical structure being further based on the derived geometric parameters, at least one of the derived geometric parameters being used as a constraint for at least one of the geometric parameters of a following part that is physically connected to a preceding part.

Description

Technical Field

[0001] This invention relates to a computer-implemented method for generating parametric models of physical structures based on point cloud data. Background Technology

[0002] Existing technologies include computer-based methods for generating parametric models of large physical structures composed of many different and physically connected parts based on point cloud data.

[0003] An example of this physical structure is a pipeline system, in which many pipes are arranged in rows and physically connected to each other. Point cloud data describes, for example, a point cloud provided by a laser scanner that has scanned the environment in which the pipeline system exists.

[0004] Some of these existing techniques rigidly create parametric models of the structure based on the obtained point cloud. However, due to this very rigorous creation of point-cloud-based models, it is possible that misalignment / misalignment may occur at the ends of the two parts to be connected during modeling. This faulty connection is particularly fatal when creating parametric models of pipeline systems, as the created model will no longer be airtight / watertight. However, such a faulty model will also be problematic for other structures, such as supported structures composed of many beams, as it will lead to a lack of stability.

[0005] However, other existing methods create parametric models by modeling subsequent parts as perfectly connected to preceding parts, but the points in the point cloud are largely ignored. In this way, a model of the structure can be created without misalignment / misalignment of the end pieces of the parts, but the deviation between the corresponding modeled parts and the points in the point cloud may continue or even add up throughout the structure, and thus the deviation can become increasingly larger as the number of modeled parts increases.

[0006] In both cases of existing technical methods, the generated model must be painstakingly reworked and corrected by the user.

[0007] Therefore, there is a need for an improved computer-implemented method for creating parametric models of physical structures. In particular, there is a need for an improved computer-implemented method that uses further standards to create parametric models in addition to physical connections. Summary of the Invention

[0008] The purpose of this invention

[0009] Therefore, the object of the present invention is to provide a computer-implemented method, reality capture device, and computer program product that overcomes the shortcomings of the prior art.

[0010] Another object of the present invention is to provide a computer-implemented method that uses another standard in addition to physical connections to create a parametric model.

[0011] These objectives are achieved by implementing at least a portion of the features of the independent claims. The dependent claims describe features for further development of the invention in an alternative or advantageous manner. Invention Overview

[0013] This invention relates to a computer-implemented method for generating a parametric model of a physical structure based on point cloud data, wherein the point cloud data describes a point cloud of points acquired in the environment in which the physical structure is located, wherein the physical structure comprises a plurality of parts, wherein the plurality of parts are arranged as interconnecting chains of physically connected parts, and the method includes: The point cloud data is acquired, and the point cloud data includes the geometric information of the environment; Obtain geometric information of a set of predefined geometric shapes associated with the plurality of parts; Perform classification on the point cloud data, in order to Based on the geometric information, detect the component among the plurality of components in the point cloud data; Assign the point set of the point cloud to each of the plurality of parts; and ο Detect the physical connection between two or more of the detected parts from the plurality of parts; and A parametric model of the physical structure is generated based on the detected physical connections of two or more of the plurality of parts and the geometric information of the set of predefined geometries. The method further includes deriving geometric parameters of each detected part from the geometric information of the set of predefined geometries, wherein the parametric model for generating the physical structure is also based on the derived geometric parameters of each detected part, wherein at least one of the derived geometric parameters of the preceding part is used as a constraint / limitation / restriction to physically connect to at least one of the geometric parameters of the subsequent part of the preceding part.

[0014] In other words, the computer-implemented method according to the present invention uses the end interfaces of parts to constrain the parameters of subsequent connected parts to construct a fully parametric model. The generation of the parametric model of the physical structure includes identifying parts / components (e.g., straight pipes, elbows, etc.) in the point cloud. This is performed automatically by appropriate algorithms. In particular, AI-based point cloud classification, semantic segmentation, and / or instance segmentation methods can be used to identify individual components / parts and the set of points assigned to each component / part. The generation of the parametric model of the physical structure also includes identifying connections between parts and combining these parts into a connected segment, as well as deriving constraints considered in optimization. For example, to ensure a smooth transition from a straight pipe (cylinder) to an elbow (segment of a torus), a "same radius" can be specified. Furthermore, the constrained "tangent point" should ensure that the endpoint of the cylinder and the starting point of the torus have exactly the same coordinates, and that the tangent at that point is identical.

[0015] The assignment of which of two sequential or consecutive parts of a structure comes first and which comes later depends on the parts of the physical structure from which its model is created / generated and via which parts are adjacent to that part. In two parts of a physical structure to be modeled along this "creation sequence" / "creation direction", the part modeled first is correspondingly the first part, and the part connected to the first part via a physical connection and modeled subsequently is the second part.

[0016] Of course, according to the method of the invention, the part following the preceding part can be another part, at least one of which will be constrained by at least one derived geometric parameter. This other part now accordingly represents the following part, while in the above example, the part assigned from the following part now represents the preceding part. This alternating assignment of preceding and following parts extends throughout the physical structure according to the modeling sequence of the physically connected parts.

[0017] The term "physically connected" or "physical connection" is understood to refer to a connection via individual connecting parts (such as pipe connectors), and direct physical contact and fixation of two parts (e.g., by welding, clamping, etc.). These two parts can also be physically connected by screwing the thread of one part onto the mating thread of another. Another example of a physical connection (which also illustrates that the term "physically connected" or "physical connection" can be understood to refer to various connection options) is connecting the respective flanges of two parts by, for example, a clamp applied to the outside of the flange.

[0018] In an exemplary embodiment of the computer-implemented method according to the present invention, at least one geometric parameter of the geometric parameters of the subsequent part physically connected to the preceding part can be constrained by at least one geometric parameter of the geometric parameters of the preceding part by setting the same equal geometric parameters in the preceding part and the subsequent part.

[0019] Optionally, the outer diameter of the rear part can be constrained by the outer diameter of the front part by setting the outer diameter of the rear part to be equal to the outer diameter of the front part; and / or the inner diameter of the rear part can be constrained by the inner diameter of the front part by setting the inner diameter of the rear part to be equal to the inner diameter of the front part.

[0020] In other words, a fully parametric model is a set of constrained parametric shapes. It has access to all free parameters and can therefore be viewed as the parametric shape itself.

[0021] In another exemplary embodiment, the computer-implemented method according to the present invention may further include a parametric model of the physical structure generated by point cloud optimization.

[0022] Optionally, optimizing the parametric model of the generated physical structure may include weighted least squares.

[0023] Based on this, the distance between a point and the full-parameter model can be calculated, allowing the fitting algorithm to evaluate the error at each step. Furthermore, the distance from each individual parametric shape to a point can be calculated. The distance to the full model is then the minimum distance to each shape. The fitting behavior can be controlled by delimiting a point to its initial closest shape. This makes distance calculations faster and the results more stable.

[0024] In another exemplary embodiment of the computer-implemented method, optimization can be performed based on constraints on at least one of the geometric parameters of a subsequent part physically connected to a preceding part. Alternatively or additionally, optimization can be performed by reducing the distance between at least one point and a predefined geometry associated with the detected part in the plurality of parts, for at least one point in a set of points assigned to the detected part in the plurality of parts, including the set of points with at least one point being assigned to the predefined geometry; and / or a parametric model of the generated physical structure.

[0025] In other words, in optimization, the parameters of the geometric elements (e.g., cylinders, cones, toroidal segments, etc.) representing parts belonging to a segment (e.g., straight pipes, reducing pipes, elbows, etc.) are determined. Therefore, for all points and geometric elements of the segment, the normal distance between points assigned to the parts corresponding to the geometric elements is minimized.

[0026] A drawback of existing computer programs for creating models of physical structures (e.g., pipeline systems) from point cloud data is that none of them can fit an entire structure consisting of two or more connected elements / parts to the point cloud at once. In particular, the processing of the structure after the separate parts have been connected and merged into a single structure poses a problem for existing solutions, as heuristic-based common propagation algorithms cannot fit the merged structure as a whole to the point cloud.

[0027] Other drawbacks of existing computer programs are that they lose the “possibility” of the reconstructed structure relative to the point cloud during the reconstruction / reconnection process, and they risk corrupting the reconstructed data by attempting to preserve that “possibility” based on heuristics.

[0028] When solving problems involving connecting two parts using joining elements (e.g., welds or separate connecting parts), a global fitting algorithm is required. Typically, the ends of the parts to be connected may not match (e.g., small differences may occur in pipe radius, position, or alignment).

[0029] Therefore, at least one of these components (e.g., pipelines) must be slightly modified so that the merged structure (e.g., pipeline system) remains watertight. However, simply aligning and resizing the pipelines to maintain the resulting watertightness may cause the pipeline model to drift further away from the points from which it was extracted and no longer match the point cloud. This offset can become significant for very long pipelines, especially due to the corrected rotational component magnified over long distances, also known as the "lever" effect.

[0030] Heuristic methods known in the prior art, which propagate connection constraints through the structure, thus keeping each individual part as aligned as possible with the point cloud before considering the next part in the structure, may lead to highly erroneous results because the errors also propagate.

[0031] The use of global fitting methods, such as those in the method according to the invention, ensures the consistency of the structure throughout the fitting process. Furthermore, under certain assumptions about the initial state, the fitting algorithm itself finds the optimal solution based on the distance to the measurement point.

[0032] In other words, the global fitting algorithm aims to provide a set of parts as an initial guess (possibly with additional constraints) for finding the optimal solution, where strong structural constraints are generated by the connections between consecutive parts.

[0033] Another advantage of this method is that it allows for the updating of any parameter of any part in the structure at any time. The fitting algorithm is able to re-evaluate all other geometric parameters to refit the resulting part to the point cloud. An exemplary use case for this is when the bend angle of a pipe is close to 90° but not exactly 90°. There is the ability to “correct” the bend at any time, for example, by forcing a fillet while keeping the entire pipe as close to the point as possible.

[0034] However, the method according to the invention is not limited to a global fitting method. In the case of connecting two parts, it is not necessary to assemble the entire structure. Assuming the first half is good enough, the method only needs to fit the second half. In practice, the end interface of the first part (e.g., a pipe) can be used as a constraint for the connecting element and inserted as an initial constraint into the fitting algorithm. In other words, the algorithm can handle partial fitting of the structure, not necessarily the entire structure.

[0035] In another exemplary implementation, additional constraints can be provided to the global fit (= optimizer), which are not constraints caused by the structure of the parts. These constraints can be from external instructions (e.g., provided by the user), rather than connections between consecutive parts / elements along the structure.

[0036] In another embodiment, the method may further include grouping two or more detected parts from a plurality of physically connected parts together to form sub-segments; grouping one or more sub-segments into sub-segment groups; and jointly optimizing all sub-segments of the sub-segment groups.

[0037] Optionally, at least two detected parts out of a plurality of parts can be grouped together and / or one or more sub-segments can be grouped into sub-segment groups by user manual input.

[0038] In other words, instead of optimizing individual parts, all parts of the structure can be the target of a global optimization approach. Typically, users flexibly combine parts into sub-segments and group them together for joint optimization. Optionally, this can be supported by a graphical user interface (e.g., a tree view) that allows for such grouping and combination in a comfortable manner.

[0039] In another embodiment, the method may further include: obtaining the value of at least one geometric parameter of at least one detected part among the plurality of parts; and setting the value of the at least one geometric parameter such that the value of the at least one geometric parameter will not be changed by the optimization.

[0040] Optionally, the value of at least one geometric parameter can be provided by manual input from the user and / or derived from a comparison of at least one detected part among a plurality of parts with a catalog of parts that can be used to construct the physical structure.

[0041] In other words, constraints can be introduced manually. For example, if the radius of the pipe is known, for instance, from a data table, that parameter can be introduced as a constraint. This means the parameter will be set to a specific value and will not be changed during optimization.

[0042] In another embodiment, the method may further include obtaining information about the orientation of at least one detected part among a plurality of parts in a Cartesian coordinate system (e.g., a two-dimensional Cartesian coordinate system or a three-dimensional Cartesian coordinate system); and setting the orientation of at least one detected part among a plurality of parts such that if the orientation of the at least one detected part among a plurality of parts meets a predefined criterion, particularly an orientation parallel to one of the axes of the Cartesian coordinate system (e.g., depending on the dimensions of the Cartesian coordinate system, an orientation parallel to the X-axis (also referred to as horizontal) or an orientation parallel to the Y-axis / Z-axis (also referred to as vertical)), the orientation of at least one detected part among a plurality of parts is not changed by optimization.

[0043] Optionally, the orientation of at least one of the detected parts in the Cartesian coordinate system can be provided by manual input.

[0044] In another implementation, during the optimization process, Two or more of the detected parts that have been physically connected cannot be separated from each other in the parametric model generated by the physical structure; and / or At least one of the detected parts can be interchanged with a part from a catalog of parts that can be used to construct a physical structure; and / or At least one of the detected parts can be removed from the parametric model generated from the physical structure; and / or Parts from the parts catalog that can be used to construct the physical structure can be inserted into the generated parametric model of the physical structure.

[0045] In other words, parts can be added / removed during the optimization process, giving the optimizer greater flexibility. This provides the optimization process with more capabilities because it is freed from the limitations imposed by the initial physical structure, which may not be suitable for finding the actual optimal solution.

[0046] In another embodiment, the classification of point cloud data may include point cloud classification. Optionally, point cloud classification may include artificial intelligence; semantic segmentation; and / or instance segmentation.

[0047] In another embodiment, the set of predetermined geometries may include: cylinders; cones, particularly truncated asymmetric cones; toroidal segments; and / or intersections.

[0048] Alternatively, if the physical structure is configured as a pipeline system, cylinders can be associated with straight pipes, cones can be associated with reducers or expansions, toroidal sections can be associated with pipe elbows, and / or intersections can be associated with pipe branches.

[0049] In other words, the algorithm automatically constructs a complete watertight parametric model for any pipeline. It relies on the ability to build parametric models of any single shape that supports pipeline reconstruction: cylinders (straight pipes), toroidals (elbows), asymmetric cones (reducing diameter pipes), self-intersecting cylinders (branchings), etc. The end interfaces of the pipe can be used to constrain the parameters of subsequent connecting pipes.

[0050] For example, two consecutive pipes must share a common radius so that the entire model is watertight. Once the first radius is set, the second radius is no longer a "free" parameter. These constraints can be constructed by relying on the fact that the connected pipes share a common interface. The end interface of a pipe is exactly the starting interface of the next pipe. In this way, for each new parametric shape in the model, a set of constraint parameters is defined that does not change during optimization. These parameters are inherited from the previous pipe in the line.

[0051] Using common "interfaces" (pipe end geometries) as constraint carriers also has the following advantages: the method according to the invention does not handle all possible pipe combinations (e.g., cylinders with elbows, elbows with reducing pipes, etc.). All pipes have rounded ends, which can be used as supports to provide constraints for the next shape, regardless of their type. This makes this implementation advantageously easy to maintain, as the user does not have to worry about combination explosion when adding new pipe types. It is sufficient to convert each pipe type into a parametric model, whose constraints can be provided as the starting interface.

[0052] The exact same approach can be used for any structured set of parametric shapes to fit a point cloud, not just pipelines. For example, the invention can be extended to steel structures. For instance, adjacency constraints can be transformed into geometric constraints applied to the parametric space. Therefore, the method is independent of the shape to be fitted. This makes it easy to extend the invention in completely different contexts (e.g., add new pipe types) or reuse it.

[0053] In another implementation, geometric parameters can be derived for cylinders, cones, toroidal segments, and / or intersections.

[0054] Alternatively or additionally, geometric parameters may include length, position, orientation, starting direction, tangent point, cylinder radius, cone radius, cone angle, truncated cone height, asymmetric cone offset, bend angle, bend inner radius and / or bend curve radius.

[0055] In another embodiment, the position of the subsequent part may be constrained by the position and / or tangent point of the preceding part, and / or the starting direction of the subsequent part may be constrained by the orientation of the preceding part. Optionally, the radius of the subsequent part may be constrained by the radius of the preceding part, particularly wherein the radius of the subsequent part may be set to be equal to the radius of the preceding part. Optionally, the cylindrical radius of the subsequent part may be constrained by the cylindrical or conical radius of the preceding part, and / or the conical radius of the subsequent part may be constrained by the conical or cylindrical radius of the preceding part.

[0056] In another embodiment, the method may further include acquiring statistical information related to at least one detected part among a plurality of parts; and displaying the statistical information. Optionally, the statistical information may include the standard deviation and / or maximum deviation of a set of points assigned to at least one detected part among a plurality of parts, and a predefined geometry associated with at least one detected part among a plurality of parts, the set of points being assigned to the predefined geometry.

[0057] In other words, for each part, statistical parameters can be displayed, namely the standard deviation or maximum deviation of the points of the geometric elements. This provides the user with an overview of how well the points assigned to the part fit their geometric representation.

[0058] In another embodiment, the method may further include: acquiring another constraint on at least one geometric parameter of the geometry of the subsequent part physically connected to the preceding part; and / or removing spurious constraints on at least one geometric parameter of the geometry of the subsequent part physically connected to the preceding part. Optionally, further constraints may be provided and / or spurious constraints may be removed by manual input from the user.

[0059] In another embodiment, the method may further include displaying constraints on at least one of the geometric parameters of the detected part and / or the subsequent part physically connected to the preceding part.

[0060] Optionally, detected parts in multiple parts and / or constraints can be displayed as an overlay representation with specific symbols.

[0061] In other words, constraints between geometric elements (cylinders, tori, etc.) can be visualized as superpositions with specific symbols, such as for the same radius or tangent point. This provides the user with an overview of which constraints have been identified and used in the optimization. Optionally, constraints should be editable, meaning the user can delete dummy constraints and manually add missing constraints.

[0062] In another embodiment, the method can be performed by a computing unit of a reality capture device, wherein the computing unit can be configured to control the reality capture device to acquire point clouds, wherein point cloud data describing the point clouds can be provided to the computing unit; and / or geometric information can include the 3D coordinates of points in the point cloud.

[0063] The present invention also relates to a reality capture device comprising: a sensor unit configured to acquire points in an environment as a point cloud; and a computing unit configured to control the acquisition of the point cloud, wherein point cloud data describing the point cloud can be provided to the computing unit, and in particular, wherein the sensor unit includes a LiDAR unit or a plurality of ToF cameras, and wherein the computing unit has program code stored for performing a method according to any of the foregoing embodiments.

[0064] The present invention also relates to a computer program product comprising program code stored on a machine-readable medium or embodied by an electromagnetic wave including program code segments, and having computer-executable instructions for performing the method according to any of the foregoing embodiments, particularly when executed in the computing unit of the reality capture device as described above. Attached Figure Description

[0065] The invention will now be described in more detail by way of example only, with reference to the schematic examples shown in the accompanying drawings. In the drawings, the same elements are labeled with the same reference numerals. The described embodiments are generally not shown to scale and should not be construed as limiting the invention.

[0066] Figure 1 This illustrates how the geometric parameters of a subsequent part are constrained by the geometric parameters of a preceding part according to an exemplary embodiment of the method of the present invention.

[0067] Figure 2A and Figure 2B A schematic diagram of the prior art method used in the prior art method is shown.

[0068] Figure 3 A parametric model of a pipeline system created using heuristic existing techniques is shown.

[0069] Figure 4A schematic diagram of the optimization steps of an exemplary embodiment of the method according to the present invention is shown.

[0070] Figure 5 An exemplary embodiment of the method according to the invention illustrates how detected geometric parameters of parts and constraints can be visualized.

[0071] Figure 6 This illustrates how to set the values ​​of geometric parameters according to an exemplary embodiment of the method of the present invention.

[0072] Figure 7 A schematic diagram of the catalog used to set geometric parameters is shown.

[0073] Figure 8 A schematic diagram illustrating an exemplary embodiment of a reality capture device according to the present invention for acquiring points in the environment where a physical structure is located.

[0074] Figure 9 A process diagram illustrating the steps of the method according to the present invention is shown. Detailed Implementation

[0075] Figure 1 This illustrates how the geometric parameters of a subsequent part 3 are constrained by the geometric parameters of the preceding part 2 according to an exemplary embodiment of method 1 of the present invention. In the example shown, the physical structure to be modeled is a pipeline system, wherein the preceding part 2 is a straight pipe and the subsequent part 3 is a curved / bent pipe. Therefore, the geometry of a cylinder can be assigned to the straight pipe 2, and the geometry of an elbow can be assigned to the bent pipe 3.

[0076] In the example shown, the cylinder is unconstrained. For example, the cylinder has geometric parameters including (internal) radius R1, length L1, position P1, and orientation D1. Orientation D1 can be determined, for example, by the extension of the cylinder's central axis 4 in three-dimensional space / environment or in a Cartesian coordinate system (assigned to three-dimensional space) (according to the following steps: obtaining information about the orientation of at least one detected part among a plurality of parts in the Cartesian coordinate system).

[0077] In other words, in addition to pipe parameters, pipe orientation / alignment D1 can also be constrained. For example, there are cases where pipes in a facility must be strictly horizontal or vertical. This general constraint can also be inserted into the optimization algorithm to give even more control over the structure of the pipe system to be constructed.

[0078] Elbow 3 has free parameters that are not constrained by the geometric parameters of cylinder 2, such as the elbow radius (also called the elbow curve radius or large radius) G or the elbow angle A. Due to the physical connection 5 of the two pipe parts 2 and 3, some geometric parameters of the elbow are restricted, such as the axis of rotation V, which is restricted by the fact that it must be orthogonal to D1, the inner radius R2 (the inner radius is set to be equal to the inner radius of cylinder R1), the position P2, or the starting direction D2 derived from direction D1.

[0079] Figure 2A and Figure 2B A schematic diagram of the method used in the prior art is shown. Figure 2A In the method shown, the parametric model 6 of the pipeline system is rigidly created based on points 7 of the point cloud. However, misalignment / misplacement of the pipeline ends 8 and 9 of the parts 2 and 3 to be connected may occur. Therefore, model 6 must then be corrected accordingly, in particular, one of the pipelines 2 and 3 must be updated in its solid so that the pipeline ends 8 and 9 can be connected.

[0080] on the other hand, Figure 2B A heuristic approach is shown in which a rigid transformation of the second pipeline 3, without considering point 7, makes pipelines 2 and 3 connectable, but the second pipeline 3 no longer matches the point cloud.

[0081] Figure 3 A parametric model of a pipeline system created using a heuristic approach based on existing techniques is shown. (Using methods such as...) Figure 2B The example illustrates a heuristic prior art method for creating a model 10 that includes pipeline component 2 and pipeline segment 11. Although these components are connected to each other without gaps, the misalignment / misalignment of the pipeline ends shown above continues, and even adds up, throughout pipeline segment 11, causing pipeline segment 2 to completely deviate from point 12 in the point cloud. Pipeline segment 2 even points in a completely different direction, which is why a logical link with pipeline segment 3 is no longer possible. Models generated in this way must be painstakingly reworked and corrected by the user.

[0082] Figure 4 A schematic diagram of optimization step 600 (the generated parametric model of the physical structure of the point cloud optimization) according to an exemplary embodiment of method 1 of the present invention is shown. The parametric model 13 of the initial creation of the pipeline system is shown on the right, which is created using the constraint method according to the present invention. Model 13 deviates slightly from points 14 of the point cloud.

[0083] As shown in the figure, the components of the pipeline system are grouped into sub-segments (according to the step: grouping two or more detected components that are physically connected to form a sub-segment together), the sub-segment is grouped into a sub-segment group (according to the step: grouping one or more sub-segments into a sub-segment group), and then the sub-segment of the sub-segment group is optimized (according to the step: jointly optimizing all sub-segments of the sub-segment group).

[0084] The results after optimization process 15 are shown on the left, where the geometric parameters (radius, length, orientation, etc.) have been re-evaluated to minimize the distance to point 14.

[0085] Figure 5 This illustration demonstrates how, according to an exemplary embodiment of Method 1 of the present invention, the constraints of detected parts and geometric parameters are visualized (according to step 700: displaying the constraints of at least one of the detected parts and / or the geometric parameters of the subsequent parts physically connected to the preceding part). For example, it illustrates two detected / identified straight pipes 16, 17 with symbol 18, an elbow 19 with symbol 20, an identified constraint "same radius / diameter" with symbol 21, and a constraint "same tangent point" with symbol 22. This exemplary illustration may be shown to a user on a display / user interface, for example.

[0086] Figure 6 This illustrates how to set the values ​​of geometric parameters according to an exemplary embodiment of method 1 of the present invention. In the illustrated exemplary embodiment, the user can manually input user-known geometric parameters 23, 24 for certain parts 25 (according to step 800: obtaining the value of at least one geometric parameter of at least one detected part among a plurality of parts, particularly wherein the value of at least one geometric parameter can be provided by the user's manual input) and set / lock them (according to step 900: setting the value of at least one geometric parameter such that the value of at least one geometric parameter will not be changed by optimization 600).

[0087] The locking parameters inner diameter 23 and bending angle 24 of the selected elbow 25, visualized by arrows 26 and 27, should not be modified by the optimization algorithm. They must remain unchanged throughout the entire reconstruction process (including the global fitting algorithm) unless they are explicitly unlocked.

[0088] In other words, in the described implementation, the user can provide some knowledge of the global fit by explicitly locking some of the pipe parameters 23, 24. For example, when pulling out the elbow 25 in the preparation step, the user can specify that the elbow angle 24 must be 90 degrees, or the diameter 23 must be exactly 9 cm. Once these parameters 23, 24 are locked, the global fit algorithm can use this information to keep these parameters 23, 24 unchanged during the optimization process.

[0089] In terms of features and user experience, this implementation gives users more control over the behavior of the fitting algorithm and over the structure of the pipeline system to be built and optimized. Users can retain known pipe values.

[0090] As the number of constraint parameters increases, the number of "free parameters" found by our optimizer decreases (because the number of parameters is constant). Therefore, this implementation makes the problems that arise easier to handle and accelerates the optimization process.

[0091] Based on this implementation, the following workflow can be performed: [1] the user clicks on the pipe to be constrained; [2] the user sets relevant values ​​for the parameters to be constrained (e.g., radius, angle, etc.); and [3] the user locks the parameters so that the optimizer knows that the corresponding parameters will not be optimized and should remain unchanged.

[0092] Figure 7 A schematic diagram of a catalog 28 for setting geometric parameters 29 is shown. In the embodiment depicted in Method 1, the user can click to open a part in catalog 28 that has potential pipe bends 32, such as pipe bends 30. The selection of the correct pipe bend and the subsequent derivation of the value 31 of geometric parameters 29 can then be performed, for example, by the user manually selecting a pipe bend 32 listed in catalog 28 or by an automatic comparison of the clicked bend 30 with the bends 32 listed in catalog 28 (according to step 1000: the value of at least one geometric parameter can be derived from a comparison of at least one detected part among a plurality of parts with a catalog of parts that can be used to construct the physical structure).

[0093] The tubing components in a factory are typically not random. They come from a set of manufactured tubing supplied by industrial vendors. Therefore, the available tubing set can be limited to well-known tubing catalogs, thus providing some clues about the tubing to be implemented in the facility.

[0094] Therefore, instead of having the user manually provide a single constraint, the user [1] clicks on the pipe to be specified; and [2] obtains a set of pipes displayed in a predefined catalog that appears to be related to the selected pipe. The user can then [3] select elements from the catalog to replace and lock the pipe parameters.

[0095] At the end of this preparation step, the global fitting method will be fed with many additional constraints that it can handle. In other words, these catalogs give users more control over the fitting algorithm.

[0096] In other words, users can bind pipes to catalog entries. Once a pipe is defined as a component within the catalog, its parameters can be updated and locked accordingly, allowing users to easily add predefined constraints (from industry-standard submissions) to the global fit.

[0097] Figure 8 A schematic diagram of an exemplary embodiment of a reality capture device 33 according to the present invention is shown, which acquires point 14 in the environment 34 where the physical structure 36 is located.

[0098] Figure 9 A flowchart illustrating the steps of a computer-implemented method 1 according to the present invention is shown. The computer-implemented method 1 for generating a parametric model 13 of a physical structure 36 based on point cloud data includes the following steps: Step 100: Obtain point cloud data, wherein the point cloud data includes geometric information of environment 34; Step 200: Obtain geometric information of a set of predefined geometries associated with multiple parts; Step 300: Perform classification on the point cloud data. Step 301: Detect a part among multiple parts in the point cloud data based on geometric information; Step 302: Assign a set of points from the point cloud to each detected part among the plurality of parts; and Step 303: Detect the physical connection 5 of two or more of the detected parts 2 and 3 among the plurality of parts; Step 400: Based on the detected physical connections 5 of two or more detected parts 2, 3 among the plurality of parts and the geometric information of the set of predefined geometries, a parametric model 13 of the physical structure 36 is generated, and Step 500: Derive the geometric parameters 23, 24, and 29 of each detected part 2 and 3 from the geometric information of the set of predefined geometries.

[0099] Step 400 of generating the parametric model 13 of the physical structure 36 is also based on the geometric parameters 23, 24, 29 derived for each of the detected parts 2, 3 among the plurality of parts, wherein, in step 501, at least one of the derived geometric parameters 23, 24, 29 of the preceding part 2 is used as a constraint to physically connect to at least one of the geometric parameters 23, 24, 29 of the following part 3 of the preceding part 2.

[0100] The point cloud data describing the point cloud includes points 14 acquired in the environment 34 where the physical structure 36 is located, wherein the physical structure 36 includes multiple parts, wherein the multiple parts are arranged as interconnect chains of physically connected parts.

[0101] Although the invention has been described above with reference to some preferred embodiments, it should be understood that many modifications and combinations of different features of the embodiments can be made. All such modifications are within the scope of the appended claims.

Claims

1. A computer-implemented method (1) for generating a parametric model (13) of a physical structure (36) based on point cloud data, wherein the point cloud data description comprises a point cloud of points (14) acquired in the environment (34) where the physical structure (36) is located, wherein, The physical structure (36) includes a plurality of parts, wherein the plurality of parts are arranged as an interconnection chain of physically connected parts, and the method (1) includes: Acquire the point cloud data (100), the point cloud data including the geometric information of the environment (34); Obtain geometric information of a set of predefined geometries associated with the plurality of parts (200). The point cloud data is classified (300) to... Based on the geometric information, detect part (2, 3) (301) among the plurality of parts in the point cloud data. ο Assign the point set (302) of the point cloud to each detected part (2, 3) of the plurality of parts; and ο Detect the physical connection (5) (303) of two or more detected parts (2, 3) among the plurality of parts; and Based on the detected physical connections (5) of two or more of the detected parts (2, 3) and the geometric information of the set of predefined geometries, a parametric model (13) (400) of the physical structure (36) is generated. Its features The method (1) further includes deriving geometric parameters (23, 24, 29) of each of the plurality of parts (2, 3) from the geometric information of the set of predefined geometries (500), wherein the parameter model (13) that generates the physical structure (36) is also based on the derived geometric parameters (23, 24, 29) of each of the plurality of parts (2, 3), wherein at least one of the derived geometric parameters (23, 24, 29) of the preceding part (2) is used as a constraint (501) on at least one of the geometric parameters (23, 24, 29) of the following part (3) that is physically connected to the preceding part (2).

2. The method (1) according to claim 1, wherein, By setting equal geometric parameters (23, 24, 29) to be the same in the preceding part (2) and the following part (3), at least one of the geometric parameters (23, 24, 29) of the following part (3) physically connected to the preceding part (2) is constrained by at least one of the geometric parameters (23, 24, 29) of the preceding part (2).

3. The method (1) according to any one of the preceding claims, the method (1) further comprising a generated parametric model (13) (600) for optimizing the physical structure (36) with respect to the point cloud, particularly wherein, The generated parametric model (13) of the physical structure (36) optimized (600) includes weighted least squares.

4. The method (1) according to claim 3, wherein, The optimization (600) is performed as follows: Based on the constraints on at least one of the geometric parameters (23, 24, 29) of the rear part (3) physically connected to the front part (2); and / or By reducing the distance between at least one point in the set of points assigned to the detected parts (2, 3) among the plurality of parts, and ο a predefined geometry associated with the detected part (2, 3) among the plurality of parts to which the set of points including the at least one point is assigned; and / or The generated parameter model (13) of the physical structure (36).

5. The method (1) according to claim 3 or 4, wherein the method (1) further comprises Two or more detected parts (2, 3) that are physically connected among the plurality of parts are grouped together to form sub-segments; Divide one or more sub-segments into sub-segment groups; and Jointly optimize all sub-segments of the sub-segment group. In particular, Grouping at least two detected parts (2, 3) from the plurality of parts together and / or grouping one or more sub-segments into sub-segment groups can be provided by manual input from the user.

6. The method (1) according to any one of claims 3 to 5, wherein the method (1) further comprises Obtain the value (800) of at least one geometric parameter (23, 24, 29) of at least one detected part (2, 3) among the plurality of parts; and Set the value of the at least one geometric parameter (23, 24, 29) such that the value of the at least one geometric parameter (23, 24, 29) is not changed (900) by the optimization (600). In particular, The value of at least one geometric parameter (23, 24, 29) can be provided by manual input by the user and / or can be derived (1000) from a comparison of at least one detected part (2, 3) among the plurality of parts with a catalog (28) of parts that can be used to construct the physical structure (36).

7. The method (1) according to any one of claims 3 to 6, wherein the method (1) further comprises: Obtain information about the orientation of at least one detected part (2, 3) among the plurality of parts in the Cartesian coordinate system; as well as If the orientation of at least one detected part (2, 3) among the plurality of parts meets a predefined criterion, specifically the orientation of one of the axes parallel to the Cartesian coordinate system, then the orientation of the at least one detected part (2, 3) among the plurality of parts is set such that the orientation of the at least one detected part (2, 3) among the plurality of parts is not changed by the optimization (600). In particular, the orientation of the at least one detected part among the plurality of parts in the Cartesian coordinate system can be provided by manual input.

8. The method (1) according to any one of claims 3 to 7, wherein, During the optimization process (600), In the generated parametric model (13) of the physical structure (36), two or more detected parts (2, 3) of the plurality of parts that have been physically connected (5) are inseparable from each other; and / or At least one of the detected parts (2, 3) can be interchanged with a part from a catalog (28) of parts that can be used to construct the physical structure (36); and / or At least one of the detected parts (2, 3) among the plurality of parts can be removed from the generated parametric model (13) of the physical structure (36); and / or Parts from the catalog (28) of parts that can be used to construct the physical structure (36) can be inserted into the generated parametric model (13) of the physical structure (36).

9. The method (1) according to any one of the preceding claims, wherein, The classification of the point cloud data includes point cloud classification, and in particular, the point cloud classification includes: AI; Semantic segmentation; and / or Instance splitting.

10. The method (1) according to any one of the preceding claims, wherein, The set of predefined geometric shapes includes Cylinder; Cones, especially truncated asymmetrical cones; toroidal section; and / or Intersection In particular, If the physical structure (36) is configured as a pipeline system, then the cylinder is associated with a straight pipe, the cone with a reduced-bore pipe or an expanded pipe, the toroidal section with a pipe elbow, and / or the intersection with a pipe branch.

11. The method according to any one of the preceding claims, wherein the method (1) further comprises Obtain statistical information related to at least one of the detected parts (2, 3) among the plurality of parts; as well as Display the statistical information; In particular, the statistical information includes the standard deviation and / or maximum deviation of the set of points assigned to the at least one detected part (2, 3) among the plurality of parts, and a predefined geometry associated with the at least one detected part (2, 3) among the plurality of parts to which the set of points is assigned.

12. The method (1) according to any one of the preceding claims, wherein the method (1) further comprises Obtain another constraint on at least one of the geometric parameters (23, 24, 29) of the rear part (3) physically connected to the front part (2); and / or Remove the pseudo-constraints on at least one of the geometric parameters (23, 24, 29) of the rear part (3) which is physically connected to the front part (2). In particular, the other constraint is one that can be provided and / or the pseudo-constraint can be deleted by the user's manual input.

13. The method (1) according to any one of the preceding claims, the method (1) further comprising a constraint (700) displaying at least one of the geometric parameters (23, 24, 29) of the detected parts (2, 3) and / or the subsequent parts (3) physically connected to the preceding part (2), particularly wherein, The detected parts (2, 3) and / or the constraints among the plurality of parts are displayed in an overlay representation with specific symbols (18, 20, 21, 22).

14. A reality capture device (33), the reality capture device comprising a sensor unit and a computing unit, the sensor unit being configured to acquire points (14) in an environment (34) as a point cloud, the computing unit being configured to control the acquisition of the point cloud, wherein, Point cloud data describing the point cloud can be provided to the computing unit, and in particular, the sensor unit includes a LiDAR unit or multiple ToF cameras. Its features The computing unit stores program code for executing the method (1) according to any one of the preceding claims.

15. A computer program product comprising program code stored on a machine-readable medium or embodied by an electromagnetic wave including a segment of program code, and the computer program product having computer-executable instructions, particularly when executed in a computing unit of a reality capture device (33) according to claim 14, the computer-executable instructions being used to perform the method (1) according to any one of claims 1 to 13.

Images