Method and apparatus for repairing missing structure point cloud of building

Abstract

The application provides a building missing structure point cloud repairing method and device, the method comprises the following steps: obtaining original point cloud data of a target building; determining missing structure features of the original point cloud data, and adjusting the posture of the target building based on the missing structure features to obtain target point cloud data; constructing a cloth structure corresponding to the target building, determining corresponding cloth node rules based on the cloth structure; performing cloth subsidence simulation on the cloth structure based on the target point cloud data and the cloth node rules to obtain a cloth simulation model; determining structure constraints corresponding to the cloth structure, determining recessed areas in the cloth simulation model that need to be repaired based on the structure constraints, and repairing the recessed areas to obtain regional repair results; and fitting the regional repair results to obtain point cloud repair results of the target building. The application can guarantee strong generalization performance and robust performance of point cloud repair.

Description

Technical Field

[0001] This invention relates to the field of point cloud data processing technology, specifically to a method and apparatus for repairing point clouds of missing building structures. Background Technology

[0002] Point cloud completion is a data augmentation technique used to automatically refine uniformly dense point clouds, ensuring their connectivity and integrity to restore the complete shape of an object. Point cloud completion is widely used in 3D reconstruction, object detection, and 3D shape classification. It is specifically designed to fill in missing parts of an object and obtain a high-quality 3D representation. However, due to the limitations of laser scanning sensor platforms during actual point cloud data acquisition, it is often difficult to acquire absolutely complete building point clouds. Furthermore, the incompleteness of these building point clouds varies, leading to differences in the performance of different point cloud completion methods.

[0003] Point cloud completion methods mainly include geometry-based completion methods and deep learning-based completion methods. Geometry-based completion methods first fit a local surface based on the geometric information of the original point cloud, and then perform interpolation. First, missing holes need to be detected and identified, and then these missing holes are filled using a smooth interpolation algorithm. The advantage of this algorithm is its ease of implementation with simple algorithms. The disadvantage is that the detection and identification of missing holes affects the efficiency and accuracy of point interpolation, and the surface fitting requires sufficient supporting edge points. Therefore, when dealing with large point cloud datasets with missing points, this algorithm needs to perform completion algorithms separately, resulting in poor generalization. Deep learning-based completion methods generate a complete point cloud based on the feature information of the original point cloud. The advantage of this algorithm is its strong generalization ability, capable of handling point clouds with different shapes and types of missing points. The disadvantage is that this algorithm relies on a predefined object set and specific defect types of point cloud missing points. In practical applications, its performance significantly degrades, even further for categories that have never appeared before. Therefore, a method is needed that can guarantee both strong generalization performance and robust point cloud repair performance. Summary of the Invention

[0004] In view of this, it is necessary to provide a method and device for repairing point clouds of missing building structures, which can ensure both strong generalization performance and robust performance of point cloud repair.

[0005] To achieve the above objectives, the present invention provides a method for repairing point clouds of missing building structures, comprising:

[0006] Obtain the raw point cloud data of the target building;

[0007] The missing structural features of the original point cloud data are determined, and the pose of the target building is adjusted based on the missing structural features to obtain the target point cloud data;

[0008] Construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure;

[0009] Based on the target point cloud data and the cloth node rules, a cloth sinking simulation is performed on the cloth structure to obtain a cloth simulation model.

[0010] Determine the structural constraints corresponding to the fabric structure, determine the concave regions in the fabric simulation model that need point cloud repair based on the structural constraints, and repair the concave regions to obtain the region repair results;

[0011] The point cloud repair results of the target building are obtained by fitting the repair results of multiple regions.

[0012] Furthermore, adjusting the posture of the target building based on the missing structural features includes:

[0013] Determine the horizontal rotation angle when the volume of the three-dimensional bounding box corresponding to the target building is minimized;

[0014] Determine the rotation angle in the vertical direction when the volume of the three-dimensional bounding box is minimized;

[0015] The orientation of the target building is adjusted based on the horizontal rotation angle when the volume of the three-dimensional bounding box is at its minimum, and the vertical rotation angle when the volume of the three-dimensional bounding box is at its minimum.

[0016] Furthermore, adjusting the posture of the target building based on the missing structural features includes:

[0017] Multiple planar structures are randomly fitted into the original point cloud, and the normal vectors and the number of planar point clouds corresponding to the multiple planar structures are determined.

[0018] Determine the angle between the normal vectors of the plurality of planar structures;

[0019] Based on the included angle of the normal vectors and the number of points in the plane, determine the optimal pair of vertical planes;

[0020] The orientation of the target building is adjusted based on the optimal vertical plane pair.

[0021] Furthermore, adjusting the orientation of the target building based on the optimal vertical plane pair includes:

[0022] Based on the optimal vertical plane pair, construct the correspondence between the vertical coordinate axis plane and the plane.

[0023] Based on the correspondence with the vertical coordinate axis, the posture of the target building is adjusted.

[0024] Further, the step of performing fabric sinking simulation on the fabric structure based on the target point cloud data and the fabric node rules to obtain a fabric simulation model includes:

[0025] Based on the target point cloud data and the placement rules, a cloth falling model and a building point cloud collision model are simulated. Based on the cloth falling model and the building point cloud collision model, a cloth simulation model is determined.

[0026] Further, determining the structural constraints corresponding to the fabric structure, determining the recessed areas in the fabric simulation model that need point cloud repair based on the structural constraints, and repairing the recessed areas to obtain the area repair result includes:

[0027] Determine the constraint forces of the fixed nodes in the fabric structure, and determine the node constraint force stiffness based on the number of constraint forces and the set of force directions of the nodes. Based on the constraint forces of the fixed nodes and the node constraint force stiffness, determine the structural constraints.

[0028] Based on the distribution direction of fixed nodes during the fabric sinking stage, traverse the unfixed nodes to determine the force constraints of the unfixed nodes.

[0029] Based on the structural constraints and the force constraints of the unfixed nodes, the rebound point location of the area to be repaired is determined;

[0030] The point cloud data corresponding to the rebound landing point is restored to the target height value to obtain the area repair result.

[0031] Further, the fitting of the repair results of multiple regions to obtain the point cloud repair result of the target building includes:

[0032] Based on the maximum sinking distance in the fabric simulation model, the cross-section of the building in the current facade direction is extracted;

[0033] Based on the cross-section of the building, determine the structural lines of the building outline;

[0034] Based on the corner points between different structural lines, the missing structural regions are filled in, and the repair results of multiple regions are fitted to obtain the point cloud repair result of the target building.

[0035] The present invention also provides a repair device for missing structural point clouds of buildings, comprising:

[0036] The acquisition module is used to acquire the raw point cloud data of the target building;

[0037] An adjustment module is used to determine the missing structural features of the original point cloud data and adjust the pose of the target building based on the missing structural features to obtain target point cloud data.

[0038] The determination module is used to construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure;

[0039] The simulation module is used to simulate the fabric sinking of the fabric structure based on the target point cloud data and the fabric node rules, so as to obtain a fabric simulation model.

[0040] The repair module is used to determine the structural constraints corresponding to the fabric structure, determine the concave areas in the fabric simulation model that need to be repaired by point cloud based on the structural constraints, and repair the concave areas to obtain the area repair results.

[0041] The fitting module is used to fit the repair results of multiple regions to obtain the point cloud repair result of the target building.

[0042] The present invention also provides an electronic device, including a memory and a processor, wherein,

[0043] The memory is used to store programs;

[0044] The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the method for repairing missing structural point clouds of buildings as described in any of the preceding claims.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for repairing missing structural point clouds of buildings as described in any of the preceding claims.

[0046] The beneficial effects of the above implementation method are as follows: The method and apparatus for repairing missing structural point clouds of buildings provided by the present invention adjust the attitude of the target building based on the missing structural features of the original point cloud data to obtain the target point cloud data; then, a cloth structure corresponding to the target building is constructed, and the corresponding cloth node rules are determined based on the cloth structure; cloth sinking simulation is performed on the cloth structure based on the target point cloud data and the cloth node rules to obtain the cloth simulation model; the structural constraints corresponding to the cloth structure are determined, and the concave areas in the cloth simulation model that need to be repaired for point cloud repair are determined based on the structural constraints, and the concave areas are repaired to obtain the area repair results; the repair results of multiple areas are fitted to obtain the point cloud repair results of the target building. The present invention determines the attitude adjustment method based on the missing structural features of the point cloud, reduces the accuracy loss caused by dimensional compression during cloth simulation projection through attitude adjustment, and then uses the structural constraints of the building to set the rules followed by the cloth nodes to ensure the accuracy of the repair points. Finally, the two stages of cloth sinking simulation and concave area repair are performed to achieve the repair of large structural missing and multi-story building structures, thereby having higher generalization performance and robustness. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 A schematic flowchart of an embodiment of the method for repairing missing structural point clouds of buildings provided by the present invention;

[0049] Figure 2 A schematic diagram of the point cloud rotation process provided by the minimum volume method of the present invention;

[0050] Figure 3 A flowchart illustrating another embodiment of the method for repairing missing structural point clouds of buildings provided by the present invention;

[0051] Figure 4 A schematic diagram of a structure of an embodiment of the repair device for missing point clouds of buildings provided by the present invention;

[0052] Figure 5 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0054] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0055] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0056] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0057] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0058] This invention provides a method and apparatus for repairing point clouds of missing structures in buildings, which will be described below.

[0059] like Figure 1 As shown, the present invention provides a method for repairing point clouds of missing building structures, comprising:

[0060] Step 110: Obtain the original point cloud data of the target building.

[0061] Understandably, 3D laser scanning can be used to obtain the original point cloud data of the target building.

[0062] Step 120: Determine the missing structural features of the original point cloud data, and adjust the pose of the target building based on the missing structural features to obtain the target point cloud data.

[0063] It is understood that the present invention can provide two posture adjustment methods: one is the minimum volume method, and the other is the vertical orientation method.

[0064] Step 130: Construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure.

[0065] Understandably, the building mesh structure corresponding to the target building is constructed first, and then the physical rules followed for repairing the building point cloud are defined, i.e., the node placement rules. Defining the physical rules, taking the XZ axis as an example, the mesh is placed along the X and Z axes, and the node placement interval in each direction is the average spacing d of the building point cloud. mean The set of nodes is NP = [np1, np2, np3, ..., np...]. n If the nodes are represented by geometric coordinates, then:

[0066] np i =(x i y i , z t )

[0067] In the formula, i represents the i-th node in the set, and z t This represents the height of the fabric at time t, which is caused by gravity.

[0068] Meanwhile, the fabric structure is a regularly spaced grid, and the nodes can be represented by grid coordinates, then:

[0069]

[0070] In the formula, r and c represent the r-th row and c-th column of the node in the grid. The time at which the node is fixed, and D and W are the length and width of the grid, respectively.

[0071] Step 140: Based on the target point cloud data and the cloth node rules, perform cloth sinking simulation on the cloth structure to obtain the cloth simulation model.

[0072] It is understandable that the fabric sinking phase simulates the gradual descent of the fabric from top to bottom under the influence of gravity alone, without the influence of collisions. The height value z of the node at time t is... t The fabric structure was simulated by fabric sinking to obtain a fabric simulation model.

[0073] Fabric drop formula z t As shown in the following formula:

[0074] z t =z max -d z ·t

[0075] In the formula, z max d represents the initial height of the fabric structure. z Let t be the step size of the fall, and t be the time of fall, where t = 1, 2, 3, ..., n.

[0076] At each time t, a building point cloud P = [p1, p2, p3, ..., p] is introduced. n The collision calculation is performed, and the collision condition formula is shown below:

[0077] [Idx, Dis]=KnnSearch(NP, P, 1)

[0078]

[0079] In the formula, KnnSearch is the distance from a point in the search set NP to the nearest point in the set P, Idx is the index position of the searched point in NP, and Dis is the distance from np. Idx The distance to the nearest point in set P.

[0080] If np Idx If the point of collision is p, then use the point of collision. i The coordinates replace np Idx The geometric coordinates are shown in the following formula:

[0081] np i =p i =(x i y i , z i )

[0082] Fixed NP Idx This point will no longer participate in the subsequent falling process. Simultaneously, np is recorded. Idx Network coordinates t i This refers to the current moment.

[0083] Step 150: Determine the structural constraints corresponding to the fabric structure, determine the concave areas in the fabric simulation model that need to be repaired by point cloud based on the structural constraints, and repair the concave areas to obtain the area repair results.

[0084] It is understandable that repairing the dented area is equivalent to performing a fabric rebound process. During the fabric rebound phase, a force constraint direction is applied; that is, the fabric nodes are only constrained by the fixed nodes between rows and columns, and there is no constraint between the unfixed nodes. The force constraint direction can be expressed by the following formula:

[0085]

[0086]

[0087]

[0088]

[0089] In the formula, r + r - and c + c - Four directions, np is the current target node q Grid coordinates

[0090] To eliminate interference from noise points, enhance the robustness of the method of this invention, and prevent the "concave" structure from being incorrectly repaired, a vertical constraint force is set, and the stiffness between the fabric nodes is set as shown in the following formula:

[0091]

[0092] In the formula, Stiffness is the stiffness of the nodal constraint force, F i Let DF be the number of constraints acting on node i. i Let be the set of force directions for node i. When the stiffness of an unfixed node is 1, the node bounces back to the height corresponding to the stiffness of 1.

[0093] After determining the structural constraints, the fabric rebound phase is initiated. During the rebound phase, for the fixed nodes (r, c) in the grid coordinates recorded during the sinking phase, the unfixed nodes are progressively traversed according to the distribution of fixed points in the row and column directions of the grid coordinates. With r - Taking direction as an example for analysis The force constraints are analyzed using the following formula:

[0094]

[0095] In the formula, For in r - Direction away The nearest fixed node At the moment of collision, np + Represents a fixed set of nodes, np - Indicates an unfixed node that needs to bounce, DF i This represents the set of forces acting on a target.

[0096] If there are fixed nodes, then the force number F i Add 1, r - Join to point The set of force directions DF i In the middle, Give Similarly, for unfixed nodes in r + c - c + The force constraints are detected sequentially in the direction.

[0097] Determine np - After constraining the forces at all nodes, according to the stiffness formula in the structural constraints, There are three cases when the stiffness is 1. The first case is when it is subject to two perpendicular constraint forces. The second case is when it is subject to three constraint forces, in which case there must be mutually perpendicular constraint forces. The third case is when it is subject to constraint forces in four directions.

[0098] Since the building facade may have a multi-layered structure, these three constraint moments could all be the bounce points of the area to be repaired. z The formula for calculating the potential bounce positions for repairing the point cloud is as follows:

[0099]

[0100]

[0101]

[0102] In the formula, t z =2 represents the moment t2 when the unfixed point is subjected to two perpendicular constraint forces. z =3 indicates the moment t3 is subjected to constraint forces in three perpendicular directions. z =4 indicates the moment t4 when the force is constrained by four vertical forces.

[0103] During point cloud repair, the height at which moment is used as the repair point cloud NP will be determined based on the building facade's layers and repair needs. i =(x i y i , z t The repair location will be determined in the experimental section, focusing on point cloud repair for multi-layered building structures, and the rebound time t will be considered. z The selection is analyzed. These nodes that recover their height values ​​after bouncing are the patched region point cloud, denoted as set NP. + .

[0104] Step 160: Fit the repair results of multiple regions to obtain the point cloud repair result of the target building.

[0105] It is understandable that the cross-section of the building in the current facade direction is intercepted at the maximum sinking distance to obtain the structural lines of the building outline, and the missing structural areas are filled by calculating the intersections between the structural lines, that is, the repair results of multiple areas are fitted.

[0106] In some embodiments, adjusting the posture of the target building based on the missing structural features includes:

[0107] Determine the horizontal rotation angle when the volume of the three-dimensional bounding box corresponding to the target building is minimized;

[0108] Determine the rotation angle in the vertical direction when the volume of the three-dimensional bounding box is minimized;

[0109] The orientation of the target building is adjusted based on the horizontal rotation angle when the volume of the three-dimensional bounding box is at its minimum, and the vertical rotation angle when the volume of the three-dimensional bounding box is at its minimum.

[0110] Understandably, the minimum volume method in attitude adjustment first solves for the horizontal rotation angle s when the volume of the 3D bounding box is minimized. 2D s 2D The calculation method is as follows:

[0111]

[0112] Loss(s 2D )=d*w

[0113] In the formula, s 2D s represents the rotation angle in the horizontal direction, and d and w represent the minimum bounding box length and width of the original point cloud after rotation by an angle s, projected onto the XY plane of the two-dimensional rectangular coordinate system.

[0114] Then, calculate the vertical rotation angle s when the volume of the 3D bounding box is minimized. 3D s 3D The calculation method is as follows:

[0115]

[0116] Loss(s) = D * W * H

[0117] In the formula, s 3D s is the rotation angle value in the vertical direction, s is the random rotation angle value, and D, W, and H are the minimum bounding box length, width, and height values ​​of the building point cloud in the spatial rectangular coordinate system after rotation by an angle s.

[0118] The formula for calculating the point cloud after attitude adjustment is as follows:

[0119]

[0120] R 2D =Rotation matrix(s) 2D )

[0121] R 3D =Rotation matrix(s) 3D )

[0122] In the formula, P = [p1, p2, p3, ..., p n ], p i =(x i y i , z i () represents the original point cloud coordinates of the building. For the set of building point clouds after attitude adjustment, R 2D R 3D For the optimal rotation angle s 2D s 3D The corresponding rotation matrix.

[0123] Figure 2 The initial orientation of the building and the resulting image after rotating the point cloud using the minimum volume method are presented. It can be seen that the initial orientation of the building is unknown in the spatial coordinate system, making it impossible to automatically select the facade to be repaired. A horizontal rotation is performed first, followed by a vertical rotation. It is found that after the orientation adjustment, the orientation of the building's main facade is basically aligned with the coordinate system's axis.

[0124] In some embodiments, adjusting the posture of the target building based on the missing structural features includes:

[0125] Multiple planar structures are randomly fitted into the original point cloud, and the normal vectors and the number of planar point clouds corresponding to the multiple planar structures are determined.

[0126] Determine the angle between the normal vectors of the plurality of planar structures;

[0127] Based on the included angle of the normal vectors and the number of points in the plane, determine the optimal pair of vertical planes;

[0128] Based on the optimal vertical plane pair, the orientation of the target building is adjusted.

[0129] Understandably, in the attitude adjustment method, the vertical elevation normals are randomly fitted to N planar structures in the original point cloud of the building using the RANSAC algorithm, obtaining the normal vectors of N planes and the number of planar point clouds, n. The angle between the normal vectors of the N planar structures, Angle, is then calculated. The formula for calculating the optimal vertical plane pair is shown below:

[0130] [Planes Plane t ] = min i＜N，j＜N (|Angle ij -90°|)∧n i >Num∧n j >Num

[0131] In the formula, Angle ij The angle between the i-th plane and the j-th plane, n i Let Num be the number of point clouds on the i-th plane, and let Num be the minimum threshold for the number of point clouds on the facade, used to ensure that the selected plane pairs are large enough to be the main facades of the building. s Plane t The optimal pair of planes is selected. In some embodiments, adjusting the orientation of the target building based on the optimal pair of vertical planes includes:

[0132] Based on the optimal vertical plane pair, construct the correspondence between the vertical coordinate axis plane and the plane.

[0133] Based on the correspondence with the vertical coordinate axis, the posture of the target building is adjusted.

[0134] Understandably, this is based on the selected plane pair. s Plane t Normal vector s Normal t ], construct the correspondence between the vertical coordinate axes and the planes, and select the corresponding axes as [1, 0, 0] and [0, 1, 0], then we have:

[0135] R1 = Rotation matrix(Normal) s [1, 0, 0])

[0136] Normal Rt =Normal t *R1

[0137] R2 = Rotation matrix(Normal) Rt [0, 1, 0])

[0138]

[0139] In the formula, R1 and R2 are rotation matrices that make the vertical plane flush with the axis, and P = [p1, p2, p3, ..., p n ], p i =(x i y i , zi ), This is a collection of point clouds representing buildings after attitude adjustment.

[0140] In some embodiments, the step of performing fabric sinking simulation on the fabric structure based on the target point cloud data and the fabric node rules to obtain a fabric simulation model includes:

[0141] Based on the target point cloud data and the placement rules, a cloth falling model and a building point cloud collision model are simulated. Based on the cloth falling model and the building point cloud collision model, a cloth simulation model is determined.

[0142] Understandably, the cloth structure is constructed first, and then the physical rules governing the repair of building point clouds are defined. Taking the XZ axis as an example, the cloth mesh is laid out along the X and Z axes, with the node spacing in each direction equal to the average spacing d of the building point clouds. mean The set of nodes is NP = [np1, np2, np3, ..., np...]. n If the nodes are represented by geometric coordinates, then we have:

[0143] np i =(x i y i , z t )

[0144] In the formula, i represents the i-th node in the set, and z t This represents the height of the fabric at time t, which is caused by gravity.

[0145] Meanwhile, the fabric structure is a regularly spaced grid, and the nodes can be represented by grid coordinates, then:

[0146]

[0147] In the formula, r and c represent the node's location in the r-th row and c-th column of the grid. The time at which the node is fixed, and D and W are the length and width of the grid, respectively.

[0148] The simulation depicts the fabric sinking phase, where the fabric descends gradually from top to bottom under the influence of gravity alone, without any collisions. The height value z of the node at time t is then calculated. t Fabric falling formula z t As shown in the following formula:

[0149] z t =z max -d z ·t

[0150] In the formula, z maxd represents the initial height of the fabric structure. z Let t be the step size of the fall, and t be the time of fall, where t = 1, 2, 3, ..., n.

[0151] At each time t, a building point cloud P = [p1, p2, p3, ..., p] is introduced. n The collision calculation is performed, and the collision condition formula is shown below:

[0152] [Idx, Dis]=KnnSearch(NP, P, 1)

[0153]

[0154] In the formula, KnnSearch is the distance from a point in the search set NP to the nearest point in the set P, Idx is the index position of the searched point in NP, and Dis is the distance from np. Idx The distance to the nearest point in set P.

[0155] If np Idx If the point of collision is p, then use the point of collision. i The coordinates replace np Idx The geometric coordinates are shown in the following formula:

[0156] np i =p i =(x i y i , z i )

[0157] Fixed NP Idx This point will no longer participate in the subsequent falling process. Simultaneously, np is recorded. Idx Network coordinates t i This refers to the current moment.

[0158] In some embodiments, determining the structural constraints corresponding to the fabric structure, determining the concave regions in the fabric simulation model that need point cloud repair based on the structural constraints, and repairing the concave regions to obtain region repair results includes:

[0159] Determine the constraint forces of the fixed nodes in the fabric structure, and determine the node constraint force stiffness based on the number of constraint forces and the set of force directions of the nodes. Based on the constraint forces of the fixed nodes and the node constraint force stiffness, determine the structural constraints.

[0160] Based on the distribution direction of fixed nodes during the fabric sinking stage, traverse the unfixed nodes to determine the force constraints of the unfixed nodes.

[0161] Based on the structural constraints and the force constraints of the unfixed nodes, the rebound point location of the area to be repaired is determined;

[0162] The point cloud data corresponding to the rebound landing point is restored to the target height value to obtain the area repair result.

[0163] It is understandable that the rebound landing point corresponds to the concave area. Restoring the point cloud data corresponding to the rebound landing point to the target height value achieves the rebound of the concave area of ​​the fabric. During the fabric rebound phase, a force constraint direction is applied, meaning that the fabric nodes are only constrained by the fixed nodes between rows and columns; there is no constraint between unfixed nodes. The force constraint direction can be expressed by the following formula:

[0164]

[0165]

[0166]

[0167]

[0168] In the formula, r + r - and c + c - Four directions, np is the current target node q Grid coordinates

[0169] To eliminate interference from noise points, enhance the robustness of the method of this invention, and prevent the "concave" structure from being incorrectly repaired, a vertical constraint force is set, and the stiffness between the fabric nodes is set as shown in the following formula:

[0170]

[0171] In the formula, Stiffness is the stiffness of the nodal constraint force, F i Let DF be the number of constraints acting on node i. i Let be the set of force directions for node i. When the stiffness of an unfixed node is 1, the node bounces back to the height corresponding to the stiffness of 1.

[0172] After determining the structural constraints, the fabric rebound phase is initiated. During the rebound phase, for the fixed nodes (r, c) in the grid coordinates recorded during the sinking phase, the unfixed nodes are progressively traversed according to the distribution of fixed points in the row and column directions of the grid coordinates. With r - Taking direction as an example for analysis The force constraints are analyzed using the following formula:

[0173]

[0174] In the formula, For in r - Direction away The nearest fixed node At the moment of collision, np + Represents a fixed set of nodes, np - Indicates an unfixed node that needs to bounce, DF i This represents the set of forces acting on a target.

[0175] If there are fixed nodes, then the force number F i Add 1, r - Join to point The set of force directions DF i In the middle, Give Similarly, for unfixed nodes in r + c - c + The force constraints are detected sequentially in the direction.

[0176] Determine np - After constraining the forces at all nodes, according to the stiffness formula in the structural constraints, There are three cases when the stiffness is 1. The first case is when it is subject to two perpendicular constraint forces. The second case is when it is subject to three constraint forces, in which case there must be mutually perpendicular constraint forces. The third case is when it is subject to constraint forces in four directions.

[0177] Since the building facade may have a multi-layered structure, these three constraint moments could all be the bounce points of the area to be repaired. z The formula for calculating the potential bounce positions for repairing the point cloud is as follows:

[0178]

[0179]

[0180]

[0181] In the formula, t z =2 represents the moment t2 when the unfixed point is subjected to two perpendicular constraint forces. z =3 indicates the moment t3 is subjected to constraint forces in three perpendicular directions. z =4 indicates the moment t4 when the force is constrained by four vertical forces.

[0182] During point cloud repair, the height at which moment is used as the repair point cloud NP will be determined based on the building facade's layers and repair needs.i =(x i y i , z t The repair location will be determined in the experimental section, focusing on point cloud repair for multi-layered building structures, and the rebound time t will be considered. z The selection is analyzed. These nodes that recover their height values ​​after bouncing are the patched region point cloud, denoted as set NP. + .

[0183] In some embodiments, fitting the repair results of the multiple regions to obtain the point cloud repair result of the target building includes:

[0184] Based on the maximum sinking distance in the fabric simulation model, the cross-section of the building in the current facade direction is extracted;

[0185] Based on the cross-section of the building, determine the structural lines of the building outline;

[0186] Based on the corner points between different structural lines, the missing structural regions are filled in, and the repair results of multiple regions are fitted to obtain the point cloud repair result of the target building.

[0187] It is understandable that the maximum sinking distance z in the fabric simulation model... min The cross-section of the building in the current facade direction is captured to obtain the structural lines of the building outline. The missing structural areas are filled by calculating the intersections between the structural lines.

[0188] Based on the cross-sectional point cloud P = [p1, p2, p3, ..., p...], ... n Construct a 2D minimum bounding box and fit its four edges. Then, use the fitted line equation... bb Add the structural lines to the set Line as the upper limit of the cross-sectional region in each direction, and solve for the four intersection points q. cc This allows for the division of the maximum fit range.

[0189] Then, the RANSAC algorithm is used to iteratively fit the structural line segments in the cross-sectional point cloud P. When the mean square error of the fitted line is greater than twice that of d... mean Termination of line segment detection. The fitted line equation is then used. s Add it to the structure line set Line. Based on the parametric equations in the structure line set Line, find the pairwise intersection points q. i , and add it to the intersection set Q.

[0190] Retrieve all intersections in Q, and delete intersections outside the minimum bounding box from Q. Calculate the distance from all intersections q in the intersection set Q to the nearest point in the cross-sectional point cloud set P. If the minimum distance is less than a set distance value D, then... maxThis indicates that the intersection point is located on the structural line and belongs to the corner point of the structural line, so it is added to the corner point set V.

[0191] Retrieve the four intersection points q of the minimum bounding box of the cross-sectional point cloud cc The distance to the nearest point in the set of corner points V is determined if the minimum distance is greater than a set distance value D. max , q cc Add the corner point set V. This ensures that, even with missing building structures, the intersection of the minimum bounding box maintains the fitted structural region as a complete polygon. Obtain the complete corner point set V.

[0192] Add the corner point V to the cross-sectional point cloud set P, and construct a polygon based on the point cloud set P. The area inside the polygon frame is the building structure area S1. Take the cloth point cloud Np obtained in the cloth simulation stage. + The two-dimensional projection is denoted as S2. The point cloud within the intersection region S = S1 ∩ S2 is the building repair point cloud. After fitting the structural region, the excess repair point cloud is removed.

[0193] In other embodiments, the flowchart of the method for repairing missing structural point clouds of buildings provided by the present invention is as follows: Figure 3 As shown.

[0194] In summary, this invention first determines the attitude adjustment method based on the missing structural features of the point cloud, reducing the accuracy loss caused by dimensional compression during cloth simulation projection through attitude adjustment. Then, it utilizes the structural constraints of the building to set the rules followed by the placement of nodes, ensuring the accuracy of the repair points. Finally, it performs two stages, "sinking" and "bounce," to repair large-scale structural defects and multi-story buildings. This invention provides a point cloud repair method for large-scale structural defects in buildings with applied structural constraints, exhibiting higher generalization and robustness compared to existing technologies.

[0195] This invention employs two methods to calculate the rotation matrix based on the characteristics of buildings, thereby adjusting the building's attitude. For buildings with an overall cubic structure, the minimum volume method is used. By rotating the building's point cloud in a Cartesian coordinate system, the minimum bounding box volume is minimized. Rotations are performed sequentially in the horizontal and vertical directions, and the rotation matrix is ​​then calculated. For buildings with consecutive missing adjacent structures, the vertical facade method is used to unify the building's facade orientation to the axial plane orientation. This requires finding a pair of vertical facades within the building structure, establishing the correspondence between the vertical facades and the vertical axial plane, and then calculating the rotation matrix. Finally, the point cloud with adjusted attitude is obtained through the rotation matrix.

[0196] After the fabric rebound is completed, the present invention first precisely divides the structural surface area to obtain an accurate point cloud of the repair area, thereby avoiding the addition of redundant structural surfaces that would affect the subsequent extraction of structural parameters.

[0197] like Figure 4 As shown, the present invention also provides a repair device 400 for missing structural point clouds of buildings, comprising:

[0198] The acquisition module 410 is used to acquire the original point cloud data of the target building;

[0199] The adjustment module 420 is used to determine the missing structural features of the original point cloud data and adjust the pose of the target building based on the missing structural features to obtain the target point cloud data.

[0200] The determination module 430 is used to construct the fabric structure corresponding to the target building and determine the corresponding fabric node rules based on the fabric structure.

[0201] Simulation module 440 is used to perform fabric sinking simulation on the fabric structure based on the target point cloud data and the fabric node rules to obtain a fabric simulation model;

[0202] The repair module 450 is used to determine the structural constraints corresponding to the fabric structure, determine the concave areas in the fabric simulation model that need to be repaired by point cloud based on the structural constraints, and repair the concave areas to obtain the area repair result.

[0203] The fitting module 460 is used to fit the repair results of multiple regions to obtain the point cloud repair result of the target building.

[0204] The building missing structural point cloud repair device provided in the above embodiments can realize the technical solution described in the above building missing structural point cloud repair method embodiments. The specific implementation principle of each module or unit can be found in the corresponding content in the above building missing structural point cloud repair method embodiments, and will not be repeated here.

[0205] like Figure 5 As shown, the present invention also provides an electronic device 500. The electronic device 500 includes a processor 501, a memory 502, and a display 503. Figure 5 Only some components of the electronic device 500 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0206] In some embodiments, memory 502 may be an internal storage unit of electronic device 500, such as a hard disk or memory of electronic device 500. In other embodiments, memory 502 may also be an external storage device of electronic device 500, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 500.

[0207] Furthermore, the memory 502 may include both internal storage units of the electronic device 500 and external storage devices. The memory 502 is used to store application software and various types of data installed on the electronic device 500.

[0208] In some embodiments, processor 501 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 502 or process data, such as the method for repairing point clouds of missing building structures in this invention.

[0209] In some embodiments, display 503 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 503 is used to display information from electronic device 500 and to display a visual user interface. Components 501-503 of electronic device 500 communicate with each other via a system bus.

[0210] In some embodiments of the present invention, when the processor 501 executes a repair program for the missing structural point cloud of a building in the memory 502, the following steps can be implemented:

[0211] Obtain the raw point cloud data of the target building;

[0212] The missing structural features of the original point cloud data are determined, and the pose of the target building is adjusted based on the missing structural features to obtain the target point cloud data;

[0213] Construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure;

[0214] Based on the target point cloud data and the cloth node rules, a cloth sinking simulation is performed on the cloth structure to obtain a cloth simulation model.

[0215] Determine the structural constraints corresponding to the fabric structure, determine the concave regions in the fabric simulation model that need point cloud repair based on the structural constraints, and repair the concave regions to obtain the region repair results;

[0216] The point cloud repair results of the target building are obtained by fitting the repair results of multiple regions.

[0217] It should be understood that when the processor 501 executes the repair program for the missing structural point cloud of the building in the memory 502, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0218] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 500 mentioned. Electronic device 500 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 500 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0219] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for repairing point clouds of missing building structures provided by the methods described above, the method comprising:

[0220] Obtain the raw point cloud data of the target building;

[0221] The missing structural features of the original point cloud data are determined, and the pose of the target building is adjusted based on the missing structural features to obtain the target point cloud data;

[0222] Construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure;

[0223] Based on the target point cloud data and the cloth node rules, a cloth sinking simulation is performed on the cloth structure to obtain a cloth simulation model.

[0224] Determine the structural constraints corresponding to the fabric structure, determine the concave regions in the fabric simulation model that need point cloud repair based on the structural constraints, and repair the concave regions to obtain the region repair results;

[0225] The point cloud repair results of the target building are obtained by fitting the repair results of multiple regions.

[0226] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0227] The above provides a detailed description of the method and apparatus for repairing point clouds of missing building structures provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for repairing point clouds of missing structural elements in a building, characterized in that, include: Obtain the raw point cloud data of the target building; The missing structural features of the original point cloud data are determined, and the pose of the target building is adjusted based on the missing structural features to obtain the target point cloud data; Construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure; Based on the target point cloud data and the cloth node rules, a cloth sinking simulation is performed on the cloth structure to obtain a cloth simulation model. Determine the structural constraints corresponding to the fabric structure, determine the concave regions in the fabric simulation model that need point cloud repair based on the structural constraints, and repair the concave regions to obtain the region repair results; The point cloud repair results of the target building are obtained by fitting the repair results of multiple regions. The adjustment of the target building's posture based on the missing structural features includes: Determine the horizontal rotation angle when the volume of the three-dimensional bounding box corresponding to the target building is minimized; Determine the rotation angle in the vertical direction when the volume of the three-dimensional bounding box is minimized; The orientation of the target building is adjusted based on the horizontal rotation angle when the volume of the three-dimensional bounding box is at its minimum, and the vertical rotation angle when the volume of the three-dimensional bounding box is at its minimum. The process of determining the structural constraints corresponding to the fabric structure, determining the concave regions in the fabric simulation model that require point cloud repair based on the structural constraints, and repairing the concave regions to obtain the region repair results includes: Determine the constraint forces of the fixed nodes in the fabric structure, and determine the node constraint force stiffness based on the number of constraint forces and the set of force directions of the nodes. Based on the constraint forces of the fixed nodes and the node constraint force stiffness, determine the structural constraints. Based on the distribution direction of fixed nodes during the fabric sinking stage, traverse the unfixed nodes to determine the force constraints of the unfixed nodes. Based on the structural constraints and the force constraints of the unfixed nodes, the rebound point location of the area to be repaired is determined; The point cloud data corresponding to the rebound landing point is restored to the target height value to obtain the area repair result.

2. The method for repairing point clouds of missing building structures according to claim 1, characterized in that, The adjustment of the orientation of the target building based on the missing structural features further includes: Multiple planar structures are randomly fitted into the original point cloud, and the normal vectors and the number of planar point clouds corresponding to the multiple planar structures are determined. Determine the angle between the normal vectors of the plurality of planar structures; Based on the included angle of the normal vectors and the number of points in the plane, determine the optimal pair of vertical planes; Based on the optimal vertical plane pair, the orientation of the target building is adjusted.

3. The method for repairing point clouds of missing building structures according to claim 2, characterized in that, The adjustment of the orientation of the target building based on the optimal vertical plane pair includes: Based on the optimal vertical plane pair, construct the correspondence between the vertical coordinate axis plane and the plane. Based on the correspondence with the vertical coordinate axis, the posture of the target building is adjusted.

4. The method for repairing point clouds of missing building structures according to claim 1, characterized in that, The fabric sinking simulation based on the target point cloud data and the fabric node rules is used to obtain a fabric simulation model, including: Based on the target point cloud data and the placement rules, a cloth falling model and a building point cloud collision model are simulated. Based on the cloth falling model and the building point cloud collision model, a cloth simulation model is determined.

5. The method for repairing point clouds of missing building structures according to any one of claims 1-4, characterized in that, The process of fitting the repair results of multiple regions to obtain the point cloud repair result of the target building includes: Based on the maximum sinking distance in the fabric simulation model, the cross-section of the building in the current facade direction is extracted; Based on the cross-section of the building, determine the structural lines of the building outline; Based on the corner points between different structural lines, the missing structural regions are filled in, and the repair results of multiple regions are fitted to obtain the point cloud repair result of the target building.

6. A device for repairing point clouds of missing structures in buildings, characterized in that, include: The acquisition module is used to acquire the raw point cloud data of the target building; An adjustment module is used to determine the missing structural features of the original point cloud data and adjust the pose of the target building based on the missing structural features to obtain target point cloud data. The determination module is used to construct the fabric structure corresponding to the target building, and determine the corresponding fabric node rules based on the fabric structure; The simulation module is used to simulate the fabric sinking of the fabric structure based on the target point cloud data and the fabric node rules, so as to obtain a fabric simulation model. The repair module is used to determine the structural constraints corresponding to the fabric structure, determine the concave areas in the fabric simulation model that need to be repaired by point cloud based on the structural constraints, and repair the concave areas to obtain the area repair results. The fitting module is used to fit the repair results of multiple regions to obtain the point cloud repair result of the target building; The adjustment of the orientation of the target building based on the missing structural features includes: Determine the horizontal rotation angle when the volume of the three-dimensional bounding box corresponding to the target building is minimized; Determine the rotation angle in the vertical direction when the volume of the three-dimensional bounding box is minimized; The orientation of the target building is adjusted based on the horizontal rotation angle when the volume of the three-dimensional bounding box is at its minimum, and the vertical rotation angle when the volume of the three-dimensional bounding box is at its minimum. The process of determining the structural constraints corresponding to the fabric structure, determining the concave regions in the fabric simulation model that require point cloud repair based on the structural constraints, and repairing the concave regions to obtain the region repair results includes: Determine the constraint forces of the fixed nodes in the fabric structure, and determine the node constraint force stiffness based on the number of constraint forces and the set of force directions of the nodes. Based on the constraint forces of the fixed nodes and the node constraint force stiffness, determine the structural constraints. Based on the distribution direction of fixed nodes during the fabric sinking stage, traverse the unfixed nodes to determine the force constraints of the unfixed nodes. Based on the structural constraints and the force constraints of the unfixed nodes, the rebound point location of the area to be repaired is determined; The point cloud data corresponding to the rebound landing point is restored to the target height value to obtain the area repair result.

7. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is configured to execute the program stored in the memory to implement the steps in the method for repairing missing structural point clouds of buildings as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for repairing the missing structural point cloud of a building as described in any one of claims 1 to 5.

Images