Mobile LiDAR data indoor plane graph reconstruction method based on scene structure elements

Abstract

The invention discloses a mobile LiDAR data indoor plane graph reconstruction method based on scene structure elements. The method comprises the following steps: preprocessing an original point cloud; segmenting the preprocessed point cloud data through a deep learning semantic segmentation method to obtain segmented structure element points; the semantic information, the geometric feature information and the position information of the segmented structure element points are integrated, and complete structure element points are extracted from the scene; using a linear fitting method to convert the extracted structural element points into linear primitives, using a preset image processing algorithm, performing expansion in a dynamic data structure, and performing reconstruction in a closed polygon form to obtain an indoor scene planar graph; and eliminating redundant polygons in the indoor scene plane graph based on an optimization function of an undirected graph, and outputting a final indoor plane graph. According to the method, the structural elements can be effectively extracted in a complex indoor scene, an accurate room planar graph is generated, and the reconstruction precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing technology, and particularly to a method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements. Background Art

[0002] The technology of reconstructing the floor plan of an indoor scene has important application values in fields such as building information modeling (BIM), indoor navigation, emergency response services, and smart buildings. Existing floor plan reconstruction methods take three-dimensional point cloud data or two-dimensional image data as input, and extract indoor structure elements (such as ceilings, floors, walls, etc.). Subsequently, these structure elements are connected through different combination strategies to generate a complete building floor plan. However, in the face of complex layouts, furniture occlusion, as well as noise and missing areas in point cloud data, there are still significant limitations in the accuracy and automation of existing algorithms. These challenges limit the accuracy and automation of existing algorithms, making the industrial community still rely on manual interaction, resulting in problems of high resource consumption and low efficiency. Therefore, developing efficient and accurate floor plan reconstruction methods has become a new research focus.

[0003] Traditional methods have made certain progress in extracting indoor structure elements and reconstructing floor plans, but they still face the interference of complex scenes and data missing. Traditional detection methods based on geometric features are known for high processing efficiency and wide applicability, but it is difficult to accurately distinguish structure elements from non-structure elements in complex scenes. Deep learning methods have better effects in dealing with complex layouts, but they still have limitations in scenes with more noise points and missing areas.

[0004] After extracting structure elements, common combination strategies include directly connecting structure elements through an optimization framework, or dividing the two-dimensional space into polygons and assigning room labels. The former is direct but time-consuming and relies on detection results; the latter has higher robustness, but has higher requirements for the accuracy of label data, increasing the complexity of data preprocessing; therefore, existing methods for reconstructing the floor plan of an indoor scene also have problems of low reconstruction accuracy and low efficiency.

[0005] Therefore, the existing technology still needs to be improved. Summary of the Invention

[0006] The technical problem to be solved by the present invention is that, aiming at the defects of the existing technology, the present invention provides a method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements to solve the problems of low reconstruction accuracy and low efficiency existing in the existing methods for reconstructing the floor plan of an indoor scene.

[0007] The technical solution adopted by the present invention to solve the technical problem is as follows: In a first aspect, the present invention provides a method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements, including: Preprocess the original point cloud; Segment the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points; Integrate the semantic information, geometric feature information, and position information of the segmented structural element points, and extract complete structural element points from the scene; Use a linear fitting method to convert the extracted structural element points into linear primitives, and use a preset image processing algorithm to expand them in a dynamic data structure and reconstruct an indoor scene floor plan in the form of a closed polygon; Based on an optimization function of an undirected graph, remove redundant polygons in the indoor scene floor plan and output the final indoor floor plan.

[0008] In one implementation, the preprocessing of the original point cloud includes: Perform downsampling on the original point cloud and resample the original point cloud using the moving least squares method to obtain the preprocessed point cloud data.

[0009] In one implementation, the segmentation of the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points includes: Segment the preprocessed point cloud data by the semantic segmentation method of deep learning, and segment the indoor point cloud into main structural elements, secondary structural elements, and non-structural elements to obtain the segmented structural element points.

[0010] In one implementation, the integration of the semantic information, geometric feature information, and position information of the segmented structural element points, and the extraction of complete structural element points from the scene includes: Analyze the similarity of geometric feature information in the point sets corresponding to the main structural elements and non-structural elements with semantic information identifiers, and identify structural candidate points; Extract the boundary points of the secondary structural elements with semantic information identifiers, and evaluate the coincidence degree of the boundary points and the structural candidate points in the two-dimensional plane; Select the complete structural element points from the structural candidate points according to the evaluated coincidence degree.

[0011] In one implementation, the use of a linear fitting method to convert the extracted structural element points into linear primitives, and the use of a preset image processing algorithm to expand them in a dynamic data structure and reconstruct an indoor scene floor plan in the form of a closed polygon includes: Based on the linear fitting method, according to the number of points in the input point cloud data, the adjusted angle threshold and distance threshold, the extracted structural element points are classified into different linear primitives; Based on the preset image processing algorithm, each linear primitive is converted into a dynamic data structure, and both ends of the dynamic data structure are extended so that the dynamic data structures intersect each other to form a closed polygon, and the indoor scene floor plan is obtained.

[0012] In one implementation, the elimination of redundant polygons in the indoor scene floor plan by the optimization function based on the undirected graph includes: Based on the optimization function number of the undirected graph, the closed polygons in the indoor scene floor plan are converted into an undirected graph, and the effective point number and endpoint degree of the edges in the undirected graph are statistically analyzed, and redundant edges are eliminated and the remaining edges are merged according to the statistical results.

[0013] In one implementation, the conversion of the closed polygons in the indoor scene floor plan into an undirected graph, the statistical analysis of the effective point number and endpoint degree of the edges in the undirected graph, and the elimination of redundant edges and the merging of the remaining edges according to the statistical results include: Each edge S in the closed polygon is divided into n segmented points, and the number of LiDAR points near each segmented point is statistically analyzed, and the segmented points with the number of LiDAR points exceeding the preset threshold T are marked as effective points; Statistically analyze the degrees of the two endpoints of each edge S in the closed polygon, and determine whether the corresponding degrees are preset values; If the corresponding degree is not the preset value, it is determined that the corresponding edge is a redundant edge formed during the expansion process, and the redundant polygons in the indoor scene floor plan are eliminated by combining the effective point number and endpoint degree of the corresponding edge.

[0014] In a second aspect, the present invention provides a mobile LiDAR data indoor floor plan reconstruction system based on scene structure elements, including: A preprocessing module for preprocessing the original point cloud; A segmentation module for segmenting the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points; An element extraction module for integrating the semantic information, geometric feature information, and position information of the segmented structural element points to extract complete structural element points from the scene; A reconstruction module for converting the extracted structural element points into linear primitives using a linear fitting method, and using a preset image processing algorithm to expand them in a dynamic data structure and reconstruct an indoor scene floor plan in the form of a closed polygon; An optimization module, configured to eliminate redundant polygons in the indoor scene floor plan based on an optimization function of an undirected graph, and output a final indoor floor plan.

[0015] In a third aspect, the present invention provides a terminal, including: a processor and a memory, where the memory stores a program for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements. When the program for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements is executed by the processor, it is used to implement the operations of the method for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements as described in the first aspect.

[0016] In a fourth aspect, the present invention further provides a medium, which is a computer-readable storage medium. The medium stores a program for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements. When the program for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements is executed by a processor, it is used to implement the operations of the method for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements as described in the first aspect.

[0017] The present invention adopts the above technical solutions and has the following effects: (1) The present invention integrates the semantic feature information and geometric feature information of point clouds to extract structure elements from indoor scene point cloud data. This enables the present invention to improve the reconstruction efficiency of indoor floor plans without relying on additional data inputs such as scanning trajectories and room heights and other prior knowledge.

[0018] (2) The dynamic data structure expansion adopted by the present invention greatly reduces the impact of missing areas caused by a cluttered indoor environment and furniture occlusion on the reconstructed floor plan.

[0019] (3) The present invention adopts an optimization function based on an undirected graph to correct and simplify the reconstructed floor plan, making the reconstruction result more accurate and complete, performing better visually, and improving the reconstruction quality of indoor floor plans. Description of the Drawings

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0021] Figure 1 It is a flowchart of the method for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements in the present invention.

[0022] Figure 2 It is a schematic diagram of all test data in the present invention.

[0023] Figure 3 It is a schematic diagram of the superimposed display of the result of the floor plan reconstruction algorithm and the original point cloud data in the present invention.

[0024] Figure 4 It is a schematic diagram of the superimposed display of the point cloud result of the indoor scene after deep learning semantic segmentation and the extracted structural element points and the original point cloud data in the present invention.

[0025] Figure 5 It is a comparison chart of the floor plan reconstruction result provided in the present invention and the results of multiple methods.

[0026] Figure 6 It is a functional schematic diagram of the terminal in one implementation manner of the present invention.

[0027] The realization, functional features and advantages of the present invention will be further described with reference to the embodiments and the accompanying drawings. Specific Embodiments

[0028] To make the objectives, technical solutions and advantages of the present invention clearer and more definite, the following further describes the present invention in detail with reference to the accompanying drawings and by way of examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0029] Exemplary Method Traditional methods have made certain progress in extracting indoor structural elements and reconstructing floor plans, but still face interference from complex scenes and data loss. Traditional detection methods based on geometric features are known for high processing efficiency and wide applicability, but it is difficult to accurately distinguish structural elements from non-structural elements in complex scenes. Deep learning methods have better effects in dealing with complex layouts, but still have limitations in scenes with more noise points and missing areas.

[0030] After extracting structural elements, common combination strategies include directly connecting structural elements through an optimization framework, or dividing the two-dimensional space into polygons and assigning room labels. The former is direct but time-consuming and depends on the detection results; the latter has higher robustness, but has higher requirements for the accuracy of label data, increasing the complexity of data preprocessing; therefore, the existing floor plan reconstruction methods for indoor scenes still have problems of low reconstruction accuracy and low efficiency.

[0031] To solve the above technical problems, an embodiment of the present invention provides a method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements. The method preprocesses the original point cloud; segments the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points; integrates the semantic information, geometric feature information, and position information of the segmented structural element points, extracts complete structural element points from the scene; uses a linear fitting method to convert the extracted structural element points into linear primitives, and uses a preset image processing algorithm to expand them in a dynamic data structure and reconstruct an indoor scene floor plan in the form of a closed polygon; and eliminates redundant polygons in the indoor scene floor plan based on an optimization function of an undirected graph, and outputs the final indoor floor plan. The embodiment of the present invention can effectively extract structural elements in a complex indoor scene, generate an accurate room floor plan, and improve the reconstruction accuracy and efficiency.

[0032] As Figure 1 shown, an embodiment of the present invention provides a method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements, including the following steps: Step S100, preprocess the original point cloud.

[0033] In this embodiment, the method can accurately identify structural elements (such as doors, windows, walls, ceilings, and floors, etc.) in an indoor scene and generate a high-precision indoor floor plan using these elements. To improve the reconstruction accuracy and efficiency of the indoor floor plan, in this embodiment, after obtaining the original point cloud data of the indoor scene, it is necessary to preprocess the original point cloud.

[0034] Specifically, in an implementation manner of this embodiment, step S100 includes the following steps: Step S101, perform downsampling processing on the original point cloud, and perform resampling processing on the original point cloud using the moving least squares method to obtain the preprocessed point cloud data.

[0035] In this embodiment, first, downsampling processing is performed on the original point cloud to reduce the number of points while retaining the structural information of the indoor point cloud, thereby improving the running efficiency of the algorithm. Subsequently, the moving least squares method (MLS) is used to resample the original point cloud to smooth the surface of the point cloud. In an indoor scene, the complexity of furniture layout often results in noise points in the collected data that are difficult to eliminate by conventional methods, and the existence of these noise points may affect the accuracy of semantic information recognition. Therefore, before using a deep learning model to perform semantic segmentation on indoor point clouds, in this embodiment, the MLS algorithm is used to smooth the point cloud data, which can effectively improve the accuracy of the semantic segmentation result. Among them, the process of the MLS algorithm is shown in formula (1): (1); Wherein, represents the polynomial surface function obtained by fitting at point , is a weight function used to calculate the weighted value of the distance between point and all its neighboring points. represents the difference between the smoothed point coordinates and the original coordinates. For each point in the input point cloud, MLS first finds a certain number of its neighboring points through a KD tree (i.e., K-D tree, a data structure for organizing multi-dimensional space data), and then uses a polynomial for surface fitting in the neighboring points, and obtains the optimal fitting surface by minimizing formula (1). Then, project point onto the surface, thereby realizing the smoothing of the point cloud surface. Considering the running efficiency and time overhead of the algorithm, in this embodiment, the degree of the polynomial function can be set to 2.

[0036] In this embodiment, through the above processing process of the MLS algorithm, the preprocessed point cloud data can be obtained.

[0037] As Figure 1 shown, an embodiment of the present invention provides a method for reconstructing an indoor floor plan of mobile LiDAR data based on scene structure elements, including the following steps: Step S200, segment the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points.

[0038] In this embodiment, for the above preprocessed point cloud data, the point cloud data is divided into three categories: main structural elements, secondary structural elements, and non-structural elements through a pre-trained deep learning model. Among them, the main structural elements include doors, windows, and walls, the secondary structural elements include ceilings and floors, and the non-structural elements include all furniture in the indoor scene, etc.

[0039] Specifically, in an implementation manner of this embodiment, step S200 includes the following steps: Step S201, segment the preprocessed point cloud data by the semantic segmentation method of deep learning, segment the indoor point cloud into main structural elements, secondary structural elements, and non-structural elements, and obtain the segmented structural element points.

[0040] In this embodiment, due to the complexity of the indoor scene environment, it is difficult to accurately extract structural elements using a single geometric feature. The main reason is that the geometric features of furniture often resemble those of structural elements. Therefore, before identifying and extracting structural elements, a pre-trained AI model (i.e., the semantic segmentation method of deep learning, such as the FCN model) is first used to perform semantic segmentation on the indoor scene point cloud, dividing the point cloud into main structural elements (such as walls, doors, windows, and columns, etc.), secondary structural elements (such as ceilings, floors), and non-structural elements (such as all furniture in the indoor scene). Since the differences among these three categories are mainly reflected in their geometric features, in this embodiment, only the spatial coordinates of the points are used. x, y, z .

[0041] In this embodiment, three types of structural elements are obtained through the above-mentioned semantic segmentation method of deep learning. A complete set of structural element points can be obtained from these structural elements to improve the reconstruction quality.

[0042] As Figure 1 shown, the embodiment of the present invention provides a method for reconstructing an indoor floor plan based on scene structural elements using mobile LiDAR data, including the following steps: Step S300, integrating the semantic information, geometric feature information, and position information of the segmented structural element points, and extracting complete structural element points from the scene.

[0043] In this embodiment, for the main structural elements, secondary structural elements, and non-structural elements obtained from the above segmentation, a complete set of structural element points is obtained by integrating various feature information (i.e., the semantic information, geometric feature information, and position information corresponding to the three types of structural element points). Specifically, the method first analyzes the similarity of geometric feature information in the point sets with semantic information labels of main structural elements and non-structural elements to identify structural candidate points. Subsequently, the method further extracts the boundary points of the secondary structural elements with semantic information labels, and by evaluating the coincidence degree of these boundary points and the structural candidate points in the two-dimensional plane, the method can accurately screen out the true structural points from the structural candidate points.

[0044] Specifically, in an implementation manner of this embodiment, step S300 includes the following steps: Step S301, analyzing the similarity of geometric feature information in the point sets corresponding to the main structural elements and non-structural elements with semantic information labels, and identifying structural candidate points; Step S302, extracting the boundary points of the secondary structural elements with semantic information labels, and evaluating the coincidence degree of the boundary points and the structural candidate points in the two-dimensional plane; Step S303, screening out the complete structural element points from the structural candidate points according to the evaluated coincidence degree.

[0045] In this embodiment, the semantic features, geometric features, and two-dimensional plane position information of the point cloud are further integrated to accurately extract the structural element points from the scene point cloud. First, for the point cloud data that has been segmented into main structural elements and non-structural elements, the normal vector information of the points is used as the geometric feature, and the geometric feature similarity between the two types of points is calculated, so as to reclassify the point cloud data. The calculation of the geometric feature similarity is shown in formula (2): (2); Wherein, and respectively represent the normal vectors of the main structural element points and non-structural element points , and is the included angle between them. Specifically, by calculating the included angle of the normal vectors of the non-structural element points and the main structural element points, if is less than the preset threshold, it is considered that the non-structural element point has a similar geometric feature to the main structural element, and then it is reclassified as a main structural element point.

[0046] However, the geometric feature similarity classification may cause some non-structural element points with similar geometric features to be misclassified as main structural element points, and these misclassified points will affect the reconstruction accuracy of the floor plan. To address this issue, this embodiment is based on the following assumption: in an indoor scene, main structural elements (such as walls, doors, windows, etc.) usually intersect with secondary structural elements (such as ceilings, floors), so the positions of the main structural element points on the two-dimensional plane usually coincide with the boundaries of the secondary structural elements. Based on this assumption, in this embodiment, the boundary points of the secondary structural elements are first extracted, and only x, y coordinate information is used to calculate the two-dimensional plane distance from the main structural element points to the boundary points of the secondary structural elements. Specifically, if the distance between the main structural element point and the boundary point of the secondary structural element exceeds the preset threshold, the point is removed to avoid the interference of non-structural element points with similar geometric features on the accuracy of the reconstruction result.

[0047] As Figure 1 shown, an embodiment of the present invention provides a method for reconstructing an indoor floor plan based on scene structural elements from mobile LiDAR data, including the following steps: Step S400, using a linear fitting method to convert the extracted structural element points into linear primitives, and using a preset image processing algorithm to expand them with a dynamic data structure, and reconstructing the indoor scene floor plan in the form of a closed polygon.

[0048] In this embodiment, for the above-extracted complete structural element points, the structural element points are converted into linear primitives through linear fitting, and the Kippi algorithm (i.e., a preset image processing algorithm) is used for expansion to obtain a floor plan of the indoor scene in the form of a closed polygon. For the linear fitting process, according to the number of points of the input data, the adjusted angle threshold, and the distance threshold, the structural element points are divided into different linear primitives. For the expansion process, the Kippi algorithm is used to convert the linear primitives into dynamic data structures, and then the two ends are extended so that they intersect each other to form a closed polygon.

[0049] Specifically, in one implementation manner of this embodiment, step S400 includes the following steps: Step S401, based on the linear fitting method, according to the number of points of the input point cloud data, the adjusted angle threshold, and the distance threshold, divide the extracted structural element points into different linear primitives; Step S402, based on the preset image processing algorithm, convert each linear primitive into a dynamic data structure, and extend the two ends of the dynamic data structure so that the dynamic data structures intersect each other to form a closed polygon, and obtain the floor plan of the indoor scene.

[0050] In this embodiment, after ensuring the accuracy of the main structural element points, the goal is to use these extracted complete structural element points to reconstruct the floor plan of the indoor scene. For this purpose, linear fitting and the Kippi algorithm are used in this embodiment to convert the main structural element points into linear primitives. This process includes: projecting the main structural element points onto a two-dimensional plane, and using the region growing algorithm, combined with parameters such as preset thresholds of angle, distance, number of points, and length, to extract simple linear primitives from the complex point cloud data.

[0051] Specifically, if the distance of a point from the points within any linear set is less than the distance threshold , and at the same time the normal vector difference is less than the angle threshold , it can be classified into this linear set. For the linear sets obtained by region growing, in this embodiment, it is further determined whether the linear sets meet the requirements according to the number of points and the length included in the linear sets. If they meet the requirements, they are retained as linear primitives, otherwise they are eliminated. Next, these retained linear primitives grow step by step from both ends through the Kippi algorithm until they intersect with the boundary or other linear primitives to form a closed polygon. This process effectively organizes the linear primitives into polygons reflecting the structure of the indoor scene, and finally realizes the accurate reconstruction of the floor plan of the indoor scene.

[0052] As Figure 1 shown, the embodiment of the present invention provides a method for reconstructing an indoor floor plan based on scene structure elements of mobile LiDAR data, including the following steps: Step S500: Based on the optimization function of the undirected graph, redundant polygons in the indoor scene floor plan are removed, and the final indoor floor plan is output.

[0053] In this embodiment, for the indoor scene floor plan reconstructed above, by converting the closed polygon into an undirected graph and counting the number of valid points and the degrees of the endpoints of the edges in the graph, redundant edges are removed and the remaining edges are merged to simplify the floor plan, thereby obtaining the final indoor floor plan.

[0054] Specifically, in one implementation manner of this embodiment, step S500 includes the following steps: Step S501: Based on the optimization function number of the undirected graph, the closed polygon in the indoor scene floor plan is converted into an undirected graph, and the number of valid points and the degrees of the endpoints of the edges in the undirected graph are counted. According to the statistical results, redundant edges are removed and the remaining edges are merged.

[0055] In this embodiment, for the reconstructed indoor scene floor plan, each edge S is divided into n segmented points, and the number of LiDAR points (i.e., lidar points) near each segmented point is counted. Among them, the segmented points with the number of points exceeding the preset threshold T are marked as valid points. Further, the degrees of the two endpoints of each edge are counted. Given that the walls in the indoor scene usually appear in the form of pairwise intersections, therefore, the degrees of the endpoints of the edges corresponding to the real walls are usually 2 (i.e., the preset value is 2). If the degree of the endpoint of the edge is not 2, then the edge may be a redundant edge formed during the expansion process. Finally, combining the number of valid points and the degrees of the endpoints of the edge, redundant polygons in the floor plan are accurately removed. The number of its valid points and the degrees of the endpoints are used to accurately remove redundant polygons in the floor plan.

[0056] In one implementation manner of this embodiment, step S501 includes the following steps: Step S501a: Each edge S in the closed polygon is divided into n segmented points, and the number of lidar points near each segmented point is counted. The segmented points with the lidar point number exceeding the preset threshold T are marked as valid points; Step S501b: The degrees of the two endpoints of each edge S in the closed polygon are counted, and it is judged whether the corresponding degree is the preset value; Step S501c: If the corresponding degree is not the preset value, it is determined that the corresponding edge is a redundant edge formed during the expansion process, and combining the number of valid points and the degrees of the endpoints of the corresponding edge, redundant polygons in the indoor scene floor plan are removed.

[0057] In this embodiment, due to the complexity of the indoor scene and data loss, the floor plan reconstructed by the Kippi algorithm may contain redundant polygons. Therefore, this embodiment designs an optimization function based on an undirected graph to remove redundant edges in the floor plan and simplify the remaining edges, ensuring that the final polygon can accurately reflect the indoor scene. The process of the removal step is shown in formula (3): (3); where is an energy function used to determine whether the linear primitive should be removed. This function includes an effective term and a degree term . The definition of the effective term is shown in formula (4) and is used to calculate the proportion of effective points of the linear primitive to be detected in the point cloud. It divides each linear primitive into different segment points according to its length, and the number is represented by ; for each segment point , the number of effective points is calculated through a preset threshold and the indicator function to determine whether the segment point is an effective point. If the proportion of effective points is larger, the energy is smaller. The indicator function is shown in formula (5).

[0058] (4); (5); In an undirected graph, the degree of an endpoint reflects the number of edges connected to that endpoint. Through observation, it is found that in most indoor scenes, walls usually appear in the form of pairwise intersections. Therefore, the degree of the endpoints of the edges corresponding to the linear primitives reflecting real walls in the undirected graph is usually 2. Based on this phenomenon, this embodiment designs the degree term , as shown in formula (6).

[0059] (6); This term measures the intersection situation of the linear primitive S by calculating the degrees of the two endpoints and of the linear primitive S. As the number of intersections of the linear primitive S with other linear primitives increases, the energy will also increase accordingly. If a certain linear primitive S intersects with multiple other primitives , it is more likely to be judged as an overextended wrong primitive rather than a linear primitive reflecting a real wall.

[0060] After removing the wrong edges, the remaining edges will be merged to simplify the floor plan. Specifically, if two edges and If they intersect and the intersection degree is 2, and satisfy the included angle condition in formula (7), they will be regarded as the same edge and merged.

[0061] (7); In this embodiment, a total of 5 indoor scene data from the UZH Rooms detection datasets and actual indoor scene datasets are used for testing. The detailed information of the 5 indoor scene datasets is shown in Table 1 and Figure 2 as shown, Figure 2 The following are the details of the five test scenes: (a) Synth 1; (b) Synth 2; (c) Synth 3; (e) Office 1; (f) Office2. They have different sizes, scene complexities, and point densities, and the main structures in the scenes are all occluded to varying degrees, which makes these scene data contain missing regions of different shapes.

[0062] As Figure 3 shown, Figure 3 The following shows the top view of the superposition comparison between the reconstruction result of this embodiment and the original point cloud data. Among them, Figure 3 the green edges and red nodes in the figure represent the results of the reconstructed floor plan, while the gray points represent the original point cloud data. The floor plan reconstruction results are: (a) Synth 1; (b) Synth 2; (c) Synth 3; (e) Office 1; (f) Office 2. It can be seen that the method proposed in this embodiment can well overcome the interference of indoor furniture and missing regions, and reconstruct a floor plan that fits the indoor scene walls.

[0063] As Figure 4 shown, Figure 4 the following shows the superposition comparison between the scene structure element points extracted through the structure element extraction step in this embodiment and the complete point cloud. Figure 4 In the figure (a), it shows the point cloud result of the indoor scene after deep learning semantic segmentation. Among them, the red represents the main structure element points, the green represents the secondary structure element points, and the blue represents the non-structure element points; Figure 4 In the figure (b), it shows the superposition display of the structure element points extracted through the multi-feature structure element extraction step of the algorithm and the original point cloud data. Among them, the gray points represent the extracted structure element points, and the blue points represent the original point cloud data. In the result of semantic segmentation, although most of the wall, door, and window points are recognized as structure element points, there are still some points that are misrecognized as non-structure element points, as shown in Figure 4 the figure (a). After integrating the multi-feature structure element extraction process, the complete scene structure element points are obtained, as shown in Figure 4As shown in (b) of the figure. In addition, the missing areas caused by scanning errors and furniture occlusion are also reconstructed and restored by the subsequent Kippi algorithm and the undirected graph optimization process.

[0064] Table 1 Details of 5 indoor scene datasets

[0065] In this embodiment, these metrics are divided into two aspects: geometric measurement and room reconstruction. In terms of geometric measurement, this embodiment uses two evaluation metrics, root mean square error (RMSE) and room area difference, to evaluate the reconstruction effect of the model. The root mean square error measures the degree of fit between the reconstructed floor plan and the actual point cloud data on the plane, and its calculation method is shown in formula (8). The calculation method of the room area difference is shown in formula (9).

[0066] (8); (9); In terms of room reconstruction, this embodiment introduces four key metrics: precision, recall, F1-score, and intersection over union (IoU). Precision measures the ratio of the overlapping area between the reconstructed room area and the true room area in the floor plan to the reconstructed room area. Recall reflects the ratio of the total area of all reconstructed rooms in the floor plan to the total area of the indoor scene. The F1-score is a metric used in statistics to measure the precision of a binary model. It takes into account both the precision and recall of the classification model and can be considered a weighted average of the model's precision and recall, with a maximum value of 1 and a minimum value of 0. The higher the F1-score, the better the model performance. The intersection over union measures the degree of overlap between the reconstructed room and the room in the actual scene. Their definitions are shown in formulas (10) to (13): (10); (11); (12); (13); The results of the geometric measurement metrics are shown in Table 2. For the room reconstruction metrics, considering that the two single-room scenarios in the test data do not well reflect the effectiveness of the method in this embodiment, therefore, only the three multi-room scenario data in the UZH Rooms detection datasets are used for testing, and four methods, Mor (morphological segmentation), Dist (distance segmentation), Vor (Voronoi diagram segmentation), and RoomFormer, are used to test the same scenarios for comparison. The precision, recall, F1-score, and intersection over union values of different algorithms are shown in Table 3 and Figure 5 as shown.

[0067] Table 2 Geometric measurement index values of five test scenarios

[0068] Table 3 Comparison of multi-room scenario reconstruction metrics

[0069] As shown in Table 3, the Mor method has a high recall rate but a low precision; the Dist method shows the highest precision but a slightly lower recall rate. The Vor method and the RoomFormer method perform relatively balanced in various metrics. However, the performance stability of these four comparison methods is not as good as the method proposed in this embodiment. The method in this embodiment shows higher robustness and accuracy in different indoor environments by effectively integrating multi-feature structural elements. As Figure 5 shown, Figure 5 further shows the comparison of the reconstruction results between this embodiment and the four comparison methods in three different room shape scenarios. The results show that this embodiment is less affected when dealing with different room shapes and demonstrates significant advantages in room boundary reconstruction and room topology structure preservation.

[0070] This embodiment achieves the following technical effects through the above technical solutions: (1) This embodiment integrates the semantic feature information and geometric feature information of the point cloud to extract structural elements from the indoor scene point cloud data. This enables this embodiment to be independent of additional data inputs such as scan trajectories and prior knowledge such as room height, improving the reconstruction efficiency of the indoor floor plan.

[0071] (2) The dynamic data structure expansion adopted in this embodiment greatly reduces the impact of missing areas caused by cluttered indoor environments and furniture occlusion on the reconstructed floor plan.

[0072] (3) This embodiment adopts an optimization function based on an undirected graph to correct and simplify the reconstructed floor plan, making the reconstruction result more accurate and complete, performing better visually, and improving the reconstruction quality of the indoor floor plan.

[0073] Exemplary device Based on the above embodiments, the present invention also provides a mobile LiDAR data indoor floor plan reconstruction system based on scene structural elements, including: A preprocessing module for preprocessing the original point cloud; A segmentation module for segmenting the preprocessed point cloud data by a semantic segmentation method of deep learning to obtain segmented structural element points; An element extraction module, configured to integrate the semantic information, geometric feature information, and position information of the segmented structural element points, and extract complete structural element points from the scene; A reconstruction module, configured to convert the extracted structural element points into linear primitives using a linear fitting method, and use a preset image processing algorithm to expand them with a dynamic data structure, and reconstruct an indoor scene floor plan in the form of a closed polygon; An optimization module, configured to eliminate redundant polygons in the indoor scene floor plan based on an optimization function of an undirected graph, and output a final indoor floor plan.

[0074] Through the above technical solutions, the present embodiment achieves the following technical effects: (1) The present embodiment integrates the semantic feature information and geometric feature information of the point cloud to extract structural elements from the indoor scene point cloud data. This enables the present embodiment to be independent of additional data inputs, such as prior knowledge of scanning trajectories, room heights, etc., and improves the reconstruction efficiency of the indoor floor plan.

[0075] (2) The dynamic data structure expansion adopted in the present embodiment greatly reduces the impact of missing areas caused by a cluttered indoor environment and furniture occlusion on the reconstructed floor plan.

[0076] (3) The present embodiment adopts an optimization function based on an undirected graph to correct and simplify the reconstructed floor plan, making the reconstruction result more accurate and complete, performing better visually, and improving the reconstruction quality of the indoor floor plan.

[0077] Based on the above embodiments, the present invention further provides a terminal, and its principle block diagram can be as Figure 6 shown.

[0078] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected through a system bus; wherein, the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect external devices; the display screen is used to display corresponding information; the communication module is used to communicate with a cloud server or other devices.

[0079] When the computer program is executed by the processor, it is used to implement the operations of the method for reconstructing an indoor floor plan based on mobile LiDAR data of scene structural elements.

[0080] Those skilled in the art can understand that Figure 6The principle block diagram shown only shows the block diagram of some structures related to the solution of the present invention, and does not constitute a limitation on the terminals to which the solution of the present invention is applied. The specific terminals may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0081] In one embodiment, a terminal is provided, which includes: a processor and a memory. The memory stores a mobile LiDAR data indoor floor plan reconstruction program based on scene structure elements. When the mobile LiDAR data indoor floor plan reconstruction program based on scene structure elements is executed by the processor, it is used to implement the operations of the above-mentioned mobile LiDAR data indoor floor plan reconstruction method based on scene structure elements.

[0082] In one embodiment, a storage medium is provided, which stores a mobile LiDAR data indoor floor plan reconstruction program based on scene structure elements. When the mobile LiDAR data indoor floor plan reconstruction program based on scene structure elements is executed by the processor, it is used to implement the operations of the above-mentioned mobile LiDAR data indoor floor plan reconstruction method based on scene structure elements.

[0083] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the above-mentioned method embodiments. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided by the present invention can include non-volatile and volatile memories.

[0084] In summary, the present invention provides a mobile LiDAR data indoor floor plan reconstruction method based on scene structure elements, including: preprocessing the original point cloud; segmenting the preprocessed point cloud data through a semantic segmentation method of deep learning to obtain segmented structure element points; integrating the semantic information, geometric feature information, and position information of the segmented structure element points to extract complete structure element points from the scene; using a linear fitting method to convert the extracted structure element points into linear primitives, and using a preset image processing algorithm to expand with a dynamic data structure and reconstruct the indoor scene floor plan in the form of a closed polygon; eliminating redundant polygons in the indoor scene floor plan based on an undirected graph optimization function and outputting the final indoor floor plan. The present invention can effectively extract structure elements in a complex indoor scene, generate accurate room floor plans, and improve the reconstruction accuracy and efficiency.

[0085] It should be understood that the application of the present invention is not limited to the above examples. For those of ordinary skill in the art, improvements or transformations can be made according to the above description, and all such improvements and transformations should fall within the protection scope of the appended claims of the present invention.

Claims

1. A method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements, characterized in that: include: Preprocess the original point cloud; The pre-processed point cloud data is segmented by the semantic segmentation method of deep learning to obtain the segmented structural element points; Integrate the semantic information, geometric feature information and position information of the segmented structural element points to extract complete structural element points from the scene; The extracted structural element points are converted into linear primitives using a linear fitting method, and are expanded with a dynamic data structure using a preset image processing algorithm, and the indoor scene plan is reconstructed in the form of a closed polygon; The redundant polygons in the indoor scene plan view are eliminated based on the optimization function of the undirected graph, and the final indoor plan view is output.

2. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 1, characterized in that: The preprocessing of the original point cloud includes: The original point cloud is downsampled and resampled using a moving least squares method to obtain the preprocessed point cloud data.

3. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 1, characterized in that: The semantic segmentation method of deep learning is used to segment the preprocessed point cloud data to obtain the segmented structural element points, including: The preprocessed point cloud data is segmented by the deep learning semantic segmentation method, and the indoor point cloud is segmented into primary structural elements, secondary structural elements and non-structural elements to obtain the segmented structural element points.

4. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 1, characterized in that: The step of integrating the semantic information, geometric feature information, and position information of the segmented structural element points to extract complete structural element points from the scene includes: Analyze the similarity of geometric feature information in the point set corresponding to the main structural elements with semantic information identification and non-structural elements, and identify the structural candidate points; Extracting boundary points of secondary structural elements with semantic information identification, and evaluating the degree of overlap between the boundary points and the structural candidate points on a two-dimensional plane; The complete structural element points are screened out from the structural candidate points according to the evaluated overlap degree.

5. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 1, characterized in that: The method of converting the extracted structural element points into linear primitives by using a linear fitting method, and using a preset image processing algorithm to expand the points with a dynamic data structure, and reconstructing the indoor scene plan in the form of a closed polygon, includes: Based on the linear fitting method, the extracted structural element points are divided into different linear primitives according to the number of points of the input point cloud data, the angle threshold and the distance threshold are adjusted; Based on the preset image processing algorithm, each linear primitive is converted into a dynamic data structure, and the two ends of the dynamic data structure are expanded so that the dynamic data structures intersect with each other to form a closed polygon, thereby obtaining the indoor scene plan view.

6. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 1, characterized in that: The undirected graph-based optimization function removes redundant polygons in the indoor scene plan view, including: Based on the optimization function number of the undirected graph, the closed polygons in the indoor scene plan view are converted into an undirected graph, and the number of valid points and the endpoint degrees of the edges in the undirected graph are counted, and the redundant edges are removed and the remaining edges are merged according to the statistical results.

7. The method for indoor floor plan reconstruction based on mobile LiDAR data based on scene structure elements according to claim 6, characterized in that: The step of converting the closed polygons in the indoor scene plan view into an undirected graph, and performing statistics on the number of valid points and the degree of endpoints of the edges in the undirected graph, and removing redundant edges and merging the remaining edges according to the statistical results, includes: Each side S in the closed polygon is divided into n segmentation points, and the number of laser radar points near each segmentation point is counted, and the segmentation points whose number of laser radar points exceeds a preset threshold T are marked as valid points; Counting the degrees of the two endpoints of each side S in the closed polygon, and determining whether the corresponding degree is a preset value; If the corresponding degree is not the preset value, the corresponding edge is determined to be a redundant edge formed during the expansion process, and the redundant polygons in the indoor scene plan are removed in combination with the number of valid points and the endpoint degree of the corresponding edge.

8. A mobile LiDAR data indoor floor plan reconstruction system based on scene structure elements, characterized in that: include: Preprocessing module, used to preprocess the original point cloud; The segmentation module is used to segment the preprocessed point cloud data through the semantic segmentation method of deep learning to obtain the segmented structural element points; An element extraction module is used to integrate the semantic information, geometric feature information and position information of the segmented structural element points to extract complete structural element points from the scene; A reconstruction module, used to convert the extracted structural element points into linear primitives using a linear fitting method, and to expand them with a dynamic data structure using a preset image processing algorithm, and to reconstruct a plan view of the indoor scene in the form of a closed polygon; The optimization module is used to remove redundant polygons in the indoor scene plan view based on an optimization function of an undirected graph, and output a final indoor plan view.

9. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a program for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements, and when the program for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements is executed by the processor, it is used to implement the operation of the method for reconstructing an indoor floor plan from mobile LiDAR data based on scene structure elements as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program for reconstructing indoor floor plans from mobile LiDAR data based on scene structure elements, and when the program for reconstructing indoor floor plans from mobile LiDAR data based on scene structure elements is executed by a processor, it is used to implement the operation of the method for reconstructing indoor floor plans from mobile LiDAR data based on scene structure elements as described in any one of claims 1 to 7.

Images