Indoor scene floor plan generation method guided by 3D point cloud semantics

Abstract

The present invention discloses a method for generating indoor scene floor plans guided by three-dimensional point cloud semantics, comprising: using a point cloud semantic segmentation network to divide indoor point clouds to obtain an indoor building main structure image; performing spatial segmentation on the building main structure image to obtain an indoor segmented structure image; performing house optimization on the indoor segmented structure image to obtain a preliminary optimized structure image; performing house boundary optimization on the preliminary optimized structure image to obtain an indoor scene floor plan. The present invention can not only greatly save the manpower and time costs of generating indoor floor plans, but also is insensitive to the missing of local point cloud data and the change of point cloud density, and can realize floor plan modeling of large-scale indoor scenes.

Description

Technical Field

[0001] The present invention belongs to the fields of computer graphics and computer-aided design, and in particular relates to a method for generating indoor scene plan views guided by three-dimensional point cloud semantics. Background Art

[0002] With the increasing demand for interior space design, indoor floor plan generation technology has attracted widespread attention as an intelligent solution. This technology is a key research direction in computer science and engineering. It aims to automatically identify objects and structures in indoor scenes and generate corresponding floor plans using techniques such as computer vision, computer graphics, and artificial intelligence. This technology not only improves interior design efficiency but also provides users with more intuitive and visual design solutions. Indoor floor plans generated by house segmentation from raw 3D point clouds are crucial for understanding and reconstructing indoor scenes. The main challenges lie in the complexity and variability of indoor environments, as well as the presence of noisy data, which makes it difficult to restore indoor structures to their exact locations. However, currently, the industry still relies entirely or partially on manual or interactive methods to generate high-quality floor plans. This labor-intensive, resource-intensive, tedious, and error-prone reconstruction method cannot effectively handle large datasets of complex indoor scenes.

[0003] Recent research reveals that mainstream indoor floor plan generation techniques include connectivity-based methods, spatial segmentation-based methods, and grammar-based methods. Connectivity-based methods connect detected isolated structural elements and, using topological rules, form a closed room boundary. However, this method relies on a post-processing step for the room boundary, which can lead to misalignment between the final floor plan and wall locations. Furthermore, due to its over-reliance on resolution, some structural details in the boundary shape may be overlooked. Spatial segmentation-based methods use geometric constraints or specific semantics to divide the indoor scene into separate functional units, over-segmenting the 2D space into spatial segments. Units with the same room instance label are then merged using connectivity rules. However, these methods are susceptible to noise points between adjacent rooms or objects such as furniture within the room, and the final segmentation performance depends on the quality of the spatial segments. Grammar-based methods utilize grammar rules to solve the indoor floor plan reconstruction task. A traditional solution is to build a structural graph for each element and then segment the room using heuristic methods. Connectivity information between rooms is also considered for segmentation. However, this approach is only applicable to applications in Manhattan world scenes.

[0004] In recent years, advances in laser radar (LiDAR) and photogrammetry technologies have enabled the efficient and real-time acquisition of indoor 3D spatial information. Both technologies can generate massive amounts (tens of millions) of discrete 3D point cloud data, enabling realistic representations of indoor 3D scenes. However, this massive amount of discrete 3D point cloud data is unstructured, posing a significant challenge to the automatic generation of indoor scene floor plans from 3D point clouds. Therefore, methods for generating indoor scene floor plans guided by the semantics of 3D point clouds are urgently needed. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a method for generating indoor scene floor plans guided by three-dimensional point cloud semantics, which can improve the automation and intelligence level of indoor floor plan generation.

[0006] To achieve the above objectives, the present invention provides a method for generating a floor plan of an indoor scene using 3D point cloud semantics guidance, comprising:

[0007] Use the point cloud semantic segmentation network to divide the indoor point cloud and obtain the main structure image of the indoor building;

[0008] Performing spatial segmentation on the main structure image of the building to obtain an indoor segmented structure image;

[0009] Performing house optimization on the indoor segmented structure image to obtain a preliminary optimized structure image;

[0010] The house boundary is optimized on the preliminary optimized structure image to obtain an indoor scene plan.

[0011] Optionally, spatially segmenting the main structure image of the building to obtain the segmented indoor structure image includes:

[0012] The three-dimensional point cloud data corresponding to the main body of the building is projected onto a two-dimensional raster image, and the distance from the main structure image of the building to the nearest boundary is calculated to obtain a distance transformation map;

[0013] The watershed algorithm is used to segment the distance transform map to obtain different areas.

[0014] Optionally, performing house optimization on the indoor segmented structure image to obtain a preliminary optimized structure image includes:

[0015] The indoor segmentation structure image is represented as an undirected graph using a Markov random field method, wherein the undirected graph includes: a preset number of spatial units and an undirected set;

[0016] The undirected graph is optimized using a maximum flow minimum cut algorithm to obtain a preliminary optimized structural image.

[0017] Optionally, the non-edge-directed set includes: edges connecting source points and sink points of spatial units, and edges connecting two spatial units.

[0018] Optionally, optimizing the undirected graph using a maximum flow minimum cut algorithm includes:

[0019]

[0020] Among them, E(L) is the total energy size in the maximum flow minimum cut algorithm, E data (l k ) represents a data item, E smooth (l k ,l j ) represents the smoothing term, λ represents the weight coefficient between the two, m is the number of rooms, k is the index of the currently selected room, j is the index value of the room adjacent to the currently selected room, l k is the label value corresponding to the currently selected room index, l j The label value corresponding to the room index adjacent to the currently selected room.

[0021] Optionally, optimizing the house boundary of the preliminary optimized structure image to obtain the indoor scene plan view includes: recursively searching for the optimal structure of the house boundary using a depth-first search to obtain the indoor scene plan view.

[0022] Optionally, recursively searching for the optimal structure of the house boundary using depth-first search includes:

[0023] Selecting a starting point and an ending point on the room boundary, determining an initial direction, and searching along the initial direction to obtain a search path;

[0024] The search path is judged until the end point is found, the energy of the search path is calculated, and the path with the minimum energy is selected as the optimal structure.

[0025] Optionally, judging the search path until the end point is found includes:

[0026] If the pixel point in the search path is a room label point and a house boundary point, continue searching along the initial direction;

[0027] If a pixel point in the search path is a blank point, two paths are generated: the first path continues searching along the initial direction, and the second path determines the search direction based on the boundary points adjacent to the blank point that have not been searched, and continues searching;

[0028] If the pixel points in the search path exceed the maximum boundary range of the room, the direction is changed and the search is continued.

[0029] Optionally, the method for calculating the energy of the search path is:

[0030] U(R i )=U corner (R i )+β*U penalty (R i )

[0031] Among them, U(R i ) is the energy size corresponding to the room boundary path, R i is the boundary corresponding to the selected room, U corner (R i ) refers to the room corner point, U penalty (R i ) refers to the room penalty term, and β refers to the weight coefficient that balances the two.

[0032] Compared with the prior art, the present invention has the following advantages and technical effects:

[0033] This method not only significantly reduces the labor and time required to generate indoor floor plans, but is also insensitive to missing local point cloud data and variations in point cloud density, enabling floor plan modeling of large-scale indoor scenes. The research findings of this invention provide a novel and efficient method for indoor floor plan generation and offer a valuable reference for the 3D modeling and visualization of indoor scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0035] Figure 1 This is a flow chart of a method for generating a floor plan of an indoor scene guided by three-dimensional point cloud semantics according to an embodiment of the present invention;

[0036] Figure 2 A projection diagram of a building main body according to an embodiment of the present invention;

[0037] Figure 3 is a distance transformation diagram according to an embodiment of the present invention;

[0038] Figure 4 This is a diagram showing the spatial segmentation results of an embodiment of the present invention;

[0039] Figure 5 This is a Markov random field optimization result diagram according to an embodiment of the present invention;

[0040] Figure 6 This is a diagram of room boundary optimization results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0042] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0043] The present invention provides a method for generating indoor scene plan graphs guided by 3D point cloud semantics. Figure 1 Shown, including:

[0044] Use the point cloud semantic segmentation network to divide the indoor point cloud and obtain the main structure image of the indoor building;

[0045] Specifically, the point cloud semantic segmentation network is used to classify indoor point clouds into four categories: floor, ceiling, vertical objects, and others;

[0046] Performing spatial segmentation on the main structure image of the building to obtain an indoor segmented structure image;

[0047] Optimizing the indoor segmented structure image using a Markov random field method to obtain a preliminary optimized structure image;

[0048] The house boundary is optimized on the preliminary optimized structure image to obtain an indoor scene plan.

[0049] Furthermore, spatially segmenting the main structure image of the building to obtain the segmented indoor structure image includes:

[0050] The 3D point cloud data corresponding to the building body is projected onto the 2D raster image ( Figure 2 ), calculate the distance from the main structure image of the building to the nearest boundary and obtain the distance transformation map;

[0051] The watershed algorithm is used to segment the distance transform map and obtain the boundaries of different regions.

[0052] Specifically, in room segmentation, the most representative indoor structures are vertical objects, primarily walls and doors. They separate interior and exterior spaces and demarcate adjacent rooms. Since the original 3D point cloud lacks semantic information, a point cloud semantic segmentation network is used to classify the point cloud into four categories: floor, ceiling, vertical objects, and others. Vertical objects are crucial for the subsequent spatial segmentation in this invention.

[0053] The watershed algorithm is an effective image segmentation method that treats an image as a terrain, where the grayscale value of a pixel represents the altitude. The image is segmented into different regions by finding the lowest point of the pixel on the gradient image. These lowest points are called watersheds, and they define the boundaries of different regions in the image. Distance Transformation Map ( Figure 3 ) provides grayscale values ​​for watershed segmentation by calculating the distance from the pixel to the nearest boundary. Finally, combining the distance transform map with the watershed algorithm can achieve the segmentation of the indoor space into multiple basic room units, such as Figure 4 shown.

[0054] Furthermore, the Markov random field method is used to optimize the indoor segmented structure image, and obtaining a preliminary optimized structure image includes:

[0055] The indoor segmentation structure image is represented as an undirected graph using a Markov random field method, wherein the undirected graph includes: a preset number of spatial units and an undirected set;

[0056] The undirected graph is optimized using the Max Flow Min Cut algorithm to obtain a preliminary optimized structural image. The Max Flow Min Cut algorithm optimizes the over-segmentation problem of the house. If there are walls between adjacent rooms, the smoothing term value will increase. If there are no walls between adjacent rooms, the smoothing term will decrease, thereby merging over-segmented rooms in the Max Flow Min Cut algorithm.

[0057] Specifically, Markov random fields are a graphical model used to model joint probability distributions. They are often used to describe the relationship between multiple variables and to process data with complex spatial structures. The Markov model can be represented as an undirected graph G =<V,E> , where V represents each spatial unit after pre-division, and E represents a set of undirected edges. The undirected edge set consists of two parts: the edges connecting the source points and the sink points, and the edges connecting the two spatial units.

[0058] The room optimization problem can be described as a multi-category label optimization problem. Given a series of spatial subdivision units F = {f k |1≤k≤m}, label the room instance L={l k |1≤k≤m} is assigned to F. In the field of computer vision, this problem can be converted into energy function minimization (as shown in formula (1)), and solved by the maximum flow minimum cut algorithm to obtain the Markov random field optimization result graph, as shown in Figure 5 shown.

[0059]

[0060] Among them, E(L) is the total energy size in the maximum flow minimum cut algorithm, E data (l k) represents a data item, E smooth (l k ,l j ) represents the smoothing term, λ represents the weight coefficient between the two, m is the number of rooms, k is the index of the currently selected room, j is the index value of the room adjacent to the currently selected room, l k is the label value corresponding to the currently selected room index, l j The label value corresponding to the room index adjacent to the currently selected room.

[0061] Data item: The data item is used to measure the degree of adaptation of a housing space unit to its corresponding label category. Specifically, it can be expressed as formula (2)

[0062]

[0063] Among them, B k Indicates the number of boundary points of the room, A k Indicates the number of adjacent points between this room and other rooms.

[0064] Smooth term: The smooth term is used to separate or merge two adjacent housing space units. When there is no wall between two adjacent space units, they should have the same label value; when there is a wall between the space units, the space units should have different label values. Therefore, the smooth term can be defined as:

[0065]

[0066] Among them, |e k ∩e j |Indicates the length of the wall between two adjacent housing units.

[0067] Furthermore, optimizing the house boundary of the preliminary optimized structure image to obtain the indoor scene plan view includes: recursively searching for the optimal structure of the house boundary using a depth-first search to obtain the indoor scene plan view.

[0068] Specifically, by taking advantage of the fact that the room topology is a simple cycle, the search for the optimal boundary of the room can be transformed into an energy minimization problem, and the optimal structure of the house boundary can be searched recursively through a depth-first search ( Figure 6 ). The specific energy function can be expressed as formula (4):

[0069] U(R i )=U corner (R i )+β*U penalty (R i )(4)

[0070] Among them, Ucorner (R i ) refers to the room corner point, U penalty (R i ) refers to the room penalty term, and β refers to the weight coefficient that balances the two.

[0071] Room corner item: The number of corners in a room boundary largely determines the complexity of the room boundary. corner (R i ) is used to measure the optimized room R i The number of corner points to include.

[0072] Room Penalty Term: The goal of this invention is to change as few points as possible to make the room boundary as regular as possible. Therefore, this invention uses the changed blank points as penalty terms, specifically:

[0073]

[0074] Among them, p refers to the pixels within the room, R i ' refers to the room range before optimization, and V(p) is used to measure the penalty value of pixels at different positions, which can be expressed as:

[0075]

[0076] Labelmap(p) is the floor plan generated by projecting the original point cloud onto the XY plane. If a grid cell contains no point cloud, its value is set to 0. Considering that some rooms may have missing points due to uneven point cloud collection, areas without point clouds may represent residential areas. Therefore, in subsequent searches, these areas without value should also be searched, but the penalty term for the corresponding pixels should be larger. Specifically, μ1 > μ2.

[0077] Search strategy: The specific search steps are as follows:

[0078] (1) First, select the starting point and the ending point on the room boundary.

[0079] (2) Determine the initial direction and search forward along the initial direction.

[0080] (3) Judgment during each search:

[0081] If the pixel is a room label point and a house boundary point, continue searching forward.

[0082] If the pixel is a blank point, two paths are generated. The first path continues searching along the direction, while the second path recalculates the direction based on the neighboring boundary points that have not been searched and continues searching.

[0083] If the point is beyond the maximum boundary of the room, change direction and continue searching.

[0084] (4) If the search reaches the end point, calculate the energy of the search path and determine the optimal solution for the house boundary based on the energy.

[0085] The above are merely preferred embodiments of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for generating indoor scene floor plans guided by 3D point cloud semantics, characterized in that: include: Use the point cloud semantic segmentation network to divide the indoor point cloud and obtain the main structure image of the indoor building; Performing spatial segmentation on the main structure image of the building to obtain an indoor segmented structure image; Performing house optimization on the indoor segmented structure image to obtain a preliminary optimized structure image; Performing house optimization on the indoor segmented structure image to obtain a preliminary optimized structure image includes: The indoor segmentation structure image is represented as an undirected graph using a Markov random field method, wherein the undirected graph includes: a preset number of spatial units and an undirected set; The undirected graph is optimized using a maximum flow minimum cut algorithm to obtain a preliminary optimized structural image; Optimizing the undirected graph using the maximum flow minimum cut algorithm includes: Among them, E(L) is the total energy size in the maximum flow minimum cut algorithm, E data (l k ) represents a data item, E smooth (l k ,l j ) represents the smoothing term, λ represents the weight coefficient between the two, m is the number of rooms, k is the index of the currently selected room, j is the index of the room adjacent to the currently selected room, l k is the label value corresponding to the currently selected room index, l j The label value corresponding to the room index adjacent to the currently selected room; Optimizing the house boundary of the preliminary optimized structure image to obtain an indoor scene plan; Optimizing the house boundary of the preliminary optimized structure image to obtain the indoor scene plan view includes: recursively searching for the optimal structure of the house boundary using a depth-first search to obtain the indoor scene plan view; The optimal structure for searching the house boundary recursively using depth-first search includes: Select a starting point and an end point on the room boundary, determine an initial direction, search along the initial direction, and obtain a search path; The search path is judged until the end point is found, the energy of the search path is calculated, and the path with the minimum energy is selected as the optimal structure.

2. The method for generating indoor scene plan graphs guided by 3D point cloud semantics according to claim 1, characterized in that: The main structure image of the building is spatially segmented to obtain the indoor segmented structure image, which includes: The three-dimensional point cloud data corresponding to the main body of the building is projected onto a two-dimensional grid image, and the distance from the main structure image of the building to the nearest boundary is calculated to obtain a distance transformation map; The watershed algorithm is used to segment the distance transform map to obtain different areas.

3. The method for generating indoor scene plan graphs guided by 3D point cloud semantics according to claim 1, characterized in that: The non-edge-directed set includes: edges of spatial units connecting source points and sink points, and edges connecting two spatial units.

4. The method for generating indoor scene plan graphs guided by 3D point cloud semantics according to claim 1, characterized in that: Judging the search path until the end point is found includes: If the pixel point in the search path is a room label point and a house boundary point, continue searching along the initial direction; If a pixel point in the search path is a blank point, two paths will be generated: the first path will continue searching along the initial direction, and the second path will determine the search direction according to the boundary points adjacent to the blank point and not searched, and continue searching; If the pixel points in the search path exceed the maximum boundary range of the room, the direction is changed and the search is continued.

5. The method for generating indoor scene plan graph guided by 3D point cloud semantics according to claim 1, characterized in that: The method for calculating the energy size of the search path is: U(R i )=U corner (R i )+β*U penalty (R i ) Among them, U(R i ) is the energy size corresponding to the room boundary path, R i is the boundary corresponding to the selected room, U corner (R i ) refers to the room corner point, U penalty (R i ) refers to the room penalty term, and β refers to the weight coefficient between the two.

Images