House type drawing reconstruction method and medium based on multi-modal semantic fusion and topology optimization

Abstract

This invention discloses a method and medium for reconstructing floor plans based on multimodal semantic fusion and topology optimization. The method includes: generating pixel-level wall masks from the original floor plan image and obtaining single-pixel-width geometric skeleton lines based on the wall masks; constructing an initial semantically null room topology network; generating a set of semantic bounding boxes composed of spatial location bounding boxes corresponding to all rooms; identifying topologically missing regions and obtaining the repaired complete floor plan topology; and fusing the repaired complete floor plan topology with room function categories to generate a vectorized floor plan model containing semantic attributes and geometric data. This floor plan reconstruction method effectively solves the problems of wall breaks and missed room detection caused by occlusion or noise in complex floor plans, significantly improving the topological closure rate and semantic recognition accuracy of the reconstructed floor plan.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent building design technology, specifically to a method and medium for reconstructing floor plans based on multimodal semantic fusion and topology optimization. Background Technology

[0002] Currently, in the fields of architectural decoration and real estate marketing, digital floor plan reconstruction is a crucial preliminary step in realizing the transformation from two-dimensional floor plans to three-dimensional virtual tours (VR property viewing), intelligent interior design layouts, and BIM (Building Information Modeling) modeling. If the original floor plan can be automatically converted into precise vectorized data, it will significantly reduce the cost of base map creation for designers and improve the efficiency of the entire design and delivery process.

[0003] However, existing floor plan vector reconstruction methods still face significant bottlenecks in accuracy under complex scenarios. Traditional image processing-based methods (such as Canny edge detection) heavily rely on edge detection and are easily affected by furniture textures and watermarks, resulting in fragmented wall lines that are difficult to form closed room areas. While existing deep learning-based segmentation methods can identify most wall pixels, they are prone to topological breaks or missed detections of the entire room when dealing with irregular floor plans or areas obscured by furniture.

[0004] Extensive practical experience has shown that floor plan reconstruction methods often result in incomplete or missing rooms and walls, requiring designers to spend considerable time manually verifying and supplementing the plans. This prevents the achievement of a truly automated "upload and use" closed loop and fails to meet the demands of large-scale industrial production. In particular, existing technologies often treat geometric reconstruction and semantic recognition as separate, sequential processes, lacking deep information interaction and collaboration. This makes it impossible to leverage high-level semantic information to reverse-check and repair underlying geometric defects.

[0005] Solving these problems is now a top priority. Summary of the Invention

[0006] To address the above technical problems, this invention provides a method and medium for reconstructing floor plans based on multimodal semantic fusion and topology optimization. This method can effectively solve the problems of wall topology breakage and room omission in complex floor plans when they are obscured by furniture, interfered with by image noise, etc., and improve the topological integrity and semantic recognition accuracy of vectorized floor plan reconstruction.

[0007] The technical solution is as follows:

[0008] The first aspect of this application relates to a method for reconstructing floor plans based on multimodal semantic fusion and topology optimization, comprising the following steps:

[0009] S1. Based on the original floor plan image, a pixel-level wall mask is generated through a deep semantic segmentation network, and a geometric skeleton line with a single pixel width is obtained based on the wall mask.

[0010] S2. Use the ELSED algorithm to detect the discrete line segment set in the geometric skeleton lines, and perform orthogonalization correction and loop closure search on the discrete line segment set based on the Manhattan rule to construct the initial semantic-free room topology network.

[0011] S3. Input the original floor plan image into the trained target detection network to identify the room function categories and generate a semantic bounding box set consisting of the spatial location bounding boxes of all rooms.

[0012] S4. Construct a geometric-semantic consistency verification mechanism, perform integrity verification on the initial semantically unrelated room topology network based on the spatial location bounding box, identify the topology missing regions, and for the topology missing regions, perform adaptive reconstruction and repair based on local geometric clues within the corresponding spatial location bounding box constraints to obtain the repaired complete apartment topology structure.

[0013] S5. Integrate the repaired complete apartment topology and room function categories to generate a vectorized apartment model containing semantic attributes and geometric data.

[0014] The above-mentioned floor plan reconstruction method based on multimodal semantic fusion and topology optimization is adopted. By integrating the semantic perception capability of deep learning with the topology construction capability of traditional geometric algorithms, a dual-stream parallel processing architecture is constructed. Extensive experiments have verified that it has the following core beneficial effects:

[0015] 1. Significantly improved accuracy: This method achieves a room recall rate of 96.8%, a topological closure rate of 99.2%, and a wall intersection-union ratio of 0.94. Compared with the traditional LSD + rule method, the room recall rate is improved by 31.6 percentage points and the topological closure rate is improved by 28.4 percentage points; compared with the pure deep learning segmentation method, the room recall rate is improved by 14.3 percentage points and the topological closure rate is improved by 15.1 percentage points, completely solving the core problem of "missing rooms and walls".

[0016] 2. A deep semantic-geometry coupled collaborative closed loop was constructed: breaking the limitation of semantic recognition as only a post-processing step in traditional methods, this approach innovatively uses YOLO semantic detection boxes as strong spatial constraints for geometric topology repair. When low-level geometric features are broken due to occlusion or noise, high-level semantic information guides geometric reconstruction, forming a virtuous cycle of "semantics guiding geometry, and geometry refining semantics," effectively solving the problem of topological breaks caused by the loss of local features in complex scenarios.

[0017] 3. A layered and progressive adaptive repair strategy is proposed: A layered strategy of "prioritizing local clue stitching and supplementing with overall bounding box as a fallback" is adopted. First, within the constraints of the semantic box, a fine-grained stitching of the freeline segments is performed adaptively to preserve the true geometric details to the greatest extent. Only in cases of extreme missing elements is the bounding box used as a topological replacement. This strategy ensures both the accuracy of the reconstruction and the extremely high robustness of the system, achieving high-precision reconstruction even in complex scenes such as severe occlusion, weak textures, and broken walls.

[0018] 4. Balancing accuracy and engineering processing efficiency: The vectorization time of this method is only 1.0s, which is shorter than the 1.2s time of pure deep learning segmentation methods and far lower than the time cost of manual drawing. The output standardized vectorized house model can be directly read by downstream design software, realizing an automated closed loop of "upload and use" and meeting the needs of large-scale industrial production.

[0019] 5. The effectiveness of the core modules has been empirically verified: The addition of the geometric-semantic consistency verification and local adaptive repair modules has improved the topological integrity from 76.5% to 98.5% and the semantic accuracy from 88.0% to 97.8%, proving the practical application value of the core innovation of this invention.

[0020] A second aspect of this application relates to an apparatus for implementing the above-described floor plan reconstruction method, comprising:

[0021] The geometry processing module is used to first generate pixel-level wall masks based on the original floor plan image through a deep semantic segmentation network, and then obtain geometric skeleton lines with a single pixel width based on the wall masks; then, it uses the ELSED algorithm to detect the discrete line segment set in the geometric skeleton line, and performs orthogonalization correction and loop closure search on the discrete line segment set based on the Manhattan rule to construct an initial semantic-free room topology network.

[0022] The semantic perception module is used to input the original floor plan image into a trained target detection network, identify the room function categories, and generate a set of semantic bounding boxes consisting of the spatial location bounding boxes of all rooms.

[0023] The consistency verification module is used to construct a geometric-semantic consistency verification mechanism, which verifies the integrity of the initial semantically insensitive room topology network based on the spatial location bounding box and identifies the missing topology regions.

[0024] An adaptive repair module is used to perform adaptive reconstruction and repair based on local geometric clues within the corresponding spatial location bounding box constraint range for the missing topology region, so as to obtain the repaired complete house topology structure.

[0025] The data fusion output module is used to fuse the repaired complete apartment topology and room function categories to generate a vectorized apartment model containing semantic attributes and geometric data.

[0026] Using the above-mentioned device, all the advantages of the floor plan reconstruction method described above are achieved.

[0027] A third aspect of this application relates to a computer-readable medium storing computer instructions that are executed by at least one processor to implement the above-described floor plan reconstruction method.

[0028] Using the above computer-readable media, all the advantages of the above-mentioned floor plan reconstruction methods are achieved. Attached Figure Description

[0029] Figure 1 The flowchart shows a floor plan reconstruction method based on multimodal semantic fusion and topology optimization.

[0030] Figure 2 This is a schematic diagram illustrating the effects of semantic segmentation and geometric topology construction. Detailed Implementation

[0031] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0032] like Figure 1 and Figure 2 As shown, a method for reconstructing floor plans based on multimodal semantic fusion and topology optimization includes the following steps:

[0033] S1. Based on the original floor plan image, a pixel-level wall mask is generated through a deep semantic segmentation network, and geometric skeleton lines with a single pixel width are obtained based on the wall mask.

[0034] Specifically, step S1 is performed according to the following steps:

[0035] S11. Input the original floor plan image into the pre-trained panoramic segmentation network to obtain multi-channel probability feature maps containing wall, door, and window categories.

[0036] In this embodiment, a panoramic segmentation network based on the Mask2Former architecture is constructed, and the Swin-Transformer is used as the backbone network to extract image features. The size is... Original floor plan image The input to the panorama segmentation network is processed by a pixel decoder and a transformer decoder, and the output is a multi-channel probability prediction map containing categories for walls, doors, windows, and background. .

[0037] Multi-channel probability prediction graph Each pixel in Category The probability is expressed as ,in, This indicates the total number of categories.

[0038] S12, Multi-channel probability prediction chart Threshold segmentation and morphological denoising are performed to extract a binarized wall mask that contains only the wall region.

[0039] In this embodiment, the "Wall" category is selected (let the index of the "Wall" category be ). The probability prediction graph is generated by setting a confidence threshold. (For example, 0.5) is binarized to generate the initial mask. :

[0040] ;

[0041] In the above formula, This indicates that the coordinates in the original floor plan image are ( The pixels belong to the "wall" category ( The predicted probability value of ).

[0042] Like noise and internal holes, for the initial mask Perform morphological closing to obtain a continuous and smooth binarized wall mask. .

[0043] The closing operation is defined as dilation followed by erosion:

[0044] ;

[0045] In the above formula, express Rectangular structural elements, This indicates an expansion operation. It indicates corrosion.

[0046] S13. Perform distance transformation and skeleton refinement operations on the binarized wall mask to obtain geometric skeleton lines with a single pixel width.

[0047] In this embodiment, the Zhang-Suen Parallel Thinning Algorithm is used to binarize the wall mask. Iterative erosion is performed. This algorithm removes edge pixels that satisfy specific connectivity conditions through two sub-iterations until the image converges to a centerline with a width of one pixel.

[0048] remember For pixels The geometric skeleton lines of the 8-neighbor pixel set, with a refined single-pixel width. satisfy:

[0049] ;

[0050] In the above formula, This represents the iterative erosion operation of the Zhang-Sun parallel thinning algorithm.

[0051] The final geometric skeleton lines with a width of one pixel are obtained. This is a binary image that retains only the geometric topology of the wall, where points with a pixel value of 1 constitute the geometric central axis of the wall, which is used for subsequent vectorization processing.

[0052] S2. The ELSED algorithm is used to detect the discrete line segment set in the geometric skeleton lines, and the discrete line segment set is orthogonalized and loop-closed based on the Manhattan rule to construct the initial semantic-free room topology network.

[0053] Specifically, step S2 is performed according to the following steps:

[0054] S21. Input the geometric skeleton lines with a single pixel width into the ELSED algorithm and output a discrete set of line segments.

[0055] In this embodiment, the geometric skeleton lines with a single pixel width obtained in step S13 are... The input is processed by the ELSED (Enhanced Line Segment Detector) algorithm. The ELSED algorithm extracts a discrete set of line segments by analyzing the consistency of the gradient direction in the image.

[0056] Output discrete set of line segments , among which, the line segment From the starting point and the end point Sure.

[0057] S22. To conform to the orthogonal characteristics of building unit types, the Manhattan hypothesis is used to analyze discrete sets of line segments. Perform orthogonalization correction (geometric correction) by following these steps:

[0058] S221. Calculate the discrete set of line segments. The direction angle of each line segment, for example, for the first... line segment Its direction angle :

[0059] ;

[0060] S222. Based on the relationship between the angle between the direction angle of each line segment and the horizontal or vertical axis and the preset tolerance threshold, each line segment is corrected to a horizontal or vertical direction, and a set of corrected line segments is obtained.

[0061] In this embodiment, an angle tolerance threshold is set. (This embodiment takes) Update the endpoint coordinates of the line segment according to the following rules:

[0062] Horizontal correction: If or If so, then the line segment is determined to be a horizontal wall.

[0063] ;

[0064] In the above formula, Indicates the first The starting ordinate of the line segment after correction. Indicates the first The ordinate of the endpoint of the line segment after correction.

[0065] Vertical correction: If If the x-coordinate is perpendicular to the wall, then the x-coordinate is determined to be the average of the two endpoints.

[0066] ;

[0067] In the above formula, Indicates the first The x-coordinate of the starting point of the line segment after correction. Indicates the first The x-coordinate of the endpoint of the line segment after correction.

[0068] For line segments that do not meet the above conditions, retain their original coordinates or remove them to obtain the corrected set of line segments. .

[0069] S23. Use the minimum closed loop search algorithm to traverse the corrected set of line segments, construct a set of candidate rooms composed of closed polygons, and define the set of candidate rooms as the initial semantically non-semantic room topology network.

[0070] In this embodiment, the corrected set of line segments Transform into an undirected planar graph ,in, Represents the set (nodes) of all line segments. Represents the set of line segments (edges). The Minimum Cycle Basis algorithm is used in undirected planar graphs. The search algorithm finds the minimum set of closed paths, and the polygon enclosed by each closed path is considered a candidate room. The initial semantically non-semantic room topology network is then used. Defined as the set of these closed polygons:

[0071] ;

[0072] In the above formula, Each represents a candidate room. A candidate room in the representation, which is a simple polygon consisting of an ordered sequence of vertices, represents an independent geometrically closed region.

[0073] S3. Input the original floor plan image into the trained object detection network to identify the room function categories and generate a set of semantic bounding boxes consisting of the spatial location bounding boxes of all rooms.

[0074] Specifically, step S3 is performed according to the following steps:

[0075] S31. A deep neural network based on the YOLO architecture (YOLOv11 in this embodiment) is used as the trained object detection network, and the original floor plan image from step S11 is processed. The input is fed into this deep neural network.

[0076] S32. Using a forward inference and boundary phase regression network, the backbone and head of the feature extraction are used to output candidate prediction boxes for each room.

[0077] For the The output vector of each detected target object. Defined as:

[0078] ;

[0079] In the above formula, The bounding box represents the spatial location of the room. and The pixel coordinates of the top-left corner of the bounding box representing the spatial location of the room. and The pixel coordinates of the bottom right corner of the room's spatial location bounding box; This indicates the confidence score (Objectness Score) that the candidate box belongs to a certain object. This represents the category probability vector.

[0080] S33. Room function category determination and post-processing.

[0081] In this embodiment, candidate prediction boxes for each room are... Decode the function to determine the room's functional category. The formula is as follows:

[0082] ;

[0083] In the above formula, This indicates taking the category index that results in the largest value within the parentheses. , This indicates that if the bounding box already contains an object, then the object belongs to a specific category. The conditional probabilities are given by Classes = {Bedroom, Living Room, Kitchen, Bathroom, ...}, where Bedroom represents the bedroom, Living Room represents the living room, Kitchen represents the kitchen, and Bathroom represents the bathroom. To remove redundant overlapping boxes from the same room, a non-maximum suppression (NMS) algorithm is performed. For any two candidate predicted boxes of the same class... and Calculate its intersection-union ratio (LOU):

[0084] ;

[0085] In the above formula, This indicates the calculation of the area of ​​the geometric region within the parentheses. Represents two candidate prediction boxes and The overlapping area Represents two candidate prediction boxes and The total area covered.

[0086] Set the poise value for the nonmaximum suppression algorithm (Typically 0.45) and confidence threshold (Usually taken as 0.25). If the crossover ratio is... Then, the boxes with lower scores are suppressed.

[0087] This generates a set of semantic bounding boxes consisting of the spatial location bounding boxes corresponding to all rooms. :

[0088] .

[0089] S4. Construct a geometric-semantic consistency verification mechanism. Based on spatial location bounding boxes, verify the integrity of the initial semantically insensitive room topology network, identify missing topology regions, and for these regions, perform adaptive reconstruction and repair based on local geometric cues within the constraints of the corresponding spatial location bounding boxes to obtain the repaired complete room topology structure. This step aims to solve the "room omission" problem caused by geometric topology breaks, and includes two stages: consistency verification and adaptive repair.

[0090] Specifically, step S4 identifies the topologically missing regions by following these steps:

[0091] In this embodiment, the semantic bounding box set obtained in step S33 is utilized. Verify the semantically null room topology network constructed in step S23. .

[0092] In this embodiment, for semantically non-semantic room topology networks Candidate prediction boxes for each room in Calculate its relation to the semantically unrelated room topology network. The maximum intersection-union ratio (Max IoU) of all candidate rooms is determined, and the matching score is defined. :

[0093] ;

[0094] In the above formula, Indicates candidate room The bounding rectangle.

[0095] Set matching threshold (0.5 is used in the example). If Then determine the candidate prediction box. No matching geometric room was found; it was marked as a Missing Room Box. These regions with unmatched bounding boxes are called topology missing regions.

[0096] In this embodiment, adaptive reconstruction and repair based on local geometric clues are performed according to the following steps:

[0097] First, retrieve the set of all unused freeline endpoints within the unmatched bounding box; then, perform cluster analysis on the endpoints of the freeline endpoint set and perform endpoint snapping and crack stitching operations.

[0098] Specifically, each unmatched bounding box Perform repairs within its spatial range:

[0099] Perform endpoint adsorption and crack closure operations:

[0100] (1) Set distance threshold (This example uses 15 pixels);

[0101] (2) Retrieve unmatched bounding boxes The set of all unused offline segment endpoints within the range For any two endpoints Calculate its Euclidean distance Make the following judgment:

[0102] If the Euclidean distance between the two endpoints is less than the distance threshold, that is: This generates a connecting line segment. Sew the two ends together;

[0103] If a certain endpoint to a certain line segment Vertical projection distance This generates a perpendicular line connecting the endpoint and the line segment, and the perpendicular line is perpendicular to the line segment, thus achieving a T-shaped connection between the endpoint and the line segment.

[0104] Next, the stitched set of line segments is generated, which is the polygon reconstruction experiment: the polygonization operator is used to attempt to construct the stitched set of line segments into a closed polygon. The relation used is:

[0105] ;

[0106] In the above formula, This function represents the transformation of a set of line segments into a set of closed polygons. This represents the set of line segments after stitching, which includes the original line segments. and the newly added suture segments (Right now ); This represents a single geometrically closed path extracted from a graph composed of a set of line segments, which represents a potential room shape; Indicates belonging to (Element of); This represents the minimum closed loop search function, which is used to extract all independent minimum closed polygons in the graph; This indicates the filtering criteria, meaning that the constructed closed polygon should primarily lie within the range of the missing bounding box (geometric constraint).

[0107] In this embodiment, a polygonization operator is used to attempt to construct a closed polygon from the stitched set of line segments, i.e., bounding box repair:

[0108] If a closed polygon is successfully constructed (i.e.) If the set is not empty, it means that a closed loop cannot be constructed. The closed polygon is then used as the repaired room topology, i.e., the one with the largest area or the one that matches the unmatched bounding box is selected. The polygon with the largest intersection-union ratio is used as the repair result;

[0109] If a closed polygon cannot be constructed (i.e. The set is empty. If the unmatched bounding box is used, the geometric contour of the unmatched bounding box is directly used as the repaired room topology. In this embodiment, the unmatched bounding box is directly used. The coordinates of the four vertices are used as the topology of the room:

[0110] ;

[0111] In the above formula, Indicates the topological structure of the room. Indicates based on unmatched bounding boxes A rectangular polygon constructed from the coordinates of its four vertices. express The set is empty, meaning that no valid closed polygon can be formed even after the stitching operation.

[0112] This step fully demonstrates the semantic-driven nature of the algorithm. Candidate prediction boxes. Here, it serves not only as a carrier of functional labels but also as a "geometric container" for topology repair. This is achieved by confining the candidate prediction box... By performing breakpoint search internally, the algorithm avoids matching ambiguities caused by full-graph search, ensuring the semantic and geometric correctness of the repair operation.

[0113] S5. Integrate the repaired complete floor plan topology and room function categories to generate a vectorized floor plan model containing semantic attributes and geometric data (precise geometric coordinates, functional attributes, and connectivity relationships). This step aims to unify all independent room data into a globally consistent coordinate system, assign semantic attributes, and output the final engineering-grade data.

[0114] Specifically, step S5 is performed according to the following steps:

[0115] S51. To eliminate minor seam errors caused by independent calculations between different rooms and ensure that all walls are strictly horizontally or vertically aligned in the global coordinate system (i.e., satisfying the Manhattan world hypothesis), one-dimensional clustering correction is performed on the horizontal and vertical coordinates of all room vertices, including:

[0116] Data collection: Collect the vertex coordinates of all the polygons in the repaired room and construct a set of x-coordinates. and the set of ordinates ,in, Indicates the first The room The x-coordinates of each vertex.

[0117] One-dimensional clustering: using the x-axis as a set For example, sorting it yields an ordered sequence. Set alignment threshold (In this embodiment, 15.0 pixels are used). The sequence is divided into several clusters. This ensures that the difference between any two adjacent elements within a cluster is less than the alignment threshold. :

[0118] ;

[0119] In the above formula, and This represents any adjacent elements within a cluster.

[0120] Coordinate update: Calculate coordinates for each cluster ( mean As a uniform correction value for all coordinates within this cluster. :

[0121] ;

[0122] The original apartment data belongs to clusters All Coordinates replaced with uniform correction values For the set of ordinates Performing the same operation yields Through this step, the wall coordinates of the entire map are snapped to a set of implicit global grid lines.

[0123] S52, classify the room function category of each room identified in step S33. The repaired room topology determined in step S4 Bind.

[0124] S53. Encapsulate the bound and merged data into a standardized vector floor plan model for output.

[0125] In this embodiment, the bound and merged data is encapsulated into a standardized vector floor plan model. This standardized vector floor plan model It consists of a series of room objects:

[0126] ;

[0127] In the above formula, Each represents a separate room object (instance) that has been constructed and incorporated with semantic information.

[0128] Final standardized vector floor plan model The preferred output format is JSON file, which can be directly read by drawing software or 3D modeling engines, realizing an automated closed loop of "upload and use" and meeting the needs of large-scale industrial production.

[0129] Example 2:

[0130] An apparatus for implementing the floor plan reconstruction method of Embodiment 1 includes a geometry processing module, a semantic awareness module, a consistency verification module, an adaptive repair module, and a data fusion output module.

[0131] The geometry processing module is used to first generate pixel-level wall masks based on the original floor plan image through a deep semantic segmentation network, and then obtain geometric skeleton lines with a single pixel width based on the wall masks. Next, the ELSED algorithm is used to detect the discrete line segment set in the geometric skeleton line, and the discrete line segment set is orthogonalized and loop closure search is performed on the discrete line segment set based on the Manhattan rule to construct an initial semantic-free room topology network.

[0132] The semantic perception module is used to input the original floor plan image into a trained object detection network, identify the functional categories of the rooms, and generate a set of semantic bounding boxes consisting of the spatial location bounding boxes of all rooms.

[0133] The consistency verification module is used to construct a geometric-semantic consistency verification mechanism, which verifies the integrity of the initial semantically insensitive room topology network based on the spatial location bounding box and identifies the missing topology regions.

[0134] The adaptive repair module is used to perform adaptive reconstruction and repair based on local geometric clues for the missing topology region within the corresponding spatial location bounding box constraint range, so as to obtain the repaired complete house topology structure.

[0135] The data fusion output module is used to fuse the repaired complete apartment topology and room function categories to generate a vectorized apartment model containing semantic attributes and geometric data.

[0136] Specifically, the consistency verification and repair module is used to identify topology missing regions by: calculating the intersection-union ratio (IoU) between each candidate room and all spatial location bounding boxes; if there is a spatial location bounding box whose IoU value with any candidate room is lower than the preset matching threshold, then the spatial location bounding box is marked as an unmatched bounding box, and its covered area is defined as a topology missing region.

[0137] Specifically, when performing adaptive reconstruction and repair, the consistency verification and repair module is used to: retrieve the floating line segments within the unmatched bounding box range, perform endpoint snapping and crack stitching, and attempt to construct a closed polygon using the polygonization operator; if a closed polygon cannot be constructed, the geometric contour of the unmatched bounding box is directly used as the repaired room topology.

[0138] The device in this embodiment can execute the floor plan reconstruction method provided in Embodiment 1 of this application, and has the corresponding functional modules and beneficial effects of the execution method.

[0139] Example 3:

[0140] An electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which enables the at least one processor to perform the aforementioned floor plan reconstruction method.

[0141] It should be noted that electronic devices are intended to represent various forms of digital computers, such as laptops, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0142] A processor can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processors include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor performs the various methods and processes described above, such as floor plan reconstruction methods.

[0143] In some embodiments, the floor plan reconstruction method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded into and / or installed on an electronic device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by a processor, one or more steps of the floor plan reconstruction method described above may be performed. Alternatively, in other embodiments, the processor may be configured to perform the floor plan reconstruction method by any other suitable means (e.g., by means of firmware).

[0144] Various implementations of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0145] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0146] Example 4:

[0147] Computer-readable storage media can be tangible media that may contain or store computer programs for use by or in conjunction with an instruction execution system, apparatus, or device. Computer-readable storage media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0148] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0149] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0150] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0151] Experimental verification:

[0152] To verify the effectiveness, superiority, and robustness of the method of this invention, three sets of comparative experiments were designed: method performance comparison experiment, core module ablation experiment, and typical difficult case repair test. The experiments were based on the same apartment image dataset (including 1000 samples of standard apartment, simple apartment, severely occluded / weak texture apartment, and multi-apartment apartment with broken walls / noise), and were tested on the same hardware platform (Intel Core i5-14600KF, NVIDIA GeForce RTX 3090). The experimental results are as follows:

[0153] Experiment 1: Comparison of Method Performance

[0154] The method of this invention was compared with the traditional LSD+rule method and the pure deep learning segmentation method. The test metrics included room recall, topological closure, wall IoU, and vectorization time. The results are shown in Table 1.

[0155] Table 1. Comparison of Method Performance

[0156]

[0157] As shown in Table 1, the method of the present invention has achieved a significant improvement in core accuracy indicators. The room recall rate and topology closure rate are close to 100%, the wall IoU reaches 0.94, and the vectorization time is only 1.0s, which is shorter than that of pure deep learning segmentation methods, and it takes into account both accuracy and engineering processing efficiency.

[0158] Experiment 2: Ablation Experiment of Core Module

[0159] To verify the effectiveness of the core modules of this invention (geometric-semantic consistency verification and local adaptive repair), three experimental groups were designed: the basic group (only the basic geometric skeleton module is retained), the improved group (basic geometric skeleton + semantic consistency verification), and the complete group (the method of this invention, basic geometric skeleton + semantic consistency verification + local adaptive repair). The test indicators were topological completeness and semantic accuracy. The results are shown in Table 2.

[0160] Table 2 Ablation Experiment Table for Core Modules

[0161]

[0162] As shown in Table 2, the addition of the semantic consistency verification module improved the topology integrity by 12.7 percentage points and the semantic accuracy by 4.5 percentage points. The addition of the local adaptive repair module further improved the topology integrity by 9.3 percentage points and the semantic accuracy by 5.3 percentage points, which proves the effectiveness and necessity of the core innovative modules of this invention.

[0163] Experiment 3: Practical Test of Repairing Typical Difficult Cases

[0164] Four typical apartment types (severe occlusion / weak texture, broken walls / high noise, standard apartment, and simple apartment) were selected to test the room recognition performance of the method of this invention and the original method (pure deep learning segmentation). The results are shown in Table 3.

[0165] Table 3. Test Results of Repairing Typical Difficult Cases

[0166]

[0167] As shown in Table 3, the method of the present invention can effectively recall missed rooms and eliminate redundant artifacts in complex and difficult scenarios such as severe occlusion and broken walls. It can also maintain high accuracy and introduce no errors in standard and simple apartment scenarios, which proves the strong robustness of the method of the present invention.

[0168] Industrial applicability:

[0169] The floor plan reconstruction method of the present invention can be directly applied to the digital reconstruction of floor plans in fields such as architectural decoration, real estate marketing, and BIM modeling. The output standardized JSON format vectorized floor plan data can be seamlessly connected to downstream application systems such as VR house viewing, intelligent soft furnishing layout, and CAD drawing.

[0170] Experimental results show that this method achieves a room recall rate of 96.8% and a topology closure rate of 99.2% in complex scenarios, with a processing time of only 1.0s. It eliminates the need for manual verification and redrawing, significantly reducing the cost of base map drawing for designers, improving the efficiency of the entire process of digital reconstruction of apartment layouts, and realizing an automated closed loop of "upload and use". It can meet the needs of large-scale industrial production in the fields of architectural decoration and real estate, and has significant industrial practicality and market application value.

[0171] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. Those skilled in the art, under the guidance of the present invention, can make various similar representations without departing from the spirit and claims of the present invention, and such modifications all fall within the protection scope of the present invention.

Claims

1. A method for reconstructing floor plans based on multimodal semantic fusion and topology optimization, characterized in that, Includes the following steps: S1. Based on the original floor plan image, a pixel-level wall mask is generated through a deep semantic segmentation network, and a geometric skeleton line with a single pixel width is obtained based on the wall mask. S2. Use the ELSED algorithm to detect the discrete line segment set in the geometric skeleton lines, and perform orthogonalization correction and loop closure search on the discrete line segment set based on the Manhattan rule to construct the initial semantic-free room topology network. S3. Input the original floor plan image into the trained target detection network to identify the room function categories and generate a semantic bounding box set consisting of the spatial location bounding boxes of all rooms. S4. Construct a geometric-semantic consistency verification mechanism, perform integrity verification on the initial semantically unrelated room topology network based on the spatial location bounding box, identify the topology missing regions, and for the topology missing regions, perform adaptive reconstruction and repair based on local geometric clues within the corresponding spatial location bounding box constraints to obtain the repaired complete apartment topology structure. S5. Integrate the repaired complete apartment topology and room function categories to generate a vectorized apartment model containing semantic attributes and geometric data.

2. The method for reconstructing floor plans according to claim 1, characterized in that, Step S1 is performed according to the following steps: S11. Input the original floor plan image into a pre-trained panoramic segmentation network to obtain a multi-channel probability feature map containing wall, door, and window categories; S12. Perform threshold segmentation and morphological denoising on the multi-channel probability feature map to extract a binarized wall mask that contains only the wall region. S13. Perform distance transformation and skeleton refinement operations on the binarized wall mask to obtain the geometric skeleton lines with a single pixel width.

3. The method for reconstructing floor plans according to claim 1, characterized in that, Step S2 is performed according to the following steps: S21. Input the geometric skeleton line with the single pixel width into the ELSED algorithm and output a discrete set of line segments; S22. Based on the Manhattan rule, orthogonalize the discrete set of line segments according to the following steps: Calculate the direction angle of each line segment in the discrete set of line segments; Based on the relationship between the angle of each line segment and the horizontal or vertical axis and the preset tolerance threshold, each line segment is corrected to a horizontal or vertical direction, and a set of corrected line segments is obtained. S23. Use the minimum closed loop search algorithm to traverse the corrected set of line segments, construct a set of candidate rooms composed of closed polygons, and define the set of candidate rooms as the initial semantically non-semantic room topology network.

4. The method for reconstructing floor plans according to claim 1, characterized in that, Step S3 is performed according to the following steps: S31. A deep neural network based on the YOLO architecture is used as a trained object detection network, and the original floor plan image is input into the deep neural network. S32. Using a forward inference and boundary regression network, extract the backbone and detection head through feature extraction, and output the candidate prediction box for each room; S33. Decode the candidate prediction boxes for each room, determine the room function category of each room, and generate a semantic bounding box set consisting of the spatial location bounding boxes corresponding to all rooms; wherein, the room function category includes at least bedroom, living room, kitchen, bathroom and balcony.

5. The method for reconstructing floor plans according to claim 1, characterized in that, In step S4, the topological missing regions are identified according to the following steps: First, calculate the maximum intersection-union ratio (IUU) between each spatial location bounding box in the semantic bounding box set and all geometric room polygons in the initial semantically unrelated room topology network, and define the matching score; Then, all spatial location bounding boxes with matching scores below the matching threshold are marked as unmatched bounding boxes, and the area covered by the unmatched bounding boxes is defined as a topological missing region.

6. The method for reconstructing floor plans according to claim 5, characterized in that, In step S4, adaptive reconstruction and repair based on local geometric cues are performed according to the following steps: First, retrieve the set of all unused offline segment endpoints within the unmatched bounding box; Then, cluster analysis is performed on the endpoints of the set of endpoints of the free line segments, and endpoint adsorption and crack stitching operations are performed to generate a set of stitched line segments; Finally, the polygonization operator is used to attempt to construct a closed polygon from the stitched set of line segments: If a closed polygon is successfully constructed, then that closed polygon will be used as the repaired room topology. If a closed polygon cannot be constructed, the geometric contour of the unmatched bounding box is directly used as the repaired room topology.

7. The method for reconstructing floor plans according to claim 6, characterized in that, Perform the endpoint adsorption and crack closure operation according to the following steps: First, set a distance threshold; Then, the following judgment is made: If the Euclidean distance between the two endpoints is less than the distance threshold, a connecting line segment is generated to stitch the two endpoints together. If the perpendicular distance from an endpoint to another line segment is less than the distance threshold, a perpendicular line is generated connecting the endpoint and the line segment, and the perpendicular line is perpendicular to the line segment.

8. The method for reconstructing floor plans according to claim 7, characterized in that, Step S5 is performed according to the following steps: S51. Perform one-dimensional clustering correction on the horizontal and vertical coordinates of all room vertices respectively; S52. Bind the room function category to the repaired room topology; S53. Encapsulate the bound and merged data into a standardized vector floor plan model for output.

9. An apparatus for implementing the floor plan reconstruction method according to any one of claims 1 to 8, characterized in that, include: The geometry processing module is used to first generate a pixel-level wall mask based on the original floor plan image through a deep semantic segmentation network, and then obtain a geometric skeleton line with a single pixel width based on the wall mask. The ELSED algorithm is then used to detect the discrete line segment set in the geometric skeleton lines, and the discrete line segment set is orthogonalized and loop closure search is performed based on the Manhattan rule to construct the initial semantic-free room topology network. The semantic perception module is used to input the original floor plan image into a trained target detection network, identify the room function categories, and generate a set of semantic bounding boxes consisting of the spatial location bounding boxes of all rooms. The consistency verification module is used to construct a geometric-semantic consistency verification mechanism, which verifies the integrity of the initial semantically insensitive room topology network based on the spatial location bounding box and identifies the missing topology regions. An adaptive repair module is used to perform adaptive reconstruction and repair based on local geometric clues within the corresponding spatial location bounding box constraint range for the missing topology region, so as to obtain the repaired complete house topology structure. The data fusion output module is used to fuse the repaired complete apartment topology and room function categories to generate a vectorized apartment model containing semantic attributes and geometric data.

10. A computer-readable medium, characterized in that, The computer-readable medium stores computer instructions that are executed by at least one processor to implement the floor plan reconstruction method according to any one of claims 1 to 8.

Images