House type contour and room information extraction method and system of house type drawing

Abstract

The application discloses a kind of house type contour and room information extraction method and system of house type drawing, it is related to image processing technical field.The house type contour and room information extraction method of house type drawing includes: obtaining the house type drawing to be handled.Each room in the house type drawing to be handled The space defined by the contour of each room is set to white, and a preliminary house type contour map is obtained.Set the pixel of each door in the preliminary house type contour map to white, and obtain the preliminary house type contour map without door.In the preliminary house type contour map without door, the position corresponding to each room is filled with the label information of each room respectively, and the final house type contour map is obtained.The center image of the house type drawing is separated from the surrounding label information, the center image is processed, and then the processed center image is spliced with the surrounding label information part, which can greatly reduce the error recognition caused by optical character recognition and improve the accuracy of the house type drawing matting result.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for extracting the floor plan outline and room information from a floor plan, a system for extracting the floor plan outline and room information from a floor plan, a computer device, and a readable storage medium. Background Technology

[0002] Image matting refers to an image processing technique that extracts the foreground object of interest from an image while filtering out the background. It belongs to image extraction technology and is beneficial to various downstream applications such as video conferencing, advertising production, and e-commerce promotion.

[0003] Based on whether or not user-assisted input is used, existing image matting methods can be divided into two categories. The first category is input-based matting methods. These methods use three-class ternary maps, sparse graffiti, background images, coarse images, or user clicks as auxiliary input to guide alpha estimation (here, alpha estimation refers to estimating the transparency of each pixel in the image to accurately separate the foreground from the background). Some methods also explore using natural language prompts to assist matting (for example, the user enters a descriptive text, such as "mold out the red flower in the image," and the algorithm performs the matting operation based on such natural language instructions). However, these methods have limited flexibility because predictions are typically made at a low level of color propagation and are very sensitive to the density of the graffiti. The second category is automatic matting methods, which can automatically extract foreground objects without any human assistance.

[0004] Current methods attempt to control the matting process by determining which objects can be extracted. For example, some methods propose automatically extracting foreground figures and all relevant objects for human-object interactions, while others suggest extracting each human instance separately, rather than extracting all human instances at once. However, these matting techniques are only suitable for specific categories, such as humans, animals, or prominent objects, and are not suitable for application scenarios such as extracting wall outlines from floor plan images.

[0005] Traditional image cutout techniques work well in certain specific scenarios, but in applications like extracting wall outlines from floor plan images, the variety of elements present—such as beds, bedside tables, and toilets—is a significant obstacle. These elements, with their diverse colors and shapes, greatly interfere with traditional cutout methods, severely limiting the effectiveness. Furthermore, floor plan images often contain symbols like Chinese characters, numbers, and arrows, which also affect the cutout results, and this useful information is frequently removed or damaged. Summary of the Invention

[0006] The technical problem to be solved by this invention is that traditional image cutout technology cannot effectively handle various elements and symbolic information contained in floor plans, resulting in poor cutout effects.

[0007] To address the aforementioned shortcomings of existing technologies, the following solutions are provided:

[0008] In a first aspect, the present invention provides a method for extracting floor plan outlines and room information from a floor plan, comprising: acquiring a floor plan to be processed; setting all pixels within the space defined by the outlines of each room in the floor plan to be processed to white to obtain a preliminary floor plan outline; setting the pixels at the locations of each door in the preliminary floor plan outline to white to obtain a preliminary floor plan outline without doors; and backfilling the annotation information of each room at the corresponding positions in the preliminary floor plan outline without doors to obtain a final floor plan outline.

[0009] Optionally, the pixels within the space defined by the outlines of each room in the floor plan to be processed are set to white to obtain a preliminary floor plan outline image. This includes: determining the location information of each room in the floor plan to be processed; splitting the floor plan to be processed into an inner floor plan outline image and an outer floor plan outline image based on the location information of each room in the floor plan to be processed; setting the pixels within the space defined by the outlines of each room in the inner floor plan outline image to white based on the location information of each room in the floor plan to be processed to obtain a black and white image of the inner floor plan outline image; and stitching the black and white image of the inner floor plan outline image and the outer floor plan outline image together to obtain a preliminary floor plan outline image. The inner floor plan outline image includes image information within the outer boundary of the floor plan outline in the floor plan to be processed, and the outer floor plan outline image includes image information outside the outer boundary of the floor plan outline in the floor plan to be processed.

[0010] Optionally, the pixels at the locations of each door in the primary floor plan outline are set to white to obtain a primary floor plan outline with doors removed. This includes: determining the position information of each door in the floor plan to be processed; and setting the pixels at the locations of each door in the primary floor plan outline to white based on the position information of each door in the floor plan to be processed to obtain a primary floor plan outline with doors removed.

[0011] Optionally, the annotation information of each room is backfilled into the primary floor plan outline map (with doors removed) corresponding to the positions of each room to obtain the final floor plan outline map. This includes: recognizing text information in the floor plan outline map to obtain the annotation information of each room; and backfilling the annotation information of each room into the primary floor plan outline map (with doors removed) to obtain the final floor plan outline map.

[0012] Optionally, determining the location information of each room and each door in the floor plan to be processed includes: acquiring multiple floor plans; labeling the room and door locations in each of the multiple floor plans to obtain multiple floor plans with room labels and multiple floor plans with door labels; training a computer vision model based on the multiple floor plans with room labels to obtain a room location information extraction model; training a computer vision model based on the multiple floor plans with door labels to obtain a door location information extraction model; and inputting the floor plan to be processed into the room location information extraction model and the door location information extraction model respectively to obtain the location information of each room and each door in the floor plan to be processed.

[0013] Optionally, the floor plan to be processed is split into an inner floor plan outline and an outer floor plan outline based on the location information of each room in the floor plan. This includes: obtaining a binary image of each room based on the location information of each room in the floor plan; performing dilation and denoising processing on the binary image of each room to obtain a binary image of the floor plan outline; and splitting the floor plan to be processed into an inner floor plan outline and an outer floor plan outline based on the binary image of the floor plan outline.

[0014] Optionally, a binary map of the rooms can be obtained based on the location information of each room in the floor plan to be processed, including: segmenting the floor plan to be processed, segmenting each room, obtaining the mask of each room, and obtaining the binary map of the rooms based on the mask of each room.

[0015] Optionally, the binary image of the room is dilated and denoised to obtain a binary image of the apartment layout, including: dilating the binary image of the room to obtain an dilated binary image; and denoising the dilated binary image to obtain a binary image of the apartment layout.

[0016] Optionally, the text information in the floor plan outline map is recognized to obtain the annotation information of each room, including: performing optical character recognition on the floor plan outline map to obtain preliminary recognition information; and filtering the preliminary recognition information according to a preset dictionary to obtain the annotation information of each room.

[0017] Secondly, this invention provides a system for extracting floor plan outlines and room information from a floor plan, comprising: a floor plan acquisition module, a primary floor plan outline acquisition module, a primary floor plan outline acquisition module with doors removed, and a floor plan outline acquisition module. The floor plan acquisition module is configured to: acquire the floor plan to be processed. The primary floor plan outline acquisition module is configured to: set all pixels within the space defined by the outlines of each room in the floor plan to be processed to white, thus obtaining a primary floor plan outline. The primary floor plan outline acquisition module with doors removed is configured to: set the pixels at the locations of each door in the primary floor plan outline to white, thus obtaining a primary floor plan outline with doors removed. The floor plan outline acquisition module is configured to: fill in the annotation information of each room in the primary floor plan outline with doors removed, corresponding to the locations of each room, to obtain the final floor plan outline.

[0018] Thirdly, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method for extracting the floor plan outline and room information of the above-mentioned floor plan.

[0019] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the processor executes the method for extracting the floor plan outline and room information of the above-mentioned floor plan.

[0020] The method and system for extracting floor plan outlines and room information provided by this invention separates the central part (inner part of the floor plan outline) of the original floor plan image (i.e., the floor plan to be processed) from the surrounding annotation information (outer part of the floor plan outline). The separated central part is then processed, and the processed central part is then stitched together with the surrounding annotation information. This avoids optical character recognition (OCR) on the surrounding annotation information, and only performs OCR on the central part, thereby greatly reducing the possible errors caused by OCR, improving the accuracy of the floor plan cutout results, and preserving symbolic information in the floor plan to the greatest extent. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating a method for extracting the floor plan outline and room information from a floor plan according to an embodiment of the present invention.

[0022] Figure 2 This is a floor plan to be processed in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0023] Figure 3 A flowchart illustrating another method for extracting floor plan outlines and room information from a floor plan in an embodiment of the present invention;

[0024] Figure 4 This is a flowchart illustrating another method for extracting floor plan outlines and room information from a floor plan in an embodiment of the present invention.

[0025] Figure 5 This is a room binary image in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0026] Figure 6 This is a flowchart illustrating a method for extracting the floor plan outline and room information from a floor plan in another embodiment of the present invention.

[0027] Figure 7 This is an inflated binary image used in a method for extracting floor plan outlines and room information in an embodiment of the present invention.

[0028] Figure 8 This is a binary image of the floor plan outline in a method for extracting floor plan outline and room information in an embodiment of the present invention.

[0029] Figure 9 This is an example of an inner view of the floor plan outline in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0030] Figure 10 This is an external view of the floor plan outline in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0031] Figure 11 This is a black and white image of the floor plan outline in a method for extracting floor plan outline and room information in an embodiment of the present invention.

[0032] Figure 12 This is a primary floor plan outline diagram in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0033] Figure 13 This is a flowchart illustrating a method for extracting the floor plan outline and room information from a floor plan in another embodiment of the present invention.

[0034] Figure 14 This is a flowchart of a method for extracting room location and door location information in a method for extracting floor plan outline and room information in an embodiment of the present invention.

[0035] Figure 15A This is a schematic diagram of the extraction result of room location information in the method for extracting the floor plan outline and room information in an embodiment of the present invention.

[0036] Figure 15B This is a schematic diagram showing the extraction result of door location information in the method for extracting floor plan outline and room information in an embodiment of the present invention.

[0037] Figure 16 This is a primary floor plan outline map with doors removed, used in a method for extracting floor plan outline and room information in an embodiment of the present invention.

[0038] Figure 17 This is a flowchart of the annotation information recognition method in a method for extracting floor plan outline and room information in an embodiment of the present invention;

[0039] Figure 18 This is a schematic diagram of the optical character recognition result in a method for extracting the floor plan outline and room information of a floor plan according to an embodiment of the present invention.

[0040] Figure 19 This is the final floor plan outline in a method for extracting floor plan outline and room information in an embodiment of the present invention.

[0041] Figure 20 This is a flowchart of an example of a method for extracting the floor plan outline and room information from a floor plan in an embodiment of the present invention.

[0042] Figure 21 This is a structural diagram of a system for extracting floor plan outlines and room information from a floor plan according to an embodiment of the present invention;

[0043] Figure 22 This is a structural diagram of a system for extracting floor plan outlines and room information from another floor plan embodiment of the present invention;

[0044] Figure 23 This is a structural diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0045] To enable those skilled in the art to better understand the technical solution of the present invention, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0046] It is understood that the specific embodiments and accompanying drawings described herein are merely for explaining the invention and are not intended to limit the invention.

[0047] It is understood that, without conflict, the various embodiments and features in the embodiments of the present invention can be combined with each other.

[0048] It is understood that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, while the parts unrelated to the present invention are not shown in the drawings.

[0049] It is understood that each unit or module involved in the embodiments of the present invention may correspond to only one entity structure, or may be composed of multiple entity structures, or multiple units or modules may be integrated into one entity structure.

[0050] It is understood that, without conflict, the functions and steps marked in the flowcharts and block diagrams of this invention may occur in a different order than that marked in the accompanying drawings.

[0051] It is understood that the flowcharts and block diagrams of this invention illustrate the possible architecture, functions, and operations of systems, apparatuses, devices, and methods according to various embodiments of this invention. Each block in the flowchart or block diagram may represent a unit, module, program segment, or code, containing executable instructions for implementing the specified function. Furthermore, each block or combination of blocks in the block diagram and flowchart can be implemented using a hardware-based system to achieve the specified function, or using a combination of hardware and computer instructions.

[0052] It is understood that the units and modules involved in the embodiments of the present invention can be implemented by software or by hardware. For example, the units and modules can be located in a processor.

[0053] Embodiments of the present invention provide a method for extracting the floor plan outline and room information from a floor plan, such as... Figure 1 As shown, the method for extracting the floor plan outline and room information includes steps 101 and 104.

[0054] Step 101: Obtain the floor plan to be processed.

[0055] Understandably, in step 101, the floor plan to be processed includes the furniture and facility legends, the floor decoration legends, the wall legends, the annotation legends, the location information of each room, and the location information of each door.

[0056] For example, such as Figure 2As shown, a floor plan is a spatial layout diagram of a residence. The furniture and facility illustrations included in the floor plan are used to visually represent the placement and approximate dimensions of furniture such as sofas, beds, wardrobes, and dining tables and chairs, helping homebuyers or users better understand how the space is utilized. For areas with pipes and facilities, such as kitchens and bathrooms, the locations of water pipes, sewers, floor drains, toilets, sinks, and shower heads are marked. These markings are usually represented by simple graphic symbols, such as circles for floor drains and rectangles with details for toilets and sinks. The floor decoration illustrations in the floor plan use simple line combinations to indicate the effect of laying flooring or tiles. In the wall illustrations, load-bearing walls are usually represented by thick solid lines or black fills to highlight their importance and non-removability, while non-load-bearing walls are generally represented by thin solid lines or hollow lines, indicating that they can be modified to a certain extent as needed. Floor plan annotations include information within each room (such as room name and area) and information outside the floor plan (such as dimension and direction annotations). For example, room function annotations clearly indicate the purpose of each room, such as "living room," "bedroom," "kitchen," "bathroom," "study," and "dining room." Some floor plans further distinguish between master bedrooms and secondary bedrooms. Dimension annotations typically use numbers and lines to represent the dimensions of walls, rooms, doors, and windows. There are usually horizontal and vertical dimension lines with specific values, typically in millimeters (mm). For example, they might indicate the length and width of a room, the thickness of a wall, or the width and height of doors and windows. Direction annotations usually include a north arrow to indicate the orientation of the house. The arrow points north, allowing you to determine other directions such as east, south, and west, facilitating understanding the lighting and ventilation of each room.

[0057] Generally, in conjunction with the floor plan and the above illustrations, the floor plan will clearly show the location of each room and each door in order to help users understand the floor plan.

[0058] Step 102: Set all pixels within the space defined by the outline of each room in the floor plan to be processed to white to obtain the initial floor plan outline.

[0059] In some embodiments, such as Figure 3 As shown, step 102 is implemented by steps 301 to 304.

[0060] Step 301: Determine the location information of each room in the floor plan to be processed.

[0061] Understandably, the location information of each room in the floor plan to be processed can be obtained through computer vision models.

[0062] Step 302: Based on the location information of each room in the floor plan to be processed, split the floor plan into an inner outline and an outer outline.

[0063] In step 302, the inner map of the floor plan includes image information located within the outer boundary of the floor plan outline in the floor plan to be processed, and the outer map of the floor plan outline includes image information located outside the outer boundary of the floor plan outline in the floor plan to be processed.

[0064] In some embodiments, such as Figure 4 As shown, step 302 is implemented by steps 401 to 403.

[0065] Step 401: Obtain the binary map of the room based on the location information of each room in the floor plan to be processed.

[0066] In some embodiments, the implementation method of step 401 includes: segmenting the floor plan to be processed, segmenting each room, obtaining the mask of each room, and obtaining a room binary map based on the mask of each room.

[0067] In image processing, a mask is a technique used to specify a particular region or pixel in an image. Essentially, it's an array or matrix with the same shape as the original image. The element values ​​control the manipulation of the original image, determining whether elements are retained, modified, or discarded—a filtering mechanism. In image segmentation, binary masks are the most common. Binary masks contain only two element values, often 0 and 1 or 0 and 255. A value of 0 indicates that the corresponding original data is masked, while 1 or 255 indicates that it is retained. They are commonly used for object extraction (extracting objects of interest from complex images) and region masking (masking unwanted parts of an image while retaining the region of interest).

[0068] For example, a mask can be generated by utilizing the difference in grayscale values ​​or other features between the target and the background in an image. First, one or more thresholds are selected based on the characteristics of the image. Then, pixels in the image with values ​​greater than or less than the threshold are marked as the target or background, respectively, thus obtaining a preliminary mask. For instance, for a grayscale image, if the grayscale values ​​of the target object are generally high, while the grayscale values ​​of the background are low, a suitable grayscale threshold can be selected, and pixels with values ​​greater than the threshold can be set as the target region (corresponding position in the mask is 1), while pixels with values ​​less than the threshold can be set as the background region (corresponding position in the mask is 0).

[0069] For example, deep learning methods can also be used to segment the floor plan, dividing it into individual rooms and obtaining masks for each room. For instance, by training on a large-scale labeled dataset, the model learns the image's feature representation and segmentation patterns. During the inference phase, the image to be segmented is input into the trained model, which directly outputs the corresponding mask. Deep learning methods can automatically learn higher-level features and perform well in image segmentation in complex scenes, but they require a large amount of labeled data and computational resources for training.

[0070] For example, a method for obtaining a binary image of a room based on the mask of each room can be as follows: In a pure black image, the portion with a mask of 1 is set to white to obtain a binary image of the room, as shown in the image below. Figure 5 As shown.

[0071] Step 402: Perform dilation and noise reduction on the binary map of the room to obtain a binary map of the apartment layout.

[0072] In some embodiments, such as Figure 6 As shown, the implementation method of step 402 includes steps 601 and 602.

[0073] Step 601: Perform a dilation operation on the room binary map to obtain a dilated binary map.

[0074] Understandably, in a room binary graph ( Figure 5 Visually, this appears as large white blocks interspersed with narrow black lines. At this point, an expansion operation can be performed on the binary image of the room to eliminate the black lines in the white blocks, making the white blocks connected, and at the same time, expanding the boundaries of the connected white blocks outward to the thickness of the wall.

[0075] For example, when selecting the dilatation kernel, set shape = cv2.MORPH_RECT, where shape is the shape of the kernel. Selecting a rectangular structure is more suitable for floor plan scenarios. Set ksize = (40, 40), where ksize is the size of the kernel. The kernel size of (40, 40) exceeds the thickness of the wall, which ensures that the boundary of the white block can expand outward beyond the thickness of the wall, thus obtaining a dilatation binary image, such as... Figure 7 As shown.

[0076] Step 602: Denoise the dilated binary map to obtain a binary map of the apartment layout outline.

[0077] Understandably, the dilation operation cannot eliminate the black within the white block, resulting in isolated black impurity blocks within the white area, such as... Figure 7 As shown, black impurity blocks may still exist inside the white connected regions in the dilated binary graph.

[0078] For example, a flood filling algorithm can be used to find the boundary of the white area, and then set all pixels within the boundary to white, thus eliminating black impurity blocks in the white connected regions of the dilated binary image.

[0079] For example, step 602 can also be implemented using the method mentioned in the paper "Progressive Contextual Aggregation Empowered by Pixel-Wise Confidence Scoring for Image Inpainting" published by Jinwoo Kim et al. in the journal IEEE Transactions on Image Processing in 2023. This method employs a hierarchical inpainting strategy, combining progressive utilization of contextual information with pixel-level confidence scoring to achieve image inpainting. Applied to this scenario, this method treats imperfection points as missing pixel blocks and combines them with surrounding pixels to generate images, thereby achieving the removal of imperfection points. Considering that the image in this scenario is not complex, a custom dataset is constructed. OpenCV is used to randomly generate polygons, and imperfection points are filled into the polygons to construct a dataset that fits this scenario. Setting N to 6, the model is trained and inferred to obtain the repaired result, such as... Figure 8 As shown.

[0080] Step 403: Based on the binary image of the apartment outline, split the apartment map to be processed into an inner outline image and an outer outline image.

[0081] Understandably, such as Figure 8 As shown, Figure 8 The boundary of the white area in the middle is the outer boundary of the apartment layout outline, in the apartment layout drawing to be processed ( Figure 2 In ) will be with Figure 8 By setting the corresponding pixels in the black area to white, you can obtain the inner outline of the floor plan, as shown below. Figure 9 As shown. In the floor plan to be processed ( Figure 2 In ) will be with Figure 8 By setting the corresponding pixels in the white area to white, you can obtain the outer outline of the floor plan, as shown below. Figure 10 As shown.

[0082] Step 303: Based on the location information of each room in the floor plan to be processed, set all pixels within the space defined by the outline of each room in the floor plan outline to white, and obtain a black and white image of the floor plan outline.

[0083] For example, a black and white image of the floor plan outline is shown below. Figure 11 As shown.

[0084] Step 304: Combine the black and white image of the inner floor plan outline with the outer floor plan outline to obtain the initial floor plan outline.

[0085] For example, Figure 10 and Figure 11 By piecing them together, we can obtain Figure 12 That is, a basic floor plan outline, such as Figure 12 As shown.

[0086] Step 103: Set the pixels at the locations of each door in the primary floor plan outline to white to obtain the primary floor plan outline with the doors removed.

[0087] In some embodiments, such as Figure 13 As shown, step 103 is implemented by steps 1301 to 1302.

[0088] Step 1301: Determine the location information of each door in the floor plan to be processed.

[0089] Understandably, the location information of each door in the floor plan to be processed can be obtained through computer vision models.

[0090] In some embodiments, such as Figure 14 As shown, the implementation methods of steps 301 and 1301 include steps 1401 to 1405.

[0091] Step 1401: Obtain multiple floor plans.

[0092] Understandably, it is possible to obtain multiple different floor plans from the internet.

[0093] Step 1402: Label the room location information and door location information of each floor plan in the multiple floor plans to obtain multiple floor plans with room labels and multiple floor plans with door labels.

[0094] Understandably, step 1402 can be achieved using manual or semi-automatic annotation methods. Manual annotation can be performed using graphic editing software (Adobe Photoshop, Illustrator, etc.) or professional annotation tools (such as Labelme, VGG Image Annotator (VIA), etc.). For example, using the brush and shape tools in graphic editing software, one can directly outline the boundaries of each room or the location of doors on the floor plan, add text labels, and set attributes such as color and transparency to distinguish different areas. Taking Labelme as an example, when using professional annotation tools for manual annotation, after opening the floor plan, one can accurately depict the outlines of the rooms using tools such as drawing polygons and rectangles, while simultaneously entering the room names (such as "Bedroom A," "Living Room," etc.) to complete the annotation. A corresponding JSON file will be generated to record the annotation information.

[0095] Step 1403: Train the computer vision model based on multiple floor plans with room annotations to obtain the room location information extraction model.

[0096] For example, the computer vision model can select the pre-trained YOLOv8 model yolov8x-seg.pt and train it using multiple floor plans with room annotations. During training, the data augmentation parameters degress are set to 180 and flipud to 0.5. The degress parameter randomly rotates the image within a specified degree range during training, improving the model's ability to recognize objects in different directions. The flipud parameter flips the image with a specified probability during training, increasing data variability without affecting object features. These two parameters provide more data during training, enhancing the model's performance after training.

[0097] Step 1404: Train the computer vision model based on multiple floor plans with door annotations to obtain the door location information extraction model.

[0098] Understandably, the method of step 1404 is the same as that of step 1403, and will not be repeated here.

[0099] Step 1405: Input the floor plan to be processed into the room location information extraction model and the door location information extraction model respectively to obtain the location information of each room and the location information of each door in the floor plan to be processed.

[0100] Understandably, step 1405 can be considered as the process of predicting the location information of each room and each door in the floor plan to be processed using the room location information extraction model and the door location information extraction model, respectively.

[0101] For example, during the prediction process, setting the minimum confidence threshold parameter `conf` to 0.75 helps reduce false alarms. The prediction results for room location information are as follows: Figure 15A As shown, Figure 15A In the diagram, a confidence level (e.g., c = 0.93) is assigned to each region where a room was predicted. The prediction results for the door's location are as follows: Figure 15B As shown, Figure 15B In the data, a confidence level (e.g., d0.90) is marked for each predicted exit area.

[0102] Step 1302: Based on the position information of each door in the floor plan to be processed, set the pixels at the location of each door in the primary floor plan outline to white to obtain the primary floor plan outline with the doors removed.

[0103] Understandably, step 1405 yields the position information of each door in the floor plan to be processed. Therefore, based on this position information, the pixels at the locations of each door in the primary floor plan outline are set to white, resulting in a primary floor plan outline with doors removed. Figure 16 As shown.

[0104] Step 104: In the initial floor plan outline drawing with doors removed, fill in the label information of each room according to the location of each room to obtain the final floor plan outline drawing.

[0105] Understandably, in step 104, the final floor plan outline includes the wall legend of the floor plan, the annotation legend of the floor plan, the location information of each room and the location information of each door. The final floor plan outline only needs to retain the floor plan outline and room layout.

[0106] In some embodiments, such as Figure 17 As shown, the implementation method of step 104 includes steps 1701 to 1702.

[0107] Step 1701: Perform optical character recognition on the floor plan outline to obtain preliminary recognition information.

[0108] Understandably, such as Figure 2 and Figure 10 As shown, the floor plan outline ( Figure 10The floor plan includes scale information and directional information around the apartment (both of which are part of the floor plan's annotation legend). This scale information is text information mixed with horizontal text, vertical text, and arrow symbols. Optical Character Recognition (OCR) can encounter recognition errors due to the vertical text and arrow symbols when recognizing these two types of information. Therefore, before OCR recognition, the outer and inner outlines of the apartment are separated, and OCR recognition is performed only on the inner outline. This eliminates the influence of characters in the outer outline on the processing results. Not only can the scale information and directional information around the apartment be completely preserved in the outer outline, but the limitations of OCR in recognizing this type of complex text information can also be avoided.

[0109] Understandably, OCR recognition can obtain text information within the floor plan outline and the location where the text information appears, for example, Figure 9 In the floor plan outline, the text information includes the name and area of ​​each room (e.g., balcony B and 2.8m). 2 The information format recognized by OCR is as follows: Figure 18 As shown.

[0110] Step 1702: Filter the preliminary recognition information according to the preset vocabulary to obtain the labeling information of each room.

[0111] Understandably, text information with a confidence level of less than 0.85 can be filtered out first. In addition, considering the problem of OCR misrecognition, the recognized text can be filtered in a preset dictionary to obtain the labeling information of each room. This can improve the accuracy of OCR recognition and selectively fill in the information recognized by OCR.

[0112] Understandably, the labeling information for each room in step 1702 refers to the labeling information located in each room on the floor plan to be processed.

[0113] For example, the final floor plan outline in step 104 is as follows: Figure 19 As shown, with Figure 2 compared to, Figure 19 It preserves information such as house size, type, and orientation to the greatest extent possible. Moreover, this information consists of original values ​​from the original drawing, rather than predicted values ​​calculated based on the graphic structure, thus greatly avoiding errors caused by calculation.

[0114] The following are Figure 2 Taking a floor plan as an example, the method for extracting the floor plan outline and room information of a floor plan proposed in this invention will be described exemplarily.

[0115] like Figure 20 As shown, the processing flow in this example is as follows:

[0116] 1. Obtain the central portion of the processed floor plan image, including the following steps:

[0117] Step 1: Mark the location of the partitions (i.e., rooms) in the floor plan image. The markings and the floor plan image are denoted as Dataset 1.

[0118] Step 2: Mark the location of the door in the floor plan image. The markings and the floor plan image are denoted as dataset 2.

[0119] Step 3: Using YOLOv8, select the pre-trained model yolov8x-seg.pt. Train the model using the same method as Dataset 1 and Dataset 2 from Steps 1 and 2. During training, set degress = 180. The degress parameter randomly rotates the image within a specified degree range during training, improving the model's ability to recognize objects in different directions. Set flipud = 0.5. The flipud parameter flips the image with a specified probability during training, increasing the variability of the data without affecting object features. These two parameters provide more data during training, enhancing the model's performance after training. The model trained on Dataset 1 is denoted as Model 1, and the model trained on Dataset 2 is denoted as Model 2.

[0120] Step 4: Use Model 1 to process the floor plan image (i.e., the floor plan to be processed). Figure 2 The prediction process is performed to obtain the locations of all rooms in the floor plan image. During prediction, `conf = 0.75` is set; `conf` is the minimum confidence threshold for prediction. Adjusting this value helps reduce false alarms. The prediction results are reflected in the image as follows: Figure 15A As shown, based on this, the corresponding positions of the cubicles in the pure black image are set to white, such as... Figure 5 As shown.

[0121] Step 5, in Figure 5 Visually, this is represented by large white blocks interspersed with narrow black lines. Figure 5 The dilation operation removes black lines from white blocks, connecting them and extending their boundaries outwards by the wall thickness. When selecting the dilation kernel, set `shape = cv2.MORPH_RECT`, where `shape` is the kernel's shape. A rectangular structure is more suitable for floor plan scenarios. Set `ksize = (40, 40)`, where `ksize` is the kernel size. Since the size (40, 40) exceeds the wall thickness, it ensures the white block boundaries extend outwards by the wall thickness. The dilated image looks like... Figure 7 As shown.

[0122] Step 6, in Figure 7 Black impurity points may still exist within white connected regions. The method described in the paper "Progressive Contextual Aggregation Empowered by Pixel-Wise Confidence Scoring for Image Inpainting" is used to address this. Figure 7 The method employs a hierarchical repair strategy, combining progressive utilization of contextual information and pixel-level confidence scoring to achieve image restoration. Applied to this scenario, the method treats imperfection points as missing pixel blocks and combines them with surrounding pixels to generate a new image, thus removing the imperfections. Considering the relatively simple nature of the images in this scenario, a custom dataset was created. OpenCV was used to randomly generate polygons, which were then filled with imperfection points to construct a dataset that fits the specific scene. With N set to 6, the model was trained and inferred to obtain the repaired results, such as... Figure 8 As shown.

[0123] Step 7, according to Figure 8 The outermost outline of the specified apartment layout is used to separate the apartment layout image. Specifically, the separation operation involves: within the apartment layout image, […]. Figure 8 The corresponding positions in the black areas are set to white, resulting in only the central portion being displayed. Figure 9 This refers to the interior outline of the apartment layout.

[0124] Step 8, for Figure 9 Perform the following processing: Based on the locations of all compartments obtained in step 4, Figure 9 The pixels containing the partition are all set to white. This eliminates irrelevant elements such as furniture and flooring in the floor plan, resulting in a basically pure floor plan outline, such as... Figure 11 As shown.

[0125] 2. Obtain the four sides of the floor plan image, including the following steps:

[0126] Step 9, according to Figure 8 The outermost outline of the specified apartment layout is used to separate the apartment layout image. The specific separation operation is as follows: In Figure 2 Lieutenant General Figure 8 The corresponding positions in the white area are set to white, at which point only the information of the surrounding rulers is obtained. Figure 10 , Figure 10 The ruler information in the text is text information that includes horizontal text, vertical text, and arrow symbols. This information is saved in this step to avoid the need for OCR to additionally recognize this type of information later, thus avoiding the limitations of OCR in recognizing this type of complex text information.

[0127] 3. The process of stitching together the center and perimeter of the floor plan image includes the following steps:

[0128] Step 10: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the Figure 11 and in step 9 Figure 10 By splicing them together, we get Figure 12 , Figure 12 It includes the floor plan outline and surrounding scale information.

[0129] 4. Remove doors from the floor plan.

[0130] Step 11: Use Model 2 to predict the location of all doors in the floor plan image. During prediction, set conf = 0.75, where conf is the minimum confidence threshold for prediction. Adjusting this value helps reduce false alarms. The prediction results are reflected in the image as follows: Figure 15B As shown, based on this, in step 9 Figure 12 In the image, the area corresponding to the door is set to white. This will remove the door from the image, revealing a pure floor plan outline. Figure 16 As shown.

[0131] 5. Use OCR to fill in the text in the compartments.

[0132] Step 12, obtained in step 7 Figure 9 OCR recognition is performed in the process because Figure 9 In the floor plan image ( Figure 2 Based on the previous version, the surrounding scale information was removed. Therefore, during OCR recognition, there will be no recognition errors caused by vertical text and arrow symbols. OCR recognition can obtain the text information in the image and the location of the text information. The text information is the name and size information of each compartment, such as Balcony B and 2.8m2. The recognized information format is as follows: Figure 18 As shown, these text messages are filled back into the locations where they appear. Figure 16 In the backfilling process, text information with a confidence level less than 0.85 is filtered out. Considering the issue of OCR misidentification, a sensitive word library is established. Recognized text is filtered within this library, and only the recognized words are selectively backfilled. The result after backfilling is... Figure 19 At this time Figure 19 Comparison of floor plan images ( Figure 2 This method preserves information such as house size, type, and orientation to the greatest extent possible. Moreover, this information consists of original values ​​from the original drawing, rather than predicted values ​​calculated based on the graphic structure, thus greatly avoiding errors caused by calculation.

[0133] This example overcomes the limitations of OCR in recognizing vertical text by processing the image in multiple parts. It preserves information such as room dimensions, type, and orientation while retaining the floor plan. This example can, to some extent, avoid sensitive characters while preserving text information. This example integrates advanced methods from research papers in the field of image generation, applying these methods to achieve image removal effects.

[0134] Embodiments of the present invention provide a system for extracting floor plan outlines and room information from a floor plan (hereinafter referred to as the extraction system), such as... Figure 21 As shown, the extraction system 2100 includes a floor plan acquisition module 2101, a primary floor plan outline acquisition module 2102, a primary floor plan outline acquisition module 2103 with doors removed, and a floor plan outline acquisition module 2104.

[0135] The floor plan acquisition module 2101 is set to acquire the floor plan to be processed.

[0136] The primary floor plan outline acquisition module 2102 is set to: set all pixels within the space defined by the outline of each room in the floor plan to be processed to white, thereby obtaining the primary floor plan outline.

[0137] In some embodiments, the primary floor plan outline acquisition module 2102 is configured to: determine the location information of each room in the floor plan to be processed; split the floor plan to be processed into an inner floor plan outline and an outer floor plan outline based on the location information of each room; set all pixels within the space defined by the outline of each room in the inner floor plan outline to white, thereby obtaining a black and white image of the inner floor plan outline; and stitch the black and white image of the inner floor plan outline and the outer floor plan outline together to obtain a primary floor plan outline. The inner floor plan outline includes image information within the outer boundary of the floor plan outline in the floor plan to be processed, and the outer floor plan outline includes image information outside the outer boundary of the floor plan outline in the floor plan to be processed.

[0138] In some embodiments, the primary floor plan outline acquisition module 2102 is configured to: obtain a room binary map based on the location information of each room in the floor plan to be processed; perform dilation and denoising processing on the room binary map to obtain a binary map of the floor plan outline; and, based on the binary map of the floor plan outline, split the floor plan to be processed into an inner floor plan outline map and an outer floor plan outline map.

[0139] In some embodiments, the primary floor plan outline acquisition module 2102 is configured to: segment the floor plan to be processed, segment out each room, obtain the mask of each room, and obtain a room binary image based on the mask of each room.

[0140] In some embodiments, the primary floor plan outline acquisition module 2102 is configured to: perform a dilation operation on the room binary map to obtain a dilated binary map; and perform noise reduction processing on the dilated binary map to obtain a binary map of the floor plan outline.

[0141] The module 2103 for obtaining the primary floor plan outline without doors is configured as follows: The primary floor plan outline without doors is configured to set the pixels at the location of each door in the primary floor plan outline to white, thereby obtaining the primary floor plan outline without doors.

[0142] In some embodiments, the primary floor plan outline acquisition module 2103 is configured to: determine the position information of each door in the floor plan to be processed; and, based on the position information of each door in the floor plan to be processed, set the pixels at the locations of each door in the primary floor plan outline to white, thereby obtaining the primary floor plan outline with doors removed.

[0143] In some embodiments, such as Figure 22 As shown, the extraction system 2100 also includes a room location and door location determination module 2105. The room location and door location determination module 2105 is configured to: acquire multiple floor plans; annotate the room location information and door location information of each floor plan in the multiple floor plans, obtaining multiple floor plans with room annotations and multiple floor plans with door annotations; train a computer vision model based on the multiple floor plans with room annotations to obtain a room location information extraction model; train a computer vision model based on the multiple floor plans with door annotations to obtain a door location information extraction model; and input the floor plan to be processed into the room location information extraction model and the door location information extraction model, respectively, to obtain the location information of each room and the location information of each door in the floor plan to be processed.

[0144] The floor plan outline acquisition module 2104 is set to: fill in the label information of each room in the primary floor plan outline (with doors removed) corresponding to the positions of each room, and obtain the final floor plan outline.

[0145] In some embodiments, the floor plan outline acquisition module 2104 is configured to: identify text information in the floor plan outline to obtain annotation information for each room; and backfill the annotation information for each room into the primary floor plan outline after removing doors to obtain the final floor plan outline.

[0146] In some embodiments, the floor plan outline acquisition module 2104 is configured to: perform optical character recognition on the floor plan outline to obtain preliminary recognition information; and filter the preliminary recognition information according to a preset dictionary to obtain annotation information for each room.

[0147] Embodiments of the present invention provide a computer device, such as... Figure 23 As shown, the computer device 2300 includes a memory 2301 and a processor 2302. The memory 2301 stores a computer program. When the processor 2302 runs the computer program stored in the memory 2301, the processor 2302 executes the above-mentioned method for extracting the floor plan outline and room information.

[0148] The specific solution and beneficial effects of the computer device provided by the embodiments of the present invention can be found in the relevant description of the method for extracting the floor plan outline and room information of a floor plan provided by the embodiments of the present invention, which will not be repeated here.

[0149] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processor performs the above-described method for extracting the floor plan outline and room information.

[0150] The specific scheme and beneficial effects of the computer-readable storage medium provided by the embodiments of the present invention can be found in the relevant description of the method for extracting the floor plan outline and room information of a floor plan provided by the embodiments of the present invention, which will not be repeated here.

[0151] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. A method for extracting the floor plan outline and room information from a floor plan, characterized in that, include: Obtain the floor plan to be processed; Set all pixels within the space defined by the outline of each room in the floor plan to be processed to white to obtain a primary floor plan outline. Set the pixels at the locations of each door in the initial floor plan outline to white to obtain the initial floor plan outline without the doors; and In the initial floor plan outline drawing with doors removed, the labeling information of each room is filled in according to the location of each room to obtain the final floor plan outline drawing.

2. The method for extracting the floor plan outline and room information from a floor plan according to claim 1, characterized in that, The step of setting all pixels within the space defined by the outlines of each room in the floor plan to be processed to white to obtain a preliminary floor plan outline includes: Determine the location information of each room in the floor plan to be processed; Based on the location information of each room in the floor plan to be processed, the floor plan to be processed is divided into an inner floor plan outline and an outer floor plan outline. The inner floor plan outline includes image information within the outer boundary of the floor plan outline, and the outer floor plan outline includes image information outside the outer boundary of the floor plan outline. Based on the location information of each room in the floor plan to be processed, all pixels within the space defined by the outlines of each room in the floor plan outline are set to white, resulting in a black and white image of the floor plan outline; and The black and white image of the inner outline of the apartment layout and the outer outline of the apartment layout are stitched together to obtain the primary apartment layout outline.

3. The method for extracting the floor plan outline and room information from a floor plan according to claim 2, characterized in that, The step of setting the pixels at the locations of each door in the primary floor plan outline to white to obtain a primary floor plan outline with the doors removed includes: Determine the position information of each door in the floor plan to be processed; and Based on the position information of each door in the floor plan to be processed, the pixels at the positions of each door in the primary floor plan outline are set to white to obtain the primary floor plan outline with the doors removed.

4. The method for extracting the floor plan outline and room information from a floor plan according to claim 2, characterized in that, The process of filling in the room labels at the corresponding positions in the initial floor plan outline (with doors removed) to obtain the final floor plan outline includes: By recognizing the text information in the floor plan outline drawing, the annotation information of each room can be obtained; and The labeling information of each room is backfilled into the initial floor plan outline drawing after removing the doors to obtain the final floor plan outline drawing.

5. The method for extracting the floor plan outline and room information from a floor plan according to claim 3, characterized in that, Determining the location information of each room and each door in the floor plan to be processed includes: Obtain multiple floor plans; Each of the multiple floor plans is labeled with room location information and door location information to obtain multiple floor plans with room labels and multiple floor plans with door labels. The computer vision model is trained based on the multiple floor plans with room annotations to obtain the room location information extraction model; The computer vision model is trained based on the multiple floor plans with door annotations to obtain a door location information extraction model; and The floor plan to be processed is input into the room location information extraction model and the door location information extraction model respectively to obtain the location information of each room and the location information of each door in the floor plan to be processed.

6. The method for extracting the floor plan outline and room information from a floor plan according to claim 2, characterized in that, Based on the location information of each room in the floor plan to be processed, the floor plan is split into an inner outline drawing and an outer outline drawing, including: A binary map of the rooms is obtained based on the location information of each room in the floor plan to be processed. The binary image of the room is dilated and denoised to obtain a binary image of the apartment layout; and The floor plan to be processed is split into an inner floor plan and an outer floor plan based on the binary image of the floor plan outline.

7. The method for extracting the floor plan outline and room information from a floor plan according to claim 6, characterized in that, The step of obtaining a binary map of the rooms based on the location information of each room in the floor plan to be processed includes: The floor plan to be processed is segmented to separate each room, and a mask for each room is obtained. The binary image of each room is then obtained based on the mask of each room.

8. The method for extracting the floor plan outline and room information from a floor plan according to claim 6, characterized in that, The process of dilating and denoising the binary image of the room to obtain a binary image of the apartment layout includes: Perform a dilation operation on the room binary map to obtain a dilated binary map; and The dilated binary image is denoised to obtain the binary image of the apartment layout outline.

9. The method for extracting the floor plan outline and room information from a floor plan according to claim 4, characterized in that, The step of identifying text information in the floor plan outline to obtain annotation information for each room includes: Optical character recognition is performed on the inner outline of the apartment to obtain preliminary recognition information; and The preliminary identification information is filtered according to a preset vocabulary to obtain the labeling information for each room.

10. A system for extracting floor plan outlines and room information from a floor plan, characterized in that, include: The floor plan acquisition module is configured to acquire the floor plan to be processed. The primary floor plan outline acquisition module is configured to: set all pixels within the space defined by the outline of each room in the floor plan to be processed to white, thereby obtaining the primary floor plan outline. The module for obtaining a preliminary floor plan outline without doors is configured to: set the pixels at the locations of each door in the preliminary floor plan outline to white, thereby obtaining a preliminary floor plan outline without doors; and The floor plan outline acquisition module is configured to: fill in the label information of each room at the corresponding position in the primary floor plan outline with doors removed, and obtain the final floor plan outline.

11. A computer device, characterized in that, The system includes a memory and a processor. The memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes a method for extracting the floor plan outline and room information of the floor plan according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the processor performs the method for extracting the floor plan outline and room information of the floor plan according to any one of claims 1 to 9.

Images