House type map processing method and device, electronic equipment and computer readable medium
By acquiring and segmenting room types and boundaries in floor plans, along with bounding box information, the problem of contour extraction when adjacent rooms in a floor plan have the same or similar colors was solved, achieving accurate room contour extraction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING ZITIAO NETWORK TECH CO LTD
- Filing Date
- 2022-01-19
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies cannot accurately extract the outline information of rooms when adjacent rooms in a floor plan are drawn with the same or very similar colors. This results in overlapping areas in the room type segmentation results and boundary segmentation results, making it impossible to accurately extract the room outline.
By obtaining the room type segmentation results, room boundary segmentation results, and detection box information of the floor plan, when it is determined that there are at least two rooms in each room type area, the area is segmented according to the detection box information to ensure that there is only one room in each segmented area. Finally, the room outline of each segmented area is extracted based on the room boundary segmentation results.
It enables accurate extraction of room outline information when adjacent rooms in a floor plan have the same or similar colors, avoiding overlapping areas and ensuring the accuracy of room outlines.
Smart Images
Figure CN116523811B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of image processing technology, and in particular to a method, apparatus, electronic device, and computer-readable medium for processing floor plans. Background Technology
[0002] In floor plan retrieval and matching tasks, vectorized digital floor plan information can provide great convenience. The accuracy of vectorized digital floor plan information largely depends on the accuracy of extracting room outlines from the floor plan image. Therefore, accurately extracting room outline information is particularly important. The room outline information includes the number, shape, and location of room outlines in the floor plan.
[0003] However, when adjacent rooms in a floor plan are drawn with the same or very similar colors, the room type segmentation results obtained by using existing technology to segment the floor plan contain overlapping areas. Furthermore, the room boundary segmentation results obtained by segmenting the floor plan by room boundary cannot effectively separate these overlapping areas. Consequently, existing technology cannot accurately extract the outline information of the rooms based on the room boundary segmentation results and the room type segmentation results. Summary of the Invention
[0004] This disclosure is provided to briefly introduce the concepts, which will be described in detail in the subsequent Detailed Description section. This disclosure is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.
[0005] This disclosure provides a floor plan processing method, apparatus, electronic device, and computer-readable medium, which achieves the technical effect of accurately extracting the outline information of rooms from a floor plan even when adjacent rooms are drawn with the same or very similar colors.
[0006] In a first aspect, embodiments of this disclosure provide a floor plan processing method, the method comprising: acquiring room type segmentation results, room boundary segmentation results, and detection box information for each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; the detection box information represents the position and size of the corresponding room in the floor plan; for each area corresponding to a room type in the room type segmentation results, determining the number of rooms located in the area corresponding to that room type; when the number is at least two, cutting the area corresponding to that room type according to the detection box information, so that each cut area contains only one room; and extracting the room outline of each cut area according to the room boundary segmentation results.
[0007] Secondly, embodiments of this disclosure provide a floor plan processing apparatus, the apparatus comprising: an acquisition unit, configured to acquire room type segmentation results, room boundary segmentation results, and detection box information for each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; and the detection box information represents the position and size of the corresponding room in the floor plan; a quantity determination unit, configured to determine the number of rooms located in the area corresponding to each room type in the room type segmentation results; an area cutting unit, configured to, when the number is at least two, cut the area corresponding to the room type according to the detection box information of the room type, so that the number of rooms in each cut area is only one; and a room contour extraction unit, configured to extract the room contour of each cut area according to the room boundary segmentation results.
[0008] Thirdly, embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the floor plan processing method as described in the first aspect.
[0009] Fourthly, embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon that, when executed by a processor, implements the steps of the floor plan processing method as described in the first aspect.
[0010] The floor plan processing method, apparatus, electronic device, and computer-readable medium provided in this disclosure acquire room type segmentation results, room boundary segmentation results, and room detection box information representing the position and size of the corresponding room in the floor plan. For each room type corresponding to the area in the room type segmentation results, when it is determined that the number of rooms in the area corresponding to the room type is at least two, the area corresponding to the room type is cut according to the detection box information of the room type, so that the number of rooms in each cut area is only one, thereby ensuring that there are no overlapping areas in the cut areas. Finally, based on the room boundary segmentation results, the room outlines of each cut area can be accurately extracted, thereby achieving the technical effect of accurately extracting the room outline information from the floor plan. Attached Figure Description
[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0012] Figure 1 This is a flowchart of one embodiment of the floor plan processing method according to the present disclosure;
[0013] Figure 2 This is a schematic diagram of one embodiment of the floor plan according to the present disclosure;
[0014] Figure 3 This is a schematic diagram of an embodiment of the room type division result based on the floor plan of this disclosure;
[0015] Figure 4 This is a schematic diagram of an embodiment of the room boundary segmentation result based on the floor plan of this disclosure;
[0016] Figure 5 This is a schematic diagram showing the distribution of room detection frames according to an embodiment of the floor plan of this disclosure;
[0017] Figure 6 This is a schematic diagram of a structure of an embodiment of the floor plan processing apparatus according to the present disclosure;
[0018] Figure 7 This is a schematic diagram of the basic structure of an electronic device provided according to an embodiment of the present disclosure. Detailed Implementation
[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0020] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.
[0021] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.
[0022] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0023] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0024] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0025] Please refer to Figure 1 This illustrates the flow of one embodiment of the floor plan processing method according to this disclosure. For example... Figure 1 The floor plan processing method shown includes steps 11-14.
[0026] Step 11: Obtain the room type segmentation results, room boundary segmentation results, and detection box information for each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; and the detection box information represents the position and size of the corresponding room in the floor plan.
[0027] Step 12: For each room type in the room type segmentation results, determine the number of rooms located in the area corresponding to that room type.
[0028] Step 13: When the number is at least two, the area corresponding to the room type is divided according to the information of the detection box corresponding to the room type, so that the number of rooms in each divided area is only one.
[0029] Step 14: Based on the room boundary segmentation results, extract the room outline of each segmented region.
[0030] The above methods will be explained in detail below.
[0031] Step 11: Obtain the room type segmentation results, room boundary segmentation results, and detection box information for each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; and the detection box information represents the position and size of the corresponding room in the floor plan.
[0032] In this embodiment, the floor plan can be a black and white image or a color image; please refer to [reference needed]. Figure 2 , Figure 2 This is a schematic diagram of the floor plan provided in this embodiment; Figure 2It includes window 201, wall 202 and room entrance 203.
[0033] In actual implementation, step 11 can be implemented as follows: directly obtain the room type segmentation results, room boundary segmentation results, and detection box information of each room from the third party.
[0034] Please refer to Figure 3 , Figure 3 This embodiment provides the room type segmentation result of the floor plan, wherein... Figure 3 The different styles of room filling areas represent the room coverage areas corresponding to different room types. Figure 3 There is an adhesion area at the mountain pass, from Figure 3 It can be seen that the living room and balcony are incorrectly treated as the same room type; the room type segmentation results include: the room coverage areas corresponding to different room types in the floor plan; room types can be: bedroom, living room, kitchen, bathroom, etc.; please refer to... Figure 4 , Figure 4 This embodiment provides room boundary segmentation results; the room boundary segmentation results include: the boundaries of different types of rooms in the floor plan, such as the walls in the floor plan ( Figure 4 (Used in bold black lines) windows ( Figure 4 The entrance to the room is represented by thin black lines. Figure 4 The boundaries of areas such as (represented by black dashed lines), archways (the unwalled area between the living room and hallway), etc., are defined by these boundaries. Figure 4 There is an adhesion area 401 at the mountain pass location.
[0035] Please refer to Figure 5 , Figure 5 This is a schematic diagram of the detection frames for each room provided in this embodiment; the number of detection frames is the same as the number of rooms in the floor plan; the detection frame can be the circumscribed rectangular detection frame of the corresponding room, such as... Figure 5 As shown; the circumscribed rectangle is the smallest rectangle containing the area of the corresponding room; the information of the detection frame may include: the position of the center point of the detection frame and the length and width detection results; the length and width detection results of the detection frame include: the length and width of the detection frame.
[0036] In one implementation, the information of the detection frame may include: the position of the top left, top right, bottom left, or bottom right corner of the detection frame, as well as the detection results of the length and width of the detection frame.
[0037] As one implementation, the detection frame can also be any other shape that includes the area where the corresponding room is located.
[0038] In one implementation, step 11 includes: step 111-step 112.
[0039] Step 111: Obtain the floor plan.
[0040] In this embodiment, to train and infer the deep learning model, if the pixel size of the floor plan is not 512*512, the floor plan can be completed into a square through image processing methods such as cropping and filling the background color. The completed image can then be further reduced or enlarged to 512*512. In other embodiments, the pixel size of the floor plan can also be reduced or enlarged to a square of other sizes.
[0041] After obtaining the floor plan, proceed to step 112.
[0042] Step 112: Input the floor plan into the pre-trained room segmentation model to obtain the room type segmentation result, the room boundary segmentation result, and the detection box information of each room.
[0043] In the above implementation process, by using a pre-trained room segmentation model to process the floor plan, the room type segmentation results, the room boundary segmentation results, and the detection box information of each room can be obtained quickly. Furthermore, even when it is impossible to obtain the room type segmentation results, the room boundary segmentation results, and the detection box information of each room from a third party, the room type segmentation results, the room boundary segmentation results, and the detection box information of each room can still be accurately obtained from the floor plan.
[0044] As one implementation, before step 112, the method further includes steps A1-A5.
[0045] Step A1: Construct an initial room segmentation model; the initial room segmentation model includes: an initial feature extraction network, an initial room segmentation branch model, and an initial room detection model.
[0046] The initial room segmentation model is a network structure used in deep learning methods.
[0047] The initial feature extraction network can be a VGG16 model or other models used to extract room features from floor plans.
[0048] The output of the initial feature extraction network is connected to the input of the initial room segmentation branch model and the input of the initial room detection model, respectively.
[0049] Step A2: Obtain the training sample set; the training sample set includes: floor plan samples, and the standard detection results of the floor plan samples.
[0050] The number of floor plan samples can be multiple, and each floor plan sample can be a black and white image or a color image; the standard detection results of the floor plan samples include: the standard results of room boundary segmentation, the standard results of room type segmentation, and the standard results of the detection boxes of each room.
[0051] In this embodiment, to train and infer the deep learning model, if the pixel size of the floor plan sample is not 512*512, the pixel size of the floor plan sample is reduced or enlarged to 512*512. In other embodiments, the pixel size of the floor plan sample can also be other sizes.
[0052] The execution order of steps A1 and A2 is not restricted.
[0053] As one implementation method, after constructing the initial room segmentation model and obtaining the training sample set, step A3 is executed.
[0054] Step A3: Use the initial feature extraction network to extract features from the floor plan sample to obtain a shared feature map.
[0055] The initial feature extraction network processes one floor plan sample at a time.
[0056] In this embodiment, if the size of a floor plan sample is 512*512, the shared feature map can include 512 feature maps, each with a size of 16*16; in other embodiments, the size of the shared feature map can also be other.
[0057] In practical implementation, A3 can be implemented as follows: a floor plan sample is input into the initial feature extraction network. The initial feature extraction network performs feature extraction, multiple convolution operations, and downsampling on the floor plan sample to extract room features from the floor plan and obtain a shared feature map. Since the shared feature map only includes room features and does not include other redundant information, it helps to reduce the complexity of room boundary segmentation, the complexity of room type segmentation, and the complexity of obtaining detection box information, and helps to improve the accuracy of room contour extraction.
[0058] Step A3: Upsample the shared feature map using the initial room segmentation branch model to obtain the room boundary segmentation prediction results and room type segmentation prediction results of the floor plan sample.
[0059] In this embodiment, the size of the room boundary segmentation prediction result of a floor plan sample is 512*512; the size of the room type segmentation prediction result of a floor plan sample is 512*512; in other embodiments, the sizes of the room boundary segmentation prediction result and the room type segmentation prediction result can also be other.
[0060] In actual implementation, step A3 can be implemented as follows: the shared feature map is upsampled multiple times using the initial room segmentation branch model to obtain the room boundary segmentation prediction results and room type segmentation prediction results of the floor plan sample.
[0061] As one implementation, the initial room segmentation branch model includes: an initial room boundary segmentation model and an initial room type segmentation model; step A3 includes: steps A31-A32.
[0062] Step A31: Upsample the shared feature map using the initial room boundary model to obtain the room boundary segmentation prediction result of the floor plan sample.
[0063] In actual implementation, step A31 can be implemented as follows: the shared feature map is upsampled multiple times using the initial room boundary model to obtain the room boundary segmentation prediction results of the floor plan sample.
[0064] Step A32: Upsample the shared feature map and the room segmentation prediction result using the initial room type model to obtain the room type segmentation prediction result.
[0065] In actual implementation, step A32 can be implemented as follows: the shared feature map and room segmentation prediction results are upsampled multiple times using the initial room type model to obtain the room type segmentation prediction results.
[0066] In the case that the initial room segmentation branch model includes an initial room boundary segmentation model and an initial room type segmentation model, step A3 can also be implemented as follows:
[0067] The method involves upsampling the shared feature map using an initial room boundary model to obtain the room boundary segmentation prediction result for the floor plan sample, and upsampling the shared feature map using an initial room type model to obtain the room type segmentation prediction result.
[0068] Step A4: Use the room detection model to upsample and convolve the shared feature map to obtain the information prediction result of the detection box of the floor plan sample.
[0069] In one implementation, the detection frame is the circumscribed rectangular detection frame of the corresponding room; the information of the detection frame includes: the position of the center point of the detection frame and the length and width detection results; the initial room detection model includes: the center point detection model and the length and width detection model; step A4 includes: steps A41-A43.
[0070] Step A41: Upsample and convolve the shared feature map using the center point detection model to obtain the center point probability prediction result of the detection box.
[0071] In actual implementation, step A41 can be implemented as follows: after upsampling the shared feature map using the center point detection model, the sampling result is then convolved to obtain the center point probability prediction result of the detection box.
[0072] The center point probability prediction results of the detection box include: center point probability prediction results of rooms of various room types.
[0073] Assuming there are a total of 9 room types in the floor plan, the center point detection model outputs a 512*512*9 feature map. The feature map has 9 layers, each representing the position of the center point of the detection box for one room type. The magnitude of the center point probability prediction result is consistent with the number of room types in the floor plan.
[0074] For the center point detection probability map of each room type, the probability value of the pixel in each room area gradually decreases outward from the center point of the room area. The probability value can be scaled to between 0 and 1 by using a sigmoid operation.
[0075] Step A42: Based on the probability threshold and pooling algorithm, process the center point probability prediction result to obtain the position prediction result of the center point of the detection box of the floor plan sample.
[0076] In actual implementation, step A42 can be implemented as follows: by setting a probability threshold, all probability values in the center point detection probability map that are less than the probability threshold are set to 0, and the center point regions of each room with probability values greater than the probability threshold are determined; using pooling operations, the position of the pixel with the highest probability in the center point region of each room is determined, where the position of the pixel with the highest probability in the center point region of each room is the predicted position of the center point of each detection box.
[0077] Step A43: Use the length and width detection model to upsample and convolve the shared feature map to obtain the length and width prediction results of the detection box of the floor plan sample.
[0078] In actual implementation, step A43 can be implemented as follows: after upsampling the shared feature map using the length and width detection model, the sampling result is then convolved to obtain the length and width prediction results of the detection box.
[0079] The predicted length and width of the detection boxes can be a 512*512*2 feature map or a feature map of other sizes. The predicted length and width are stored numerically at each pixel position in the feature map. Here, 2 represents the number of layers in the feature map, with one layer representing the predicted length of each detection box and another layer representing the predicted width of each detection box. The value of the length and width of each detection box is scaled to between 0 and 1 using a sigmoid operation, representing the ratio of the length (width) of the room detection box to the length (width) of the entire floor plan.
[0080] Step A5: Optimize the initial feature extraction network, the initial room segmentation branch model, and the initial room detection model using the room boundary segmentation prediction results, the room type segmentation prediction results, the information prediction results, and the corresponding standard detection results to obtain the trained room segmentation model.
[0081] In actual implementation, step A5 can be implemented as follows: using the backpropagation algorithm, room boundary segmentation prediction results, room type segmentation prediction results, information prediction results, and corresponding standard detection results, optimize the initial feature extraction network, the initial room segmentation branch model, and the initial room detection model until the determined loss value is less than the preset value, or the number of optimizations reaches the preset number, then stop the optimization and take the room segmentation model with the smallest loss value as the final trained room segmentation model.
[0082] Step 12: For each room type in the room type segmentation results, determine the number of rooms located in the area corresponding to that room type.
[0083] In actual implementation, step 12 can be implemented as follows: for each room type in the room type segmentation results, determine the number of detection boxes located in the area corresponding to that room type based on the information of each detection box.
[0084] Step 13: When the number is at least two, the area corresponding to the room type is divided according to the information of the detection box corresponding to the room type, so that the number of rooms in each divided area is only one.
[0085] In one implementation, step 13 includes: step 131-step 132.
[0086] Step 131: For each detection box corresponding to the room type, determine the area intersection ratio between the detection box and the area corresponding to the room type based on the information of the detection box.
[0087] Step 132: Based on the area intersection-union ratio and detection box information corresponding to the room type, the area corresponding to the room type is divided so that the number of rooms in each divided area is only one.
[0088] In actual implementation, step 132 can be implemented as follows: for each room type, sort the area intersection-union ratio of each detection box corresponding to that room type by size to obtain the sorting result. Based on the sorting result and the information of the room detection boxes, cut the area corresponding to that room type so that there is only one room in each cut area, and obtain the area belonging to each room detection box.
[0089] Step 14: Based on the room boundary segmentation results, extract the room outline of each segmented region.
[0090] In practice, step 14 can be implemented as follows: based on the room boundary segmentation results, use OpenCV's findContours function or other functions to extract the room contours of each segmented region. It is worth noting that the room contours extracted from the floor plan include the following information: the number of rooms in the floor plan, and the position and shape of each room contour.
[0091] As one implementation, the method further includes step 15.
[0092] Step 15: Obtain the location of the room names in each room of the floor plan; wherein each room name corresponds one-to-one with each room area.
[0093] In practice, step 15 can be implemented as follows: obtain the location of each room name in the floor plan from a third party. Each room area in the floor plan includes one room name.
[0094] In one implementation, step 15 includes: step 151-step 152.
[0095] Step 151: Obtain the floor plan.
[0096] Step 152: Identify the room names in the floor plan to obtain the location of each room name.
[0097] As one implementation method, step 152 can be carried out as follows: using optical character recognition technology, the room names in the floor plan are identified to obtain the location of each room name.
[0098] Among them, optical recognition technology can be OCR recognition technology or other technologies that can recognize text content and location.
[0099] As one implementation method, step 152 can be carried out as follows: using a pre-trained room name recognition model, the room names in the floor plan are identified to obtain the location of each room name.
[0100] If the location of the room names in each room of the floor plan is obtained, step 13 can also be implemented as follows: for each room type in the room type segmentation result, determine the number of room names in the area corresponding to that room type; wherein, the number of room names corresponding to that room type is the number of rooms corresponding to that room type.
[0101] Specifically, for each room type in the room type segmentation results, the number of room names located in the area corresponding to that room type is determined based on the location of each room name.
[0102] In one implementation, after step 15, the method further includes step 13.
[0103] Step 13: Determine the room name corresponding to each detection box based on the location of each room name and the information of each detection box.
[0104] In one implementation, step 13 includes: step 131-step 132.
[0105] Step 131: Determine the Euclidean distance between each room name and each detection frame based on the location of each room name and the information of each detection frame.
[0106] In actual implementation, step 131 can be implemented as follows: determine the Euclidean distance between each room name and each detection frame based on the location of each room name and the location of the center point of each detection frame.
[0107] As one implementation method, step 131 can also be implemented as follows: determine the Euclidean distance between each room name and each detection frame based on the position of each room name and the position of the upper left, upper right, lower left or lower right corner of each detection frame.
[0108] Step 132: Determine the room name corresponding to each detection box based on the Euclidean distance and the Hungarian algorithm.
[0109] Correspondingly, once the room names corresponding to each detection box have been determined, step 13 can also be implemented as follows: based on the information of the detection box corresponding to the room type and the position of the room name, the area corresponding to the room type is divided so that there is only one room in each divided area.
[0110] Specifically, based on the information of the detection box corresponding to the room type, the area corresponding to the room type is segmented so that there is only one room name in each segmented area.
[0111] As one implementation, after step 12, the method further includes: for each room type, when the number of rooms in the area corresponding to the room type is one, extracting the room outline of the area corresponding to the room type based on the room boundary segmentation result to obtain the room outline.
[0112] Further reference Figure 6 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a floor plan processing device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.
[0113] like Figure 6 As shown, the floor plan processing device of this embodiment includes: an acquisition unit 610, a quantity determination unit 620, a region cutting unit 630, and a room outline extraction unit 640; wherein, the acquisition unit 610 is used to acquire the room type segmentation result, the room boundary segmentation result, and the detection box information of each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; the detection box information represents the position and size of the corresponding room in the floor plan; the quantity determination unit 620 is used to determine the number of rooms located in the region corresponding to each room type in the room type segmentation result; the region cutting unit 630 is used to cut the region corresponding to the room type according to the detection box information of the room type when the number is at least two, so that the number of rooms in each cut region is only one; the room outline extraction unit 640 is used to extract the room outline of each cut region according to the room boundary segmentation result.
[0114] In this embodiment, the specific processing of the floor plan processing device’s acquisition unit 610, quantity determination unit 620, area cutting unit 630 and room outline extraction unit 640 can be referred to the above method embodiment, and will not be repeated here.
[0115] As one implementation, the area cutting unit 630 includes: an area intersection-union ratio determination unit, used to determine the area intersection-union ratio between the detection frame and the area corresponding to the room type for each detection frame corresponding to the room type, based on the information of the detection frame; and a cutting subunit, used to cut the area corresponding to the room type based on the area intersection-union ratio and the information of the detection frames, so that the number of rooms in each cut area is only one.
[0116] In one embodiment, the device further includes: a location determination unit, which obtains the location of the room names in each room of the floor plan; wherein each room name corresponds one-to-one with each room area; wherein the quantity determination unit 620 is specifically used to determine the number of room names located in the area corresponding to each room type in the room type segmentation result; wherein the number of room names corresponding to the room type is the number of rooms corresponding to the room type.
[0117] In one implementation, the location determination unit includes: a floor plan acquisition unit for acquiring the floor plan; and an identification unit for identifying the room names in the floor plan to obtain the location of each room name.
[0118] In one implementation, the recognition unit is specifically used to identify the room names in the floor plan using optical character recognition technology, thereby obtaining the location of each room name.
[0119] In one embodiment, the device further includes: a matching unit, configured to determine the room name corresponding to each detection frame based on the location of each room name and the information of each detection frame; wherein, the area cutting unit 630 is further configured to cut the area corresponding to the room type based on the information of the detection frame corresponding to the room type and the location of the room name, so that the number of rooms in each cut area is only one.
[0120] In one implementation, the matching unit is specifically used to determine the Euclidean distance between each room name and each detection box based on the location of each room name and the information of each detection box; and to determine the room name corresponding to each detection box based on each Euclidean distance and the Hungarian algorithm.
[0121] In one implementation, the acquisition unit 610 is specifically used to acquire the floor plan; and input the floor plan into a pre-trained room segmentation model to obtain the room type segmentation result, the room boundary segmentation result, and the detection box information of each room.
[0122] In one embodiment, the apparatus further includes: a model building unit for building an initial room segmentation model; the initial room segmentation model includes: an initial feature extraction network, an initial room segmentation branch model, and an initial room detection model; a sample acquisition unit for acquiring a training sample set; the training sample set includes: floor plan samples and standard detection results of the floor plan samples; a feature extraction unit for extracting features from the floor plan samples using the initial feature extraction network to obtain a shared feature map; a segmentation unit for upsampling the shared feature map using the initial room segmentation branch model to obtain room boundary segmentation prediction results and room type segmentation prediction results of the floor plan samples; a detection box detection unit for upsampling and convolving the shared feature map using the initial room detection model to obtain information prediction results of the detection boxes of the floor plan samples; and a model optimization unit for optimizing the initial feature extraction network, the initial room segmentation branch model, and the initial room detection model using the room boundary segmentation prediction results, the room type segmentation prediction results, the information prediction results, and the corresponding standard detection results to obtain the trained room segmentation model.
[0123] In one implementation, the segmentation unit is specifically used to upsample the shared feature map using the initial room boundary model to obtain the room boundary segmentation prediction result of the floor plan sample; and to upsample the shared feature map and the room segmentation prediction result using the initial room type model to obtain the room type segmentation prediction result.
[0124] In one implementation, the detection box detection unit is specifically used to upsample and convolve the shared feature map using the center point detection model to obtain the center point probability prediction result of the detection box; and to process the center point probability prediction result based on a probability threshold and pooling algorithm to obtain the position prediction result of the center point of the detection box of the floor plan sample; and to upsample and convolve the shared feature map using the length and width detection model to obtain the length and width prediction result of the detection box of the floor plan sample.
[0125] The following is for reference. Figure 7 This document illustrates a structural diagram of an electronic device (terminal device or server) suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.
[0126] like Figure 7 As shown, the electronic device may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0127] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.
[0128] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of embodiments of this disclosure.
[0129] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0130] In some implementations, clients and servers may communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0131] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0132] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: acquire room type segmentation results, room boundary segmentation results, and detection box information for each room in a floor plan; wherein, the room type represents the category to which the corresponding room belongs; the detection box information represents the position and size of the corresponding room in the floor plan; for each room type in the room type segmentation results, determine the number of rooms located in the area corresponding to that room type; when the number is at least two, cut the area corresponding to that room type according to the detection box information so that each cut area contains only one room; and extract the room outlines of each cut area according to the room boundary segmentation results.
[0133] Computer program code for performing the operations of this disclosure can be written in one or more programming languages or a combination thereof, including but not limited to object-oriented programming languages such as Java, procedural Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0134] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0135] The units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the units are not, in some cases, intended to limit the specific unit.
[0136] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application-Specific Standard Products (ASICs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0137] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0138] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.
[0139] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0140] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for processing floor plans, characterized in that, The method includes: The system obtains the room type segmentation results, room boundary segmentation results, and detection box information for each room from the floor plan; wherein, the room type represents the category to which the corresponding room belongs; and the detection box information represents the position and size of the corresponding room in the floor plan. For each room type in the room type segmentation results, determine the number of rooms located in the area corresponding to that room type; When the number is at least two, for each detection frame corresponding to the room type, the area intersection ratio between the detection frame and the area corresponding to the room type is determined based on the information of the detection frame; Based on the area intersection-union ratio and detection box information corresponding to the room type, the area corresponding to the room type is divided so that each divided area contains only one room. Based on the room boundary segmentation results, extract the room outline of each segmented region.
2. The method according to claim 1, characterized in that, The method further includes: Obtain the location of the room names in each room of the floor plan; wherein each room name corresponds one-to-one with each room area; Specifically, determining the number of rooms located within the region corresponding to each room type in the room type segmentation result includes: For each room type in the room type segmentation result, determine the number of room names located in the area corresponding to that room type; wherein, the number of room names corresponding to that room type is the number of rooms corresponding to that room type.
3. The method according to claim 2, characterized in that, The step of obtaining the location of the room names in each room of the floor plan includes: Obtain the floor plan; The room names in the floor plan are identified to obtain the location of each room name.
4. The method according to claim 3, characterized in that, The process of identifying the room names in the floor plan and obtaining the location of each room name includes: Optical character recognition technology is used to identify the room names in the floor plan and obtain the location of each room name.
5. The method according to claim 4, characterized in that, The method further includes: Based on the location of each room name and the information of each detection box, determine the room name corresponding to each detection box; The step of dividing the area corresponding to the room type according to the information of the detection box corresponding to the room type, so that each divided area contains only one room, includes: Based on the information of the detection box corresponding to the room type and the location of the room name, the area corresponding to the room type is divided so that there is only one room in each of the divided areas.
6. The method according to claim 2, characterized in that, The step of determining the room name corresponding to each detection box based on the location of each room name and the information of each detection box includes: Based on the location of each room name and the information of each detection frame, determine the Euclidean distance between each room name and each detection frame. Based on the Euclidean distances and the Hungarian algorithm, the room names corresponding to each detection box are determined.
7. The method according to claim 1, characterized in that, The process of obtaining the room type segmentation results, room boundary segmentation results, and detection box information for each room in the floor plan includes: Obtain the floor plan; The floor plan is input into a pre-trained room segmentation model to obtain the room type segmentation results, the room boundary segmentation results, and the detection box information for each room.
8. The method according to claim 7, characterized in that, Before inputting the floor plan into a pre-trained room segmentation model to obtain the room type segmentation result, the room boundary segmentation result, and the detection box information for each room, the method further includes: Construct an initial room segmentation model; the initial room segmentation model includes: an initial feature extraction network, an initial room segmentation branch model, and an initial room detection model; Obtain a training sample set; the training sample set includes: floor plan samples, and standard detection results of the floor plan samples; The initial feature extraction network is used to extract features from the floor plan sample to obtain a shared feature map; The shared feature map is upsampled using the initial room segmentation branch model to obtain the room boundary segmentation prediction results and room type segmentation prediction results of the floor plan sample; The shared feature map is upsampled and convolved using the initial room detection model to obtain the information prediction results of the detection boxes of the floor plan sample; Using the room boundary segmentation prediction results, the room type segmentation prediction results, the information prediction results, and the corresponding standard detection results, the initial feature extraction network, the initial room segmentation branch model, and the initial room detection model are optimized to obtain the trained room segmentation model.
9. The method according to claim 8, characterized in that, The initial room segmentation branch model includes: an initial room boundary segmentation model and an initial room type segmentation model; the upsampling of the shared feature map using the initial room segmentation branch model to obtain the room boundary segmentation prediction result and the room type segmentation prediction result of the floor plan sample includes: The shared feature map is upsampled using the initial room boundary segmentation model to obtain the room boundary segmentation prediction result of the floor plan sample; The shared feature map and the room boundary segmentation prediction results are upsampled using the initial room type segmentation model to obtain the room type segmentation prediction results.
10. The method according to claim 8, characterized in that, The detection frame is the outer rectangular detection frame of the corresponding room; The information of the detection box includes: the position of the center point of the detection box and the length and width detection results; the initial room detection model includes: a center point detection model and a length and width detection model; the prediction result of the information of the detection box of the floor plan sample by upsampling and convolving the shared feature map using the initial room detection model includes: The shared feature map is upsampled and convolved using the center point detection model to obtain the probability prediction result of the center point of the detection box. Based on probability thresholding and pooling algorithms, the probability prediction results of the center point are processed to obtain the position prediction results of the center point of the detection box of the floor plan sample. The shared feature map is upsampled and convolved using the aforementioned length and width detection model to obtain the length and width prediction results of the detection box of the floor plan sample.
11. A floor plan processing device, characterized in that, The device includes: The acquisition unit is used to acquire the room type segmentation results, room boundary segmentation results, and detection box information of each room in the floor plan; wherein, the room type represents the category to which the corresponding room belongs; and the detection box information represents the position and size of the corresponding room in the floor plan; The quantity determination unit is used to determine the number of rooms located in the area corresponding to each room type in the room type segmentation result. The region segmentation unit is used, when the number is at least two, to determine the area intersection ratio between the detection frame and the region corresponding to the room type for each detection frame corresponding to the room type, based on the information of the detection frame; and to segment the region corresponding to the room type based on the area intersection ratio and the information of the detection frame, so that the number of rooms in each segmented region is only one. The room outline extraction unit is used to extract the room outline of each cut area based on the room boundary segmentation results.
12. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-10.
13. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-10.