A deep learning-based super-large architectural drawing sanitary ware identification method

Abstract

The application provides a kind of based on deep learning's super large building drawing sanitary ware identification method, it solves the problems such as large quantities of building drawing component identification, which includes the following steps: S1: data preparation, building drawing in practical application is given to building designer to carry out the annotation of sanitary ware;S2: drawing pretreatment and data set construction, rasterization and sliding window sampling to drawing;S3: network training, after data enhancement, it is sent into detection framework, and detection model is obtained by optimizing loss function iteration;S4: network prediction, image filling and overlapping slice are carried out, and then result fusion is carried out;S5: post-processing, screening is carried out to predicted result;S6: experimental result, evaluate sanitary ware identification.The application has the advantages of high component identification accuracy, without establishing template library and the like.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of building engineering, and particularly relates to a super-large building drawing sanitary ware recognition method based on deep learning. BACKGROUND

[0002] Drainage system design is an important part in the field of building design, mainly positioning sanitary ware to further position water point position, connecting water points to arrange water pipes, thereby completing subsequent water consumption calculation. These operations are based on CAD drawings. The components in the CAD drawing are usually packaged into a block, which is difficult to separate, and the actual drawing has many layers of wall, column network, beam and other non-sanitary information. It is time-consuming and laborious to rely on manual positioning of sanitary ware. Therefore, the type and position of the sanitary ware are detected automatically by the computer, thereby preparing for the next step of positioning the water point and arranging the water pipe. At present, the component recognition method for building drawings is mostly based on template matching. This technology corresponds the coordinates of the current to-be-recognized building drawing to each type of matching module, finds the corresponding key information area of the to-be-recognized building drawing, judges whether the information area matches the type template, and further matches the building components using a sub-component template. This technology has the following problems: a large number of template libraries need to be established manually, but the number of templates and the types of components are limited, and the recognition accuracy of components not saved in the template library is low; a large amount of geometric information needs to be preprocessed in the CAD drawing before template matching, which is computationally intensive and inefficient; template matching has its own limitations, mainly in that it can only be translated, and the algorithm has low robustness to rotated or scaled targets.

[0003] In order to solve the problems in the prior art, people have made long-term exploration and proposed various solutions. For example, Chinese patent document discloses an intelligent recognition method for building engineering drawings [202010455992.2], and the specific steps are as follows: loading the CAD drawing; judging whether the drawing is a standard drawing of a predetermined format, if not, converting it into a predetermined format; dividing the work area and setting the type of data information to be extracted; recognizing the drawing and constructing a drawing model; integrating and connecting the material model.

[0004] The above-mentioned scheme solves the problem of building drawing recognition to some extent, but the scheme still has many problems, such as the need to establish a template library for component recognition in a large number of building drawings. SUMMARY

[0005] The purpose of the present application is to solve the above problems, and to provide a design reasonable, without the need for manual establishment of a template library, based on deep learning, super-large building drawing sanitary ware recognition method.

[0006] In order to achieve the above object, the following technical scheme is adopted in the present application: A super large building drawing sanitary ware recognition method based on deep learning comprises the following steps:

[0007] S1: data preparation, the building drawing in actual application is handed over to the building designer to mark the sanitary ware;

[0008] S2: drawing pretreatment and data set construction, rasterization and sliding window sampling of the drawing;

[0009] S3: network training, after data enhancement, the detection framework is sent, and the detection model is obtained through optimization of the loss function iteration;

[0010] S4: network prediction, the image is filled and overlapped, and then the result fusion is carried out;

[0011] S5: post-processing, the prediction result is screened;

[0012] S6: experimental result, the sanitary ware recognition is evaluated. The rasterized drawing bitmap data is preprocessed, the neural network for target detection task is trained, and can be applied to the water pipe arrangement and drainage calculation of the building.

[0013] In the above-mentioned super large building drawing sanitary ware recognition method based on deep learning, the content marked in step S1 includes the type and the circumscribed rectangle frame of the sanitary ware, so as to obtain the target of network learning.

[0014] In the above-mentioned super large building drawing sanitary ware recognition method based on deep learning, in step S2, the range to be recognized of the CAD drawing is manually framed, the CAD self-printing function is used to print the black and white binary image in mono-chrome mode, the contrast ratio of the CAD model space and the image space is 4 units = 1 pixel; in step S2, the size of the sliding window is determined according to the size range of the sanitary ware, the sampling containing the sanitary ware target is regarded as a positive sample, the sampling not containing the target is regarded as a negative sample, and the negative samples are extracted according to the proportion; in step S2, the coordinates of the two diagonal points of the recognition range in the CAD model space are recorded, which are used for subsequent calculation.

[0015] In the above-mentioned super large building drawing sanitary ware recognition method based on deep learning, in step S2, when the data set is constructed, after the rasterization obtains the raster image, a part of the image is divided into a test set, and another part outside the test set is randomly divided into a training set and a verification set after the sliding window sampling. By establishing the training set and the verification set, subsequent network training is facilitated.

[0016] In the deep learning-based super-large architectural drawing sanitary ware recognition method, the data enhancement in step S3 expands the original data when loading the training set; the detection framework in step S3 is divided into three modules: a feature extraction module, a feature fusion module and a detection module. The enhanced data is sent to the built detection framework, the network model is continuously iterated and optimized, and finally a detection model with good performance is obtained.

[0017] In the deep learning-based super-large architectural drawing sanitary ware recognition method, the loss function in step S3 calculates the difference between the network prediction result and the true value for each predicted target, and continuously iterates to guide the network to minimize the difference, so as to obtain a recognition result closer to the true label.

[0018] In the deep learning-based super-large architectural drawing sanitary ware recognition method, the image padding in step S4 fills the blank around the image; in the overlapping slice in step S4, the slice step is increased, and does not exceed the slice edge length.

[0019] In the deep learning-based super-large architectural drawing sanitary ware recognition method, the result fusion in step S4 converts the result list of the slice into three mask slices m label , m score , m box , takes the center area of the mask slice, and seamlessly splices into a mask M label , M score , M box , and finally calculates the range of each bounding box according to the mask; when splicing the mask slice, the degree of edge coincidence of the current slice and the adjacent left and upper slices is calculated, and if the degree of coincidence reaches a set threshold, the bounding boxes of the adjacent slices can be merged.

[0020] In the deep learning-based super-large architectural drawing sanitary ware recognition method, step S5 uses the diagonal point coordinates recorded in the rasterization process to map the network prediction result to the CAD model space. According to the existing data, the prediction result is screened according to the size of the bounding box, and the results obviously outside the prediction category size distribution are excluded.

[0021] In the deep learning-based super-large architectural drawing sanitary ware recognition method, the evaluation of the sanitary ware recognition in step S6 involves IoU, Precision, Recall and mAP.

[0022] Compared with the prior art, the advantages of the present application are that a large number of template libraries do not need to be established manually, and the data types that have not been seen can be inferred through learning a large number of drawings; the components not saved in the template library have high recognition accuracy; it is robust to various transformations such as rotation and scaling, and can process sanitary wares composed of scattered graphics in CAD. BRIEF DESCRIPTION OF DRAWINGS

[0023] Figure 1 is a project flowchart of the present application;

[0024] Figure 2 is a Swin Transformer network architecture diagram of the present application;

[0025] Figure 3 is a Mask R-CNN network architecture diagram of the present application;

[0026] Figure 4 is a drawing local, rasterized image, drawing space result mapping and image space recognition result on the test set of the present application. DETAILED DESCRIPTION

[0027] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0028] As shown in Figure 1 , a deep learning-based super-large building drawing sanitary ware recognition method comprises the following steps:

[0029] S1: data preparation, the building drawings in actual application are handed over to the building designers for the annotation of sanitary ware, so as to obtain the target for network learning.

[0030] S2: drawing preprocessing and dataset construction, rasterization and sliding window sampling of the drawing, the purpose is to convert the vector graphics into raster images which can be input into the target detection neural network.

[0031] S3: network training, after data enhancement, it is sent into the detection framework, and a good performance detection model is obtained through the iteration of the optimization loss function;

[0032] S4: network prediction, the input of network inference is also limited, while the resolution of the rasterized image is high, usually above 12000000 pixels, the image is filled and overlapped with slices, and then the result is fused;

[0033] S5: post-processing, the prediction results are screened;

[0034] S6: Evaluate the experimental results for sanitary ware recognition. By manually framing the image range that needs to be recognized, rasterizing it into an image that can be used as input to the neural network, and after cutting it into the network and post-processing the results, the category and location of the sanitary ware are obtained. Finally, the coordinates of the sanitary ware in the image space are mapped to the model space of the CAD. Compared with the template matching method, which has low robustness for rotated and scaled sanitary ware targets, and the variety of sanitary ware styles, the deep learning method can quickly learn from a large number of architectural drawings without the need for manual establishment of a corresponding template library, and can analyze the layout of the building and predict the variety of sanitary ware styles.

[0035] Specifically, the content labeled in step S1 includes the type of sanitary ware and the bounding rectangle. The types of sanitary ware include toilets, urinals, squat toilets, wash basins, sinks, mop pools, washing machines, floor drains, shower rooms, bathtubs, and shower heads.

[0036] In depth, step S2 rasterizes the image by manually framing the CAD drawing to be recognized. The CAD comes with a printing function that prints a black and white binary image in mono-chrome mode. The ratio of the CAD model space to the image space is 4 units = 1 pixel. The purpose of using mono-chrome mode is that different designers use different colors and drawing layer orders when drawing the drawing, and color printing mode is not robust for the variety of sanitary ware colors. Using black and white printing can unify the colors of different components and reduce false positives.

[0037] Because the rasterized image has a high resolution, the video memory limits the input of the neural network to a fixed and not very large image, so the original image needs to be sampled using a sliding window, and the sampling block is used as the input for training and inference. The size of the sanitary ware in the image ranges from 14 to 580, and the sanitary ware is usually concentrated in the bathroom, balcony, and kitchen area. The size of the sliding window is set to 1200*1200, and the sliding step is 600. In step S2, the size of the sliding window is determined according to the size range of the sanitary ware. The sampling block containing the sanitary ware target is considered as a positive sample, and the sampling block not containing the target is considered as a negative sample. The negative samples are extracted in proportion, and the number of negative samples is large because the target is usually concentrated. One option is to make the positive sample: negative sample = 1:3. Step S2 records the coordinates of the two opposite corners of the recognition range in the CAD model space, which is used for subsequent calculations.

[0038] As Figure 4As shown, in step S2, when the data set is constructed, after rasterizing to obtain the grid image, a part of the image is divided into a test set, and another part outside the test set is randomly divided into a training set and a validation set in proportion after sliding window sampling. 1 / 10 of the images are divided into a test set for final effect testing. The remaining images are randomly divided into a training set and a validation set in a ratio of 8:1 after sliding window sampling.

[0039] Further, the building drawings that the building designers can actually provide are limited, and more training data is beneficial to the improvement of the performance of the neural network. In order to improve the accuracy of the sanitary ware recognition, a data enhancement strategy is introduced. In step S3, the data enhancement is performed on the original data when the training set is loaded; the overfitting problem caused by too little data is avoided, including rotation, scaling, flipping, random cutting, etc. Especially in practical applications, there are sanitary wares with arbitrary angle inclination, and rotation enhancement can improve the robustness of the network to the rotated target. The rotation angle is in the interval [0, 90], and the interval is 10°.

[0040] In step S3, the detection framework is divided into three modules: a feature extraction module, a feature fusion module, and a detection module. Three network models of Swin Transformer, FPN, and Mask R-CNN are respectively optimized for combination, but other feature extraction networks and detection networks are also applicable to the framework. The feature network includes but is not limited to ResNet, EfficientNet, MobileNet, etc., and the detection network is, for example, Faster R-CNN, Cascade R-CNN, R-FCN, etc.

[0041] The feature extraction module learns the semantic information of the image and models the relationship between pixels and pixels, which is called a backbone network. The actually adopted Swin Transformer structure extracts multi-level features through image blocking, linear embedding, block merging, W-MSA (Windows Multi-head Self Attention) and SW-MSA (Shifted Windows Multi-head Self Attention) etc. The Swin Transformer network architecture is as shown in Figure 2 .

[0042] The feature fusion module processes and reasonably utilizes the multi-level important features extracted by the backbone network, which is beneficial to the learning of the next step of detection, that is, to fuse the low-level spatial information and the high-level semantic information.

[0043] The Mask R-CNN network architecture used by the detection module sends the feature map into a region proposal network (RPN) to obtain a plurality of regions of interest, and then into a Head network, i.e., into a RoiAlign layer for pooling to obtain uniformly sized features, and then into a classification branch, a regression branch, and a mask branch, to obtain class and position results in each branch. The Mask R-CNN network architecture is as shown in Figure 3

[0044] In addition, in step S3, the loss function calculates the difference between the network prediction result and the true value for each predicted target, and continuously iterates to guide the network to minimize the difference, to obtain a recognition result closer to the true label. The total loss is composed of the loss of the RPN network and the loss of the Head network. The RPN network proposes k anchor boxes and obtains 2k classification confidences and 4k anchor box coordinates. The Head network obtains 12 classification confidences and position parameters of a plurality of detection targets, and generates 12 m x m binary masks. The loss function of the Mask R-CNN is as follows:

[0045]

[0046] The loss function of the RPN network is composed of a binary classification loss and a regression position loss, as shown in the following formula:

[0047]

[0048] In the formula, i represents the anchor box index, p i represents the probability of being predicted as foreground, represents the true value (the true value is assigned as 1 for foreground and 0 for background), t i represents the predicted box coordinate parameter, represents the true box coordinate parameter.

[0049] The loss function of the Head network is composed of a multi-classification loss, a regression position loss, and a mask loss, as shown in the following formula:

[0050]

[0051] and is a cross-entropy loss, is a Smooth L1 loss for the predicted position parameter and the true position parameter.

[0052] The cross-entropy loss measures the difference between the true value and the predicted value of the classification, and is calculated as follows:

[0053] ​

[0054] where p i represents the true distribution, q i represents the predicted distribution, i represents the category when the bounding box is predicted. The Smooth L1 loss makes the position parameters (x, y, w, h) of the bounding box predicted by the network approach the true label, and the calculation method is as follows:

[0055]

[0056] In the formula, u represents the parameter-by-parameter difference between the predicted bounding box and the true label.

[0057] At the same time, the image filling in step S4 fills the blank around the image, so that the edge slice can still use the context information; the overlapping slice in step S4 is similar to the sliding window sampling in data preprocessing, and the slice size is 1200*1200, and the slice overlap ratio is set to 15%, that is, the step is 1020. In order to speed up the inference speed, the step can be appropriately increased, but not more than the slice length.

[0058] It can be seen that the result inferred by the detection model is on the slice cut out by the sliding window, and is not a global coordinate, so it is necessary to fuse the results output by the detector. Due to the slice operation, part of the target is located at the edge in the current slice, and only a few pixels, while in the adjacent slice, it is close to the center position, and the prediction box has higher accuracy. In step S4, the result fusion converts the result list of the slice into three mask slices m label , m score , m box , which takes the center area of the mask slice, and seamlessly splices into a mask M label , M score , M box , and finally calculates the range of each bounding box according to the mask; when splicing the mask slice, the degree of overlap of the edge of the current slice with the left and upper adjacent slices is calculated, and if the degree of overlap reaches a set threshold, the bounding boxes of the adjacent slices can be merged.

[0059] It is obvious that, according to data statistics, there are differences in size between various categories, and some categories belong to large targets, such as shower rooms and bathtubs, and some categories belong to small targets, such as floor drains. In this step, according to the existing data, the size of the bounding box of the prediction result is screened, and the results obviously outside the size distribution of the prediction category are excluded. The result predicted by the network is in the coordinate system of the raster image, and step S5 uses the diagonal point coordinates recorded in the rasterization process to map the network prediction result to the CAD model space.

[0060] Preferably, the evaluation of the sanitary ware recognition of step S6 involves IoU, Precision, Recall and mAP. The IoU represents the intersection of the current two rectangular frames and the ratio of the union, and the threshold is usually set in the interval [0.5, 0.95]. When the IoU of the predicted frame and the real frame reaches the threshold, it is considered that the predicted frame is correctly predicted. Precision is the accuracy, which represents the ratio of correctly predicted samples to all predicted samples. Recall is the recall rate, which represents the number of positive samples correctly predicted to all positive samples. mAP (mean Average Precision) is the average precision AP of each recall rate based on the PR curve for each class. The average AP of all classes is obtained by averaging the AP of all classes, which is the mAP. This index combines the class and position prediction to comprehensively measure the average precision of each class at a specific IoU threshold. The quantitative results of the sanitary ware recognition are shown in the following table.

[0061]

[0062] In summary, the principle of the present embodiment is to manually frame the drawing range that needs to be recognized, rasterize it into an image that can be input into a neural network, send it into the network after cutting, and perform post-processing on the results to obtain the category and position of the sanitary ware. Finally, the coordinates of the sanitary ware in the image space are mapped to the model space of the CAD.

[0063] The specific embodiments described herein are merely illustrative of the spirit of the present application. Those skilled in the art of the present application can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, but will not deviate from the spirit of the present application or exceed the scope defined by the appended claims.

[0064] Although the terms such as network model, detection framework, etc. are used more frequently herein, the possibility of using other terms is not excluded. The use of these terms is only to facilitate the description and explanation of the essence of the present application; any interpretation of them as any kind of additional limitation is contrary to the spirit of the present application.

Claims

1. A deep learning-based super-large architectural drawing fixture identification method, characterized in that, It comprises the following steps: S1: data preparation, the building drawing in the actual application is given to the building designer for the annotation of sanitary wares; S2: drawing preprocessing and data set construction, rasterization of the drawing and sliding window sampling; S3: network training, after data enhancement, the data is sent into the detection framework, and the detection model is obtained through the optimization of the loss function iteration; S4: network prediction, the image is filled and overlapped with slices, and then the result is fused; The image filling in step S4 fills the blank around the image; the slice step length is increased in the overlapping slice in step S4, and does not exceed the slice edge length; The result fusion in step S4 converts the result list of the slice into three mask slices m label , m score , m box , takes the central region of the mask slice, and seamlessly splices the mask M label , M score , M box , and finally calculates the range of each bounding box according to the mask; when splicing the mask slice, the degree of overlap of the edge of the current slice with the left and upper adjacent slices is calculated, and if the degree of overlap reaches a set threshold, the bounding boxes of the adjacent slices are merged; S5: post-processing, the prediction result is screened; S6: experimental results, the sanitary ware recognition is evaluated. 2.The method of claim 1, wherein the method further comprises: The annotation in step S1 includes the type and the circumscribed rectangular frame of the sanitary wares. 3.The method of claim 1, wherein the method further comprises: determining a coordinate of the identified fixture in the image based on the coordinate of the identified fixture in the blueprint. In step S2, the rasterization of the drawing is realized by manually framing the CAD drawing to be recognized, and a black and white binary image in jpg format is printed in mono-chrome mode by using the printing function of CAD; the contrast ratio between the CAD model space and the image space is 4 units=1 pixel; in step S2, the size of the sliding window is determined according to the size range of the sanitary wares; the sampling containing the sanitary ware target is regarded as a positive sample, and the sampling without the target is regarded as a negative sample; the negative samples are extracted according to the proportion; in step S2, the coordinates of the two opposite corners of the recognition range in the CAD model space are recorded. 4.The method of claim 3, wherein the method further comprises: In step S2, when the rasterization obtains the raster image, a part of the image is divided into a test set, and another part outside the test set is randomly divided into a training set and a verification set after sliding window sampling.

5. The method for recognizing sanitary ware on ultra-large architectural drawings based on deep learning according to claim 1, characterized in that, In step S3, the data enhancement expands the original data when loading the training set; the detection framework in step S3 is divided into three modules: a feature extraction module, a feature fusion module and a detection module.

6. The method for recognizing sanitary ware in ultra-large architectural drawings based on deep learning according to claim 5, characterized in that, In step S3, the loss function calculates the difference between the network prediction result and the true value for each predicted target, and continuously iterates to guide the network to minimize the difference, so as to obtain a recognition result closer to the true annotation.

7. The method for recognizing sanitary ware in ultra-large architectural drawings based on deep learning according to claim 1, characterized in that, In step S5, the diagonal point coordinates recorded in the rasterization process are used to map the network prediction result to the CAD model space.

8. The method for recognizing sanitary ware in ultra-large architectural drawings based on deep learning according to claim 1, characterized in that, The evaluation of the sanitary ware recognition in step S6 involves IoU, Precision, Recall and mAP.

Images