Methods, devices, electronic equipment, and storage media for identifying building components

Abstract

This application proposes a method, apparatus, electronic device, and storage medium for identifying building components. The method involves extracting pixel features from a wall detail drawing, determining a first prediction result for the area covered by each pixel based on these pixel features, including the probability that the area covered by the pixel is a wall area in the wall detail drawing; extracting image features from the wall detail drawing, determining a second prediction result based on these image features, including a target area in the wall detail drawing and the probability that the target area is a wall area; and determining the positions of components such as walls, windows, and railings in the wall detail drawing based on the first and second prediction results. This technical solution can automatically identify the position of a wall from a wall detail drawing based on pixel features and image features, eliminating the need for manual identification by those reading the wall detail drawing, thus achieving high identification efficiency.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a method, apparatus, electronic device and storage medium for recognizing building components. Background Technology

[0002] Wall detail drawings depict the precise locations and dimensions of components such as walls, windows, railings, and balconies, serving as crucial information during construction. These drawings may be created by different designers, and the significant stylistic differences between their work can make it difficult for readers to quickly identify the walls within the drawings.

[0003] Therefore, how to quickly identify the wall from the wall details is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] In view of this, this application proposes a method, apparatus, electronic device and storage medium for identifying building components, which can quickly identify a wall from a wall detail drawing.

[0005] The technical solution proposed in this application is as follows:

[0006] In a first aspect, embodiments of this application provide a method for identifying building components, including:

[0007] Extract pixel features from unit pixels in the wall detail drawing, and determine a first prediction result for the area covered by the unit pixel based on the pixel features; the first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail drawing;

[0008] Extract image features from the wall detail drawing, and determine a second prediction result for the wall detail drawing based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located;

[0009] Based on the first prediction result and the second prediction result, the positions of the wall components in the wall detail drawing are determined.

[0010] Secondly, embodiments of this application provide a device for identifying building components, including:

[0011] The first extraction module is used to extract pixel features of unit pixels in the wall detail drawing, and determine a first prediction result for the area covered by the unit pixel based on the pixel features; the first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail drawing;

[0012] The second extraction module is used to extract image features of the wall detail and determine a second prediction result for the wall detail based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located.

[0013] The determination module is used to determine the position of the wall components in the wall detail drawing based on the first prediction result and the second prediction result.

[0014] Thirdly, embodiments of this application provide an electronic device, including:

[0015] Memory and processor;

[0016] The memory is used to store programs;

[0017] The processor is configured to implement any of the above methods by running a program in the memory.

[0018] Fourthly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the method described in any of the above-mentioned embodiments.

[0019] The proposed method for identifying building components involves extracting pixel features from a wall detail drawing and determining a first prediction result for the area covered by the pixel. The first prediction result includes the probability that the area covered by the pixel is the location of a wall component in the wall detail drawing. Next, image features are extracted from the wall detail drawing, and a second prediction result is determined based on these features. The second prediction result includes the target area where the wall component is located, and the probability that the target area is the location of the wall component. Based on the first and second prediction results, the position of the wall component in the wall detail drawing is determined. In other words, the technical solution of this application can automatically determine the position of a wall from a wall detail drawing based on the pixel features of the pixel and the image features of the wall detail drawing, eliminating the need for manual identification by those reading the wall detail drawing, thus offering high speed and efficiency. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating a method for identifying building components provided in an embodiment of this application;

[0022] Figure 2 This is a schematic diagram of the wall location extraction model provided in the embodiments of this application;

[0023] Figure 3 This is a schematic diagram illustrating the effect of extracting wall components according to an embodiment of this application;

[0024] Figure 4 A schematic diagram of the window and railing location extraction model provided in this application embodiment;

[0025] Figure 5 This is a schematic diagram illustrating the effect of extracting window components and window railing components according to an embodiment of this application;

[0026] Figure 6 This is a structural schematic diagram of a building component identification device provided in an embodiment of this application;

[0027] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solution of this application embodiment is applicable to the application scenario of identifying building components in wall detail drawings. By adopting the technical solution of this application embodiment, the location of building components can be automatically identified from the wall detail drawings, and the identification efficiency is high.

[0029] Wall detail drawings depict the precise locations and dimensions of components such as walls, windows, railings, and balconies, serving as crucial information during construction. However, because these drawings may originate from different designers with varying drawing approaches and methods, their formats are inconsistent. Furthermore, the walls themselves often contain numerous similar, miscellaneous lines, making it difficult for readers to quickly identify individual building components. For instance, when reviewing wall detail drawings to determine compliance with design specifications, the inability to quickly identify the components slows down the review process.

[0030] To assist readers in identifying various building components from wall detail drawings, optical character recognition (OCR) technology is used to identify the text annotations of each building component in the wall detail drawings. Based on the text annotations of each building component, the position of each building component in the wall detail drawing is determined. However, since the positions of the text annotations of each building component are not uniform, it is impossible to accurately determine the position of each building component.

[0031] Traditional deep learning models such as Mask R-CNN and DeepLabv3+ have good segmentation capabilities for images in natural scenes. However, the lines in detailed wall drawings are complex, and the segmentation requires high precision. Traditional models often result in missegmentation, model overfitting, and bounding box loss. The obtained mask images also often differ from the test comparison results.

[0032] Therefore, how to quickly identify the wall from the wall details is a technical problem that urgently needs to be solved by those skilled in the art.

[0033] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0034] This application proposes a method for identifying building components. This method can be executed by an electronic device, which can be any device with data and instruction processing capabilities, such as a computer, smart terminal, or server. See also... Figure 1 As shown, the method includes:

[0035] S101. Extract the pixel features of unit pixels in the wall detail drawing, and determine the first prediction result for the area covered by the unit pixel based on the pixel features.

[0036] The aforementioned wall detail drawing refers to an image showing the detailed location and dimensions of the building components. The building components include at least one of the following: wall components, window components, window railing components, balcony components, balcony railing components, and window sill tread components.

[0037] The wall detail drawing is obtained by converting the architectural design drawing into a different format. Specifically, the architectural design drawing is created using computer software and includes the detailed location and dimensions of the building components. The architectural design drawing is then converted into an image format (such as JPG or PNG) to obtain the aforementioned wall detail drawing. The computer software mentioned includes AutoCAD, Adobe Illustrator, CorelDRAW, etc., but this embodiment does not limit the specific software used.

[0038] The aforementioned pixel features refer to a specific value or attribute of each pixel in an image, used to describe and distinguish different images. In some embodiments, pixel features include at least one of color attributes, line texture attributes, position attributes, and boundary gradient attributes. The aforementioned unit pixel refers to one or more pixels; that is, one or more pixels can be considered as a unit pixel, and this embodiment does not impose any limitation.

[0039] The aforementioned first prediction result includes the probability that the area covered by a unit pixel is the area where the wall component is located in the wall detail drawing. Specifically, in the embodiments of this application, pixel features of each unit pixel in the wall detail drawing are extracted so as to predict the probability that the area covered by each unit pixel is the area where the wall component is located in the wall detail drawing based on the pixel features of each unit pixel in the wall detail drawing.

[0040] In some embodiments, a pixel classification model can be pre-trained. The pixel classification model is used to extract the pixel features of each unit pixel in the wall detail drawing and predict the probability that the area covered by the unit pixel in the wall detail drawing is the area where the wall component is located in the wall detail drawing based on the pixel features.

[0041] A large number of detailed wall drawings can be obtained as training samples, and the positions of the walls in the training samples can be marked as training labels. The training process of the pixel classification model is as follows: the training samples are input into the pixel classification model to obtain the prediction results output by the pixel classification model. By comparing the prediction results output by the pixel classification model with the training labels, the loss value of the pixel classification model is determined. The parameters of the pixel classification model are adjusted with the goal of reducing the loss value of the pixel classification model. Then, the above training process is repeated until the loss value of the pixel classification model is less than a set value. The above set value can be set according to the actual situation, and this embodiment does not limit it.

[0042] The pixel classification model can use a convolutional neural network model as the basic model, and this embodiment does not limit it.

[0043] The wall detail drawing is input into the trained pixel classification model so that the pixel classification model can predict the probability that the area covered by each unit pixel is the area where the wall component is located in the wall detail drawing.

[0044] S102. Extract the image features of the wall detail drawing and determine the second prediction result for the wall detail drawing based on the image features.

[0045] The aforementioned image features refer to the effective information used to describe and represent the details of the wall. In some embodiments, image features include color features, texture features, shape features, and spatial relationship features.

[0046] The aforementioned second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located. The location corresponding to the target area refers to the predicted location of the wall component when predicting the wall detail based on its image features. Specifically, in the embodiments of this application, image features of the wall detail are extracted to determine each target area of ​​the wall detail and the probability that each target area is the area where the wall component is located, based on these image features.

[0047] In some embodiments, a region classification model can be pre-trained. The region classification model is used to extract image features of the wall details, predict each target region of the wall details based on the image features, and the probability that each target region is the region where the wall component is located.

[0048] A large number of detailed wall images can be obtained as training samples for the region classification model, and the positions of the walls in the training samples can be marked as training labels for the region classification model. The training process of the region classification model is the same as that of the pixel classification model in the above embodiments, and those skilled in the art can refer to the description in the above embodiments, which will not be repeated here. It should be noted that the region classification model can use a convolutional neural network model as the base model, and this embodiment does not limit it.

[0049] The wall detail drawing is input into the trained region classification model so that the region classification model can predict each target region in the wall detail drawing, and the probability that each target region is the region where the wall component is located.

[0050] It should be noted that in the embodiments of this application, the execution order of steps S101 and S102 is not limited. Step S101 can be executed first, followed by step S102; step S102 can be executed first, followed by step S101; or steps S101 and S102 can be executed simultaneously.

[0051] S103. Based on the first and second prediction results, determine the positions of the wall components in the wall detail drawing.

[0052] In the embodiments of this application, the position of the wall components in the wall detail drawing can be further determined by fusing the first prediction result and the second prediction result.

[0053] Specifically, the second prediction result includes each target area in the wall detail drawing and the probability that each target area is the location of a wall component. The second prediction result can be corrected using the first prediction result. In some embodiments, for each target area, the product of the probability that each target area is the location of a wall component and the probability that the area covered by the target unit pixel is the location of a wall component in the wall detail drawing can be calculated. Here, the target unit pixel refers to the unit pixel covering the target area. If the product is greater than a set value, the location of the target area can be determined to be the location of a wall component in the wall detail drawing. If the product is less than or equal to the set value, the location of the target area can be determined not to be the location of a wall component in the wall detail drawing.

[0054] The above-mentioned settings can be set according to the actual situation, and this embodiment does not impose any limitations.

[0055] In the above embodiments, by extracting pixel features from unit pixels in the wall detail image, a first prediction result is determined based on the pixel features for the area covered by the unit pixel; the first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail image; image features are extracted from the wall detail image, and a second prediction result is determined based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located; based on the first prediction result and the second prediction result, the position of the wall component in the wall detail image is determined. In other words, the embodiments of this application can automatically determine the position of the wall from the wall detail image based on the pixel features of unit pixels and the image features of the wall detail image, without requiring manual identification by the reader of the wall detail image, resulting in high speed and efficiency.

[0056] Moreover, the embodiments of this application are based on pixel features of a unit pixel, which can separate the wall body pixel by pixel, thereby improving the accuracy of wall body component identification.

[0057] As an optional implementation, another embodiment of this application discloses that the steps of the above embodiments—extracting pixel features of unit pixels in the wall detail image, determining a first prediction result for the area covered by the unit pixel based on the pixel features, extracting image features of the wall detail image, determining a second prediction result for the wall detail image based on the image features, and determining the position of the wall component in the wall detail image based on the first and second prediction results—may specifically include the following steps:

[0058] The wall detail drawing is input into a pre-trained wall location extraction model, which extracts pixel features of unit pixels in the wall detail drawing. Based on the pixel features, a first prediction result is determined for the area covered by the unit pixel. Image features of the wall detail drawing are extracted. Based on the image features, a second prediction result is determined for the wall detail drawing. Based on the first and second prediction results, the position of the wall component in the wall detail drawing is determined and output.

[0059] The aforementioned wall location extraction model was trained using sample wall detail drawings as training samples and the locations of wall components in the sample wall detail drawings as training labels.

[0060] A large number of sample wall detail drawings can be obtained as training samples. The positions of wall components in the sample wall detail drawings are marked as training labels to train the wall position extraction model. The training process of the wall position extraction model is the same as the training process of the pixel classification model in the above embodiments. Those skilled in the art can refer to the description in the above embodiments, and it will not be repeated here.

[0061] After the wall location extraction model is trained, the wall detail drawing can be input into the wall location extraction model to obtain the position of the wall components output by the wall location extraction model.

[0062] In the above embodiments, by using the wall position extraction model, the position of the wall components can be extracted quickly and accurately from the wall details.

[0063] As an optional implementation method, such as Figure 2 As shown in another embodiment of this application, the wall location extraction model of the above embodiments is disclosed, which may specifically include: a first backbone network module, an image encoding module (Mask Decoder Module), a pixel parsing module (Pixel Decoder Module), and a semantic segmentation module (Semantic Segment Module).

[0064] The first backbone network module is used to extract image features from the wall details.

[0065] Specifically, considering both the speed of convolutional inference and model size, this embodiment selects the C2f (Context to Fusion) module as the backbone network. The C2f module is a neural network module for extracting multi-scale features. It mainly consists of two parts: an upsampling module and a multi-layer feature fusion module. The upsampling module downsamples high-level features to the same resolution as low-level features for feature fusion. The multi-layer feature fusion module fuses features from different levels, allowing the model to utilize information from both high-level and low-level features simultaneously. Within the C2f module, the multi-layer feature fusion module employs a special attention mechanism to learn and select features at different levels, thereby improving semantic segmentation performance. This attention mechanism automatically learns the importance of features at different levels and strengthens or weakens the contribution of features at specific levels as needed.

[0066] The pixel analysis module is used to extract the pixel features of each unit pixel in the wall detail drawing based on the image features of the wall detail drawing, and to determine the first prediction result for the area covered by each unit pixel based on the pixel features.

[0067] Specifically, such as Figure 2 As shown, the image features of the wall details are input into the pixel decoder unit of the pixel parsing module. 2*2 upsampling is performed to obtain pixel-by-pixel embedding of the image. Pixel features such as color attributes, line texture attributes, position attributes, and boundary gradient attributes are extracted. Then, the pixel features are converted into binary masks. The fully connected layer of the pixel parsing module is used to assign a category to each mask. Finally, the ReLU function is used for prediction to obtain the first prediction result.

[0068] The image encoding module is used to determine a second prediction result for the wall detail based on the image features of the wall detail.

[0069] Specifically, such as Figure 2 As shown, the image encoding module splits the image features of the wall details into a 16*16 patch sequence and converts it into a one-dimensional vector. Four serial Mask Encoders (Mask Decoders) extract image feature embeddings for learnable locations. The Mask Encoder consists of a self-attention structure and an independent feedforward neural network. The self-attention structure performs vector dot product on each patch sequence, establishing an attention relationship between each image sequence. Finally, a softmax activation function is used to generate a probability prediction result for each location, which serves as the second prediction result.

[0070] The semantic segmentation module is used to determine and output the positions of wall components in the wall detail drawing based on the first and second prediction results.

[0071] Specifically, such as Figure 2 As shown, in the semantic segmentation module, the image embedding and pixel embedding features are combined by dot product, and a 1*1 CNN network is used to uniformly predict the number of channels for classification. Finally, the sigmoid activation function in the semantic segmentation module is used to generate inference results for each location.

[0072] In some embodiments, Figure 3 The image shows the locations of wall components extracted from wall details using a wall location extraction model. Figure 3 The area indicated by the gray detection box is the location of the wall component.

[0073] In the above embodiments, the network structure selection of the first backbone network module and the fully connected layer significantly improves the inference speed of the model, saving the segmentation and inference time for individual wall detail drawings when the wall detail drawings are complex. Furthermore, the abstract graphic features extracted by the decoder unit of the pixel parsing module play a supervisory role in pixel-by-pixel recognition, increasing the accuracy of wall segmentation for hierarchical sequences, and exhibiting good robustness against various interfering lines such as viewing lines, windows, railing lines, and insulation layers. It is also suitable for recognizing walls with missing text and various layers and colors.

[0074] As an optional implementation, another embodiment of this application discloses that the method of the above embodiments may further include the following steps:

[0075] The locations of window components and window railing components are extracted from the edges of the masked area in the masked image; the masked image is obtained by masking the locations of wall components in the wall detail drawing.

[0076] Specifically, in the wall detail drawing, window components and window railing components have high feature similarity. Traditional classification algorithms such as K-Nearest Neighbor (KNN) and support vector machines (SVM) are difficult to classify them. Classic classification models in deep learning such as VGG and ResNet do not perform well in classifying window components and window railing components drawn in different ways, and are prone to missed detections or false detections.

[0077] To address the aforementioned technical problems, in the embodiments of this application, the positions of wall components in the wall detail drawing can be masked to obtain a masked image. Masking the wall components prevents the lines of the wall from affecting the identification of the positions of window components and window railing components. Furthermore, since both window components and window railing components are located at the edges of the wall components, attention can be focused on the edges of the masked area in the masked image, and the positions of the window components and window railing components can be extracted from these edges.

[0078] In some embodiments, a window and railing location extraction model can be pre-trained, and a masked image can be input into the window and railing location extraction model so that the window and railing location extraction model can extract and output the location of the window component and the location of the window and railing component from the edge of the masked area in the masked image.

[0079] A large number of detailed wall drawings can be obtained. The locations of wall components in these drawings are then masked to obtain sample masked images. These sample masked images are used as training samples, and the locations of window components and window railing components within these images are used as training labels to train the window and railing location extraction model. The specific training process for the window and railing location extraction model is the same as that for the pixel classification model described in the above embodiments. Those skilled in the art can refer to the descriptions in the above embodiments, and will not be repeated here. It should be noted that the window and railing location extraction model can use a convolutional neural network model as its base model; this embodiment does not limit this approach.

[0080] In the above embodiments, by using the window and railing position extraction model to process the masked image, the positions of the window components and the window railing components in the wall detail drawing can be extracted quickly and accurately.

[0081] As an optional implementation method, such as Figure 4 As shown in another embodiment of this application, the window and railing position extraction model of the above embodiments is disclosed, which may specifically include: a second backbone network module, a feature segmentation module and a self-attention module.

[0082] The second backbone network module is used to extract image features from the masked image to obtain a feature map of the masked image. In some embodiments, the second backbone network module may use a ResNet backbone.

[0083] The feature segmentation module is used to segment the feature map into different scales and extract the image features corresponding to the feature maps at different scales. In some embodiments, such as... Figure 4As shown, the feature segmentation module can use the FPN (Feature Pyramid Network) network and utilize the image pyramid to extract richer and more representative features (Prediction b) from feature maps (Features a) at different scales.

[0084] The self-attention module is used to focus attention on the edges of the masked area, and extracts and outputs the locations of window components and window railing components based on the image features corresponding to feature maps of different sizes.

[0085] like Figure 4 As shown, the self-attention module takes the edge of the masked region in the masked image as the attention area, and performs feature fusion with the image features corresponding to the feature maps of different scales extracted by the feature segmentation module after passing through the max pooling layer and the average pooling layer. Finally, the prediction result is generated by the sigmoid activation function.

[0086] In some embodiments, Figure 5 The image shows the locations of the window components (win) and the window railing components (gan) extracted from the masked image using a window and railing location extraction model.

[0087] In the above embodiments, by using the edge of the masked area in the masked image as the area of ​​attention, the location of the window components and the window railing components in the wall detail drawing can be extracted quickly and accurately.

[0088] As an optional implementation, another embodiment of this application discloses that the method of the above embodiments may further include the following steps:

[0089] Identify balcony annotation text from the wall detail drawing; based on the balcony annotation text, determine the location of balcony components and balcony railing components in the wall detail drawing.

[0090] In the embodiments of this application, balcony label text can be identified from the wall detail drawing. For example, OCR technology can be used to identify balcony label text from the wall detail drawing, or other text recognition technologies in the prior art can be used to identify balcony label text from the wall detail drawing; this embodiment is not limited to any particular method. In some embodiments, the balcony label text includes the text "balcony" in the wall detail drawing.

[0091] Then, based on the balcony annotation text, the location of the balcony components is located from the wall detail drawing, and the corresponding railing is bound to the space as the balcony railing component, that is, the location of the railing in the balcony area is determined as the location of the balcony railing component.

[0092] In the above embodiments, the positions of balcony components and balcony railing components can be quickly located from the wall details based on OCR technology.

[0093] As an optional implementation, another embodiment of this application discloses that the method of the above embodiments may further include the following steps:

[0094] Based on the positional relationship between the window components and the window sill tread components, the positional relationship between the wall components and the window sill tread components, and the position of the window components and the wall components, the position of the window sill tread components in the wall detail drawing is determined.

[0095] Specifically, in the wall detail drawing, the positional relationship between the window component and the window sill step component, and the positional relationship between the wall component and the window sill step component are generally fixed. Therefore, the position of the window sill step component can be identified from the wall detail drawing based on the positional relationship between the window component and the window sill step component, the positional relationship between the wall component and the window sill step component, and the position of the window component and the wall component.

[0096] In some embodiments, a zero-shot learning model can be constructed based on the positional relationship between the window component and the window sill tread component, the positional relationship between the wall component and the window sill tread component, and the position of the window component and the wall component, so as to infer the position of the window sill tread component based on the zero-shot learning model.

[0097] In the above embodiments, the position of the window sill step component can be inferred based on the positional relationship between the window component and the window sill step component, and the positional relationship between the wall component and the window sill step component. This eliminates the need to train a large-volume model, reduces memory usage, and improves computation speed.

[0098] As an optional implementation, another embodiment of this application discloses that the method of the above embodiments may further include the following steps:

[0099] Based on the location of the building components in the wall detail drawing, check the design dimensions of the building components; the building components include at least one of the following: wall components, window components, window railing components, balcony components, balcony railing components, and window sill tread components; determine whether the design dimensions of the building components conform to the design specifications.

[0100] Specifically, after determining the location of the building components in the wall detail drawing, the dimensions of the building components in the wall detail drawing can be further checked. By comparing the dimensions of the building components in the wall detail drawing with the standard dimension comparison table of the building components, it can be determined whether the design dimensions of the building components conform to the design specifications.

[0101] The standard dimension comparison table for setting building components generally specifies the size range of the building components. If the size of the building component in the wall detail drawing is within the specified size range of the standard dimension comparison table, it means that the design size of the building component conforms to the design specifications. If the size of the building component in the wall detail drawing is not within the specified size range of the standard dimension comparison table, it means that the design size of the building component does not conform to the design specifications.

[0102] In some embodiments, the dimensions of wall components, window components, window railing components, balcony components, balcony railing components, and window sill tread components can be determined according to the steps of the above embodiments. The window number information can be found according to the floor plan and the door and window schedule, and the relevant building design specifications can be reviewed.

[0103] In the above embodiments, the wall detail drawings can be automatically reviewed to determine whether they conform to the design specifications, thereby reducing the manpower required for the wall detail drawing review process.

[0104] Corresponding to the above-mentioned method for identifying building components, this application also discloses a device for identifying building components, see [link to relevant documentation]. Figure 6 As shown, the device includes:

[0105] The first extraction module 100 is used to extract pixel features of unit pixels in the wall detail drawing and determine a first prediction result for the area covered by the unit pixel based on the pixel features; the first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail drawing.

[0106] The second extraction module 110 is used to extract image features of the wall detail drawing and determine a second prediction result for the wall detail drawing based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located.

[0107] The determination module 120 is used to determine the position of the wall components in the wall detail drawing based on the first prediction result and the second prediction result.

[0108] As an optional implementation, another embodiment of this application discloses that, in the above embodiments, the first extraction module 100 extracts pixel features of unit pixels in the wall detail drawing and determines a first prediction result for the area covered by the unit pixel based on the pixel features; the second extraction module 110 extracts image features of the wall detail drawing and determines a second prediction result for the wall detail drawing based on the image features; and the determining module 120, when determining the position of the wall component in the wall detail drawing based on the first and second prediction results, is specifically used for:

[0109] The wall detail drawing is input into a pre-trained wall location extraction model, which extracts the pixel features of unit pixels in the wall detail drawing, determines a first prediction result for the area covered by the unit pixel based on the pixel features, extracts the image features of the wall detail drawing, determines a second prediction result for the wall detail drawing based on the image features, and determines and outputs the position of the wall component in the wall detail drawing based on the first and second prediction results.

[0110] The wall location extraction model is trained using sample wall detail drawings as training samples and the locations of wall components in the sample wall detail drawings as training labels.

[0111] As an optional implementation, another embodiment of this application discloses the wall location extraction model of the above embodiments, including: a first backbone network module, an image encoding module, a pixel parsing module, and a semantic segmentation module;

[0112] The first backbone network module is used to extract image features from the wall details;

[0113] The pixel analysis module is used to extract the pixel features of each unit pixel in the wall detail drawing based on the image features of the wall detail drawing, and to determine the first prediction result for the area covered by each unit pixel based on the pixel features.

[0114] The image encoding module is used to determine a second prediction result for the wall detail drawing based on the image features of the wall detail drawing;

[0115] The semantic segmentation module is used to determine and output the positions of wall components in the wall detail drawing based on the first and second prediction results.

[0116] As an optional implementation, another embodiment of this application discloses that the apparatus of the above embodiments further includes:

[0117] The third extraction module is used to extract the location of the window component and the window railing component from the edge of the masked area in the masked image; the masked image is obtained by masking the location of the wall component in the wall detail drawing.

[0118] As an optional implementation, another embodiment of this application discloses that the third extraction module of the above embodiments, when extracting the positions of the window components and the window railing components from the edges of the masked area in the masked image, is specifically used for:

[0119] The masked image is input into the window and railing location extraction model so that the window and railing location extraction model can extract and output the location of the window component and the location of the window and railing component from the edge of the masked area in the masked image.

[0120] The window and railing location extraction model is trained using sample mask images as training samples and the locations of window components and railing components in the sample mask images as training labels.

[0121] As an optional implementation, another embodiment of this application discloses the window and railing position extraction model of the above embodiments, including: a second backbone network module, a feature segmentation module, and a self-attention module;

[0122] The second backbone network module is used to extract image features of the masked image to obtain the feature map of the masked image;

[0123] The feature segmentation module is used to segment the feature map into different sizes and extract the image features corresponding to the feature maps at different scales;

[0124] The self-attention module is used to focus attention on the edges of the masked area. Based on the image features corresponding to feature maps of different scales, it extracts and outputs the locations of window components and window railing components.

[0125] As an optional implementation, another embodiment of this application discloses that the apparatus of the above embodiments further includes:

[0126] The recognition module is used to identify balcony annotation text from wall detail drawings;

[0127] The first determination module is used to determine the location of balcony components and balcony railing components in the wall detail drawing based on the balcony annotation text.

[0128] As an optional implementation, another embodiment of this application discloses that the apparatus of the above embodiments further includes:

[0129] The second determining module is used to determine the position of the window sill step component in the wall detail drawing based on the positional relationship between the window component and the window sill step component, the positional relationship between the wall component and the window sill step component, and the position of the window component and the wall component.

[0130] As an optional implementation, another embodiment of this application discloses that the apparatus of the above embodiments further includes:

[0131] The review module is used to check the design dimensions of building components based on the location of the building components in the wall detail drawing; the building components include at least one of wall components, window components, window railing components, balcony components, balcony railing components and window sill tread components; and to determine whether the design dimensions of the building components conform to the design specifications.

[0132] Specifically, the specific working content of each unit of the above-mentioned building component identification device can be found in the above method embodiment, and will not be repeated here.

[0133] Corresponding to the above-mentioned method for identifying building components, this application also discloses an electronic device, see [link to relevant documentation]. Figure 7 As shown, the electronic device includes:

[0134] Memory 200 and processor 210;

[0135] The memory 200 is connected to the processor 210 and is used to store programs;

[0136] The processor 210 is configured to implement the building component identification method disclosed in any of the above embodiments by running a program stored in the memory 200.

[0137] Specifically, the aforementioned electronic device may further include: a bus, a communication interface 220, an input device 230, and an output device 240.

[0138] The processor 210, memory 200, communication interface 220, input device 230, and output device 240 are interconnected via a bus. Among them:

[0139] A bus can include a pathway for transmitting information between various components of a computer system.

[0140] The processor 210 can be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application. It can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0141] Processor 210 may include a main processor, as well as a baseband chip, modem, etc.

[0142] The memory 200 stores a program for executing the technical solution of this application, and may also store an operating system and other critical business functions. Specifically, the program may include program code, which includes computer operation instructions. More specifically, the memory 200 may include read-only memory (ROM), other types of static storage devices capable of storing static information and instructions, random access memory (RAM), other types of dynamic storage devices capable of storing information and instructions, disk storage, flash memory, etc.

[0143] Input device 230 may include a device for receiving user input data and information, such as a keyboard, mouse, camera, scanner, light pen, voice input device, touch screen, pedometer, or gravity sensor.

[0144] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0145] The communication interface 220 may include a device that uses any transceiver to communicate with other devices or communication networks, such as Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0146] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement the various steps of the building component identification method provided in the above embodiments of this application.

[0147] In addition to the methods and devices described above, embodiments of this application may also be computer program products, which include computer program instructions that, when executed by a processor, cause the processor to perform the various steps of the building component identification method provided in the above embodiments.

[0148] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this application. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0149] Furthermore, embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the various steps of the building component identification method provided in the above embodiments.

[0150] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0151] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0152] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For apparatus embodiments, since they are basically similar to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0153] The steps in the methods of the various embodiments of this application can be adjusted, merged, or deleted in order according to actual needs, and the technical features described in each embodiment can be replaced or combined.

[0154] The modules and sub-modules in the apparatus and terminal in the various embodiments of this application can be merged, divided, and deleted according to actual needs.

[0155] It should be understood, in the several embodiments provided in this application, that the disclosed terminals, devices, and methods can be implemented in other ways. For example, the terminal embodiments described above are merely illustrative; for instance, the division of modules or sub-modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple sub-modules or modules may be combined or integrated into another module, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0156] The modules or submodules described as separate components may or may not be physically separate. The components that constitute a module or submodule may or may not be physical modules or submodules; that is, they may be located in one place or distributed across multiple network modules or submodules. Some or all of the modules or submodules can be selected to achieve the purpose of this embodiment's solution, depending on actual needs.

[0157] Furthermore, the functional modules or sub-modules in the various embodiments of this application can be integrated into one processing module, or each module or sub-module can exist physically separately, or two or more modules or sub-modules can be integrated into one module. The integrated modules or sub-modules described above can be implemented in hardware or in the form of software functional modules or sub-modules.

[0158] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0159] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software unit executed by a processor, or a combination of both. The software unit can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0160] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0161] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for identifying building components, characterized in that, include: Extract pixel features from the wall detail image, and determine a first prediction result for the area covered by the unit pixel based on the pixel features; The first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail drawing; Extract image features from the wall detail drawing, and determine a second prediction result for the wall detail drawing based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located; Based on the first prediction result and the second prediction result, determine the position of the wall components in the wall detail drawing; The step of determining the position of the wall components in the wall detail drawing based on the first prediction result and the second prediction result includes: For the target area, calculate the product of the probability that the target area is the area where the wall component is located and the probability that the area covered by the target unit pixel is the area where the wall component is located in the wall detail drawing. If the product is greater than a set value, determine that the position of the target area is the position of the wall component in the wall detail drawing; if the product is less than or equal to the set value, determine that the position of the target area is not the position of the wall component in the wall detail drawing. Here, the target unit pixel refers to the unit pixel that covers the target area. The method also includes: The window and railing positions are extracted from the edges of the masked area in the masked image using a window and railing position extraction model; the masked image is obtained by masking the positions of the wall components in the wall detail drawing. Specifically, the window and railing location extraction model is used to: extract image features from the masking image to obtain a feature map of the masking image; segment the feature map into different sizes and extract image features corresponding to feature maps of different scales; focus attention on the edges of the masking area, and extract and output the location of the window component and the location of the window railing component based on the image features corresponding to feature maps of different scales.

2. The method according to claim 1, characterized in that, Extracting pixel features from unit pixels in the wall detail image, determining a first prediction result for the area covered by the unit pixel based on the pixel features, extracting image features from the wall detail image, determining a second prediction result for the wall detail image based on the image features, and determining the position of the wall component in the wall detail image based on the first prediction result and the second prediction result, including: The wall detail drawing is input into a pre-trained wall location extraction model, which extracts pixel features of unit pixels in the wall detail drawing, determines a first prediction result for the area covered by the unit pixel based on the pixel features, extracts image features of the wall detail drawing, determines a second prediction result for the wall detail drawing based on the image features, and determines and outputs the position of the wall component in the wall detail drawing based on the first prediction result and the second prediction result. The wall location extraction model is trained using sample wall detail drawings as training samples and the positions of wall components in the sample wall detail drawings as training labels.

3. The method according to claim 2, characterized in that, The wall location extraction model includes: a first backbone network module, an image encoding module, a pixel parsing module, and a semantic segmentation module; The first backbone network module is used to extract image features from the wall detail drawing; The pixel analysis module is used to extract pixel features of unit pixels in the wall detail image based on the image features of the wall detail image, and determine a first prediction result for the area covered by the unit pixel based on the pixel features; The image encoding module is used to determine a second prediction result for the wall detail based on the image features of the wall detail. The semantic segmentation module is used to determine and output the positions of wall components in the wall detail drawing based on the first prediction result and the second prediction result.

4. The method according to claim 1, characterized in that, include: The window and railing location extraction model is trained using sample masking images as training samples and the locations of window components and window railing components in the sample masking images as training labels.

5. The method according to claim 1, characterized in that, The window and railing location extraction model includes: a second backbone network module, a feature segmentation module, and a self-attention module; The second backbone network module is used to extract the image features of the masking image to obtain the feature map of the masking image; The feature segmentation module is used to segment the feature map into different sizes and extract image features corresponding to feature maps of different scales; The self-attention module is used to focus attention on the edge of the masking area, and extract and output the location of the window component and the location of the window railing component based on the image features corresponding to feature maps of different scales.

6. The method according to claim 1, characterized in that, Also includes: Identify the balcony annotation text from the wall detail drawing; Based on the balcony annotation text, the positions of the balcony components and the balcony railing components in the wall detail drawing are determined.

7. The method according to claim 1, characterized in that, Also includes: Based on the positional relationship between the window component and the window sill tread component, the positional relationship between the wall component and the window sill tread component, and the position of the window component and the wall component, the position of the window sill tread component in the wall detail drawing is determined.

8. The method according to claim 1, characterized in that, Also includes: Based on the location of the building components in the wall detail drawing, the design dimensions of the building components are detected. The specified building components include at least one of wall components, window components, window railing components, balcony components, balcony railing components, and window sill tread components; Determine whether the design dimensions of the specified building components conform to the design specifications.

9. A device for identifying building components, characterized in that, include: The first extraction module is used to extract pixel features of unit pixels in the wall detail drawing, and determine a first prediction result for the area covered by the unit pixel based on the pixel features; the first prediction result includes the probability that the area covered by the unit pixel is the area where the wall component is located in the wall detail drawing; The second extraction module is used to extract image features of the wall detail and determine a second prediction result for the wall detail based on the image features; the second prediction result includes the target area where the wall component is located, and the probability that the target area is the area where the wall component is located. A determining module is used to determine the position of a wall component in the wall detail drawing based on the first prediction result and the second prediction result; wherein, determining the position of the wall component in the wall detail drawing based on the first prediction result and the second prediction result includes: for the target area, calculating the product of the probability that the target area is the area where the wall component is located and the probability that the area covered by the target unit pixel is the area where the wall component is located in the wall detail drawing; if the product is greater than a set value, determining that the position of the target area is the position of the wall component in the wall detail drawing; if the product is less than or equal to the set value, determining that the position of the target area is not the position of the wall component in the wall detail drawing; wherein, the target unit pixel refers to the unit pixel covering the target area; The device also includes: The third extraction module is used to extract the locations of window components and window railing components from the edges of the masked area in the masked image using a window and railing location extraction model. The masked image is obtained by masking the locations of the wall components in the wall detail drawing. Specifically, the window and railing location extraction model is used to: extract image features from the masked image to obtain a feature map of the masked image; segment the feature map into different sizes and extract image features corresponding to feature maps of different scales; focus on the edges of the masked area, and extract and output the locations of the window components and window railing components based on the image features corresponding to feature maps of different scales.

10. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is configured to implement the method as described in any one of claims 1 to 8 by running a program in the memory.

11. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.

Images