Method, device and electronic equipment for identifying components in construction engineering drawings

Abstract

Embodiments of the present application provide a kind of identification method, device and electronic equipment of component in construction engineering drawing, the method comprises: the building engineering drawing to be detected is divided into multiple image slices;Adopt the inclined component identification model based on rotating ROI to the inclined direction component in the image slice is identified, in the case where there is inclined component in the image slice, first identification result is obtained;The first identification result includes the identification frame of inclined component and the identification category of inclined component;Based on horizontal vertical component identification model, the component in horizontal direction and vertical direction in the image slice is identified, and second identification result is obtained;The first identification result and second identification result are synthesized according to preset synthesis rule, and the target identification result of multi-angle component is obtained.The breakthrough of multi-angle component identification in modern building engineering drawing examination is realized, and the development of building industry is promoted.

Description

Technical Field

[0001] This invention relates to the field of architectural engineering, and more particularly to a method, apparatus, and electronic device for identifying components in architectural drawings. Background Technology

[0002] Architectural drawings are a fundamental and crucial type of data in the construction industry, reflecting a range of domain knowledge and basic data information about a building from its initial design and construction to its later operation and maintenance. Since signed and stamped PDF pixel-art architectural drawings are currently the only legally valid form of delivered drawings, automated and intelligent review of PDF pixel-art architectural drawings is a current research hotspot in the industry.

[0003] In the field of component recognition in architectural drawings, existing technologies can only identify components in architectural drawings that are parallel to the x-axis and y-axis of the Cartesian coordinate system. They cannot be directly applied to identify components that are at arbitrary angles to the x-axis and y-axis of the Cartesian coordinate system. Summary of the Invention

[0004] This invention provides a method, apparatus, and electronic device for identifying components in architectural engineering drawings, which can identify components in architectural engineering drawings that are at any angle to the x-axis and y-axis of a Cartesian coordinate system.

[0005] In a first aspect, embodiments of the present invention provide a method for identifying components in architectural drawings, the method comprising:

[0006] The architectural drawing to be inspected is divided into multiple image slices;

[0007] A tilted component recognition model based on rotated ROI is used to identify tilted components in the image slice. When tilted components exist in the image slice, a first recognition result is obtained. The first recognition result includes the recognition border of the tilted component and the recognition category of the tilted component.

[0008] The horizontal and vertical components in the image slice are identified based on the horizontal and vertical component recognition model to obtain a second recognition result;

[0009] The first and second identification results are combined according to a preset synthesis rule to obtain target identification results of multi-angle components; the synthesis rule is used to filter redundant component identification results.

[0010] Secondly, embodiments of the present invention provide a device for identifying components in architectural drawings, comprising:

[0011] The slicing module is used to slice the architectural engineering drawing to be inspected into multiple image slices;

[0012] The tilted component recognition module is used to recognize tilted components in the image slice using a tilted component recognition model based on rotation ROI. When tilted components exist in the image slice, a first recognition result is obtained. The first recognition result includes the recognition border of the tilted component and the recognition category of the tilted component.

[0013] The horizontal and vertical component recognition module identifies horizontal and vertical components in the image slice based on the horizontal and vertical component recognition model, and obtains a second recognition result;

[0014] The synthesis module is used to synthesize the first recognition result and the second recognition result according to a preset synthesis rule to obtain the target recognition result of the multi-angle component; the synthesis rule is used to filter redundant component recognition results.

[0015] Thirdly, embodiments of the present invention provide an electronic device, including: a processor and a memory, wherein the processor executes a computer program stored in the memory to implement the method as described in the first aspect.

[0016] Compared with prior art, the present invention has the following advantages:

[0017] In this embodiment of the invention, the architectural engineering drawing is first segmented, and then tilted component identification and horizontal and vertical component identification are performed on the segmented image slices respectively. The tilted component identification adopts a tilted component identification model based on rotated ROI, and innovatively uses the e2cnn function library for convolution processing in the feature extraction part. Then, combined with the RPN network and ROI Transformer network, rotated ROI generation and corresponding feature extraction are performed to generate a first feature map. The first feature map is then innovatively aligned to generate a second feature map, achieving consistency of the component's feature map in space and direction. Based on the second feature map, the tilted component is classified and the identification bounding box is regressed, thus completing the tilted component identification. Then, combined with the identification results of horizontal and vertical components, the results are merged according to preset rules to finally obtain the target identification results of multi-angle components. This greatly improves the efficiency and accuracy of automated review of architectural engineering drawings and promotes the development of the construction industry. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Furthermore, it should be noted that the various data-related processes in the embodiments of this application are all carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0019] Figure 1 A flowchart illustrating an embodiment of a method for identifying components in architectural drawings according to the present invention is shown;

[0020] Figure 2 A flowchart illustrating an embodiment of a method for synthesizing component identification results in architectural engineering drawings according to the present invention is shown;

[0021] Figure 3 A structural block diagram of an embodiment of a component identification device in architectural drawings according to the present invention is shown;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Architectural drawings are a fundamental and crucial type of data in the construction industry, reflecting a range of domain knowledge and basic data information about a building from its initial design and construction to its later operation and maintenance. Since signed and stamped PDF pixel-art architectural drawings are currently the only legally valid form of delivered drawings, automated and intelligent review of PDF pixel-art architectural drawings is a current research hotspot in the industry.

[0025] In the prior art, only components parallel to the x-axis and y-axis of the Cartesian coordinate system in architectural drawings can be identified. It cannot be directly applied to identify components at arbitrary angles to the x-axis and y-axis of the Cartesian coordinate system. In order to solve the above problems, the present invention proposes a method, device, electronic device and readable storage medium for identifying components in architectural drawings.

[0026] Reference Figure 1The flowchart illustrates an embodiment of a method for identifying components in architectural drawings according to the present invention. The method may include:

[0027] Step 101: Divide the architectural engineering drawing to be detected into multiple image slices.

[0028] Architectural engineering drawings are diagrams drawn according to relevant national engineering construction standards, using projection methods to represent the shape and size of engineering objects. They accurately express the design content and technical requirements of a building's architecture, structure, and equipment. The architectural engineering drawings to be tested in this embodiment of the invention include, but are not limited to, image formats such as JPG, PNG, and GIF, or PDF format.

[0029] In embodiments of the present invention, multi-angle recognition is performed based on pixels in architectural drawings. A pixel refers to a small square in the image, each with a defined location and assigned color value. The color and position of the square determine the appearance of the image. A pixel can be considered an indivisible unit or element within the entire image. Indivisible means it cannot be further divided into smaller units or elements; it exists as a single-colored square. Each bitmap image contains a certain number of pixels, which determine the size of the image displayed on the screen.

[0030] Because the entire architectural drawing has too many pixels, overall recognition would result in excessive computation and long recognition time. Therefore, it is necessary to balance recognition accuracy and efficiency. The architectural drawing to be detected is segmented in the horizontal and vertical directions, and the architectural drawing to be detected is cut into multiple image slices.

[0031] A slice of 1024 pixels in length and width can be used, with a step size of 400 pixels. Taking a JPG architectural drawing A with a length of 13339 pixels and a width of 9494 pixels as an example, when segmenting drawing A, the first slice B corresponds to a rectangular area in A bounded by the coordinates of the top left corner (0, 0) and the bottom right corner (1024, 1024). The next slice C to the right of slice B corresponds to a rectangular area in A bounded by the coordinates of the top left corner (400, 0) and the bottom right corner (1424, 1024). The next slice D below slice B corresponds to a rectangular area in A bounded by the coordinates of the top left corner (0, 400) and the bottom right corner (1024, 1424). To ensure that each slice is the same size, when segmenting engineering drawing A, the area in A corresponding to the rightmost slice E, which is located on the same horizontal direction as slice B, is a rectangular area enclosed by the coordinates of the upper left corner (12315, 1024) and the lower right corner (13339, 1024). The above example is only for illustrating the segmentation process of architectural engineering drawing images. The pixel size, step size, and pixel size of the architectural engineering drawing are not limited by the above and can be adjusted according to the actual situation.

[0032] Step 102: Use a tilted component recognition model based on rotation ROI to identify tilted components in the image slice. If tilted components exist in the image slice, obtain a first recognition result. The first recognition result includes the recognition border of the tilted component and the recognition category of the tilted component.

[0033] The inclined component refers to a component in an architectural drawing that is not parallel to the x-axis and y-axis of a Cartesian coordinate system.

[0034] The ROI (Region of Interest) refers to the region of interest. In machine vision and image processing, the region to be processed from the image is delineated using rectangles, circles, ellipses, irregular polygons, etc., and is called the ROI. Generally, the ROI can be delineated with a rectangle. When the longer side of the delineated rectangle is parallel to the x-axis in a Cartesian coordinate system, it can be called a horizontal ROI. When the longer side of the delineated rectangle is not parallel to the x-axis in the same Cartesian coordinate system, or when it is at an angle to the x-axis in the same Cartesian coordinate system, it can be called a rotated ROI.

[0035] When there are tilted components in the image slice, this embodiment of the invention extracts features from the image slice to generate a feature map of the image slice. Then, based on the extracted feature map of the image slice, a candidate region for recognition of the component in the image slice, i.e., a rotational region of interest (ROI), is generated. Based on the rotational ROI, a first feature map of the component in the rotational ROI is extracted from the feature map of the image slice. Then, the first feature map is input into a classification model to output the category of the component. The rotational ROI is added to obtain the recognition result of the tilted component, i.e., the first recognition result.

[0036] Optionally, in order to identify the category of the component more accurately, the first feature map can be aligned in space and direction to obtain a second feature map of the rotated ROI. This allows the classification model to classify the component without having to consider the different spatial positions and orientations of the component in the slice, thus improving the efficiency and accuracy of classification.

[0037] Optionally, the feature maps, first feature maps, and second feature maps of the image slices described above can all be represented in matrix form.

[0038] This invention provides an embodiment of a tilted component recognition model based on a rotated ROI, which is used to identify tilted structures in image slices using the method described above.

[0039] Optionally, the tilted component recognition model may include a feature extraction unit, a rotated ROI generation unit, a rotated ROI alignment unit, and a classification and bounding box regression unit.

[0040] The feature extraction unit is used to extract features from the image slices and generate feature maps of the image slices. In this embodiment of the invention, pixel-based feature extraction of the image slices can be performed using convolutional methods, such as residual networks like ResNet18 and ResNet50, or Feature Pyramid Networks (FPN).

[0041] The rotation ROI generation unit is used to generate candidate recognition regions, i.e., rotation ROIs, of components in the image slices based on the feature maps of the image slices generated by the feature extraction unit. Optionally, this unit can be implemented using a Region Proposal Network (RPN) model and an RoI Transformer network model.

[0042] The rotation ROI alignment unit is used to ensure that all components in an image slice maintain spatial and directional consistency, thereby obtaining a second feature map of the rotation ROI of each component. Alignment refers to ensuring that all components maintain spatial and directional consistency, thus reducing directional and spatial variables in the features used for component identification and classification, leading to more accurate classification. Optionally, alignment can be achieved through channel transformation in the feature map; the specific alignment method should be selected based on the model's training performance.

[0043] The classification and bounding box regression unit is used to fine-tune the rotation ROI of the component based on the second feature map and to classify the component, obtaining the identification bounding box and identification category of the tilted component in the engineering drawing, i.e., the first identification result. The classification result also includes the probability, i.e., the confidence level, corresponding to the identification category of the tilted component.

[0044] Optionally, the inclined component recognition model can be trained using various methods such as stochastic gradient descent with impulse and L2 norm regularization. The model's sample set is a dataset constructed based on actual architectural floor plans used to guide construction, for training the inclined component recognition model and the horizontal and vertical component recognition model. For example, each image in the dataset can be 0.24 pixels x 10.24 pixels. Components in each image in the dataset are manually labeled using labelImg and labelme software. The labeled component categories can include at least one of the following: double doors, single doors, sliding doors, windows, columns, stairs, elevators, etc., but are not limited to these seven categories. When using stochastic gradient descent with impulse for model training, the optimizer for model training can be the SGD optimizer, and the learning rate, impulse, and weight decay coefficient can be adjusted as needed. Preferably, for example, in this embodiment of the invention, a learning rate of 0.001, an impulse of 0.9, and a weight decay coefficient of 0.0001 can achieve good recognition results.

[0045] Optionally, the loss function in model training may include various loss functions such as cross-entropy loss function and smooth L1 loss function. The training cutoff condition may be that the training period reaches a certain value or the accuracy of model training. This embodiment of the invention does not limit this, and the training cutoff condition that performs better in actual practice shall prevail.

[0046] Step 103: Based on the horizontal and vertical component recognition model, identify the horizontal and vertical components in the image slice to obtain the second recognition result.

[0047] The second recognition result includes the recognition borders of horizontal and vertical components and the recognition categories of horizontal and vertical components.

[0048] The horizontal and vertical component recognition model is used to identify horizontal and vertical components in image slices. It is constructed using the standard Faster-RCNN model, with ResNet50 and FPN networks used in the feature extraction network part. Other parts can be the same as the standard Faster-RCNN. The image slices are input into the horizontal and vertical component recognition model for processing to obtain the recognition result, which is the second recognition result.

[0049] Optionally, step 103, which involves identifying horizontal and vertical components in the image slice based on a horizontal and vertical component recognition model to obtain a second recognition result, includes:

[0050] Step S301: Extract feature maps of the image slices sequentially using a residual network and an FPN network.

[0051] Residual networks are a type of convolutional neural network characterized by their ease of optimization and the ability to improve accuracy by increasing their depth. Therefore, in this embodiment of the invention, residual networks are first used to perform pixel-based convolution processing on the image slices. Commonly used residual networks include ResNet50 and ResNet18.

[0052] Considering that image slice data may be lost after convolution through a residual network, this embodiment of the application inputs the output of the residual network into an FPN network to generate feature maps of the image slices. The FPN network not only compensates for the data loss in the image slice data after convolution, but also the feature maps of the image slices extracted by the FPN network are unaffected by the size of the target to be identified in the image slice. For example, if two cars of different sizes are in the same image, the feature maps extracted by the FPN network will retain the features of both cars.

[0053] After processing by residual networks and FPN networks, the feature maps of the extracted image slices can retain the data of the image slices to the maximum extent, and can reduce the error of different recognition results of the same type of components due to different sizes, thereby improving the accuracy of component recognition.

[0054] Step S302: Based on the feature map of the image slice generated in step S301, the ROI of the horizontal and vertical components in the input image slice is generated using the RPN network.

[0055] The ROI is a candidate bounding box for horizontal and vertical components in an image slice, represented by four parameters (x, y, w, h), where x and y represent the horizontal and vertical coordinates of the center point of the bounding box, and w and h represent the width and height of the bounding box.

[0056] Step S303: Spatial pooling is performed on each bounding box generated in step S302.

[0057] Spatial pooling refers to extracting a fixed-length feature vector from the feature map region of the corresponding image slice for each candidate bounding box. Each feature vector is fed into a fully connected layer sequence, which branches into two peer output layers. One layer performs classification, outputting the class and confidence score of each ROI in the image slice; the other layer outputs four real values ​​for each class of K objects. Each set of four values ​​encodes the precise bounding box position of each of the K classes, i.e., the candidate bounding box parameters, used to adjust the ROI of the component generated in step S302.

[0058] Step S304: Using the fixed-size feature vector generated in step S303, fine-tune the candidate box generated in step 10 to generate the recognition borders and corresponding category determinations for horizontal and vertical components in the image slice.

[0059] In this embodiment of the invention, the feature vector generated in step S303 is used to predict the accurate position of the target by regressing the offset and scaling scale of the target recognition bounding box relative to the candidate recognition bounding box, thereby generating the recognition bounding boxes of horizontal and vertical components in the image slice; at the same time, the softmax method is used to classify the images in the candidate bounding boxes to obtain the recognition results of the horizontal and vertical components, i.e., the second recognition result.

[0060] In the process of identifying the horizontal and vertical components, steps 302, 303, and 304 are the same as the implementation process of the Faster-RCNN model, except that the FPN model based on residual network is used in the feature extraction part of step 301.

[0061] Step 104: Combine the first and second identification results according to a preset synthesis rule to obtain target identification results for multi-angle components. The synthesis rule is used to filter out redundant component identification results.

[0062] The synthesis process refers to merging the first and second identification results together, and then filtering and deleting redundant identification results to finally generate multi-angle component identification results of the target architectural engineering drawing.

[0063] The redundant component identification results may include repeated and redundant identification results for the same component. In this embodiment of the invention, such redundant results can be filtered and deleted by calculating the overlap ratio between components or by the confidence level corresponding to the component category.

[0064] The redundant component identification results may also include the identification results of inclined components in the architectural drawings in the identification of horizontal and vertical components. Of course, there are also special cases, such as the identification results of double doors containing the identification results of single doors, which should also be filtered and deleted.

[0065] Optionally, step 102 involves using a tilted component recognition model based on rotated ROI to identify tilted components in the image slice. If tilted components exist in the image slice, a first recognition result is obtained, including:

[0066] Step S201: Use a residual network and a feature pyramid network to extract feature maps of the image slices.

[0067] Similar to the horizontal and vertical component recognition model, this step can also use a feature pyramid network (FPN) based on residual networks to extract features from the image slices. The feature map generated by this method can effectively preserve the features of tilted components in the image slices and will not cause excessive data loss due to processing such as convolution.

[0068] Step S202: Based on the feature map of the image slice, a region generation network and a ROI Transformer network are used to generate a rotated ROI result of the component in the image slice; the rotated ROI result includes the rotated ROI of the component in the image slice and the first feature map of the rotated ROI.

[0069] In the Faster-RCNN network, the Region Proposal Network (RPN) is used to extract candidate bounding boxes for target objects. In this embodiment of the invention, candidate bounding boxes for components are extracted from image slices based on the feature maps of the image slices, generating horizontal ROIs for the components.

[0070] The RoI Transformer is a three-stage detection model for rotating target detection. Its core idea is to convert the horizontal RoI output by RPN into a rotating RoI, which mainly includes two parts: RRoI Leaner and RRoI Wrapping.

[0071] This invention generates rotated RoIs by using the RRoI Learner part of the RoI Transformer network model to model the rotation angle of the horizontal ROIs generated by the RPN. The rotated RoI is represented by five parameters (x, y, w, h, θ), where x and y represent the horizontal and vertical coordinates of the candidate bounding box center point, w and h represent the width and height of the candidate bounding box, and θ represents the rotation angle of the candidate bounding box edge. Then, the RRoI Wrapping part of the RoI Transformer network model is used to extract the feature results corresponding to the rotated RoI from the feature map of the image slice, i.e., the first feature map of the rotated ROI. The first feature map of the rotated ROI extracted by RRoI Wrapping contains the parameter information of the rotated ROI and the feature information used for component classification.

[0072] Step S203: Based on the first feature of the rotated ROI, perform alignment processing to obtain the second feature map of the rotated ROI.

[0073] The alignment process refers to processing the feature map to make the various components in the image slice consistent in space and direction. As the name suggests, it makes the spatial and directional information in the second feature map of different components the same, retaining more of the relevant feature information of the components themselves, and improving the accuracy and efficiency of subsequent classification and recognition.

[0074] Step S204: Using the second feature map, obtain the first identification result of the inclined component.

[0075] The first identification result includes the identification category and identification border of the tilted component.

[0076] Optionally, in this embodiment, a matrix straightening operation is performed on the second feature map to obtain a first feature vector for further adjusting the rotational ROI of the component and a second feature vector for classifying the component. The first feature vector contains more detailed and accurate information than the five parameters (x, y, w, h, θ) of the rotational ROI generated by the ROI Transformer, allowing for fine-tuning of the component's rotational ROI. The second feature vector contains the component's own feature information and is used for component classification. Finally, the first and second feature vectors are used to obtain the first identification result of the tilted component.

[0077] Optionally, step S201, which involves extracting feature maps of the image slices based on residual networks and feature pyramid networks, includes:

[0078] Step S2011: Convolve the image slices based on the residual network and the e2cnn function library to obtain convolutional data;

[0079] Step S2012: Based on the feature pyramid network and the e2cnn function library, the convolutional data is processed to generate the feature map of the image slice.

[0080] Optionally, this embodiment of the invention provides a feasible FPN network model combined with the e2cnn function library. In step S2011, the calculation results f1 from layer 11, f2 from layer 23, f3 from layer 41, and f4 from layer 50 of the ResNet50 network are retained and transmitted to the FPN network. The FPN network first uses the convolution operation functions provided by the e2cnn Python function library to perform convolution operations on f1, f2, f3, and f4 respectively, obtaining calculation results c1, c2, c3, and c4. The convolution kernel size is 1x1, and the output channels are 256. Then, using the upsampling functions provided by the e2cnn Python function library, c4 is upsampled to obtain calculation result u4, with a scale factor of 2. Bitwise addition is performed on c3 and u4 to obtain p3. Upsampling is then performed on p3 to obtain calculation result u3, with a scale factor of 2. Bitwise addition is then performed on u3 and c2. The calculation process is as follows: p2 is obtained; upsampling is performed on p2 to obtain u2, with a scale factor of 2; bitwise addition is performed on u2 and c1 to obtain p1; convolution operations are performed on p1, p2, p3, and c4 using the convolution function provided by the Python library e2cnn, respectively, to obtain o1, o2, o3, and o4; max pooling is performed on o4 using the max pooling function provided by the Python library e2cnn to obtain p4, with a kernel size of 1x1 and a stride of 2; finally, o1, o2, o3, o4, and p4 are collectively used as the output of the FPN network. The shape of o1 is (K, N, H, W), where K is the number of image channels, N is the number of orientation channels, H is the height, and W is the width. The shapes of o2, o3, o4, and p4 are the same as o1, but they differ in specific values. The output of the FPN network is the feature extraction results of different levels of the input slice, i.e., the feature map of the image slice.

[0081] In this embodiment of the invention, the e2cnn function library is used for convolution and computation in feature extraction steps S2011 and S2012. By using the e2cnn function library, the specified transformation behavior of the input feature space is guaranteed after the input undergoes transformations such as convolution, so that the feature map of the image slice effectively preserves the rotational and equivariant features of components of different sizes in the input slice.

[0082] When the output of operation f with respect to T(x) is the same as the output of T with respect to f(x), i.e., f(T(x)) = T(f(x)), where T represents a transformation, then operation f exhibits equivariance. This rotational equivariance means that when the feature information of a component undergoes a rotational transformation, the component also undergoes a corresponding change. Rotational equivariance ensures the rationality of subsequent alignment processing of components in space and direction, ensuring that the alignment processing of the first feature map only maintains consistency between different components in space and direction, without affecting other feature information in the feature map.

[0083] Optionally, step S202, which involves generating rotated ROI results for components in the image slice based on the feature map of the image slice using a region generation network and a ROITransformer network, includes:

[0084] Step S2021: Based on the feature map of the image slice, a region generation network is used to generate the horizontal ROI of the component.

[0085] The region generation network is the same as the region generation network in the horizontal and vertical component recognition. It extracts candidate bounding boxes of components from the image slices based on the feature maps of the image slices and generates the horizontal ROI of the components.

[0086] Step S2022: Based on the horizontal ROI, generate the rotated ROI of the component in the image slice using the RRoI Learner part in the ROI Transformer network.

[0087] Step S2023: Based on the RRoI Wrapping part in the ROI Transformer network, extract the first feature map of the rotated ROI from the feature map of the image slice; the first feature map is the feature map corresponding to the rotated ROI.

[0088] The first feature map is the feature map of the component corresponding to the rotated ROI, which includes five parameter information of the rotated ROI (x, y, w, h, θ) and feature information for component classification.

[0089] Optionally, step S203, which involves aligning the first feature map of the rotated ROI to obtain a second feature map of the rotated ROI, includes:

[0090] Step S2031: Based on the rotated ROI and the first feature map, calculate the channel index on the directional channel of the first feature map, and adjust the directional channel corresponding to the channel index to the position of the first directional channel. The calculation formula is as follows:

[0091]

[0092] Where r is the channel index on the first feature map direction channel, θ is the rotation angle of the rotated ROI, and N is the total number of direction channels in the first feature map of the rotated ROI.

[0093] Step S2032: Based on the channel index, perform interpolation calculation on each directional channel in the first feature map according to the following formula to obtain the second feature map:

[0094]

[0095] in, This represents the feature result corresponding to the i-th channel in the directional channel; This represents the calculation result corresponding to the i-th channel in the interpolated direction channel, where i, i+1∈[1,N]; α is the weight parameter.

[0096] The embodiments of the present invention achieve the alignment of different components in space and direction through the above steps, and step S2032 makes the parameter information of the rotated ROI more accurate through the difference method, which is used for the subsequent regression of the identification border of the tilted component.

[0097] Optionally, step S204, which involves using the second feature map to obtain the first identification result of the inclined member, includes:

[0098] Step S2041: Perform matrix straightening operation on the second feature map to obtain the first feature vector and the second feature vector.

[0099] The matrix straightening operation refers to transforming a matrix into a long column vector as needed, thereby extracting the required vector information. In this embodiment of the invention, matrix straightening yields a first feature vector and a second feature vector. The first feature vector, as described above, contains parameter information for the rotated ROI and is used for the regression of the identification bounding box of the tilted component; the second feature vector is used for component classification.

[0100] Step S2042: Based on the first feature vector, the rotated ROI is fine-tuned to obtain the identification bounding box of the tilted component.

[0101] The fine-tuning refers to adjusting the parameters of the rotated ROI generated by the RRoI Learner part using the parameter information of the rotated ROI of the component in the first feature vector, so as to obtain the recognition bounding box of the tilted component.

[0102] Step S2043: Use the cross-entropy loss function classifier to classify the tilted component according to the second feature vector, and obtain the category and corresponding confidence level of the tilted component.

[0103] Alternatively, when classifying inclined components, cross-entropy loss function classifier, mean squared error loss function classifier (MSE) or other classifiers can be used for component classification. In actual use, the optimal classifier can be selected based on the classification effect.

[0104] Optionally, step 104, which involves synthesizing the first and second recognition results according to a preset synthesis rule to obtain target recognition results for multi-angle components, includes:

[0105] Step S401: Merge the first recognition result and the second recognition result to obtain the first merged recognition result.

[0106] The second recognition result includes the recognition category of the horizontal and vertical components and the recognition border of the horizontal and vertical components.

[0107] The merging of the first and second identification results refers to merging the set of identification results for horizontal and vertical components and the set of identification results for inclined components on the engineering drawing and placing them on the same architectural engineering drawing.

[0108] Optionally, during the merging process, the merged identification results can be preprocessed, which may include filtering and deleting the identification results of inclined components from the identification results of horizontal components.

[0109] Step S402: Based on the first merged recognition result, for components of the same category, calculate the overlap ratio between the recognition borders of the components of the same category. When the overlap ratio is greater than the first overlap ratio, delete the recognition results corresponding to the components whose confidence level of the category is less than the first confidence level, and obtain the second merged recognition result.

[0110] The overlap ratio of the identification borders between the components is calculated using the following formula:

[0111]

[0112] Where Ratio represents the overlap ratio between the recognition bounding boxes of the two components, and OBB1 and OBB2 represent the regions enclosed by the recognition bounding boxes corresponding to the recognition results of the two components, respectively.

[0113] When the overlap ratio of two components of the same category is too large, it indicates that the same component may have been identified repeatedly. Generally, the identification results of components with lower category confidence can be deleted, or the first confidence threshold can be set in advance for filtering and deletion.

[0114] Step S403: Using the nonmaximum suppression algorithm and combining it with the classification of building components, the second merged recognition result is deduplicated to obtain the target recognition result of multi-angle components.

[0115] The deduplication process refers to removing any duplicate recognition results that may exist in the second merged recognition result. The deduplication process is performed according to a preset synthesis rule, which may include limitations on any one of the following: the angle between the recognition border of the component and the x-axis, the recognition category of the component, the confidence level of the recognition category of the component, and the overlap ratio between the recognition borders of the components.

[0116] The core idea of ​​the non-maximum suppression algorithm is to search for local maxima and suppress maxima. Therefore, the embodiments of the present invention can use the non-maximum suppression algorithm to control the variables in the synthesis rules to achieve deduplication of the second merging recognition result.

[0117] The reason for combining the classification of building components is that there may be overlapping component categories, such as single-leaf doors and double-leaf doors, which can lead to inaccurate identification. Therefore, if the identification result of a double-leaf door also contains the identification result of a single-leaf door, then only the identification result of the double-leaf door can be retained.

[0118] Optionally, the deduplication process includes at least one of the following:

[0119] Delete component identification results in the second merged identification result whose identification category confidence is lower than the second confidence level; the second confidence level is a pre-set value used to remove identification results with low confidence in the second merged identification result;

[0120] Only retain one component identification result from the second merged identification result that belongs to the first category, has the same identification category, and has overlapping identification borders;

[0121] Delete the component identification results in the second merged identification results whose identification category is single door and whose identification border overlap ratio with the identification category of double door is greater than the second overlap ratio; the second overlap ratio is preset and used to remove the identification results of possible duplicate single doors in double doors.

[0122] The component identification result includes the component identification category and the component identification border.

[0123] Optionally, after obtaining the target recognition result of the multi-angle component as described in step 104, the method further includes:

[0124] Step 105: Based on the target recognition results of the multi-angle components, and combined with the architectural knowledge base, obtain the specific semantic information of the multi-angle components.

[0125] Based on the target recognition results of multi-angle components in the engineering drawings, and combined with domain knowledge, various specific semantic information such as "width" and "bounding box" are calculated for each type of component.

[0126] For example, to illustrate the calculation process of specific semantic information in this step, the following explanation uses the identification of "double-leaf door" and "staircase" components in architectural drawings as examples, but this example is not intended to limit the invention. Based on knowledge in the architectural field, after obtaining the identification borders of the "double-leaf door" and "staircase," the lengths of the long and short sides of the identification borders are calculated respectively. The length of the long side of the "double-leaf door" identification border is the value corresponding to its "width" semantic information; the length of the short side of the "staircase" identification border is the value corresponding to its "width" semantic information. After obtaining the identification border of the "double-leaf door," sampling is performed on the four sides of the identification border to obtain sampling points. With authorization, combined with existing or publicly available pixel engineering drawing wall identification results, the intersection of the sampling points and the wall identification results can be determined, further determining the edges connecting the "double-leaf door" identification border and the wall identification results, and further calculating the "door opening direction" semantic information of the "double-leaf door."

[0127] Optionally, in step 105, based on the target recognition result of the multi-angle component and combined with a knowledge base in the architectural field, specific semantic information of the multi-angle component is obtained, and the method further includes:

[0128] Step 106: Structure the semantic information data and the target recognition result data of the multi-angle components.

[0129] The semantic information obtained in step 105 is added to the target recognition results of the multi-angle components of the architectural drawings obtained in step 104. Then, all the data is organized by component to generate corresponding structured JSON data or other data that can be used for visualization.

[0130] Step 107: Based on the generated structured JSON data or other structured data, return the JSON data or other structured data and the visualization results to the front end for display on the front end interface.

[0131] Optionally, such as Figure 2 As shown, according to the synthesis process described in step 104, this embodiment of the invention provides a specific implementation method, including:

[0132] Corresponding to step S401, merging the first identification result and the second identification result to obtain the first merged identification result may include:

[0133] Step S4011: Select an identification result from the set of first identification results and calculate the minimum value of the angles between the four sides of the identified border and the x-axis in the identification result.

[0134] Step S4012: If the minimum value of the included angle is greater than 15°, delete the recognition results that intersect with the recognition result in the set of second recognition results, and update the set of second recognition results. If the minimum value of the included angle is not greater than 15 degrees, no processing is performed.

[0135] Step S4013: Following the method described in steps S4011 and S4012, traverse all recognition results in the set of first recognition results until the second recognition set is no longer updated.

[0136] Step S4014: Merge the set of first recognition results and the set of updated second recognition results to obtain the set of first merged recognition results.

[0137] Corresponding to step S402, based on the first merged recognition result, for components of the same category, the overlap ratio between the recognition bounding boxes of the components of the same category is calculated. When the overlap ratio is greater than a first overlap ratio, the recognition results corresponding to the categories of components with confidence scores less than the first confidence score are deleted, and a second merged recognition result is obtained, which may include:

[0138] Step S4021: Calculate the overlap ratio of the recognition borders between components of the same category in the set of the first merged recognition results according to the following formula (1):

[0139]

[0140] Where Ratio represents the overlap ratio between the recognition bounding boxes of the two components, and OBB1 and OBB2 represent the regions enclosed by the recognition bounding boxes corresponding to the recognition results of the two components, respectively.

[0141] Step S4022: When the overlap ratio between two components of the same category is greater than 0.7, delete the identification result with the lower category confidence between the two to obtain a set of second merged identification results.

[0142] Corresponding to step S403, using a non-maximum suppression algorithm combined with the classification of building components, the second merged recognition result is deduplicated to obtain target recognition results for multi-angle components, which may include:

[0143] Step S4031: Select an identification result a from the set of second merged identification results. If the confidence level of the identification result a is less than 0.6, delete the identification result a from the second merged identification results and update the set of second merged identification results. If the confidence level of the identification result is not less than 0.6, continue to the next step.

[0144] Step S4032: If the recognition category of recognition result a is "window", "stairs" or "sliding door", delete all recognition results with the same recognition category as recognition result a, and set the recognition border of a to the smallest outer recognition border among all recognition results of the same category as a and recognition result a. At the same time, set the confidence of a to the maximum confidence of all recognition results of the same category as a and recognition result a. Update the set of the second merged recognition results. If the recognition category of recognition result a is not any of "window", "stairs" or "sliding door", continue to the next step.

[0145] Step S4033: When the recognition category of recognition result a is double door, use the above formula (1) to calculate the overlap ratio between the recognition border of all single door categories in the set of the second merged recognition results and the recognition border of a. Keep the recognition results with an overlap ratio greater than 0.7 with the recognition border of a, and delete the recognition results with an overlap ratio of less than 0.7 with the recognition border of a. When the recognition category of recognition result a is not double door, keep recognition result a and do not process it.

[0146] Step S4034: Following the methods described in steps S4031, S4032 and S4033, traverse all recognition results in the set of second merged recognition results until the set of second merged recognition results is no longer updated, and output the updated set of second merged recognition results as the final target recognition result of the multi-angle component.

[0147] In summary, this invention provides a method for identifying components in architectural engineering drawings. In this embodiment, the architectural engineering drawing is first segmented, and then tilted component identification and horizontal / vertical component identification are performed on the segmented image slices. The tilted component identification employs a tilted component identification model based on rotated ROIs, and innovatively uses the e2cnn function library for convolution processing in the feature extraction part. Then, it combines an RPN network and an ROI Transformer network to generate rotated ROIs and extract corresponding features to generate a first feature map. The first feature map is then innovatively aligned to generate a second feature map, achieving consistency in the component's feature map in space and direction. Based on the second feature map, the tilted component is classified and the identification bounding box is regressed, thus completing the tilted component identification. Then, the identification results of horizontal and vertical components are combined according to preset rules and merged to finally obtain the target identification results of multi-angle components. This greatly improves the efficiency and accuracy of automated review of architectural engineering drawings and promotes the development of the construction industry.

[0148] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0149] Reference Figure 3 The diagram illustrates a structural block diagram of an embodiment of a component identification device in architectural drawings according to the present invention. The device 200 may include:

[0150] Slicing module 201 is used to slice the architectural engineering drawing to be detected into multiple image slices;

[0151] The tilted component recognition module 202 is used to recognize tilted components in the image slice using a tilted component recognition model based on rotation ROI. When tilted components exist in the image slice, a first recognition result is obtained. The first recognition result includes the recognition border of the tilted component and the recognition category of the tilted component.

[0152] The horizontal and vertical component recognition module 203 identifies horizontal and vertical components in the image slice based on the horizontal and vertical component recognition model, and obtains a second recognition result;

[0153] The synthesis module 204 is used to synthesize the first recognition result and the second recognition result according to a preset synthesis rule to obtain the target recognition result of the multi-angle component; the synthesis rule is used to filter redundant component recognition results.

[0154] Optionally, the tilted component identification module may include:

[0155] An image slice feature map extraction module is used to extract feature maps of the image slices using a residual network and a feature pyramid network.

[0156] The rotation ROI module is used to generate rotation ROI results for components in the image slice based on the feature map of the image slice using a region generation network and an ROITransformer network; the rotation ROI results include the rotation ROI of the component in the image slice and the first feature map of the rotation ROI;

[0157] The alignment module is used to perform alignment processing based on the first feature of the rotated ROI to obtain the second feature map of the rotated ROI;

[0158] The first recognition result acquisition module is used to acquire the first recognition result of the inclined component using the second feature map.

[0159] Optionally, the image slice feature map extraction module may include:

[0160] The residual network module is used to perform convolution on the image slices based on the residual network and the e2cnn function library to obtain convolution data;

[0161] The FPN network module is used to perform operations on the convolutional data based on the feature pyramid network and the e2cnn function library to generate feature maps of the image slices.

[0162] Optionally, the rotating ROI module may include:

[0163] A horizontal ROI generation module is used to generate horizontal ROIs of components based on the feature maps of the image slices using a region generation network.

[0164] A rotation ROI generation module is used to generate rotation ROIs of components in the image slice based on the horizontal ROI using the RRoILearner part of the ROI Transformer network;

[0165] The first feature map extraction module is used to extract the first feature map of the rotated ROI from the feature map of the image slice based on the RRoI Wrapping part in the ROI Transformer network; the first feature map is the feature map corresponding to the rotated ROI.

[0166] Optionally, the alignment module may include:

[0167] The directional channel module is used to calculate the channel index on the directional channel of the first feature map based on the rotated ROI and the first feature map, and adjust the directional channel corresponding to the channel index to the position of the first directional channel. The calculation formula is as follows:

[0168]

[0169] Where r is the channel index on the first feature map direction channel, θ is the rotation angle of the rotated ROI, and N is the total number of direction channels in the first feature map of the rotated ROI;

[0170] The interpolation module is used to perform interpolation operations on each directional channel in the first feature map based on the channel index according to the following formula to obtain the second feature map:

[0171]

[0172] in, This represents the feature result corresponding to the i-th channel in the directional channel; This represents the calculation result corresponding to the i-th channel in the interpolated direction channel, where i, i+1∈[1,N]; α is the weight parameter.

[0173] Optionally, the first recognition result acquisition module may include:

[0174] The matrix operation module is used to perform matrix straightening operation on the second feature map to obtain the first feature vector and the second feature vector;

[0175] The tilted component identification bounding box generation module is used to fine-tune the rotated ROI based on the first feature vector to obtain the tilted component identification bounding box.

[0176] The tilted component classification module is used to classify the tilted component based on the second feature vector using a cross-entropy loss function classifier, and obtain the category and corresponding confidence level of the tilted component.

[0177] Optionally, the synthesis module may include:

[0178] The merging module is used to merge the first identification result and the second identification result to obtain the first merged identification result;

[0179] The simplified processing module is used to calculate the overlap ratio between the recognition borders of components of the same category based on the first merged recognition result. When the overlap ratio is greater than the first overlap ratio, the recognition result corresponding to the category of the component with a confidence level less than the first confidence level is deleted, and the second merged recognition result is obtained.

[0180] The deduplication module is used to perform deduplication processing on the second merged recognition result by using a nonmaximum suppression algorithm combined with the classification of building components, so as to obtain the target recognition result of multi-angle components.

[0181] Optionally, the deduplication process includes at least one of the following:

[0182] Delete the component identification results in the second merged identification results where the confidence level of the identification category is lower than the second confidence level;

[0183] Only retain one component identification result from the second merged identification result that belongs to the first category, has the same identification category, and has overlapping identification borders;

[0184] Delete the component identification results in the second merged identification results whose identification category is single door and whose identification border overlap ratio with the identification category of double door is greater than the second overlap ratio;

[0185] The component identification result includes the component identification category and the component identification border.

[0186] Optionally, the device may further include:

[0187] The semantic information acquisition module is used to acquire specific semantic information of the multi-angle components based on the target recognition results of the multi-angle components and in conjunction with a knowledge base in the architectural field.

[0188] Optionally, the device may further include:

[0189] The data structuring module is used to add the semantic information obtained from the semantic information acquisition module to the target recognition results of multi-angle components in the architectural engineering drawings obtained from the synthesis module. Then, all the data is organized by component to generate corresponding structured JSON data or other data that can be used for visualization.

[0190] The visualization module is used to return the generated structured JSON data or other structured data, along with the visualization results, to the front end for display on the front-end interface.

[0191] In summary, this invention provides a device for identifying components in architectural engineering drawings. In this embodiment, the architectural engineering drawing is first segmented, and then tilted component identification and horizontal / vertical component identification are performed on the segmented image slices respectively. The tilted component identification employs a tilted component identification model based on rotated ROIs, and innovatively uses the e2cnn function library for convolution processing in the feature extraction part. Then, it combines an RPN network and an ROI Transformer network to generate rotated ROIs and extract corresponding features to generate a first feature map. The first feature map is then innovatively aligned to generate a second feature map, achieving consistency in the component's feature map in space and direction. Based on the second feature map, the tilted component is classified and the identification bounding box is regressed, thus completing the tilted component identification. Then, the identification results of horizontal and vertical components are combined according to preset rules and merged to finally obtain the target identification results of multi-angle components. This greatly improves the efficiency and accuracy of automated review of architectural engineering drawings and promotes the development of the construction industry.

[0192] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0193] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0194] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0195] refer to Figure 4 This document illustrates an electronic device 500 provided in an embodiment of this application, comprising: a processor 501, a memory 502, and a computer program stored on the memory 502 and executable on the processor 501. When the computer program is executed by the processor 501, it implements the steps of the method for identifying components in architectural drawings as described in the method embodiment.

[0196] Embodiments of this disclosure also provide a readable storage medium that, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the component identification method in the architectural drawings described in the foregoing embodiments.

[0197] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0198] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this disclosure are not directed to any particular programming language. It should be understood that the embodiments of this disclosure described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of the embodiments of this disclosure.

[0199] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the present disclosure may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0200] Similarly, it should be understood that, in order to streamline this disclosure and aid in understanding one or more of the various inventive aspects, in the foregoing description of exemplary embodiments of the present disclosure, various features of the embodiments of the present disclosure are sometimes grouped together in a single embodiment, figure, or description thereof. However, this approach to disclosure should not be construed as reflecting an intention that the claimed embodiments of the present disclosure require more features than expressly recited in each claim. Rather, as reflected in the following claims, inventive aspects lie in fewer than all features of a single foregoing disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the present disclosure.

[0201] Those skilled in the art will understand that modules in the device of the embodiments can be adaptively changed and placed in one or more devices different from that embodiment. Modules, units, or components in the embodiments can be combined into a single module, unit, or component, and further, they can be divided into multiple sub-modules, sub-units, or sub-components. Except where at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or device so disclosed. Unless expressly stated otherwise, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0202] The various component embodiments of this disclosure can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the sorting device according to the embodiments of this disclosure. Embodiments of this disclosure can also be implemented as device or apparatus programs for performing some or all of the methods described herein. Such programs implementing embodiments of this disclosure can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0203] It should be noted that the above embodiments are illustrative of embodiments of this disclosure and not restrictive of embodiments of this disclosure, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. Embodiments of this disclosure can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0204] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0205] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the embodiments of the present disclosure. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the embodiments of the present disclosure should be included within the protection scope of the embodiments of the present disclosure.

[0206] The above description is merely a specific implementation of the embodiments of this disclosure, but the protection scope of the embodiments of this disclosure is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the embodiments of this disclosure should be included within the protection scope of the embodiments of this disclosure. Therefore, the protection scope of the embodiments of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method of identifying components in a construction engineering drawing, characterized by, The method comprises the following steps: cutting a building engineering drawing to be detected into a plurality of image slices; identifying a tilted direction component in the image slice by using a tilted component identification model based on a rotating ROI, and obtaining a first identification result in the case that there is a tilted component in the image slice; the first identification result comprises an identification bounding box of the tilted component and an identification category of the tilted component; identifying a horizontal direction component and a vertical direction component in the image slice based on a horizontal and vertical component identification model, and obtaining a second identification result; synthesizing the first identification result and the second identification result according to a preset synthesis rule to obtain a target identification result of a multi-angle component; the synthesis rule is used for screening redundant component identification results; the tilted component identification model based on the rotating ROI comprises the following steps: extracting a feature map of the image slice by using a residual network and a feature pyramid network; generating a rotating ROI result of a component in the image slice by using a region generation network and an ROI Transformer network based on the feature map of the image slice; the rotating ROI result comprises a rotating ROI of the component in the image slice and a first feature map of the rotating ROI; performing alignment processing based on the first feature map of the rotating ROI to obtain a second feature map of the rotating ROI; obtaining the first identification result of the tilted component by using the second feature map; the step of extracting the feature map of the image slice by using the residual network and the feature pyramid network comprises the following steps: performing convolution on the image slice based on a residual network combined with a function library e2cnn to obtain convolution data; generating the feature map of the image slice by performing operation on the convolution data based on a feature pyramid network combined with the function library e2cnn; the step of performing alignment processing based on the first feature map of the rotating ROI to obtain the second feature map of the rotating ROI comprises the following steps: based on the rotating ROI and the first feature map, calculating a channel subscript on a direction channel of the first feature map, adjusting the direction channel corresponding to the channel subscript to a first direction channel position, and the calculation formula is as follows: wherein r is a channel subscript on the first feature map direction channel, is a rotation angle of the rotated ROI, and N is a total number of direction channels of the first feature map of the rotated ROI. based on the channel subscript, performing interpolation operation on each direction channel in the first feature map according to the following formula to obtain the second feature map: wherein, represents the feature result corresponding to the i-th channel of the directional channel; represents the calculation result corresponding to the i-th channel of the interpolated directional channel, is a weight parameter, .​ 2. The method of claim 1, wherein, the step of generating the rotating ROI result of the component in the image slice by using the region generation network and the ROI Transformer network based on the feature map of the image slice comprises the following steps: generating a horizontal ROI of the component by using the region generation network based on the feature map of the image slice; generating the rotating ROI of the component in the image slice by using an RRoI Learner part in the ROI Transformer network based on the horizontal ROI; The first feature map of the rotated ROI is extracted from the feature map of the image slice based on the RRoI Wraping part in the ROI Transformer network; the first feature map is a feature map corresponding to the rotated ROI.

3. The method of claim 1, wherein, The first feature vector and the second feature vector are obtained by performing a matrix straightening operation on the second feature map. The first feature vector is used to fine-tune the rotated ROI, so as to obtain the recognition bounding box of the inclined member. The second feature vector is used to classify the inclined member by using a cross-entropy loss function classifier, so as to obtain the category and corresponding confidence of the inclined member. The second recognition result includes the recognition category of the horizontal and vertical member and the recognition bounding box of the horizontal and vertical member.

4. The method of claim 1, wherein, The first recognition result and the second recognition result are combined according to a preset combination rule to obtain the target recognition result of the multi-angle member, including: The first recognition result and the second recognition result are combined to obtain a first combined recognition result. Based on the first combined recognition result, the overlap ratio between the recognition bounding boxes of the members of the same category is calculated, and when the overlap ratio is greater than a first overlap ratio, the recognition result corresponding to the category of the member with a confidence less than a first confidence is deleted to obtain a second combined recognition result. The second combined recognition result is processed by using a non-maximum suppression algorithm in combination with the classification of the member in the building field to obtain the target recognition result of the multi-angle member. The de-duplication processing at least includes any one of the following:

5. The method of claim 4, wherein, The member recognition result with a confidence less than a second confidence in the recognition category in the second combined recognition result is deleted. Only one of the member recognition results in the second combined recognition result is retained, which has the same recognition category and overlapping recognition bounding box. The member recognition result in the second combined recognition result is deleted, which has a recognition category of a single door and an overlap ratio of the recognition bounding box with a member with a recognition category of a double door greater than a second overlap ratio. The member recognition result includes the recognition category of the member and the recognition bounding box of the member. After obtaining the target recognition result of the multi-angle member, the following steps are further included:

6. The method of claim 1, wherein, Based on the target recognition result of the multi-angle member, specific semantic information of the multi-angle member is obtained in combination with a knowledge base in the building field. The system includes:

7. An apparatus for identifying components in construction engineering drawings, characterized by A slicing module is configured to split a building engineering drawing to be detected into a plurality of image slices. An inclined member recognition module is configured to recognize an inclined direction member in the image slice by using an inclined member recognition model based on a rotated ROI, and obtain a first recognition result in the case that the image slice includes an inclined member; the first recognition result includes a recognition bounding box of the inclined member and a recognition category of the inclined member. A horizontal and vertical member recognition module is configured to recognize a horizontal direction and vertical direction member in the image slice based on a horizontal and vertical member recognition model, and obtain a second recognition result. ​ The synthesis module is configured to synthesize the first recognition result and the second recognition result according to a preset synthesis rule to obtain a target recognition result of the multi-angle component. The synthesis rule is configured to filter redundant component recognition results. The inclined component recognition module comprises: The image slice feature map extraction module is configured to extract a feature map of the image slice using a residual network and a feature pyramid network. The rotation ROI module is configured to generate a rotation ROI result of a component in the image slice based on the feature map of the image slice using a region generation network and an ROI Transformer network; the rotation ROI result comprises a rotation ROI of the component in the image slice and a first feature map of the rotation ROI. The alignment module is configured to perform alignment processing based on the first feature map of the rotation ROI to obtain a second feature map of the rotation ROI. The first recognition result acquisition module is configured to acquire a first recognition result of an inclined component using the second feature map. The image slice feature map extraction module comprises: The residual network module is configured to perform convolution on the image slice based on a residual network combined with a function library e2cnn to obtain convolution data. The FPN network module is configured to perform operation on the convolution data based on a feature pyramid network combined with the function library e2cnn to generate the feature map of the image slice. The alignment module comprises: The direction channel module is configured to calculate a channel subscript on a direction channel of the first feature map based on the rotation ROI and the first feature map, adjust a direction channel corresponding to the channel subscript to a first direction channel position, and calculate a formula as follows: wherein r is a channel subscript on the first feature map direction channel, is a rotation angle of the rotated ROI, and N is a total number of direction channels of the first feature map of the rotated ROI. The interpolation module is configured to perform interpolation operation on each direction channel in the first feature map based on the channel subscript according to a formula as follows to obtain the second feature map: wherein, represents the feature result corresponding to the i-th channel of the directional channel; represents the calculation result corresponding to the i-th channel of the interpolated directional channel, is a weight parameter, .​ 8. An electronic device, comprising: The processor and the memory, the processor executes the computer program stored in the memory, realizes the method in any one of claims 1-6. The computer instructions are stored in the readable storage medium, and the computer instructions are executed by the processor to realize the method in any one of claims 1-6.

9. A readable storage medium, characterized by, ​

Images