Component identification method and device, training set construction method and device, equipment and storage medium

[0055]In order to make the objects, technical solutions, and advantages of the present application, the technical solutions in the present application embodiment will be described in connext of the present application embodiment, and It is a part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative labor are the scope of the present application.

[0056] In practice, it is found that different components are different in the number of architectures, resulting in imbalance of training data samples, and the number of different components present long-tail distribution, especially those with high frequency, very small, so that identification The model cannot be well trained, the recognition accuracy is low.

[0057] In view of the above technical problems found in the actual work, the present application provides a method of building training sets, such as figure 1 As shown, the method includes the following steps:

[0058] Step 101, determine that there is a target member in the first architectural drawing;

[0059] Specifically, when it is determined whether there is a target member in the first architect, it can be passed by the following method:

[0060] Get the target category label; get the first category of the first architectural drawings; determine that there is a target category tag in the first category label file; the component corresponding to the target category tag is used as the target component. Among them, the method of obtaining the target category tag includes obtaining a second category file for at least one architectural drawing; where at least one architectural drawing includes the first architectural paper; statistical second category of non-marking files, each component Total number of category tags; determine a category label that the total number is lower than the preset value, as the target category tag.

[0061] First, it is necessary to explain that when the specific implementation, obtain the image corresponding to each architectural drawing, the data format of the image can be solved, and the prior art relies on CAD (computer aided design, CAD-Computer AidedDesign) resolution results. The problem of layer specification, improves the drawing utilization; and distinguishes the DWG format of the architectural drawing in the prior art, there is no need to analyze by CAD, while the data format is image, or does not rely on component related layer information.

[0062] When obtaining the category label file, use the detection annotation tool, for example: labelimg, labeled multiple architectural drawings, get the category file of each drawing, you can store the category ID file as a JSON file, where the JSON file contains buildings. The category label of various components in the image of the drawings, and the coordinate information of the detection box Box, where the coordinate information is determined by the left point and the lower right point of the detection box Box, and the format of the Box is {'Box' :( Xmin, Ymin, Xmax, ymax)}, where Xmin represents the abscissa of the upper left point of the detection box Box, Ymin represents the ordinate of the upper left point of the detection box Box; XMAX represents the horizontal point of the detection box Box, YMAX represents the detection box BOX The ordinate of the lower right point.

[0063] For each architectural drawing, all of the category label files are obtained, all categories labels, for example, in architectural paper 1, the number of category labels of the member 1 is 5; In the architectural drawing 2, the number of times the category label appears is 10; in the construction drawing 3, the number of times the component label is 8; the total number of category tags of the member 1 is 5 + 10 + 8 = 23. After obtaining the total number of each component, the total number of times is lower than the class label of the preset, as the target category tag, the component corresponding to the target category tag is used as the target component, and the number of target components in the drawing is relatively small, and the target needs to be The sample data of the component is enhanced.

[0064] However, since the number of times the target member occurs less, it is likely to exist in some architectural drawings, and for architectural drawings that do not have the target component, there is no enhancement to improve the distribution of different components in the original sample data. In the specific implementation, the first architectural drawing of the target member is screened from multiple architectural drawings, and when it is judged whether or not the target component is included in a piece of architect, it is a target category that determines if the category of the section is labeled. Tags can be, if there is a target category tag in the sheet of architectures, there is a target component in which the architectural drawings are determined.

[0065] Step 102, data is enhanced to the first architectural drawing, resulting in the drawing collection of the first architectural drawing;

[0066] Among them, a method of data enhancement to the first architectural drawings includes: using geometric transformations such as translation, flipping, rotation), data enhancement, the image is enhanced using randomly adjusting brightness; also uses a random adjustment contrast The method enhances the image. The above data enhancement method can be used singly or in combination. For example, a combination enhancement method for translation and rotation of the first architectural drawings. In addition, it is also necessary to use other data enhancements. Put the enhanced first architectural drawings and the original first architectural drawings as the first architectural drawings.

[0067] Step 103, for each drawing of the drawing set, select at least two different starting positions on the drawings, and separate the drawings according to the start position, and obtain multiple sub-maps;

[0068] It will be noted here that at least two different starting positions are selected for the original first architectural drawings and each of the enhanced first architectures. For example, after step 102, the enhanced first architectural drawings are 20 sheets, coupled with the original first architectural drawings, a total of 21 sheets, then select 5 different from these 21 enhanced architectural drawings. The starting position is divided, that is, separate 5 times for each drawing.

[0069] In the specific implementation, the number of starting positions can be determined as needed, in order to increase the number of samples of the target member as much as possible, some different starting positions can be selected, and the number of samples of the other components should be considered to achieve the sample uniform. It is a case where it is not uniformly distributed in different components.

[0070] Since the architectural drawings are high-resolution image data, the picture size is too large, and the use of depth learning recognition model cannot be properly detected correctly, and the construction drawing can be scattered into a smaller sub-map to be identified. Specifically, the drawings are separately separated by different starting positions in different starting positions in different starting positions in different starting positions in different starting positions, in different starting positions, in different starting positions, in different starting positions. In the process of separating the first building drawings by the first sliding window, the size of the overlapping region present in the adjacent region selected in the first sliding window.

[0071] The size of the preset first dimension of the first sliding window can refer to the input size of the identification model used. The setting of the preset first overlap ratio should ensure that each component in the architectural drawings will exist. For example, if figure 2 As shown, the width of the first architectural drawing is W, which is H, and the width of the first sliding window is W1, which is H1, and the preset first overlap ratio is set to 0.2.

[0072] The method of the above-described divided architectural drawings, according to the coordinates, the number of times is large, and the number of sections is large, and since the number of times the target member has occurred, in a piece of architectural drawings, the sub-mapped There is only a small portion of the target sub-map, and the other divided sub-maps are abandoned, and the segmentation is relatively low.

[0073] In the present application embodiment, another method of cutting the molecain is also provided, and the specific is as follows:

[0074] Gets the category file of each of the drawings of the drawing; The first sliding window and the preset first overlap ratio are separately separated by different initial positions.

[0075] When implementing, obtain the first category file of the first architectural drawing, obtain the category file of each enhanced first architectural drawing according to the data enhanced transformation rules to get the category of each drawing in the entire drawing collection. Note file. In the category label file, the coordinate information of each component is recorded. According to this feature, from the category label file, the coordinate information of the target component is obtained, and the cutoff is performed within the preset range of the coordinate information, and there is no target sub-map. The drawing portion does not make a cut, which can more efficiently obtain a sub-map containing the target member to obtain a target sub-map faster and efficient.

[0076] Step 104, from a plurality of subgraphs, acquire a target sub-figure containing the target member; wherein the target sub-map is used to construct a training set of the target component.

[0077] In a specific implementation, the target sub-map can be selected from multiple sub-maps as follows. The first, obtain the category file of each drawing of the drawings collection; for each drawing, from the category label file, obtain the coordinate information of the target component; according to the coordinate information, determine the target component into the sub-figure, will The sub-map is a target sub-figure; wherein the method of obtaining a class label file for each drawing in the drawing collection can refer to step 103. Second, the target device is used to determine the target member, and the sub-map is used as a target sub-figure. In the specific implementation, the target detection method is not limited, and any target detection method can be employed.

[0078] In the present application embodiment, data is enhanced for the first architerat of the target member, and the drawing collection of the first architectural drawing is obtained. On each of the drawings, at least two different starting positions are selected separately. Separate according to different starting positions, from the subprogramme obtained from the division, select the target sub-map of the target member. In the present application embodiment, the number of samples of the target member is increased by data enhancing and selecting different starting positions, and the number of samples is effectively supplemented, and the number of samples can be effectively improved the identification model training results. Spend. Further, the method of the sample expansion provided herein, without changing the sample characteristics, increasing the diversity of the sample, improves the uneven distribution of different components in the architectural drawings.

[0079] For architectural drawings that do not exist, separate according to the first sliding window of the preset first size and the preset first overlap ratio, the sample expansion is not done, and the target sub-map is coupled to the sample data set. Some sample data in the obtained sample data set as the training set, the rest as the test set, for example: sample data in the sample data set is divided into training set and test set in accordance with the proportion of 9: 1. Among them, the training set is used to train the identification model, while the test set is used to test the training-trained recognition model to determine if the identification model is well trained.

[0080] In the specific implementation, the identification model can be used in the YOLOV4 model, where 9 anchor values ​​(anchor values) are initialized using KMeans cluster, using the public data set training, and use a preheating policy (WARMUP) when the learning rate is set. The maximum number of iterations is 200EPOCH, in which all data of the training set is used to make a complete training of the identification model, called EPOCH. In addition, it will also be noted that the recognition model can also employ other neural network models.

[0081] After training, it is obtained from a well-trained identification model, and the architectural drawings are identified using the training good recognition model. The specific method is as follows:

[0082] Get images of the second architectural drawings to be identified;

[0083] Separate the image to obtain a plurality of to-pending sub-maps, and record the respective conversion matrices from the image to each of the subgraphs to be identified;

[0084]Among them, when the image of the second architectural drawing is pressed, the image is separate according to the second sliding window of the second size, and the image is divided, wherein the preset second overlap ratio is used for Laborating the size of the overlap region in the adjacent region selected by the second sliding window by using the second sliding window sections. The preset second dimension of the second sliding window can be the same as the preset first dimension, or may be different; the preset second overlap ratio can be the same as the preset first overlap ratio, or may be different.

[0085] The individual peaks to be identified to the recognition model to obtain at least one preliminary recognition box of each member; where the identification model is used in advance using the training set training of the target member, the acquisition method of the training set of the target member includes: determination There is a target component in a building drawing; the first architectural drawings are enhanced to obtain a collection of drawings of the first architectural drawings; on each of the drawings of the drawings, at least two different starting positions are selected; according to each The starting position separates each of the drawings of the drawing set, obtains a plurality of subgraphs; from multiple sub-maps, it acquires a target sub-map containing the target component; where the target sub-map is used to build the training set of the target component;

[0086] According to the conversion matrix, the preliminary identification box of each component is converted to the image;

[0087] For each component, the preliminary identification box of the image in the image is removed from the non-polar large value suppression algorithm, and the identification result of the component is obtained.

[0088] In the presence of the overlapping region, the components of the overlap region may exist in multiple sub-maps, and in the sub-map after dividing, the components of the overlap region will have multiple detection results, so they pass non-polar The value suppression algorithm (NMS) removes the redundant preliminary identification box.

[0089] Only the overlapping regions between the two identification frames are considered in the conventional NMS, that is, the redundant recognition box is suppressed by IOU (integration ratio) index. However, in the actual drawings, the component distribution is more complex, there is a case where two or more components are blocked, and when the IOU threshold is too high, because the combined overlapping area component identification box cannot be removed, when the IOU threshold is set too low The identification frame is easily removed in the components that face each other. The present application provides an improved NMS algorithm, including:

[0090] Get a preset intersection and ratio threshold; preset intersection and threshold can be set according to empirical value.

[0091] Select the initial recognition box in each initial recognition box, the initial identification box with the highest confidence is selected as the reference recognition box, and the remaining initial recognition box is used as an alternate recognition box;

[0092] After identifying the image of the second architectural drawings by the identification model, each preliminary recognition box obtained corresponds to a confidence, the higher the confidence, the more reliable.

[0093] For each alternate identification box, calculate the European distance of the reference recognition box and the center point of the alternate recognition box, and the diagonal length of the minimum external rectangle of the reference recognition box and the standby identification box; calculate the European distance and the diagonal The ratio of the length of the line; uses a preset intersection and threshold ratio to subtract the ratio, resulting in the intersive ratio threshold;

[0094] Remove the alternate recognition box that is less than the hypotenal ratio threshold corresponding to the preset threshold.

[0095] In the present application embodiment, not only the overlapping area of ​​the two identification frames, but also considered the European distance of the central point of the two recognition boxes and the maximum external rectangular rectangular diagonal length of the two identification frames, calculating the European distance and the pair The ratio of the length of the angle, using the preset intersection and the difference between the threshold and the ratio, minimizes the normalization distance between the center points of the two recognition boxes, so that the center of the recognition box is maximized to the target, avoiding legacy Many, excessive recognition boxes.

[0096] After the identification result of the component, it also includes: the identification result is contoured and the polygon is fitted, and the precise coordinates of the component are obtained.

[0097] The coordinates corresponding to the identification result of the component may still have an error, and the quadratic correction is performed twice by contour detection and polygonal fitting identification results, and finally obtains accurate component location information. In the present application embodiment, the identification result is further regained by contour detection and polygon fitting method, and the accuracy of the component coordinates is improved.

[0098] Based on the same concept, a device for constructing a training set is provided in the present application embodiment, which may be referred to in the description of the method, and the repetition is not described. image 3 As shown, the device mainly includes:

[0099] The determination module 301 is used to determine that there is a target member in the first architectural drawing;

[0100] The enhancement module 302 is used to enhance the first architectural drawing to obtain a drawing collection of the first architectural drawing;

[0101] Separation module 303 for each drawing of the drawing set, selects at least two different starting positions on the drawing, and separates the drawings according to the start position, and obtains multiple sub-maps;

[0102] The acquisition module 304 is used to acquire a target sub-figure containing the target member from a plurality of subgraphs; where the target sub-map is used to construct the training set of the target component.

[0103] In a specific embodiment, the determination module 301 is used to acquire the target category tag; obtain the first category of the first architectural drawing; determine that there is a target category tag in the first category label; the components corresponding to the target category tag are used as Target components.

[0104] In a specific embodiment, the determination module 301 is specifically used to acquire a second category file for at least one architectural drawing; where at least one architeo paper includes the first architectural paper; statistical second category labeling file, each The total number of category tags of a component; determines a category tag of the total number below the preset value, as the target category tag.

[0105] In a specific embodiment, the division module 303 is configured to separate the drawings in different starting positions in different starting positions in different starting positions in different starting positions in different starting positions in different starting positions in different starting positions in different starting positions in different starting positions, in different starting positions, in different starting positions, in different starting positions. The preset first overlap ratio is used to characterize the size of the overlap region in the adjacent region selected by the first sliding window in the process of separating the first architectural drawing using the first sliding window; or acquisition The category of each drawing of the drawings set; from the class label file, the coordinate information of the target member is obtained; within the preset range of the coordinate information, according to the first size of the first size A sliding window and a preset first overlap ratio are separated by different starting positions.

[0106] In a specific embodiment, the module 304 is acquired to obtain a class label file for each of the drawings of the drawing collection; for each drawing, from the category label file, obtain the coordinate information of the target component; according to the coordinate information, determine The target member falls into the sub-figure, and the sub-map is used as the target sub-figure; or the target detecting method determining the sub-figure contains the target member, the sub-map is used as the target sub-figure.

[0107] Based on the same concept, an electronic device is also provided in the examples of the present application. Figure 4 As shown, the electronic device mainly includes: processor 401, memory 402, and communication bus 403, wherein processor 401 and memory 402 perform communication between each other through communication bus 403. The memory 402 stores a program that can be performed by the processor 401, and the processor 401 performs a program stored in the memory 402, and implements the following steps:

[0108] Determines the existence of a target component in the first architectural drawing;

[0109] Data is enhanced to the first architectural drawings, and the drawings collection of the first architectural drawings are obtained;

[0110] For each of the drawings of the drawings, at least two different starting positions are selected on the drawing, and the drawings are separately separated according to the start position, and multiple sub-maps are obtained;

[0111] From multiple subgraphs, the target subgraph containing the target component is acquired; where the target sub-map is used to build the training set of the target component;

[0112] or,

[0113] Get images of the second architectural drawings to be identified;

[0114] Separate the image to obtain a plurality of to-pending sub-maps, and record the respective conversion matrices from the image to each of the subgraphs to be identified;

[0115] The individual peaks to be identified to the recognition model to obtain at least one preliminary recognition box of each member; where the identification model is used in advance using the training set training of the target member, the acquisition method of the training set of the target member includes: determination There is a target member in a building drawing; the first architectural drawing is enhanced to obtain a set of drawings of the first architectural drawing; for each drawing of the drawing set, at least two different starts are selected on the drawing. Location, and separate the drawings according to the starting position, a plurality of sub-maps; from multiple sub-maps, the target sub-map containing the target member is acquired; wherein the target sub-map is used to build the training of the target component. set;

[0116] According to the conversion matrix, the preliminary identification box of each component is converted to the image;

[0117] For each component, the preliminary identification box of the image in the image is removed from the non-polar large value suppression algorithm, and the identification result of the component is obtained.

[0118] The communication bus 403 mentioned in the above electronic device can be a peripheralcomponent interconnect, a referred to as a PCI code bus or an Extended IndustryStandard Architecture, a referred to as an EISA bus. The communication bus 403 can be divided into an address bus, a data bus, a control bus, and the like. For ease of expression, Figure 4 It is only represented by a thick line, but does not mean that there is only one bus or a type of bus.

[0119] Memory 402 can include a random access memory (RAM), or may include a non-volatile memory (Non-Volatile Memory), such as at least one disk memory. Alternatively, the memory can also be at least one storage device located away from the processor 401.

[0120] The processor 401 described above can be a general purpose processor, including a central processor (CPU), a network processor (NP), etc., can also be Digital Signal Processing, referred to as DSP) Application Specific Intercuit, ASIC, Field Programmable Gate Array, Field-ProGramMable Gate Array, Fields, or other programmable logic devices, separate doors or transistor logic devices, discrete hardware components.

[0121] In yet another embodiment of the present application, a computer readable storage medium is also provided, and a computer program is stored in the computer readable storage medium, and when the computer program is running on the computer, the computer performs the above embodiment. A method or component of the construction training set is described.

[0122]In the above embodiment, it can be achieved through software, hardware, firmware, or any combination thereof in whole or in part. When implemented using software, you can fully or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When loading and executing the computer instruction on a computer, all or partially generated the flow or function described in accordance with the present application embodiment. The computer can be a general purpose computer, a dedicated computer, a computer network, or another programmable device. The computer instruction can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, a computer instruction from a website site, computer, server, or data center (for example Coaxial cable, fiber, digital subscriber line (DSL)) or wireless (eg, infrared, microwave, etc.) is transmitted to another website site, computer, server, or data center. The computer readable storage medium can be any available medium that the computer can access or a data storage device including one or more available media integrated servers, data centers. The available medium can be a magnetic medium such as a floppy disk, a hard disk, a tape, or the like, a photologies (e.g., a DVD) or a semiconductor medium (e.g., a solid-state hard disk).

[0123] It should be noted that in this article, a relationship term such as "first" and "second", etc. is only used to distinguish an entity or operation with another entity or an operational area, not necessarily or implied. There is any such actual relationship or order between entities or operations. Moreover, the term "comprising", "comprising" or any other variable is intended to cover non-exclusive contained, thereby enabling a process, method, article, or device including a series of elements, not only those elements, but also not expressly listed. Other elements, or elements that are also inherent to such processes, methods, items, or equipment. In the absence of more restrictions, the elements defined by the statement "include a ...", and there is no additional same elements in the process, method, item, or apparatus including the element.

[0124] The above is only the embodiments of the present invention, and those skilled in the art will appreciate or implemented the invention. A variety of modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Accordingly, the present invention will not be limited to these embodiments shown herein, but rather consistent with the widest range of the principles and novel features described herein.