Drawing recognition method and drawing recognition device

Abstract

To provide a drawing recognition method and a drawing recognition device capable of extracting a symbol and an annotation from a drawing with high accuracy and associating them with each other to create data.SOLUTION: A drawing recognition method for extracting a symbol and an annotation contained in a drawing file performs: a drawing file creation step of creating a symbol extraction drawing file S2 and a plurality of line type drawing files S3 from an original drawing file S1; a symbol extraction step of extracting a symbol from the symbol extraction drawing file S2; an annotation extraction file setting step of setting an annotation extraction file S6; and an annotation extraction step of extracting, in the annotation extraction file S6 determined in the annotation extraction file setting step, an annotation corresponding to each symbol from a figure detected in a detection area SS corresponding to an annotation search area SA centered on each symbol extracted in the symbol extraction step.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a drawing recognition method, and more particularly to a drawing recognition method and drawing recognition device for extracting various symbols and characters written near the symbols that supplement the specifications of facilities, equipment, etc. represented by the symbols from various drawings in association with each other. [Background technology]

[0002] Currently, CAD software is commonly used to create various drawings. When drawings created using CAD software are provided, they are sometimes provided not as CAD data but as data converted into PDF format or as paper-based drawings (hereinafter simply referred to as paper-based drawings, etc.), and it may be necessary to obtain various information contained in these drawings. For example, architectural design drawings contain a large amount of information such as equipment symbols (hereinafter simply referred to as symbols), text and figures related to the symbols (hereinafter simply referred to as side notes), and wiring, and it may be necessary to extract the symbols and side notes. However, architectural design drawings often include symbols, side notes, and wiring, etc., overlapping with each other or in close proximity to each other, making it difficult to accurately extract the symbols and side notes. Therefore, technologies for accurately extracting symbols and side notes from paper-based drawings, etc. have been developed (see Patent Documents 1 to 5). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-259076 [Patent Document 2] Japanese Patent Application Laid-Open No. 2006-227824 [Patent Document 3] Patent No. 7234719 [Patent Document 4] Japanese Patent Application Laid-Open No. 2001-92967 [Patent Document 5] Japanese Patent Application Publication No. 11-126216 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques of Patent Documents 1 to 5 extract figures and characters in drawings by template matching, but in drawings that contain a mixture of wiring, symbols, etc., it is difficult to perform appropriate template matching, making it difficult to accurately extract symbols, etc. In particular, when symbols and annotations are written in association with each other, it is difficult to accurately extract the symbols and annotations while keeping them associated with each other.

[0005] In view of the above circumstances, an object of the present invention is to provide a drawing recognition method and drawing recognition device that can accurately extract symbols and annotations from a drawing and associate them with each other to create data. [Means for solving the problem]

[0006] <Drawing recognition method> The drawing recognition method of the first invention is a drawing recognition method for extracting symbols and annotations contained in a drawing file, and is characterized by carrying out the following steps: a drawing file formation step for forming a symbol extraction drawing file used for symbol extraction and a plurality of line type drawing files classified by line type from the drawing file; a symbol extraction step for extracting symbols from the symbol extraction drawing file formed in the drawing file formation step; a annotation extraction file setting step for determining an annotation extraction file which is a line type drawing file from which annotations are extracted from the plurality of line type drawing files formed in the drawing file formation step; and an annotation extraction step for extracting annotations corresponding to each symbol from figures detected in a detection area corresponding to an annotation search area centered on each symbol extracted in the symbol extraction step from the annotation extraction file determined in the annotation extraction file setting step. The drawing recognition method of the second invention is characterized in that, in the first invention, in the file setting step for extracting side notes, OCR processing is performed on the plurality of line type drawing files, and the line type drawing file among the plurality of line type drawing files that contains the most detected character data is set as the file for extracting side notes. A drawing recognition method according to a third aspect of the present invention is characterized in that, in the second aspect of the present invention, the OCR processing is AI-OCR processing. The drawing recognition method of the fourth invention is characterized in that, in the first invention, it includes a side note search area setting step for setting a side note search area centered on each of the multiple symbols extracted in the symbol extraction step, and in the side note extraction file setting step, for the multiple line type drawing files formed in the drawing file formation step, a comparison area corresponding to the side note search area set in the side note search area setting step is set for a representative symbol, and the side note extraction file is determined based on the figure detected in the comparison area. The drawing recognition method of the fifth invention is characterized in that, in the fourth invention, in the file setting process for extracting side notes, a file for extracting side notes is determined for multiple line type drawing files formed in the drawing file formation process based on the similarity between the side note templates registered in the side note template list and the figures detected in the comparison area. The drawing recognition method of the 6th invention is characterized in that, in the 5th invention, in the file setting process for extracting annotations, a matching process is performed between the annotation template registered in the annotation template list and the figure detected in the comparison area while changing the size of the annotation template, and an extraction template size is determined, which is the size of the annotation template that gives the highest similarity in the annotation extraction file, and in the annotation extraction process, an annotation template of the extraction template size determined in the file setting process for extracting annotations is used to extract the annotations included in the detection area. The drawing recognition method of the seventh invention is characterized in that, in the first invention, in the side note extraction process, multiple figures included in the side note search area are extracted as side note candidates, and among the multiple side note candidates, the side note candidate that is closest to the symbol is linked to the symbol. The drawing recognition method of the 8th invention is characterized in that, in the 1st invention, in the side note extraction process, multiple figures included in the side note search area are extracted as side note candidates, and for the multiple extracted side note candidates, a side note candidate to be linked to a symbol is determined based on a management list that stores the relationship between the symbol that set the side note search area and the side note. <Drawing recognition device> The drawing recognition device of the 9th invention is a drawing recognition device that extracts symbols and annotations contained in a drawing file, and is characterized in that it comprises: a drawing file formation unit that forms a symbol extraction drawing file used for symbol extraction and a plurality of line type drawing files classified by line type from the drawing file; a symbol extraction unit that extracts symbols from the symbol extraction drawing file formed by the drawing file formation unit; a annotation extraction file setting unit that determines a line type drawing file from which annotations are extracted from the plurality of line type drawing files formed by the drawing file formation unit; and an annotation extraction unit that extracts annotations corresponding to each symbol from figures included in a detection area corresponding to an annotation search area centered on each symbol extracted by the symbol extraction unit, from the annotation extraction file determined by the annotation extraction file setting unit. The drawing recognition device of the 10th invention is characterized in that, in the 9th invention, the file setting unit for extracting side notes has the function of performing OCR processing on the multiple line type drawing files and setting the line type drawing file that contains the most detected character data among the multiple line type drawing files as the file for extracting side notes. The drawing recognition device of an eleventh aspect of the present invention is the drawing recognition device of the tenth aspect of the present invention, characterized in that the OCR processing is AI-OCR processing. The drawing recognition device of the 12th invention is characterized in that, in the 9th invention, it is equipped with a side note search area setting unit that sets side note search areas centered on the symbols for multiple symbols from the symbols extracted by the symbol extraction unit, and the side note extraction file setting unit has the function of setting a comparison area corresponding to the side note search area set by the side note search area setting unit for a representative symbol for multiple line type drawing files formed by the drawing file formation unit, and determining the side note extraction file based on the figure detected in the comparison area. The drawing recognition device of the 13th invention is characterized in that, in the 12th invention, the file setting unit for annotating extraction has the function of determining a file for annotating extraction based on the similarity between the annotating template registered in the annotating template list and the figure detected in the comparison area for multiple line type drawing files formed by the drawing file formation unit. The drawing recognition device of the 14th invention is characterized in that, in the 13th invention, the annotative extraction file setting unit has the function of performing a matching process between the annotative template registered in the annotative template list and the figure detected in the comparison area while changing the size of the annotative template, and determining the extraction template size, which is the size of the annotative template that results in the highest similarity in the annotative extraction file, and the annotative extraction unit has the function of extracting annotative included in the detection area using an annotative template of the extraction template size determined in the annotative extraction file setting unit. The drawing recognition device of the 15th invention is characterized in that, in the 9th invention, the annotative extraction unit has a function of extracting multiple figures included in the annotative search area as annotative candidates, and a function of linking the annotative candidate among the multiple annotative candidates that is closest to the symbol to the symbol. The drawing recognition device of the 16th invention is characterized in that, in the 9th invention, the annotative extraction unit has a function of extracting multiple figures included in the annotative search area as annotative candidates, and has a function of determining the annotative candidate to be linked to a symbol for the multiple extracted annotative candidates based on a management list that stores the relationship between the symbol that set the annotative search area and the annotative. [Effects of the Invention]

[0007] <Drawing recognition method> According to the first aspect of the present invention, symbols and side notes are extracted separately, so that the precision of extraction of symbols and side notes can be increased, and related symbols and side notes can be appropriately linked and stored. According to the second to sixth aspects of the present invention, the extraction accuracy of the side notes can be improved. According to the seventh aspect of the present invention, a symbol can be easily linked to an annotative note that is highly related to the symbol. According to the eighth aspect of the present invention, it is possible to increase the accuracy of linking a symbol with an annotative note that is highly related to the symbol. <Drawing recognition device> According to the ninth aspect of the present invention, symbols and side notes are extracted separately, so that the precision of extraction of symbols and side notes can be increased, and related symbols and side notes can be appropriately linked and stored. According to the tenth to fourteenth aspects of the present invention, the extraction accuracy of the side notes can be improved. According to the fifteenth aspect of the present invention, a symbol can be easily linked to an annotative note that is highly related to the symbol. According to the sixteenth aspect of the present invention, it is possible to increase the accuracy of linking a symbol with an annotative note that is highly related to the symbol. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 is a flowchart of a drawing recognition method according to the first embodiment. [Figure 2] 1 is a schematic block diagram of a drawing recognition device 1 of the present embodiment. [Figure 3](A) is a diagram showing an example of an original drawing file S1, (B) is a diagram showing an example of a drawing file for symbol extraction (original SVG file), (C) is a diagram showing an example of a list of multiple original line type image files, (D) is a diagram showing an example of a drawing file for symbol extraction S2, and (E) is a diagram showing an example of multiple line type drawing files S3. [Figure 4] 10A is a diagram showing an example of a state in which a side note search area SA is set for each symbol included in the symbol extraction drawing file S2, and FIG. 10B is a diagram showing an example of a method for selecting a side note extraction file S6. [Figure 5] (A) is a diagram showing an example of a state in which a side note search area SA has been set for a selected symbol included in the symbol extraction drawing file S2, and (B) is a diagram showing an example of a state in which a detection area SS corresponding to the side note search area SA of (A) has been set. [Figure 6] This figure shows an example of a method for selecting a file S6 for extracting side notes, where (A) shows an example of a side note search area SA for a representative symbol, and (B) shows an example of a state in which a comparison area TS corresponding to the side note search area SA of (A) is set. [Figure 7] FIG. 10 is a flowchart of a drawing recognition method according to a second embodiment. [Figure 8] FIG. 10 is a flowchart of a drawing recognition method according to a third embodiment. [Figure 9] 10A is a diagram showing an example of symbol data SD, FIG. 10B is a diagram showing an example of a side note template list TP, and FIG. 10C is a diagram showing an example of a management list CR. DETAILED DESCRIPTION OF THE INVENTION

[0009] The drawing recognition method of this embodiment is a method for extracting symbols and annotations from drawing files such as construction drawings, and is characterized by its ability to extract symbols and annotations with high accuracy while maintaining the relationship between the symbols and annotations.

[0010] The drawing file from which symbols and the like are extracted by the drawing recognition method of this embodiment may be a vector-format drawing file and is not particularly limited. Examples of drawing files include drawing files created by scanning or otherwise digitizing drawings printed on paper (drawing files in which the image is composed of pixel data) and drawing files created by converting drawings created using CAD (drawing files in which symbols and the like in the data each contain individual information). For example, the former file type includes raster-format drawing files such as raster PDF files and PNG files, while the latter file type includes vector-format drawing files such as vector PDF files and SVG files. Of the raster-format drawing files and vector-format drawing files mentioned above, the drawing file from which symbols and the like are extracted by the drawing recognition method of this embodiment is a vector-format drawing file. Hereinafter, the drawing file from which symbols and the like are extracted will simply be referred to as the original drawing file.

[0011] In the following explanation, the original drawing from which symbols and the like are extracted will be described as a construction drawing, but the drawings from which symbols and the like are extracted by the drawing recognition method of this embodiment are not particularly limited. Furthermore, in the following explanation, since the original drawing from which symbols and the like are extracted is a construction drawing, the term "symbol" refers to a symbol that diagrammatically represents an electrical or mechanical component used in the installation of a building. Furthermore, the term "sidenote" refers to a character or symbol that supplements the specifications of the electrical or mechanical component represented by the symbol and is written adjacent to the symbol.

[0012] <Drawing recognition device 1 of this embodiment> First, a drawing recognition device 1 for implementing the drawing recognition method of this embodiment will be described. As shown in Figure 2, the drawing recognition device 1 may have a display function for displaying drawing files, etc. formed by each unit described later on a display DS, and an input function for inputting necessary information and instructing necessary operations using input devices IS such as a keyboard and a mouse.

[0013] <Data Entry Section 2> As shown in Fig. 2, an apparatus 1 (hereinafter simply referred to as drawing recognition apparatus 1) for implementing the drawing recognition method of this embodiment has a data input unit 2 that creates an original drawing file S1 (see Fig. 3(A)). This data input unit 2 has a function (data receiving function) of accepting data of a vector-format drawing file, such as a vector PDF file, that has already been created, from a storage medium such as a USB or receiving it via a line such as the Internet. The drawing file received by the data input unit 2 is transmitted to and stored in a storage unit 10 as the original drawing file S1.

[0014] <Drawing file creation section 3> As shown in FIG. 2, the drawing recognition device 1 has a drawing file forming unit 3. The drawing file forming unit 3 has a function of forming a plurality of drawings from an original drawing file S1 received by the data input unit 2. Specifically, the drawing file forming unit 3 has a function of forming a symbol extraction drawing file S2 (see FIG. 3(D)) and a plurality of line type drawing files S3 (see FIG. 3(E)) from the original drawing file S1. The plurality of line type drawing files are files classified by line thickness. The drawing file forming unit 3 has a function of transmitting the created symbol extraction drawing file S2 and the plurality of line type drawing files S3 to the storage unit 10, and the symbol extraction drawing file S2 and the plurality of line type drawing files S3 transmitted to the storage unit 10 are stored in the storage unit 10.

[0015] There is no particular limitation on the method by which the drawing file forming unit 3 forms the symbol extraction drawing file S2 and the plurality of line type drawing files S3 from the original drawing file S1. For example, they can be formed by the following method.

[0016] Since the original drawing file S1 is a drawing file in a vector format such as a vector PDF file, the drawing file forming unit 3 forms a symbol extraction drawing file S2 and a plurality of line type drawing files S3 as follows. First, the drawing file forming unit 3 forms an original SVG file (see FIG. 3(B)) from the original drawing file S1, and then forms a symbol extraction drawing file S2, which is a PNG file, by extracting data such as symbols, annotations, and wiring contained in the original SVG file. In this case, the symbol extraction drawing file S2 is a PNG file that includes all of the symbols, annotations, wiring, etc. The drawing file creation unit 3 also creates a plurality of original line type drawing files, which are SVG files, based on the line thickness information contained in the original SVG file (see FIG. 3(C)), and creates a plurality of line type drawing files S3, which are PNG files, from these plurality of original line type drawing files. These plurality of line type drawing files S3 are files classified by line thickness. In other words, each line type drawing file S3 is a PNG file that includes symbols, annotations, wiring, etc. with the same line thickness, and does not include symbols, annotations, wiring, etc. with different line thicknesses.

[0017] 3(E) shows three line type drawing files S3 (A) to (C), but the number of line type drawing files S3 to be formed is not particularly limited. An appropriate number of line type drawing files S3 may be formed depending on the lines, symbols, annotations, etc. contained in the original drawing file S1.

[0018] <Symbol Extraction Unit 4> As shown in FIG. 2, the drawing recognition device 1 has a symbol extraction unit 4. The symbol extraction unit 4 has a function of extracting symbols from the drawing file S2 for symbol extraction created by the drawing file creation unit 3. The method by which the symbol extraction unit 4 extracts symbols is not particularly limited, but symbols can be extracted by a known method such as pattern matching based on the symbol data SD (see FIG. 9(A)) stored in the storage unit 10. Pattern matching can be performed using AI (artificial intelligence), but the means for performing pattern matching is not particularly limited. The symbol extraction unit 4 also has a function of creating an extracted symbol file S4 for each extracted symbol, in which symbol data obtained by cutting out an image of an area containing the extracted symbol is associated with location information of the area (i.e., information such as the coordinates of the area in the drawing file S2 for symbol extraction). Each created extracted symbol file S4 is transmitted to and stored in the storage unit 10.

[0019] If symbol extraction is performed using pattern matching with AI (artificial intelligence), the extracted symbol file S4 also includes information about the confidence level of the symbol extraction (expressed as a number: the closer to 1, the higher the confidence level). Alternatively, the extracted symbol file S4 may be an image of an area containing the extracted symbol, and the file name of the extracted symbol file S4 may include location information and the confidence level, thereby associating the area image with the location information and the confidence level. This method is advantageous because it allows the extracted symbol files S4 to be sorted on the display DS in descending (or descending) order of confidence level, allowing for efficient deletion of erroneously detected extracted symbol files S4. For example, for an image of a rectangular area, the file name of the extracted symbol file S4 may be set to 0.9224@17560×4782_17634×4854_0.png. In the file name mentioned above, "0.9224" is the confidence level, and "17560×4782_17634×4854" is the X-axis and Y-axis coordinates (position information) of the diagonal corners of the rectangular area image.

[0020] <Search area setting section 5> As shown in FIG. 2, the drawing recognition device 1 has an annotated search area setting unit 5. The annotated search area setting unit 5 has a function of setting an area (annotated search area SA) for searching for annotated notes associated with a symbol. Specifically, the annotated search area setting unit 5 has a function of setting a certain area centered on each symbol as the annotated search area SA for all symbols extracted by the symbol extraction unit 4. Specifically, the annotated search area setting unit 5 has a function of setting a certain area centered on each symbol as the annotated search area SA for each symbol based on information included in each extracted symbol file S4 (see FIG. 4(A)). When setting the annotated search area SA for each symbol, the annotated search area setting unit 5 sets a range that is expected to include both each symbol and annotated notes associated with that symbol as the annotated search area SA. For example, an area calculated based on the size of each extracted symbol may be set as the annotated search area SA. Furthermore, when a management list CR (see FIG. 9(C)) in which notes corresponding to each symbol (i.e., used for each symbol) are registered in advance is stored in the storage unit 10 (or may be stored in the side note search area setting unit 5), the side note search area SA may be calculated based on the number of characters (number of characters in the side note character string) registered in the management list CR. The information on this side note search area SA includes the coordinates of the side note search area SA (for example, if a rectangular area is set as the side note search area SA, coordinate information of the corners, etc.). Once the information on this side note search area SA is calculated, it is transmitted from the side note search area setting unit 5 to the storage unit 10 and stored in the storage unit 10 in association with extracted symbol information of each symbol for which the side note search area SA is set. Note that the information on the side note search area SA may include information on the corresponding extracted symbol.

[0021] FIG. 9C shows a list of a plurality of annotations corresponding to one symbol, and the management list CR stores such an annotation list in association with each symbol.

[0022] <File Setting Unit 6 for Margin Extraction> As shown in FIG. 2, the drawing recognition apparatus 1 has a file setting unit 6 for margin extraction. The file setting unit 6 for margin extraction has a function of selecting a line type drawing file S3 suitable for margin extraction from a plurality of line type drawing files S3 formed by the drawing file forming unit 2 and setting it as the file S6 for margin extraction. When the file S6 for margin extraction is selected, the information of the file S6 for margin extraction is transmitted from the file setting unit 6 for margin extraction to the storage unit 10 and stored therein. The method by which the file setting unit 6 for margin extraction determines (selects) the file S6 for margin extraction is not particularly limited, and for example, the following method can be adopted.

[0023] <Method Using OCR Processing> The file setting unit 6 for margin extraction can adopt a method of performing OCR (Optical Character Recognition / Reader) processing on a plurality of line type drawing files S3 and determining the file S6 for margin extraction from the plurality of line type drawing files S3 based on the character data obtained for each line type drawing file S3 by the OCR processing. For example, in a design drawing, the characters described on the drawing including the margin are often described in the same line type (line thickness), so it is considered likely that the characters including the margin are included in the same line type drawing file S3. In other words, it is considered likely that the line type drawing file S3 containing a large number of characters contains the margin. Therefore, among the plurality of line type drawing files S3, the line type drawing file S3 with a large number of detected characters when the OCR processing is performed can be set as the file S6 for margin extraction.

[0024] Furthermore, it is desirable that the OCR processing in the marginal note extraction file setting unit 6 be performed using AI-OCR processing. AI-OCR uses a machine learning algorithm to learn characters and sentences from large amounts of data, and is therefore characterized by its ability to grasp the characteristics of characters and documents and perform flexible and effective recognition. Therefore, by performing AI-OCR processing that uses machine learning (such as deep learning) to learn marginal notes obtained from many mechanical design drawings (such as the original drawing file S1), the accuracy of marginal note detection can be improved.

[0025] <When using a representative symbol> As shown in FIG. 6, the sidenote extraction file setting unit 6 may select multiple representative symbols from the symbols extracted by the symbol extraction unit 4, set a comparison area TS corresponding to the sidenote search area SA set by the sidenote search area setting unit 5 for the selected symbols, and determine the sidenote extraction file S6 using the figures detected in the comparison area TS. That is, the figures detected in the comparison area TS may be compared with the sidenote templates registered in the sidenote template list TP using a known method such as pattern matching, and the line type drawing file S3 with the highest matching result may be used as the sidenote extraction file S6. The matching result may be the similarity between the figure and the sidenote template. For example, the similarity (or confidence level in the case of AI matching) between the figures matched in multiple comparison areas TS and the sidenote template may be calculated, and the line type drawing file S3 containing many highly similar figures may be used as the sidenote extraction file S6.

[0026] Here, the comparison area TS means an area in each line type drawing file S3 that has coordinates corresponding to the coordinates of the sideline search area SA in the symbol extraction drawing file S2. For example, in the case of Figures 6(A) and 6(B), the comparison area TS(1) in Figure 6(B) corresponds to the sideline search area SA(1) in Figure 6(A), and the comparison area TS(2) in Figure 6(B) corresponds to the sideline search area SA(2) in Figure 6(A).

[0027] Furthermore, the matching between the figure and the annotated template may be performed while changing the size of the annotated template. In this case, the size of the annotated template that provides the highest similarity (optimal template size) is stored in the storage unit 10 in association with the annotated extraction file S6, or the size of the annotated template alone.

[0028] The method for selecting the representative symbol is not particularly limited. For example, the annotated extraction file setting unit 6 may automatically select the symbol by selecting symbols in descending order of confidence when the symbols are extracted by the symbol extraction unit 4. Alternatively, the worker may select the representative symbol by checking the image of the extracted symbol (i.e., the image included in the extracted symbol file S4 or the image in the extraction drawing file S2) and the confidence level, etc.

[0029] <sidenote extraction part 7> As shown in FIG. 2, the drawing recognition device 1 has a side note extraction unit 7. The side note extraction unit 7 has a function of setting a detection area SS corresponding to the side note search area SA formed by the side note search area setting unit 5 in a side note extraction file S6 based on the extracted symbol file S4 formed by the symbol extraction unit 4, and extracting side notes from this detection area SS, and a linking function of linking the extracted side notes with symbols. Specifically, the side note extraction unit 7 sets, for all symbol files S4, detection areas SS corresponding to the side note search area SA set by the side note search area setting unit 5 in the side note extraction file S6. Then, for figures included in the detection area SS in the side note extraction file S6, pattern matching is performed using the side note template list TP, and figures (Z in FIG. 5) that are highly similar to the side note template are extracted as side note candidates. If only one side note candidate is extracted, the side note candidate is treated as a side note related to the symbol for which the side note search area SA is set, and the side note candidate and the symbol are linked together and transmitted to the memory unit 10 as symbol data for storage.

[0030] Furthermore, when there are multiple side note candidates, the side note extraction unit 7 has a function of linking the side note candidate that is considered to be most relevant to the symbol from the multiple side note candidates to the symbol, and creating symbol data. The method by which the side note extraction unit 7 selects the side note candidate that is considered to be highly relevant is not particularly limited, but for example, the side note candidate that is closest in distance from the symbol to the side note candidate (for example, the distance from the center of the symbol to the center of the side note candidate) may be linked to the symbol.

[0031] In addition, when the side note extraction file S6 is selected using the matching process in the side note extraction file setting unit 6 and an optimal template size suitable for matching is required, it is desirable to perform pattern matching using the optimal template size. In other words, when performing pattern matching between a figure contained in the side note search area SA and a side note template, it is desirable to perform pattern matching using a side note template of the optimal template size. Performing pattern matching using a side note template of the optimal template size makes it possible to properly extract side notes from figures present in the side note search area SA, and makes it easier to prevent figures other than side notes from being mistakenly extracted as side notes.

[0032] <Extraction and linking of symbols and annotations> The operation of extracting symbols and annotations by the drawing recognition device 1 described above (that is, the drawing recognition method of the first embodiment) will be described (see FIGS. 1 and 2).

[0033] <Drawing file creation process> As shown in FIGS. 1 and 2, the drawing file forming unit 3 receives an original drawing file S1 that is formed or received by the data input unit 2 and stored in the memory unit 10. The drawing file forming unit 3 forms a symbol extraction drawing file S2 and a plurality of line type drawing files S3 from the original drawing file S1 (see FIGS. 3(D) and 3(E)). The formed plurality of line type drawing files S3 are transmitted to and stored in the memory unit 10, but the symbol extraction drawing file S2 may be transmitted directly from the drawing file forming unit 3 to the symbol extraction unit 4 or the side note extraction unit 7. Similarly, the formed plurality of line type drawing files S3 are also transmitted to and stored in the memory unit 10, but the formed plurality of line type drawing files S3 may be transmitted directly from the drawing file forming unit 3 to the side note extraction file setting unit 6 or the side note extraction unit 7.

[0034] <Symbol extraction process> 1 and 2, when the symbol extraction unit 4 receives the symbol extraction drawing file S2 from the storage unit 10 (or when the symbol extraction drawing file S2 is sent from the drawing file formation unit 3 to the symbol extraction unit 4), the symbol extraction unit 4 extracts symbols from the symbol extraction drawing file S2 by pattern matching using AI or a known pattern matching method based on the symbol extraction drawing file S2. Specifically, an extracted symbol file S4 is formed, which includes symbol data obtained by cutting out an image of an area including the extracted symbol, location information of the area, a confidence level, etc. This extracted symbol file S4 is sent to the storage unit 10 and stored therein.

[0035] The extracted symbol file S4 may be sent directly to the side search area setting unit 5. Furthermore, extracted symbol files S4 may be transmitted to storage unit 10 or ancillary search area setting unit 5 once extraction of all symbols has been completed, or extracted symbol files S4 may be transmitted to storage unit 10 or ancillary search area setting unit 5 each time a symbol is extracted and an extracted symbol file S4 is formed. Furthermore, once a certain number of extracted symbol files S4 have been formed, the certain number of extracted symbol files S4 may be transmitted collectively to storage unit 10 or ancillary search area setting unit 5.

[0036] <Side note: Search area setting process> 1 and 2, when the ancillary search area setting unit 5 receives the extracted symbol file S4 from the memory unit 10 (or when the extracted symbol file S4 is sent from the symbol extraction unit 4 to the ancillary search area setting unit 5), the ancillary search area setting unit 5 sets an ancillary search area SA for each symbol. That is, the ancillary search area setting unit 5 sets a certain area centered on the symbol as the ancillary search area SA based on the information included in the extracted symbol file S4. Then, information about the set ancillary search area SA is sent from the ancillary search area setting unit 5 to the memory unit 10 and stored therein.

[0037] The information on the side note search area SA may be sent directly from the side note search area setting unit 5 to the side note extraction file setting unit 6 or the side note extraction unit 7 . Furthermore, the information on the annotative search area SA may be transmitted to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7 once the formation of the information on the annotative search area SA for all symbols has been completed, or the information on the annotative search area SA may be transmitted to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7 each time the information on the annotative search area SA for each symbol is formed. Furthermore, once the information on the annotative search area SA for a certain number of symbols has been formed, the information on a certain number of the annotative search areas SA may be transmitted collectively to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7.

[0038] <File setting process for extracting side notes> As shown in FIGS. 1 and 2, when the file setting unit 6 for side note extraction receives multiple line type drawing files S3 from the storage unit 10 (or when multiple line type drawing files S3 are sent from the drawing file formation unit 3 to the file setting unit 6 for side note extraction), the file setting unit 6 for side note extraction performs AI-OCR processing or the like and selects a file for side note extraction S6 from the multiple line type drawing files S3 based on the character data. For example, the line type drawing file S3 with the most detected characters is selected as the file for side note extraction S6. Then, information about the selected file for side note extraction S6 is stored in the storage unit 10. Note that the information about the file for side note extraction S6 may be sent directly from the file setting unit 6 for side note extraction to the file extraction unit 7.

[0039] When determining the sidenote extraction file S6 by a method such as pattern matching (in the case of the drawing recognition method of the second embodiment), as shown in FIG. 7, the sidenote extraction file setting unit 6 acquires information on the multiple line type drawing files S3 and the sidenote search area SA from the storage unit 10 (when information on the multiple line type drawing files S3 and the sidenote search area SA is transmitted from the drawing file forming unit 3 and the sidenote search area setting unit 5), and then acquires the sidenote template list TP stored in the storage unit 10. The sidenote extraction file setting unit 6 may also acquire the sidenote template list TP stored in the storage unit 10 in advance. Then, it selects multiple representative symbols, and for each of the multiple line type drawing files S3, sets comparison areas TS corresponding to the sidenote search areas SA of the selected representative symbols in the multiple line type drawing files S3, and matches graphics such as sidenotes included in the comparison areas TS with the sidenote templates registered in the sidenote template list TP to calculate the matching results. A matching result is created for each line type drawing file S3, and based on the matching result, the line type drawing file S3 with the highest result is selected as the side note extraction file S6, and information on the side note extraction file S6 is stored in the storage unit 10. If the size of the side note template is changed during matching, information on the optimal template size is also stored in the storage unit 10 along with information on the side note extraction file S6. The information on the side note extraction file S6 and the information on the optimal template size may also be sent directly from the side note extraction file setting unit 6 to the side note extraction unit 7.

[0040] <sidenote extraction process> 1 and 2, when the information on the side note search area SA and the information on the side note extraction file S6 are acquired from the storage unit 10 (when the information on the side note search area SA and the information on the side note extraction file S6 are transmitted from the side note search area setting unit 5 and the side note extraction file setting unit 6), the side note extraction unit 7 acquires the side note template list TP stored in the storage unit 10. Note that the side note extraction unit 7 may have acquired the side note template list TP stored in the storage unit 10 in advance.

[0041] When the sidenote template list TP is acquired, the sidenote extraction unit 7 sets the detection area SS corresponding to the sidenote search area SA for all symbols in the sidenote extraction file S6, and matches the figures included in this detection area SS with the sidenote template list TP to extract sidenote candidates. If there is one sidenote candidate and the similarity (confidence in AI matching) is equal to or greater than a certain level, or if multiple sidenote candidates are extracted but only one has a similarity equal to or greater than a certain level, the sidenote candidate is determined to be a sidenote of the symbol to which the detection area SS is set (in other words, the symbol to which the sidenote search area SA is set). The sidenote candidate determined to be a sidenote of the symbol is then linked to the symbol to which the detection area SS (sidenote search area SA) is set, and the linked data is transmitted to the storage unit 10 for storage.

[0042] Furthermore, when multiple annotative candidates with a similarity equal to or higher than a certain level are extracted, the annotative extraction unit 7 determines that the annotative candidate that is considered to be most related to the symbol is the annotative of the symbol, links this annotative candidate with the symbol for which the annotative search area SA is set, and transmits and stores it as symbol data to the storage unit 10. For example, it determines that the annotative candidate that is closest (shortest) to the symbol is the annotative candidate that is considered to be highly related to the symbol, links this annotative candidate with the symbol for which the annotative search area SA is set, and transmits and stores it as symbol data to the storage unit 10.

[0043] Since symbols and annotations are extracted in the above-described manner, the precision of extraction of symbols and annotations can be increased, and related symbols and annotations can be appropriately linked and stored.

[0044] <About the Sidenote Template List TP> The font, color, size, etc. of the side notes stored in the side note template list TP (see FIG. 9(B)) used for side note matching are not particularly limited. The characters constituting the side notes may be registered as a whole, or each character constituting the side note (for example, a single alphanumeric character) may be registered. The side note template list TP may contain multiple side notes that are expressed in multiple fonts and have the same constituent characters or numbers.

[0045] Furthermore, a management list CR may be provided in addition to the side note template list TP, which stores the number of side note characters used in symbols (see FIGS. 2 and 9(C)), and this management list CR may be stored in the storage unit 10. Then, a function (verification function) for determining whether side notes selected by matching using the management list CR are appropriate may be provided in the side note extraction file setting unit 6 and the side note extraction unit 7. By providing such a management list CR and verification function, it is possible to prevent shapes that are not suitable for symbols from being selected as side notes in the side note extraction file setting unit 6 and the side note extraction unit 7.

[0046] For example, when multiple symbols are written adjacent to each other, the comparison area TS or the detection area SS may contain a footnote (hereinafter referred to as a false footnote) for the adjacent symbol. In this case, depending on how the footnote is written in the drawing, the false footnote may have a higher matching reliability than the footnote actually associated with the symbol, or the distance between the false footnote and the symbol may be closer than the distance between the actual footnote and the symbol. This situation is particularly likely to occur when a footnote template consists of only a single alphanumeric character. Therefore, if a footnote is selected by matching the shape included in the comparison area TS or the detection area SS with the footnote template, and then a verification function is used to verify whether the footnote is suitable for the symbol, this can prevent the footnote extraction file setting unit 6 or the footnote extraction unit 7 from selecting a footnote that is inappropriate for the symbol.

[0047] <About setting up the file S6 for extracting side notes> In the above example, the description was given of a case where the side note extraction file setting unit 6 automatically determines the side note extraction file S6 from multiple line type drawing files S3. The side note extraction file S6 may also be determined by an operator visually checking the multiple line type drawing files S3. For example, the side note extraction file setting unit 6 may be provided with a function to display multiple line type drawing files S3 on a display DS or the like, and a function to select one line type drawing file S3 from the displayed multiple line type drawing files S3 and save that line type drawing file S3 as the side note extraction file S6. In addition, even when an operator visually checks multiple line type drawing files S3 and determines the file for extracting side notes S6, the processes other than the process of determining the file for extracting side notes S6 are performed by each function of the drawing recognition device 1, as in the drawing recognition method of the first embodiment and the drawing recognition method of the second embodiment. Furthermore, when the worker determines the file S6 for side note extraction, the side note search area setting unit 5 described above does not necessarily have to be provided, and the symbol extraction unit 3 or the side note extraction unit 7 may be provided with a function similar to that of the side note search area setting unit 5. In other words, the drawing recognition device 1 may be configured with the data input unit 2, the drawing file creation unit 3, the symbol extraction unit 4, the file setting unit 6 for side note extraction, the side note extraction unit 7, and the memory unit 10. The method in which the worker extracts the side notes by adopting the method for determining the side note extraction file S6 is the drawing recognition method of the third embodiment (see FIG. 8). [Industrial Applicability]

[0048] The drawing recognition method of the present invention is suitable as a method for extracting symbols and characters from drawings containing a large number of symbols, characters, and lines, such as construction drawings. [Explanation of symbols]

[0049] 1 Drawing recognition device 2 Data entry section 3. Drawing File Creation Department 4 Symbol Extraction 5. Side note search area setting section 6. File settings for extracting side notes 7 Paragraph extraction part 10 Storage section S1 Original drawing file S2 Symbol extraction drawing file S3 Linetype Drawing File S4 Extracted Symbol File S6 File for extracting side notes SA Sidenote Search Area SS detection area TS comparison area TP Sidenote Template List CR Management List

Claims

1. A drawing recognition method for extracting symbols and annotations contained in a drawing file, comprising: a drawing file creation step of creating a symbol extraction drawing file used for symbol extraction and a plurality of line type drawing files classified by line type from the drawing file; a symbol extraction step of extracting symbols from the symbol extraction drawing file formed in the drawing file formation step; a side note extraction file setting step for determining a side note extraction file, which is a line type drawing file for extracting side notes from the plurality of line type drawing files formed in the drawing file forming step; a side note extraction step of extracting side notes corresponding to each symbol from a figure detected in a detection area corresponding to a side note search area centered on each symbol extracted in the symbol extraction step, from the side note extraction file determined in the side note extraction file setting step; A drawing recognition method characterized by:

2. In the side note extraction file setting step, An OCR process is performed on the plurality of line type drawing files, and a line type drawing file that contains a large amount of detected character data is set as the side note extraction file.

2. The drawing recognition method according to claim 1.

3. The OCR processing is AI-OCR processing.

3. The drawing recognition method according to claim 2.

4. a side search area setting step for setting side search areas each centered on a symbol for each of the plurality of symbols extracted in the symbol extraction step, In the side note extraction file setting step, For the plurality of line type drawing files formed in the drawing file forming step, a comparison area corresponding to the side search area set in the side search area setting step is set for a representative symbol, and the side extracting file is determined based on the figure detected in the comparison area.

2. The drawing recognition method according to claim 1.

5. In the side note extraction file setting step, For the plurality of line type drawing files formed in the drawing file forming step, a file for extracting side notes is determined based on the similarity between the side note templates registered in the side note template list and the figures detected in the comparison area.

5. The drawing recognition method according to claim 4.

6. In the side note extraction file setting step, A matching process is performed between the side note template and the figure detected in the comparison area while changing the size of the side note template registered in the side note template list, and an extraction template size is determined, which is the size of the side note template that has the highest similarity in the side note extraction file; In the side note extraction step, The side note included in the detection area is extracted using a side note template of the extraction template size determined in the side note extraction file setting step.

6. The drawing recognition method according to claim 5.

7. In the side note extraction step, extracting a plurality of figures included in the detection area as side note candidates; Among the plurality of side note candidates, the side note candidate that is closest to the symbol is associated with the symbol.

2. The drawing recognition method according to claim 1.

8. In the side note extraction step, extracting a plurality of figures included in the detection area as side note candidates; For the extracted plurality of side note candidates, a side note candidate to be linked to the symbol is determined based on a management list storing associations between the symbol and the side note, in which the side note search area corresponding to the detection area from which the side note candidate was extracted is set.

2. The drawing recognition method according to claim 1.

9. A drawing recognition device that extracts symbols and annotations included in a drawing file, The drawing recognition device a drawing file forming unit that forms, from the drawing file, a symbol extraction drawing file used for symbol extraction and a plurality of line type drawing files classified by line type; a symbol extraction unit that extracts symbols from the symbol extraction drawing file formed by the drawing file formation unit; a side note extraction file setting unit that determines a side note extraction file, which is a line type drawing file that extracts side notes from the plurality of line type drawing files formed by the drawing file forming unit; a side note extraction unit that extracts side notes corresponding to each symbol from a figure included in a detection area corresponding to a side note search area centered on each symbol extracted by the symbol extraction unit, from the side note extraction file determined by the side note extraction file setting unit. A drawing recognition device characterized by:

10. The side note extraction file setting unit The method has a function of performing OCR processing on the plurality of line type drawing files, and determining a line type drawing file that contains a large amount of detected character data from the plurality of line type drawing files as the file for side note extraction.

10. The drawing recognition device according to claim 9.

11. The OCR processing is AI-OCR processing.

11. The drawing recognition device according to claim 10.

12. a side search area setting unit that sets side search areas each centered on a symbol extracted by the symbol extraction unit for a plurality of symbols, The side note extraction file setting unit For the plurality of line type drawing files formed by the drawing file forming unit, a comparison area corresponding to the sideline search area set by the sideline search area setting unit is set for a representative symbol, and the sideline extraction file is determined based on the figure detected in the comparison area.

10. The drawing recognition device according to claim 9.

13. The side note extraction file setting unit The file for extracting side notes is determined based on the similarity between the side note template registered in the side note template list and the figure detected in the comparison area for the plurality of line type drawing files formed by the drawing file forming unit.

13. The drawing recognition device according to claim 12.

14. The side note extraction file setting unit The method has a function of changing the size of the sidenote template registered in the sidenote template list while performing a matching process between the sidenote template and a figure detected in the comparison area, and determining an extraction template size that is the size of the sidenote template that maximizes the similarity in the sidenote extraction file, The side note extraction unit The file setting unit for extracting the side notes has a function of extracting the side notes included in the detection area by using a side note template of the extraction template size determined by the side note extraction file setting unit.

14. The drawing recognition device according to claim 13.

15. In the side note extraction unit, A function of extracting a plurality of figures included in the detection area as side note candidates; and a function of linking the side note candidate that is closest to the symbol among the plurality of side note candidates with the symbol.

10. The drawing recognition device according to claim 9.

16. In the side note extraction unit, The method has a function of extracting a plurality of figures included in the detection area as side note candidates, For the extracted plurality of side note candidates, a function of determining side note candidates to be linked to a symbol based on a management list storing associations between the symbols and side notes that set the side note search areas corresponding to the detection areas from which the side note candidates were extracted.

10. The drawing recognition device according to claim 9.

Images