Drawing recognition method and drawing recognition device

Abstract

A drawing recognition method and device are provided that can accurately extract symbols and annotations from a drawing and associate the two to digitize them. [Solution] A drawing recognition method for extracting symbols and annotations contained in a drawing file, comprising: a drawing file formation process for forming a drawing file S2 for symbol extraction and a plurality of line type drawing files S3 from an original drawing file S1; a symbol extraction process for extracting symbols from the drawing file S2 for symbol extraction; a file setting process for file setting a file S6 for annotation extraction; and a file extraction process for extracting annotations corresponding to each symbol from figures detected in a detection area SS corresponding to an annotation search area SA centered on each symbol extracted in the symbol extraction process, in the file S6 for annotation extraction determined in the file setting process for file extraction.

Description

[Technical field]

[0001] The present invention relates to a drawing recognition method and more particularly to a drawing recognition method and device for extracting, from various drawings, various symbols and characters or the like that are written near the symbols and that supplement the specifications of facilities, equipment, or the like represented by the symbols in association with each other. [Background technology]

[0002] Currently, it is common to create various drawings using CAD software. When drawings created using CAD software are provided, they may be provided not as CAD data but as data converted into PDF format or as paper-based drawings (hereinafter simply referred to as paper-based drawings, etc.), and it may be necessary to obtain various information described in these drawings. For example, architectural design drawings contain a large amount of information such as equipment symbols (hereinafter simply referred to as symbols), characters and figures related to the symbols (hereinafter simply referred to as side notes), wiring, etc., and it may be necessary to extract symbols and side notes. However, architectural design drawings are written in a state where symbols, side notes, and wiring are overlapped or very close to each other, so it is difficult to extract symbols and side notes with high accuracy, and techniques have been developed for extracting symbols and side notes with high accuracy from paper-based drawings, etc. (see Patent Documents 1 to 5). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-259076 A [Patent Document 2] JP 2006-227824 A [Patent Document 3] Patent No. 7234719 [Patent Document 4] JP 2001-92967 A [Patent Document 5] Japanese Patent Application Publication No. 11-126216 Summary of the Invention [Problem to be solved by the invention]

[0004] The techniques of Patent Documents 1 to 5 extract figures and characters in a drawing by template matching, but in a drawing that contains a mixture of wiring, symbols, etc., it is difficult to perform appropriate template matching, and it is difficult to accurately extract symbols, etc. In particular, when a symbol and a side note are described in association with each other, it is difficult to accurately extract the symbol or side note in the associated state.

[0005] In view of the above circumstances, an object of the present invention is to provide a drawing recognition method and a drawing recognition device that can accurately extract symbols and annotations from a drawing and associate the two with each other to convert them into data. [Means for solving the problem]

[0006] <How to recognize drawings> The drawing recognition method of the first invention is a drawing recognition method for extracting symbols and annotations contained in a drawing file, and is characterized by carrying out the following steps: a drawing file formation step for forming a symbol extraction drawing file used for symbol extraction and a plurality of line type drawing files classified by line type from the drawing file; a symbol extraction step for extracting symbols from the symbol extraction drawing file formed in the drawing file formation step; a annotation extraction file setting step for determining an annotation extraction file which is a line type drawing file from which annotations are extracted from the plurality of line type drawing files formed in the drawing file formation step; and a annotation extraction step for extracting annotations corresponding to each symbol from figures detected in a detection area corresponding to an annotation search area centered on each symbol extracted in the symbol extraction step from the annotation extraction file determined in the annotation extraction file setting step. The drawing recognition method of the second invention is characterized in that, in the first invention, in the file setting step for extracting side notes, OCR processing is performed on the multiple line type drawing files, and the line type drawing file among the multiple line type drawing files that contains the most detected character data is set as the file for extracting side notes. A drawing recognition method according to a third aspect of the present invention is characterized in that, in the second aspect of the present invention, the OCR processing is AI-OCR processing. The drawing recognition method of the fourth invention, in the first invention, includes a side note search area setting step for setting a side note search area centered on each of the multiple symbols extracted in the symbol extraction step, and in the side note extraction file setting step, a comparison area corresponding to the side note search area set in the side note search area setting step is set for a representative symbol for the multiple line type drawing files formed in the drawing file formation step, and the side note extraction file is determined based on the figure detected in the comparison area. The drawing recognition method of the 5th invention is characterized in that, in the 4th invention, in the file setting process for side note extraction, a file for side note extraction is determined for a plurality of line type drawing files formed in the drawing file formation process based on the similarity between the side note templates registered in the side note template list and the figures detected in the comparison area. The drawing recognition method of the 6th invention is characterized in that, in the 5th invention, in the file setting process for extracting annotation, a matching process is performed between the annotation template registered in the annotation template list and the figure detected in the comparison area while changing the size of the annotation template, and an extraction template size is determined, which is the size of the annotation template that gives the highest similarity in the file for extracting annotation, and in the annotation extraction process, annotation included in the detection area is extracted using a annotation template of the extraction template size determined in the file setting process for extracting annotation. The drawing recognition method of the seventh invention is characterized in that, in the sidenote extraction process, a plurality of figures included in the sidenote search area are extracted as sidenote candidates, and among the plurality of sidenote candidates, the sidenote candidate that is closest to the symbol is linked to the symbol. The drawing recognition method of the 8th invention is characterized in that, in the sidenote extraction step, a plurality of figures included in the sidenote search area are extracted as sidenote candidates, and for the extracted plurality of sidenote candidates, a sidenote candidate to be linked to a symbol is determined based on a management list in which the relationship between the symbol that set the sidenote search area and the sidenote is stored. <Drawing recognition device> The drawing recognition device of the 9th invention is a drawing recognition device that extracts symbols and annotations contained in a drawing file, and is characterized in that it comprises a drawing file formation unit that forms a drawing file for symbol extraction used for symbol extraction and a plurality of line type drawing files classified by line type from the drawing file, a symbol extraction unit that extracts symbols from the drawing file for symbol extraction formed by the drawing file formation unit, a file setting unit for annotation extraction that determines a line type drawing file from which annotations are extracted from the plurality of line type drawing files formed by the drawing file formation unit, and a file extraction unit that extracts annotations corresponding to each symbol from a figure included in a detection area corresponding to an annotation search area centered on each symbol extracted by the symbol extraction unit, from the file for annotation extraction determined by the file setting unit for annotation extraction. The drawing recognition device of the 10th invention is characterized in that, in the 9th invention, the file setting unit for annotative extraction has the function of performing OCR processing on the multiple line type drawing files, and setting the line type drawing file that contains the most detected character data among the multiple line type drawing files as the file for annotative extraction. The drawing recognition device of an eleventh aspect of the present invention is the drawing recognition device of the tenth aspect of the present invention, characterized in that the OCR processing is AI-OCR processing. The drawing recognition device of the 12th invention, in the 9th invention, is characterized in that it is equipped with a side note search area setting unit which sets side note search areas centered on a symbol for each of a plurality of symbols from the symbols extracted by the symbol extraction unit, and the side note extraction file setting unit has the function of setting a comparison area corresponding to the side note search area set by the side note search area setting unit for a representative symbol for a plurality of line type drawing files formed by the drawing file formation unit, and determining the side note extraction file based on a figure detected in the comparison area. The drawing recognition device of the 13th invention is characterized in that, in the 12th invention, the file setting unit for extracting side notes has a function of determining a file for extracting side notes for a plurality of line type drawing files formed by the drawing file formation unit based on the similarity between the side note templates registered in the side note template list and the figures detected in the comparison area. The drawing recognition device of the 14th invention is characterized in that, in the 13th invention, the file setting unit for extracting annotation has a function of performing a matching process between the annotation template registered in the annotation template list and a figure detected in the comparison area while changing the size of the annotation template, and determining an extraction template size, which is the size of the annotation template that results in the highest similarity in the annotation extraction file, and the annotation extraction unit has a function of extracting annotation included in the detection area using a annotation template of the extraction template size determined in the file setting unit for extracting annotation. The drawing recognition device of the 15th invention is characterized in that, in the 9th invention, the side note extraction unit has a function of extracting a plurality of figures included in the side note search area as side note candidates, and a function of linking the side note candidate among the plurality of side note candidates that is closest to the symbol to the symbol. The drawing recognition device of the 16th invention is characterized in that, in the 9th invention, the footnote extraction unit has a function of extracting a plurality of figures included in the footnote search area as footnote candidates, and has a function of determining the footnote candidate to be linked to a symbol for the extracted plurality of footnote candidates based on a management list in which the relationship between the symbol that sets the footnote search area and the footnote is stored. Effect of the Invention

[0007] <How to recognize drawings> According to the first aspect of the present invention, since the symbols and the side notes are extracted separately, it is possible to increase the extraction accuracy of the symbols and the side notes, and to appropriately associate and store the related symbols and side notes. According to the second to sixth aspects of the present invention, the extraction precision of the side notes can be improved. According to the seventh aspect of the present invention, a symbol can be easily linked to a side note that is highly related to the symbol. According to the eighth aspect of the present invention, it is possible to increase the accuracy of linking a symbol with a side note that is highly related to the symbol. <Drawing recognition device> According to the ninth aspect of the present invention, since the symbols and the side notes are extracted separately, it is possible to improve the extraction accuracy of the symbols and the side notes, and to appropriately associate and store the related symbols and side notes. According to the tenth to fourteenth aspects of the invention, the extraction precision of the side notes can be improved. According to the fifteenth aspect of the invention, a symbol can be easily linked to an annotated comment that is highly related to the symbol. According to the sixteenth aspect of the present invention, it is possible to increase the accuracy of linking a symbol with a side note that is highly related to the symbol. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a flow diagram of a drawing recognition method according to the first embodiment. [Diagram 2] 1 is a schematic block diagram of a drawing recognition device 1 of the present embodiment. [Diagram 3](A) is a diagram showing an example of an original drawing file S1, (B) is a diagram showing an example of a drawing file for symbol extraction (original SVG file), (C) is a diagram showing an example of a list of multiple original line type image files, (D) is a diagram showing an example of a drawing file for symbol extraction S2, and (E) is a diagram showing an example of multiple line type drawing files S3. [Figure 4] 13A is a diagram showing an example of a state in which an annotative search area SA is set for each symbol included in a drawing file S2 for symbol extraction, and FIG. 13B is a diagram showing an example of a method for selecting an annotative extraction file S6. [Diagram 5] (A) is a diagram showing an example of a state in which a side note search area SA has been set for a selected symbol included in a drawing file S2 for symbol extraction, and (B) is a diagram showing an example of setting a detection area SS corresponding to the side note search area SA in (A). [Figure 6] This figure shows an example of a method for selecting a side note extraction file S6, where (A) shows an example of a side note search area SA for a representative symbol, and (B) shows an example of a state in which a comparison area TS corresponding to the side note search area SA of (A) is set. [Figure 7] FIG. 11 is a flow diagram of a drawing recognition method according to a second embodiment. [Figure 8] FIG. 11 is a flow diagram of a drawing recognition method according to a third embodiment. [Figure 9] 13A is a diagram showing an example of symbol data SD, FIG. 13B is a diagram showing an example of a side note template list TP, and FIG. 13C is a diagram showing an example of a management list CR. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] The drawing recognition method of this embodiment is a method for extracting symbols and annotations from drawing files such as construction drawings, and is characterized by its ability to extract symbols and annotations with high accuracy while maintaining the association between the symbols and annotations.

[0010] The drawing file from which symbols and the like are extracted by the drawing recognition method of this embodiment may be a vector format drawing file, and is not particularly limited. Drawing files include drawing files (drawing files in which images are composed of pixel data) in which drawings printed on paper are digitized by scanning, and drawing files (drawing files in which symbols and the like in the data have individual information) created by converting drawings drawn by CAD. For example, the former file includes raster format drawing files such as raster PDF files and PNG files, and the latter file includes vector format drawing files such as vector PDF files and SVG files. The drawing file from which symbols and the like are extracted by the drawing recognition method of this embodiment is a vector format drawing file among the above-mentioned raster format drawing files and vector format drawing files. In the following, the drawing file from which symbols and the like are extracted is simply called an original drawing file.

[0011] In the following description, a case where the drawing from which the symbols are extracted is a construction drawing will be described as a representative example, but the drawing from which the symbols are extracted by the drawing recognition method of this embodiment is not particularly limited. In the following description, since the drawing from which the symbols are extracted is a construction drawing, the term "symbol" refers to a symbol that diagrammatically represents an electrical / mechanical component used in the facilities of a building. In addition, the term "side note" refers to a character or symbol that supplements the specifications of the electrical / mechanical component represented by the symbol described above and is written adjacent to the symbol.

[0012] <Drawing recognition device 1 of this embodiment> First, a drawing recognition device 1 for implementing the drawing recognition method of this embodiment will be described. As shown in FIG. 2, the drawing recognition device 1 may have a display function for displaying drawing files formed by each unit described later on a display DS, and an input function for inputting necessary information and instructing necessary operations using an input device IS such as a keyboard or a mouse.

[0013] <Data Entry Section 2> As shown in Fig. 2, an apparatus 1 for implementing the drawing recognition method of this embodiment (hereinafter simply referred to as drawing recognition apparatus 1) has a data input unit 2 for creating an original drawing file S1 (see Fig. 3(A)). This data input unit 2 has a function (data receiving function) for accepting data of a vector-format drawing file, such as a vector PDF file, that has already been created, from a storage medium such as a USB or receiving it via a line such as the Internet. The drawing file received by the data input unit 2 is transmitted to and stored in the storage unit 10 as the original drawing file S1.

[0014] <Drawing file creation section 3> As shown in FIG. 2, the drawing recognition device 1 has a drawing file forming unit 3. The drawing file forming unit 3 has a function of forming a plurality of drawings from the original drawing file S1 received by the data input unit 2. Specifically, the drawing file forming unit 3 has a function of forming a symbol extraction drawing file S2 (see FIG. 3(D)) and a plurality of line type drawing files S3 (see FIG. 3(E)) from the original drawing file S1. The plurality of line type drawing files are files classified according to line thickness. The drawing file forming unit 3 has a function of transmitting the created symbol extraction drawing file S2 and the plurality of line type drawing files S3 to the storage unit 10, and the symbol extraction drawing file S2 and the plurality of line type drawing files S3 transmitted to the storage unit 10 are stored in the storage unit 10.

[0015] The method by which the drawing file forming unit 3 forms the symbol extraction drawing file S2 and the multiple line type drawing files S3 from the original drawing file S1 is not particularly limited. For example, they can be formed in the following manner.

[0016] Since the original drawing file S1 is a drawing file in a vector format such as a vector PDF file, the drawing file forming unit 3 forms a symbol extraction drawing file S2 and a plurality of line type drawing files S3 as follows. First, the drawing file forming unit 3 forms an original SVG file (see FIG. 3B) from the original drawing file S1, and forms a symbol extraction drawing file S2, which is a PNG file, by extracting data such as symbols, annotations, wiring, etc., contained in the original SVG file. In this case, the symbol extraction drawing file S2 is a PNG file that includes all of the symbols, annotations, wiring, etc. Furthermore, the drawing file forming unit 3 forms a plurality of original line type drawing files, which are SVG files, based on the line thickness information included in the original SVG file (see FIG. 3(C)), and forms a plurality of line type drawing files S3, which are PNG files, from the plurality of original line type drawing files. The plurality of line type drawing files S3 are files classified according to line thickness. In other words, each line type drawing file S3 is a PNG file that includes symbols, bylines, wiring, etc. with the same line thickness, and does not include symbols, bylines, wiring, etc. with different line thicknesses.

[0017] In addition, in Fig. 3(E), three line type drawing files S3 (a) to (c) are shown, but the number of line type drawing files S3 is not particularly limited. An appropriate number of line type drawing files S3 may be formed according to the lines, symbols, notes, etc. included in the original drawing file S1.

[0018] <Symbol Extraction Part 4> As shown in FIG. 2, the drawing recognition device 1 has a symbol extraction unit 4. The symbol extraction unit 4 has a function of extracting a symbol from the drawing file S2 for symbol extraction created by the drawing file creation unit 3. The method by which the symbol extraction unit 4 extracts a symbol is not particularly limited, but the symbol can be extracted by a known method such as pattern matching based on the symbol data SD (see FIG. 9(A)) stored in the storage unit 10. The pattern matching can be performed using pattern matching using AI (artificial intelligence), but the means for performing the pattern matching is not particularly limited. The symbol extraction unit 4 also has a function of forming an extracted symbol file S4 for each extracted symbol, in which symbol data obtained by cutting out an image of an area including the extracted symbol is associated with location information of the area (i.e., information on the coordinates of the area on the drawing file S2 for symbol extraction). Each of the formed extracted symbol files S4 is transmitted to the storage unit 10 and stored therein.

[0019] In addition, when symbol extraction is performed by pattern matching using AI (artificial intelligence), information on the confidence level of symbol extraction (expressed as a number: the closer to 1, the higher the confidence level) is also included in the extracted symbol file S4. In addition, the extracted symbol file S4 may be an image of an area including an extracted symbol, and the file name of the extracted symbol file S4 may include the position information and the confidence level, so that the image of the area is associated with the position information and the confidence level. If this method is adopted, it becomes possible to sort the extracted symbol files S4 in descending (or descending) order of confidence level on the display DS, which is desirable in that extracted symbol files S4 that have been erroneously detected can be efficiently deleted. For example, in the case of an image of a rectangular area, the file name of the extracted symbol file S4 may be set as 0.9224@17560×4782_17634×4854_0.png. In the file name mentioned above, "0.9224" is the confidence level, and "17560x4782_17634x4854" is the X-axis and Y-axis coordinates (position information) of the diagonal corners of the rectangular area image.

[0020] <Search area setting section 5> As shown in FIG. 2, the drawing recognition device 1 has a side note search area setting unit 5. The side note search area setting unit 5 has a function of setting an area (side note search area SA) for searching side notes associated with a symbol. Specifically, the side note search area setting unit 5 has a function of setting a certain area centered on each symbol as the side note search area SA for all symbols extracted by the symbol extraction unit 4. Specifically, based on information included in each extracted symbol file S4, the side note search area setting unit 5 has a function of setting a certain area centered on each symbol as the side note search area SA of each symbol (see FIG. 4(A)). When setting the side note search area SA of each symbol, the side note search area setting unit 5 sets a range that is assumed to include both each symbol and side notes related to that symbol as the side note search area SA. For example, an area calculated based on the size of each extracted symbol may be set as the side note search area SA. In addition, when a management list CR (see FIG. 9(C)) in which a note corresponding to each symbol (i.e., used for each symbol) is registered in advance is stored in the storage unit 10 (may be stored in the side note search area setting unit 5), the side note search area SA may be calculated based on the number of characters (the number of characters of the side note character string) registered in the management list CR. The information on the side note search area SA includes the coordinates of the side note search area SA (for example, when a rectangular area is set as the side note search area SA, the coordinate information of the corners, etc.). When the information on the side note search area SA is calculated, the information on the side note search area SA is transmitted from the side note search area setting unit 5 to the storage unit 10 and stored in the storage unit 10 in association with the extracted symbol information of each symbol for which the side note search area SA is set. The information on the side note search area SA may include information on the corresponding extracted symbol.

[0021] FIG. 9C shows a list of a plurality of annotations corresponding to one symbol, and the management list CR stores such an annotation list in association with each symbol.

[0022] <File Setting Unit 6 for Margin Extraction> As shown in FIG. 2, the drawing recognition apparatus 1 has a file setting unit 6 for margin extraction. The file setting unit 6 for margin extraction has a function of selecting a line type drawing file S3 suitable for margin extraction from a plurality of line type drawing files S3 formed by the drawing file forming unit 2 and setting it as the margin extraction file S6. When the margin extraction file S6 is selected, the information of the margin extraction file S6 is transmitted from the file setting unit 6 for margin extraction to the storage unit 10 and stored therein. The method by which the file setting unit 6 for margin extraction determines (selects) the margin extraction file S6 is not particularly limited. For example, the following method can be adopted.

[0023] <Method Using OCR Processing> The file setting unit 6 for margin extraction can adopt a method of performing OCR (Optical Character Recognition / Reader) processing on a plurality of line type drawing files S3 and determining the margin extraction file S6 from the plurality of line type drawing files S3 based on the character data obtained for each line type drawing file S3 by the OCR processing. For example, in a design drawing, characters described in a drawing including margins are often described in the same line type (line thickness). Therefore, it is considered likely that the characters including margins are included in the same line type drawing file S3. In other words, it is considered likely that the line type drawing file S3 containing a large number of characters contains margins. Therefore, among the plurality of line type drawing files S3, the line type drawing file S3 with a large number of detected characters when OCR processing is performed can be set as the margin extraction file S6.

[0024] In addition, the OCR process in the file setting unit 6 for extracting side notes is preferably performed by AI-OCR processing. AI-OCR uses a machine learning algorithm to learn characters and sentences from a large amount of data, and is therefore characterized by being able to grasp the characteristics of characters and documents and perform flexible and effective recognition. Therefore, by performing AI-OCR processing in which side notes obtained from many mechanical design drawings (such as the original drawing file S1) are machine-learned using machine learning (such as deep learning), the accuracy of detecting side notes can be improved.

[0025] <When using a representative symbol> Also, as shown in FIG. 6, the file setting unit 6 for extracting sidenotes may select a plurality of representative symbols from the symbols extracted by the symbol extraction unit 4, set a comparison area TS corresponding to the sidenote search area SA set by the sidenote search area setting unit 5 for the selected symbols, and use the figures detected in the comparison area TS to determine the file S6 for extracting sidenotes. That is, the figure detected in the comparison area TS may be compared with the sidenote template registered in the sidenote template list TP by a known method such as pattern matching, and the line type drawing file S3 with a high matching result may be set as the file S6 for extracting sidenotes. As the result of matching, the similarity between the figure and the sidenote template may be adopted. For example, the similarity between the figure matched in a plurality of comparison areas TS and the sidenote template (confidence level in the case of AI matching) may be obtained, and the line type drawing file S3 containing many figures with high similarity may be set as the file S6 for extracting sidenotes.

[0026] Here, the comparison area TS means an area in each line type drawing file S3 that has coordinates corresponding to the coordinates of the sideline search area SA in the symbol extraction drawing file S2. For example, in the case of Fig. 6(A) and (B), the comparison area TS(1) in Fig. 6(B) corresponds to the sideline search area SA(1) in Fig. 6(A), and the comparison area TS(2) in Fig. 6(B) corresponds to the sideline search area SA(2) in Fig. 6(A).

[0027] In addition, the matching between the figure and the annotative template may be performed while changing the size of the annotative template. In this case, the size of the annotative template that provides the highest similarity (optimal template size) is stored in the storage unit 10 in association with the annotative extraction file S6, or the size of the annotative template alone.

[0028] The method of selecting the representative symbol is not particularly limited. For example, the file setting unit 6 for extracting side notes may automatically select the symbol by selecting the symbol in descending order of the confidence level when the symbol is extracted by the symbol extracting unit 4. Alternatively, the worker may select the representative symbol by checking the image of the extracted symbol (i.e., the image included in the extracted symbol file S4 or the image in the drawing file S2 for extraction) and the confidence level.

[0029] <sidenote extraction part 7> As shown in FIG. 2, the drawing recognition device 1 has a side note extraction unit 7. The side note extraction unit 7 has a function of setting a detection area SS corresponding to the side note search area SA formed by the side note search area setting unit 5 in the side note extraction file S6 based on the extracted symbol file S4 formed by the symbol extraction unit 4, and extracting side notes from the detection area SS, and a linking function of linking the extracted side note with the symbol. Specifically, the side note extraction unit 7 sets the detection area SS corresponding to the side note search area SA set by the side note search area setting unit 5 for all symbol files S4 in the side note extraction file S6. Then, for the figures included in the detection area SS of the side note extraction file S6, pattern matching is performed using the side note template list TP, and figures (Z in FIG. 5) that are highly similar to the side note template are extracted as side note candidates. Then, when only one side note candidate is extracted, the side note candidate is treated as a side note related to the symbol for which the side note search area SA is set, and the side note candidate and the symbol are linked together and transmitted to and stored in the memory unit 10 as symbol data.

[0030] In addition, when there are multiple side note candidates, the side note extraction unit 7 has a function of linking the side note candidate that is considered to be most related to the symbol from among the multiple side note candidates to the symbol to create symbol data. The method by which the side note extraction unit 7 selects the side note candidate that is considered to be highly related is not particularly limited, but for example, the side note candidate that is closest in distance between the symbol and the side note candidate (for example, the distance from the center of the symbol to the center of the side note candidate) may be linked to the symbol.

[0031] In addition, when the sidenote extraction file S6 is selected using the matching process in the sidenote extraction file setting unit 6, and an optimal template size suitable for matching is required, it is preferable to perform pattern matching using the optimal template size. In other words, when performing pattern matching between a figure included in the sidenote search area SA and a sidenote template, it is preferable to perform pattern matching with a sidenote template of the optimal template size. If pattern matching is performed using a sidenote template of the optimal template size, sidenotes can be appropriately extracted from figures present in the sidenote search area SA, and it becomes easier to prevent figures other than sidenotes from being erroneously extracted as sidenotes.

[0032] <Extraction and linking of symbols and annotations> The operation of extracting symbols and annotations by the above-mentioned drawing recognition device 1 (that is, the drawing recognition method of the first embodiment) will be described (see FIGS. 1 and 2).

[0033] <Drawing file creation process> As shown in FIG. 1 and FIG. 2, the drawing file forming unit 3 receives the original drawing file S1 formed or received by the data input unit 2 and stored in the storage unit 10. The drawing file forming unit 3 forms a symbol extraction drawing file S2 and a plurality of line type drawing files S3 from the original drawing file S1 (see FIG. 3(D) and (E)). The formed plurality of line type drawing files S3 are transmitted to the storage unit 10 and stored therein, but the symbol extraction drawing file S2 may be directly transmitted from the drawing file forming unit 3 to the symbol extraction unit 4 or the side note extraction unit 7. Similarly, the formed plurality of line type drawing files S3 are also transmitted to the storage unit 10 and stored therein, but the formed plurality of line type drawing files S3 may be directly transmitted from the drawing file forming unit 3 to the side note extraction file setting unit 6 or the side note extraction unit 7.

[0034] <Symbol extraction process> 1 and 2, when the symbol extraction unit 4 receives the drawing file S2 for symbol extraction from the storage unit 10 (or when the drawing file S2 for symbol extraction is transmitted from the drawing file formation unit 3 to the symbol extraction unit 4), the symbol extraction unit 4 extracts a symbol from the drawing file S2 for symbol extraction by pattern matching using AI or a known pattern matching method based on the drawing file S2 for symbol extraction. Specifically, an extracted symbol file S4 is formed, which includes symbol data obtained by cutting out an image of an area including the extracted symbol, location information of the area, a confidence level, and the like. This extracted symbol file S4 is transmitted to the storage unit 10 and stored therein.

[0035] The extracted symbol file S4 may be sent directly to the side search area setting unit 5. Furthermore, all extracted symbol files S4 may be transmitted to storage unit 10 or ancillary search area setting unit 5 once extraction of all symbols has been completed, or each time a symbol is extracted and an extracted symbol file S4 is formed, extracted symbol file S4 may be transmitted to storage unit 10 or ancillary search area setting unit 5. Furthermore, when a certain number of extracted symbol files S4 are formed, the certain number of extracted symbol files S4 may be transmitted collectively to storage unit 10 or ancillary search area setting unit 5.

[0036] <Search area setting process> As shown in Figures 1 and 2, when the sideline search area setting unit 5 receives the extracted symbol file S4 from the memory unit 10 (or when the extracted symbol file S4 is transmitted from the symbol extraction unit 4 to the sideline search area setting unit 5), the sideline search area setting unit 5 sets a sideline search area SA for each symbol. That is, the sideline search area setting unit 5 sets a certain area centered on the symbol as the sideline search area SA based on the information contained in the extracted symbol file S4. Then, information on the set sideline search area SA is transmitted from the sideline search area setting unit 5 to the memory unit 10 and stored therein.

[0037] The information on the sidenote search area SA may be directly transmitted from the sidenote search area setting unit 5 to the sidenote extraction file setting unit 6 or the sidenote extraction unit 7 . Furthermore, the information on the annotative search area SA may be transmitted to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7 when the formation of the information on the annotative search area SA for all symbols is completed, or the information on the annotative search area SA may be transmitted to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7 each time the information on the annotative search area SA for each symbol is formed. Moreover, when the information on the annotative search area SA for a certain number of symbols is formed, the information on a certain number of the annotative search areas SA may be transmitted collectively to the memory unit 10, the annotative extraction file setting unit 6, or the annotative extraction unit 7.

[0038] <File setting process for extracting side notes> As shown in FIG. 1 and FIG. 2, when the file setting unit 6 for extracting paranote receives a plurality of line type drawing files S3 from the storage unit 10 (or when a plurality of line type drawing files S3 are transmitted from the drawing file forming unit 3 to the file setting unit 6 for extracting paranote), the file setting unit 6 for extracting paranote performs AI-OCR processing or the like, and selects a file S6 for extracting paranote from the plurality of line type drawing files S3 based on the character data. For example, the line type drawing file S3 with the most detected characters is selected as the file S6 for extracting paranote. Then, information on the selected file S6 for extracting paranote is stored in the storage unit 10. Note that the information on the file S6 for extracting paranote may be transmitted directly from the file setting unit 6 for extracting paranote to the file extracting unit 7.

[0039] In addition, when the file S6 for extracting the sidenote is determined by a method such as pattern matching (in the case of the drawing recognition method of the second embodiment), as shown in FIG. 7, when the file setting unit 6 for extracting the sidenote acquires information on the multiple line type drawing files S3 and the sidenote search area SA from the storage unit 10 (when the information on the multiple line type drawing files S3 and the sidenote search area SA is transmitted from the drawing file forming unit 3 and the sidenote search area setting unit 5), it acquires the sidenote template list TP stored in the storage unit 10. In addition, the file setting unit 6 for extracting the sidenote may acquire the sidenote template list TP stored in the storage unit 10 in advance. Then, a plurality of representative symbols are selected, and for each of the multiple line type drawing files S3, a comparison area TS corresponding to the sidenote search area SA of the selected representative symbols is set in the multiple line type drawing files S3, and a figure such as a sidenote included in this comparison area TS is matched with a sidenote template registered in the sidenote template list TP to calculate a matching result. A matching result is created for each line type drawing file S3, and based on the matching result, the line type drawing file S3 with the highest result is selected as the side note extraction file S6, and information on the side note extraction file S6 is stored in the storage unit 10. Note that, if matching is performed while changing the size of the side note template, information on the optimal template size is also stored in the storage unit 10 together with information on the side note extraction file S6. Also, information on the side note extraction file S6 and information on the optimal template size may be directly transmitted from the side note extraction file setting unit 6 to the side note extraction unit 7.

[0040] <sidenote extraction process> 1 and 2, when the information on the sidenote search area SA and the information on the sidenote extraction file S6 are acquired from the storage unit 10 (when the information on the sidenote search area SA and the information on the sidenote extraction file S6 are transmitted from the sidenote search area setting unit 5 and the sidenote extraction file setting unit 6), the sidenote extraction unit 7 acquires the sidenote template list TP stored in the storage unit 10. Note that the sidenote extraction unit 7 may acquire the sidenote template list TP stored in the storage unit 10 in advance.

[0041] When the sidenote template list TP is acquired, the sidenote extraction unit 7 sets the detection area SS corresponding to the sidenote search area SA for all symbols in the sidenote extraction file S6, and matches the figures included in the detection area SS with the sidenote template list TP to extract sidenote candidates. If there is one sidenote candidate and the similarity (confidence in AI matching) is equal to or higher than a certain level, or if multiple sidenote candidates are extracted but only one has a similarity equal to or higher than a certain level, the sidenote candidate is determined to be a sidenote of the symbol with the detection area SS set (in other words, the symbol with the sidenote search area SA set). Then, the sidenote candidate determined to be a sidenote of the symbol is linked to the symbol with the detection area SS (sidenote search area SA) set, and is transmitted to the storage unit 10 as symbol data and stored therein.

[0042] Furthermore, when multiple side note candidates with a certain degree of similarity or higher are extracted, the side note extraction unit 7 determines that the side note candidate that is considered to be most related to the symbol is the side note of the symbol, links this side note candidate to the symbol in which the side note search area SA is set, and transmits and stores it as symbol data in the storage unit 10. For example, it determines that the side note candidate that is closest (shortest) to the symbol is the side note that is considered to be highly related to the symbol, links this side note candidate to the symbol in which the side note search area SA is set, and transmits and stores it as symbol data in the storage unit 10.

[0043] Since symbols and annotations are extracted in the above-described manner, the precision of extraction of symbols and annotations can be increased, and related symbols and annotations can be appropriately linked and stored.

[0044] <About the Sidenote Template List TP> The font, color, size, etc. of the annotative notes stored in the annotative note template list TP (see FIG. 9(B)) used for matching the annotative notes is not particularly limited. The characters constituting the annotative notes may be registered as a whole, or each character constituting the annotative note (for example, one alphanumeric character) may be registered. A plurality of annotative notes having the same constituent characters or numbers expressed in a plurality of fonts may be included in the annotative note template list TP.

[0045] Furthermore, a management list CR that stores the number of characters of the side note used in the symbol and the like may be provided separately from the side note template list TP (see Fig. 2 and Fig. 9(C)), and this management list CR may be stored in the storage unit 10. Then, a function (verification function) that uses the management list CR to determine whether the side note selected by matching is appropriate may be provided in the side note extraction file setting unit 6 and the side note extraction unit 7. By providing such a management list CR and verification function, it is possible to prevent shapes that are not suitable for symbols from being selected as side notes in the side note extraction file setting unit 6 and the side note extraction unit 7.

[0046] For example, when multiple symbols are written adjacently, the sidenotes of the adjacent symbols (hereinafter referred to as false sidenotes) may be included in the comparison area TS or the detection area SS. In this case, depending on the state in which the sidenotes are written in the drawing, the false sidenotes may have a higher matching reliability than sidenotes that are actually related to the symbol, or the distance between the false sidenotes and the symbol may be closer than the distance between the actual sidenotes and the symbol. In particular, when only one alphanumeric character is used as the sidenote template, such a situation is likely to occur. Therefore, if the sidenote is selected by matching the figure included in the comparison area TS or the detection area SS with the sidenote template, and then the verification function is set to verify whether the sidenote is suitable for the symbol, it is possible to prevent the sidenote extraction file setting unit 6 and the sidenote extraction unit 7 from selecting a sidenote that is not suitable for the symbol.

[0047] <About the settings for the file S6 for extracting side notes> In the above example, the case where the file setting unit 6 for extracting side notes automatically determines the file S6 for extracting side notes from a plurality of line type drawing files S3 has been described. The file S6 for extracting side notes may be determined by an operator visually checking the plurality of line type drawing files S3. For example, the file setting unit 6 for extracting side notes may be provided with a function for displaying a plurality of line type drawing files S3 on a display DS or the like, and a function for selecting one line type drawing file S3 from the plurality of displayed line type drawing files S3 and saving the selected line type drawing file S3 as the file S6 for extracting side notes. In addition, even when an operator visually checks multiple line type drawing files S3 to determine the file for extracting side notes S6, processes other than the process of determining the file for extracting side notes S6 are performed by each function of the drawing recognition device 1, as in the drawing recognition method of the first embodiment and the drawing recognition method of the second embodiment. Furthermore, when the worker determines the file S6 for side note extraction, the above-mentioned side note search area setting unit 5 does not necessarily have to be provided, and the symbol extraction unit 3 or the side note extraction unit 7 may be provided with a function similar to that of the side note search area setting unit 5. In other words, the drawing recognition device 1 may be composed of the data input unit 2, the drawing file formation unit 3, the symbol extraction unit 4, the file setting unit 6 for side note extraction, the side note extraction unit 7, and the storage unit 10. The method of extracting side notes by adopting the method in which the worker determines the side note extraction file S6 becomes the drawing recognition method of the third embodiment (see FIG. 8). [Industrial Applicability]

[0048] The drawing recognition method of the present invention is suitable as a method for extracting symbols and characters from drawings containing a large number of symbols, characters, and lines, such as construction drawings. [Explanation of symbols]

[0049] 1 Drawing recognition device 2 Data Entry Section 3. Drawing File Creation Department 4 Symbol Extraction 5. Side note search area setting section 6. File settings for extracting side notes 7 Paragraph extraction part 10 Storage section S1 Original drawing file S2 Drawing file for symbol extraction S3 Linetype Drawing File S4 Extracted Symbol File S6 File for extracting side notes SA Sidenote Search Area SS Detection Area TS comparison area TP Sidenote Template List CR Management List

Claims

1. A drawing recognition method for extracting symbols and annotations contained in a drawing file, comprising the steps of: a drawing file forming step of forming a symbol extraction drawing file used for symbol extraction from the drawing file and a plurality of line type drawing files classified according to line types; a symbol extraction step of extracting a symbol from the symbol extraction drawing file formed in the drawing file formation step; A file setting process for extracting side notes from the plurality of line type drawing files formed in the drawing file forming process, which is a file for extracting side notes, is a line type drawing file; a sidenote extraction step of setting a detection area corresponding to the sidenote search area centered on each symbol extracted in the symbol extraction step in the sidenote extraction file determined in the sidenote extraction file setting step, and extracting sidenotes corresponding to each symbol from figures detected in the detection area of ​​the sidenote extraction file. A drawing recognition method comprising:

2. In the side note extraction file setting step, An OCR process is performed on the plurality of line type drawing files, and a line type drawing file that contains a large amount of detected character data among the plurality of line type drawing files is set as the file for side note extraction.

2. The drawing recognition method according to claim 1.

3. The OCR processing is AI-OCR processing.

3. The drawing recognition method according to claim 2.

4. a side search area setting step of setting side search areas each having a center of a symbol for each of the plurality of symbols extracted in the symbol extraction step, In the side note extraction file setting step, For the plurality of line type drawing files formed in the drawing file forming step, a comparison area corresponding to the sideline search area set in the sideline search area setting step is set for a representative symbol, and the sideline extraction file is determined based on the figure detected in the comparison area.

2. The drawing recognition method according to claim 1.

5. In the side note extraction file setting step, For the plurality of line type drawing files formed in the drawing file forming step, a file for extracting side notes is determined based on the similarity between the side note templates registered in the side note template list and the figures detected in the comparison area.

5. The drawing recognition method according to claim 4.

6. In the side note extraction file setting step, A matching process is performed between the sidenote template and the figure detected in the comparison area while changing the size of the sidenote template registered in the sidenote template list, and an extraction template size is determined, which is the size of the sidenote template that has the highest similarity in the sidenote extraction file; In the side note extraction step, The side note included in the detection area is extracted using the side note template of the extraction template size determined in the side note extraction file setting step.

6. The drawing recognition method according to claim 5.

7. In the side note extraction step, Extracting a plurality of figures included in the detection area as side note candidates; Among the plurality of side note candidates, the side note candidate having the shortest distance from the symbol is associated with the symbol.

2. The drawing recognition method according to claim 1.

8. In the side note extraction step, Extracting a plurality of figures included in the detection area as side note candidates; For the extracted plurality of side note candidates, a side note candidate to be linked to a symbol is determined based on a management list in which the association between the symbol and the side note is stored, the side note search area being set corresponding to the detection area in which the side note candidate is extracted.

2. The drawing recognition method according to claim 1.

9. A drawing recognition device for extracting symbols and annotations included in a drawing file, comprising: The drawing recognition device includes: a drawing file forming unit for forming a symbol extraction drawing file used for symbol extraction from the drawing file and a plurality of line type drawing files classified according to line types; a symbol extraction unit that extracts a symbol from the symbol extraction drawing file formed by the drawing file formation unit; A file setting unit for extracting side notes determines a file for extracting side notes, which is a line type drawing file for extracting side notes from the multiple line type drawing files formed by the drawing file forming unit; a sidenote extraction unit that sets a detection area corresponding to a sidenote search area centered on each symbol extracted by the symbol extraction unit in the sidenote extraction file determined by the sidenote extraction file setting unit, and extracts sidenotes corresponding to each symbol from a figure included in the detection area of ​​the sidenote extraction file. A drawing recognition device characterized by the above.

10. The side note extraction file setting unit includes: A function of performing OCR processing on the plurality of line type drawing files, and determining a line type drawing file that contains a large amount of detected character data from among the plurality of line type drawing files as the file for side note extraction.

10. The drawing recognition device according to claim 9.

11. The OCR processing is AI-OCR processing.

11. The drawing recognition device according to claim 10.

12. a side search area setting unit for setting side search areas each having a center on a symbol extracted by the symbol extraction unit for each of a plurality of symbols, The side note extraction file setting unit includes: The file forming unit has a function of setting a comparison area corresponding to the sideline search area set by the sideline search area setting unit for a representative symbol for a plurality of line type drawing files formed by the sideline file forming unit, and determining the sideline extraction file based on a figure detected in the comparison area.

10. The drawing recognition device according to claim 9.

13. The side note extraction file setting unit includes: The file extracting section has a function of determining a file for extracting side notes based on a similarity between a side note template registered in a side note template list and a figure detected in the comparison area for a plurality of line type drawing files formed by the drawing file forming section.

13. The drawing recognition device according to claim 12.

14. The side note extraction file setting unit includes: A function of performing a matching process between the sidenote template and a figure detected in the comparison area while changing the size of the sidenote template registered in the sidenote template list, and determining an extraction template size that is the size of the sidenote template that has the highest similarity in the sidenote extraction file, The side note extraction unit is The file setting unit has a function of extracting a side note included in the detection area by using a side note template having the extraction template size determined by the side note extraction file setting unit.

14. The drawing recognition device according to claim 13.

15. In the side note extraction unit, A function of extracting a plurality of figures included in the detection area as side note candidates; and a function of linking, among the plurality of side note candidates, a side note candidate that is closest to the symbol with the symbol.

10. The drawing recognition device according to claim 9.

16. In the side note extraction unit, A function of extracting a plurality of figures included in the detection area as side note candidates, For the extracted multiple side note candidates, a function of determining a side note candidate to be linked to a symbol based on a management list in which the association between the symbol and the side note, which sets the side note search area corresponding to the detection area in which the side note candidate is extracted, is stored.

10. The drawing recognition device according to claim 9.

Images