Symbol identification method

Abstract

To provide a symbol identification method allowing for improving accuracy in identification of a symbol.SOLUTION: A computer performs: first processing of filling a region surrounded with lines in image data; second processing of subjecting the filled region to expansion processing; third processing of extracting a contour of the filled region; and fourth processing of identifying the filled region as a prescribed symbol if a result of comparison between the extracted contour and a contour of the prescribed symbol shows that the extracted contour and the contour of the prescribed symbol are approximate to each other, that two straight lines intersecting with each other are detected by performing linear transformation on a plurality of contour points constituting the extracted contour, and that a difference between an area of a region surrounded with the extracted contour and an area of the prescribed symbol is equal to or less than a prescribed value or twice the area of the region surrounded with the extracted contour is substantially equal to an area of a rectangular symbol.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a method for identifying symbols. [Background technology]

[0002] Conventionally, a symbol extraction processing method for extracting a triangular symbol is known (see, for example, Patent Document 1). Also conventionally known are a technique for recognizing a triangular finishing symbol and displaying the surface texture of a three-dimensional model (see, for example, Patent Document 2), a technique for matching a symbol in a drawing with a template pattern (see, for example, Patent Document 3), a technique for extracting a symbol by pattern matching (see, for example, Patent Document 4), and a technique for selecting a triangle from a menu prepared in advance to determine the geometrical components of a symbol (see, for example, Patent Document 5). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 61-109375 [Patent Document 2] Japanese Patent Application Publication No. 5-165923 [Patent Document 3] JP 2004-234424 A [Patent Document 4] Patent Publication No. 2022-21031 [Patent Document 5] Japanese Patent Application Publication No. 4-51366 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when a symbol is connected to other lines or characters, it is unclear how to separate the symbol from the other lines or characters, making it difficult to recognize the symbol.

[0005] Therefore, an object of the present invention is to provide a symbol identification method capable of improving the accuracy of symbol identification. [Means for solving the problem]

[0006] A symbol identification method according to the present invention is characterized in that a computer executes a first process of filling in an area surrounded by lines in image data, a second process of performing an expansion process on the area filled in by the first process, a third process of extracting the contour of the area filled in by the first process, and a fourth process of identifying the area filled in by the first process as the specified symbol when a result of comparing the contour extracted by the third process with the contour of a specified symbol shows that the contour extracted by the third process is similar to the contour of the specified symbol, two straight lines that intersect with each other are detected by linearly converting a plurality of contour points that constitute the contour extracted by the third process, and the area difference between the area of ​​the area surrounded by the contour extracted by the third process and the area of ​​the specified symbol is equal to or less than a predetermined value or twice the area of ​​the area surrounded by the contour extracted by the third process is approximately the same as the area of ​​a rectangular symbol.

[0007] A computer that executes the present invention may obtain the result of symbol identification using a learning model that receives image data as input and outputs the result of symbol identification. The learning model is generated using a known machine learning algorithm such as a neural network. Effect of the Invention

[0008] According to the present invention, it is possible to improve the accuracy of identifying symbols. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the configuration of an information processing device. [Diagram 2] FIG. 2 is a flowchart showing the symbol identification process executed by the information processing device. [Diagram 3]Fig. 3(A) is a diagram showing an example of a filling process corresponding to step S2 in Fig. 2. Fig. 3(B) is a diagram showing an example of an expansion process corresponding to step S3 in Fig. 2. Fig. 3(C) is a diagram showing an example of a contour extraction process corresponding to step S4 in Fig. 2. Fig. 3(D) is a diagram showing an example of a line process corresponding to step S6 in Fig. 2. Figs. 3(E) and 3(F) are diagrams showing examples of an area comparison process corresponding to step S7 in Fig. 2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] Fig. 1 is a block diagram showing the configuration of an information processing device. The information processing device 100 in Fig. 1 includes a CPU (Central Processing Unit) 101 as a processor, a RAM (Random Access Memory) 102 and a ROM (Read Only Memory) 103 as memories, a HDD (Hard Disk Drive) 104, an interface (IF) 105, a network interface (IF) 106, and a display device 107. The information processing device 100 can be realized by, for example, a computer, a server, or a smartphone. The CPU 101 is connected to the RAM 102, the ROM 103, the HDD 104, the IF 105, the network IF 106, and the display device 107 via a bus 108.

[0012] The IF 105 is, for example, a USB (Universal Serial Bus) and is connected to a camera 110 as an imaging device. In this embodiment, the camera 110 is connected to the outside of the information processing device 100, but may be built into the information processing device 100. The network IF 106 is an interface for connecting the information processing device 100 to an external network. The camera 110 may be connected to the information processing device 100 via the network IF 106. The display device 107 displays an application that executes a symbol identification process. The symbol identification process may be included in a character recognition process that identifies letters, numbers, and symbols. In addition, the display device 107 displays a symbol that can be edited by an application for executing the symbol identification process at a position corresponding to the symbol in the image data, or displays a message to that effect when there is no symbol that can be edited by an application for executing the symbol identification process that corresponds to the symbol in the image data.

[0013] The HDD 104 stores applications or programs for executing various types of image processing and symbol identification processing. Furthermore, the HDD 104 stores drawings created by a drawing creation application in the information processing device 100, drawings photographed by the camera 110, and drawings input from an external device or external memory via the network IF 106. The above programs correspond to a flowchart described later. Furthermore, the HDD 104 stores the results of various types of image processing and the results of symbol identification processing. Instead of the HDD 104, an SSD (Solid State Drive) may be adopted.

[0014] Fig. 2 is a flowchart showing the symbol identification process executed by the information processing device 100. The symbol identification process in Fig. 2 is realized by the CPU 101 based on a program stored in the HDD 104. Fig. 3(A) shows an example of the filling process corresponding to step S2 in Fig. 2, Fig. 3(B) shows an example of the expansion process corresponding to step S3 in Fig. 2, Fig. 3(C) shows an example of the contour extraction process corresponding to step S4 in Fig. 2, Fig. 3(D) shows an example of the straight line process corresponding to step S6 in Fig. 2, and Figs. 3(E) and 3(F) show examples of the area comparison process corresponding to step S7 in Fig. 2.

[0015] First, the CPU 101 converts the drawing stored in the HDD 104 into image data (step S1). Specifically, the drawing format (e.g., pdf or a format specific to an application) is converted into an image data format (e.g., bmp, gif, jpeg, tiff). At this time, color data is converted into binarized black and white image data. The converted image data is saved in the RAM 102 or the HDD 104.

[0016] Next, CPU 101 fills in the area surrounded by lines in the image data (step S2). For example, as shown in Fig. 3(A), when a triangular symbol is connected to a line, CPU 101 fills in the triangular area. Note that in Fig. 3(A) to Fig. 3(F), the filled area is shown by hatching.

[0017] Next, CPU 101 performs an expansion process on the filled area (step S3). The expansion process is a process in which if the pixel to be converted and the surrounding pixels surrounding the pixel to be converted are white, the pixel to be converted is converted to white, and if the pixel to be converted and the surrounding pixels are not white, the pixel to be converted is maintained as black. When the expansion process is performed on this filled area, the triangular area becomes smaller as shown in FIG. 3(B), but most of the filled area remains. The filled area becomes a black mass separated from the lines in FIG. 3(A). In this way, the filled area can be made into a black mass separated from the lines, thereby improving the accuracy of symbol identification.

[0018] Next, CPU 101 extracts the contour of the area where the expansion process has been performed (step S4). Here, as shown in FIG. 3(C), CPU 101 selects a group of contour points at the boundary between the area where the expansion process has been performed (black) and the area that has not been filled in (white), and sets the contour of the area where the expansion process has been performed as the contour of the area where the expansion process has been performed. As shown in FIG. 3(C), a straight line in the horizontal part of the area where the expansion process has been performed is expressed by two points, so that the contour of the area where the expansion process has been performed becomes V-shaped. In addition, the number of contour points indicating the contour of the area where the expansion process has been performed is three or more, and is not limited to the examples of FIG. 3(C) to FIG. 3(E).

[0019] Next, CPU 101 determines whether the contour extracted in step S4 and subjected to the expansion process resembles the contour of a predetermined symbol (here, an equilateral triangle) (step S5). Here, if a triangle formed by three contour points at the top left, top right, and bottom edge shown in Fig. 3C resembles an equilateral triangle, that is, if the three interior angles are within 60 degrees ±5 degrees, CPU 101 determines that the contour extracted in step S4 resembles the contour of the predetermined symbol (YES in step S5).

[0020] If the contour extracted in step S4 is not similar to the contour of the specified symbol (NO in step S5), CPU 101 determines that there is no symbol corresponding to the area filled in in step S2, displays on display device 107 a message indicating that there is no symbol corresponding to the filled in area (step S9), and terminates this process.

[0021] If the contour extracted in step S4 is similar to the contour of the predetermined symbol (YES in step S5), CPU 101 performs straight-line transformation on the multiple contour points constituting the contour extracted in step S4 to determine whether or not two intersecting straight lines are detected (step S6). Here, as shown in Fig. 3(D), it is determined whether or not two intersecting straight lines can be detected by straight-line transformation (Hough transform). In this process, it is determined from multiple angles whether or not the extracted contour is a triangle.

[0022] If two intersecting straight lines are not detected (NO in step S6), the process proceeds to step S9.

[0023] If two intersecting straight lines are detected (YES in step S6), CPU 101 determines whether the difference between the area of ​​the region surrounded by the contour extracted in step S4 and the area of ​​the predetermined symbol is equal to or smaller than a predetermined value, or whether twice the area of ​​the region surrounded by the contour extracted in step S4 is substantially the same as the area of ​​the rectangular symbol (step S7). Note that it is determined that they are substantially the same when twice the area of ​​the region surrounded by the contour extracted in step S4 is within the range of 90 to 110% of the area of ​​the rectangular symbol.

[0024] In step S7, as shown in Fig. 3(E), an area comparison process is performed between the area of ​​the region surrounded by the contour extracted in step S4 and the area of ​​a predetermined symbol. Alternatively, as shown in Fig. 3(F), an area comparison process is performed between a shape obtained by multiplying the area of ​​the region surrounded by the contour extracted in step S4 by two and a rectangular symbol. In the area comparison process in Fig. 3(F), the letter T can be excluded as a symbol identification result.

[0025] If the area difference between the area of ​​the area surrounded by the contour extracted in step S4 and the area of ​​the specified symbol is not equal to or less than a specified value, or if twice the area of ​​the area surrounded by the contour extracted in step S4 is not approximately equal to the area of ​​the rectangular symbol (NO in step S7), proceed to step S9 above.

[0026] If the area difference between the area of ​​the area surrounded by the contour extracted in step S4 and the area of ​​the specified symbol is equal to or less than a specified value, or if twice the area of ​​the area surrounded by the contour extracted in step S4 is approximately the same as the area of ​​the rectangular symbol (YES in step S7), CPU 101 identifies the area filled in in step S2 as the specified symbol and displays the specified symbol at a position in the image data corresponding to the filled area (step S8), and ends this process.

[0027] In step S8, the symbol is displayed in the application for executing the symbol identification process, so that the predetermined symbol is displayed in a color other than black and white (for example, red) so as to overlap the filled-in area in the image data.

[0028] The processes in steps S1 to S9 in Fig. 2 may be performed by machine learning. In this case, the information processing device 100 acquires the result of symbol recognition using a learning model that receives, for example, image data converted from a drawing as input and outputs the result of symbol recognition processing. The learning model is generated using a known machine learning algorithm such as a neural network. The teacher data to be input to the machine learning algorithm can be created, for example, by preparing various image data in advance in the HDD 104 and linking the result of the symbol recognition processing to the image data.

[0029] As described above, according to this embodiment, the symbol identification method executed by the information processing device 100 includes a first process (step S2) of filling in an area surrounded by lines in image data, a second process (step S3) of performing an expansion process on the area filled in by the first process, a third process (step S4) of extracting a contour of the area filled in by the first process, and a fourth process (step S8) of identifying the area filled in by the first process as the predetermined symbol when a result of comparing the contour extracted by the third process with the contour of a predetermined symbol shows that the contour extracted by the third process is similar to the contour of the predetermined symbol, two straight lines intersecting each other are detected by linearly converting a plurality of contour points constituting the contour extracted by the third process, and the area difference between the area of ​​the area surrounded by the contour extracted by the third process and the area of ​​the predetermined symbol is equal to or smaller than a predetermined value or twice the area of ​​the area surrounded by the contour extracted by the third process is approximately the same as the area of ​​the rectangular symbol (YES in steps S5 to S7).

[0030] By executing the above-mentioned first to fourth processes, the information processing device 100 can separate the filled-in area from the straight lines, characters, etc. connected to the filled-in area, making it easier to distinguish between the straight lines and characters, thereby improving the accuracy of symbol identification.

[0031] Although a preferred embodiment of the present invention has been described in detail above, the present invention is not limited to such a specific embodiment, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims. [Explanation of symbols]

[0032] 100 Information processing device 101 CPU 102 RAM 103 ROM 104 HDD 105 Interface 106 Network Interface 107 Display device 108 Bus 110 Camera

Claims

[Claim 1] The computer A first operation of filling in an area enclosed by lines in the image data; a second process for performing an expansion process on the area filled in by the first process; a third process for extracting the contour of the area filled in by the first process; a fourth process for identifying the area filled in by the first process as the predetermined symbol when a result of comparing the contour extracted by the third process with the contour of a predetermined symbol shows that the contour extracted by the third process is similar to the contour of the predetermined symbol, two straight lines intersecting each other are detected by linearly converting a plurality of contour points constituting the contour extracted by the third process, and an area difference between an area surrounded by the contour extracted by the third process and an area of ​​the predetermined symbol is equal to or less than a predetermined value or twice the area of ​​the area surrounded by the contour extracted by the third process is approximately the same as an area of ​​a rectangular symbol; A symbol identification method comprising:

Images