Image analysis device, image analysis method, and image analysis program

Abstract

This image analysis device (1) comprises: an image recognition unit (10) that performs recognition processing for recognizing elements that constitute a drawing from image data (D0) on the drawing, and outputs a recognition result (D10) indicating the result of the recognition processing; and a non-detection sensing unit (20) that uses reference data stored in a knowledge database (23) and performs analysis processing for sensing, from the recognition result (D10), a non-detected element that is an undetected element that was not recognized by the image recognition unit (10), and outputs an analysis result (D20) indicating the result of the analysis processing.

Description

Image analysis device, image analysis method, and image analysis program

[0001] The present disclosure relates to an image analysis device, an image analysis method, and an image analysis program.

[0002] Various techniques have been proposed for detecting (also called "recognizing" or "extracting") elements from images (e.g., images of sequence drawings) in which symbols and line segments are drawing elements (see, for example, Patent Documents 1 and 2).

[0003] JP-A-2004-234424 JP-A-4-241067

[0004] However, in the above-described conventional technology, even if there is an element that has not been detected (i.e., an element that has not been detected), the user is not notified of the element that has not been detected (i.e., an element that has not been detected).

[0005] The present disclosure aims to provide an image analysis device, an image analysis method, and an image analysis program that can notify that an element of a drawing has not been detected.

[0006] The image analysis device disclosed herein is characterized by having an image recognition unit that performs recognition processing to recognize elements that make up a drawing from image data of the drawing and outputs a recognition result indicating the result of the recognition processing, and an undetected detection unit that performs analysis processing to detect undetected elements, which are elements that were not recognized by the image recognition unit, from the recognition result using reference data stored in a knowledge database and outputs an analysis result indicating the result of the analysis processing.

[0007] The image analysis method disclosed herein is a method executed by an image analysis device, and is characterized by comprising the steps of: performing a recognition process to recognize elements that constitute a drawing from image data of the drawing, and outputting a recognition result indicating the result of the recognition process; and performing an analysis process using reference data stored in a knowledge database to detect undetected elements, which are elements that were not recognized and missed detection, from the recognition result, and outputting an analysis result indicating the result of the analysis process.

[0008] According to the device, method, and program of the present disclosure, it is possible to notify that an element in a drawing has not been detected.

[0009] 6 is a functional block diagram showing the configuration of an image analysis device according to embodiment 1. FIG. 7 is a diagram showing an example of the hardware configuration of an image analysis system including the image analysis device according to embodiment 1. FIG. 8 is a diagram showing another example of the hardware configuration of an image analysis system including the image analysis device according to embodiment 1. FIG. 9 is a diagram showing an example of image data input to an image recognition unit of the image analysis device according to embodiment 1. FIG. 10 is a diagram showing, in a table format, an example of a recognition result output from the image recognition unit of the image analysis device according to embodiment 1. FIG. 11 is a functional block diagram showing the configuration of an undetection detection unit of the image analysis device according to embodiment 1. FIG. 12 is a diagram showing, in a table format, an example of conditions between symbols and the number of connections referenced by the undetection detection unit of FIG. 6. FIG. 13 is a diagram showing an example of an analysis result notified by the notification unit of the undetection detection unit of FIG. 6. FIG. 14 is a diagram showing another example of an analysis result notified by the notification unit of the undetection detection unit of FIG. 6. FIG. 15 is a flowchart showing the operation of the undetection detection unit of the image analysis device according to embodiment 1. FIG. 16 is a functional block diagram showing the configuration of an image analysis device according to embodiment 2. FIG. 17 is a diagram showing an example of image data input to the image recognition unit of the image analysis device according to embodiment 2. FIG. 18 is a diagram showing, in a table format, an example of a recognition result output from the image recognition unit of the image analysis device according to embodiment 2. 19 is a diagram showing an example of conditions referenced by the undetection detection unit of the image analyzing device according to embodiment 2. FIG. 20 is a flowchart showing the operation of the undetection detection unit of the image analyzing device according to embodiment 2. FIG. 21 is a functional block diagram showing the configuration of an image analyzing device according to embodiment 3. FIG. 22 is a functional block diagram showing the configuration of the undetection detection unit of the image analyzing device according to embodiment 3. FIG. 23 is a diagram showing an example of image data input to the image recognition unit of the image analyzing device according to embodiment 3. FIG. 24 is a diagram showing, in table form, an example of recognition results output from the image recognition unit of the image analyzing device according to embodiment 3. FIG. 25 is a diagram showing an example of a graph list used by the undetection detection unit of the image analyzing device according to embodiment 3. FIG. 26 is a diagram showing an example of a graph constructed from image data by the undetection detection unit of the image analyzing device according to embodiment 3. FIG. 27 is a diagram showing an example of analysis results notified by the notification unit of the undetection detection unit of FIG. 18. FIG. 28 is a diagram showing another example of analysis results notified by the notification unit of the undetection detection unit of FIG. 18.11 is a flowchart showing the operation of the non-detection detection unit of the image analyzing device according to the third embodiment.

[0010] An image analysis device, an image analysis method, and an image analysis program according to embodiments will be described below with reference to the drawings. The following embodiments are merely examples, and the embodiments can be appropriately combined and modified. In the drawings, the same reference numerals are used to designate the same components or components having the same functions.

[0011] 1 is a functional block diagram showing the configuration of an image analysis device 1 according to the first embodiment. The image analysis device 1 is a device capable of executing an image analysis method according to the first embodiment. The image analysis device 1 is also a device (e.g., a computer) capable of executing an image analysis program according to the first embodiment. As shown in FIG. 1, the image analysis device 1 includes an image recognition unit 10 and an undetection detection unit 20.

[0012] The image recognition unit 10 performs a recognition process to recognize elements constituting a drawing from image data D0 of the drawing to be analyzed, and outputs a recognition result D10 indicating the result of the recognition process. The image data D0 of the drawing is, for example, digital image data generated by optically reading a drawing drawn on paper.

[0013] The undetected element detection unit 20 performs an analysis process using reference data stored in a knowledge database (knowledge DB 23 shown in FIG. 6 , which will be described later) to detect undetected elements, which are elements that were not recognized by the image recognition unit 10, from the recognition result D10 obtained by the image recognition unit 10, and outputs an analysis result D20 indicating the result of the analysis process. In the first embodiment, the drawing to be analyzed includes symbols and line segments as elements. The drawing to be analyzed is, for example, a sequence diagram including symbols and line segments as elements.

[0014] Fig. 2 is a diagram showing an example of the hardware (HW) configuration of an image analysis system including the image analysis device 1. The image analysis system in Fig. 2 has a processing circuit 51 as the image analysis device 1, an image capture device 54 that captures an image and converts it into image data (for example, an image reading device that optically reads a drawing and converts it into digital image data), a storage device 55 (i.e., storage) such as an HDD (Hard Disk Drive) or SSD (Solid State Drive), and a display 56 as a visual information presentation device.

[0015] FIG. 3 is a diagram showing another example of the HW configuration of an image analysis system including the image analysis device 1. The image analysis system of FIG. 3 includes a processor 52 and a memory 53 that constitute the image analysis device 1, an image capture device 54, a storage device 55, and a display 56. The processor 52, the memory 53, and the image capture device 54 of FIG. 3 may be part of the same computer. The processor 52 can execute a program (e.g., an image analysis program) as software stored in the memory 53. The HW configurations of FIGS. 2 and 3 are merely examples, and the HW configuration of the image analysis system is not limited to the examples of FIGS. 2 and 3.

[0016] The processor 52 is, for example, a CPU (Central Processing Unit) that executes a program. The program may be provided to the image analysis device 1 via a network, or may be recorded on a recording medium such as an optical disk or a magnetic disk and provided to the image analysis device 1. The image analysis system may also include a GPU (Graphics Processing Unit). Alternatively, the image analysis system may be configured with a processing circuit such as an FPGA (Field Programmable Gate Array).

[0017] FIG. 4 is a diagram showing an example of image data D0 input to the image recognition unit 10 of the image analyzing device 1. The image recognition unit 10 of the image analyzing device 1 recognizes (also referred to as "extracting" or "detecting") symbol 101 (square), symbol 102 (circle), symbol 103 (triangle), symbol 104 (circle), symbol 105 (black dot), line segment 111, line segment 112, line segment 113, and line segment 114 from the input image data D0. Line segment 111 connects symbol 102 (circle) and symbol 103 (triangle). Line segment 112 connects symbol 101 (square) and symbol 103 (triangle). Line segment 113 connects symbol 101 (square) and symbol 105 (black dot). Line segment 114 connects symbol 104 (circle) and symbol 105 (black dot). The recognition process executed by the image recognition unit 10 may be performed using the techniques described in Patent Document 3 or Non-Patent Document 1, for example.

[0018] Patent No. 6076128

[0019] Shogo Yoneyama, Kei Hirano, and Yasuhiro Okada, "Study on Symbol Extraction Methods in Drawing Images," Institute of Electronics, Information and Communication Engineers General Conference, March 2006

[0020] 5 is a diagram showing, in table form, an example of the recognition result D10 output from the image recognition unit 10 of the image analyzing device 1. The recognition result D10 is data including, for example, element labels (e.g., symbols and line segments), start coordinates indicating the start position of each element in the image, the width W and height H of the element, the characteristics (shape) of the symbol, and the number of line segment connections for each symbol. The coordinates are shown as coordinates (x, y) in a coordinate system in which the upper left corner of the image in FIG. 4 is the origin (0, 0), the x-axis is positive in the right direction from the origin, and the y-axis is positive in the downward direction from the origin.

[0021] 6 is a functional block diagram showing the configuration of the undetected object detection unit 20 of the image analyzing device 1. The undetected object detection unit 20 has a determination unit 21 that makes a determination using reference data stored in a knowledge DB 23, and a notification unit 22 that outputs notification information. The knowledge DB 23 may be stored in a storage device of an external device, or may be stored in a storage device (e.g., storage device 55 in FIGS. 2 and 3 ) that is part of the undetected object detection unit 20.

[0022] The determination unit 21 performs a determination, which is an analysis process, to detect undetected symbols as undetected elements, which are elements that were not recognized by the image recognition unit 10, from the recognition result D10 using reference data stored in the knowledge DB 23. For example, the determination unit 21 determines whether the number of connections is appropriate by referring to the recognition result in Fig. 5 and the reference list in Fig. 7. In Fig. 5, symbols 101 to 105 have an appropriate number of connections, but the black dot of symbol 105 has the number of connections of "2" in Fig. 8, even though the number of connections is "3 or more" according to the connection number condition, and it can be determined that there is an undetected element.

[0023] The notification unit 22 outputs an analysis result D20 indicating the result of the analysis process (i.e., the determination result) performed by the determination unit 21. Note that, although Fig. 5 shows a path 22a for outputting the analysis result without passing through the notification unit 22, the determination result by the determination unit 21 may be displayed on a display (for example, the display 56 in Figs. 2 and 3) via the notification unit 22.

[0024] FIG. 7 is a diagram showing, in table form, an example of a condition (reference list) between symbols and the number of connections referenced by the undetected element detection unit 20. As shown in FIG. 7 , the reference data stored in the knowledge DB 23 includes, for example, a connection number condition, which is a condition that the number of line segment connections for each symbol type (e.g., square, triangle, circle, black dot) must satisfy. Furthermore, if there is a symbol that does not satisfy the connection number condition, the undetected element detection unit 20 outputs notification information indicating the presence of an undetected element and the location of the undetected element (i.e., the undetected location) as the analysis result D20. In FIG. 7 , for example, the number of connections for each symbol may be a fixed number such as an upper or lower limit (e.g., "1 or more" or "3 or less") or a numerical value (e.g., "1" or "2"). Alternatively, the number of connections may be listed for all symbols, or may be listed for only some symbols, with the remaining symbols being marked as "no limit" or "1 or more if no limit is specified."

[0025] Fig. 8 is a diagram showing an example of an analysis result D20 notified by the notification unit 22 of the undetected symbol detection unit 20. In Fig. 8, the undetected symbol detection unit 20 outputs display data that displays a shaded oval region on the display image, and provides the user with text information such as "A line segment is connected to the undetected symbol" as an explanation in the display data.

[0026] 9 is a diagram showing another example of the analysis result D20 notified by the notification unit 22 of the undetection detection unit 20. In Fig. 9, the undetection detection unit 20 displays a dashed line segment connecting symbols 103 and 105 in the display image, and provides the user with text information as an explanation in the display data, such as "In the drawing to be analyzed, symbols may be connected by a line segment."

[0027] FIG. 10 is a flowchart illustrating the operation of the undetected element detection unit 20 of the image analysis device 1. In the first embodiment, a drawing includes symbols and line segments as elements, and the reference data stored in the knowledge DB 23 includes a connection number condition that must be satisfied by the number of line segments for each symbol type. Upon receiving the recognition result D10 from the image recognition unit 10, the undetected element detection unit 20 checks whether the number of line segments connected to each symbol satisfies a predetermined connection number condition based on the symbol and line segment recognition result (step ST11). Next, the undetected element detection unit 20 determines whether there are any symbols that do not satisfy the connection number condition, and if so, determines whether there are any undetected elements (step ST12). Next, the undetected element detection unit 20 determines the content of the notification information based on this determination result (step ST13). However, the operation of the undetected element detection unit 20 is not limited to the example of FIG. 10 .

[0028] As described above, according to the first embodiment, by performing undetection processing based on the number of line segment connections for each symbol, even if there is an end point of a line segment connected to a symbol, if the line segment is undetected (i.e., if a line segment has not been detected), the user is notified of the presence or absence of an undetected element and its position information (for example, by re-scanning the image), thereby making it possible to properly detect elements on the drawing. This makes it possible to improve the accuracy of image analysis and the sophistication of the analysis result D20.

[0029] <<Embodiment 2>> In embodiment 1, undetected elements (i.e., elements that have not been detected) are determined based on the number of line segments connected to a symbol. However, in embodiment 2, undetected elements are determined using drawing conditions, which are knowledge about the drawing contained in the image data (i.e., conditions that elements should have in the drawing).

[0030] 11 is a functional block diagram showing the configuration of an image analysis device 2 according to embodiment 2. The image analysis device 2 is a device capable of executing the image analysis method according to embodiment 2. The image analysis device 2 is also a device (e.g., a computer) capable of executing the image analysis program according to embodiment 2. As shown in FIG. 11 , the image analysis device 2 includes an image recognition unit 10 and an undetected state detection unit 30.

[0031] The undetected element detection unit 30 performs an analysis process using reference data stored in a knowledge DB (knowledge DB 33 shown in FIG. 12 described later) to detect undetected elements, which are elements not recognized by the image recognition unit 10, from the recognition result D10 obtained by the image recognition unit 10, and outputs an analysis result D30 indicating the result of the analysis process. In the second embodiment, the drawing to be analyzed includes symbols and line segments as elements. The drawing to be analyzed is, for example, a sequence diagram including symbols and line segments as elements.

[0032] The image analysis device 2 and the HW configuration of the image analysis system including the image analysis device 2 are the same as those described in the first embodiment.

[0033] 12 is a functional block diagram showing the configuration of the undetected state detection unit 30 of the image analyzing device 2. The undetected state detection unit 30 has a determination unit 31 that makes a determination using reference data stored in a knowledge DB 33, and a notification unit 32 that outputs notification information. The knowledge DB 33 may be stored in a storage device of an external device, or may be stored in a storage device as part of the undetected state detection unit 30.

[0034] The reference data stored in the knowledge DB 33 includes drawing conditions that symbols and line segments must satisfy, and when there is a symbol or line segment that does not satisfy the drawing conditions, the undetected detection unit 30 outputs notification information indicating that there is an undetected element as the analysis result D30.

[0035] The judgment unit 31 uses reference data stored in the knowledge DB 33 to perform a judgment, which is an analytical process, to detect undetected symbols as undetected elements, which are elements that were not recognized by the image recognition unit 10, from the recognition result D10.

[0036] 13 is a diagram showing an example of image data D0 input to the image recognition unit 10 of the image analyzing device 2. The image recognition unit 10 of the image analyzing device 2 recognizes (also referred to as "extracting" or "detecting") symbol 201 (hatched rectangle), symbol 202 (triangle), symbol 203 (circle), symbol 204 (circle), symbol 205 (square), symbol 206 (circle), symbol 207 (circle), line segment 211, line segment 212, line segment 213, line segment 214, and line segment 215 from the input image data D0. Line segment 211 connects symbol 201 (hatched rectangle) and symbol 202 (triangle). Line segment 212 connects symbol 202 (triangle) and symbol 203 (circle). Line segment 213 connects symbol 201 (hatched rectangle) and symbol 204 (circle). A line segment 214 connects the symbol 205 (square) and the symbol 206 (circle). A line segment 215 connects the symbol 205 (square) and the symbol 207 (circle). The recognition process executed by the image recognition unit 10 is the same as that in the first embodiment.

[0037] 14 is a diagram showing, in table form, an example of the recognition result D10 output from the image recognition unit 10 of the image analyzing device 2. The recognition result D10 is data including, for example, elements (e.g., symbols and line segments), start coordinates indicating the start position of each element in the image, the width W and height H of the element, and the characteristics (shape) of the symbol. The coordinates are shown as coordinates (x, y) in a coordinate system in which the upper left corner of the image in FIG. 13 is the origin (0, 0), the x-axis is positive in the right direction from the origin, and the y-axis is positive in the downward direction from the origin.

[0038] FIG. 15 is a diagram showing examples of conditions referenced by the undetected state detection unit 30 of the image analysis device 2. As shown in FIG. 15, the drawing conditions are stored in the storage device as a list of conditions that a drawing should have as a knowledge DB 33. The drawing conditions may be conditions related to line segment connections, or may be conditions that a specific symbol is included. Examples of drawing conditions shown in the list in FIG. 15 include "a line segment is connected to the hatched rectangle," and "the number of hatched rectangles is one or less."

[0039] The determination unit 31 determines the presence or absence of undetected elements from the recognition result of Fig. 14 by referring to the drawing conditions included in the list of Fig. 15. In Fig. 14, there is one shaded rectangle, which matches the drawing conditions of Fig. 15, but there are symbols (symbols 203, 205, 206, and 207 in Fig. 13) that are not connected to the shaded rectangle, and it can be determined that there are undetected elements. The notification unit 32 is the same as the notification unit 22 in the first embodiment.

[0040] 16 is a flowchart showing the operation of the undetected element detection unit 30 of the image analysis device 2. The undetected element detection unit 30 refers to the table showing the drawing conditions in FIG. 15, and if an element (symbol, line segment, etc.) of the drawing does not match the drawing condition, it determines that "an undetected element exists" (step ST21). If "an undetected element exists," the notification unit 32 notifies the user of information that "an undetected element exists" for the element that matches the drawing condition. If there is no symbol that matches the drawing condition, the undetected element detection unit 30 notifies the user of the condition, and if "an undetected element does not exist," it outputs the recognition result of the image recognition unit alone (step ST22).

[0041] As described above, according to the second embodiment, by using the drawing conditions that the drawing to be analyzed must have as reference data to determine whether or not there are undetected elements and identify undetected locations, it becomes possible to perform appropriate undetection processing tailored to the drawing in question. This makes it possible to improve the accuracy of image analysis and enhance the analysis result D30.

[0042] It is also possible to combine the first and second embodiments. In this case, the reference data stored in the knowledge DB 33 includes connection number conditions that must be satisfied by the number of line segment connections for each type of symbol, and drawing conditions that must be satisfied by the symbols and line segments, and the undetection detection unit 30 outputs, as an analysis result, notification information indicating the presence of an undetected element when there is a symbol that does not satisfy the connection number condition and there is a symbol or line segment that does not satisfy the drawing condition.

[0043] In the first embodiment, the determination of undetection is made based on the number of line segments connected to a symbol, but in the third embodiment, the determination of undetection is made using a reference graph in a reference graph list that stores in advance a graph configured by elements (symbols and line segments) in a drawing included in image data. Note that the determination of undetection may include not only determining the presence or absence of an undetected element, which is an element that has not been detected, but also determining the position of the undetected element.

[0044] 17 is a functional block diagram showing the configuration of an image analysis device 3 according to embodiment 3. The image analysis device 3 is a device capable of executing the image analysis method according to embodiment 3. The image analysis device 3 is also a device (e.g., a computer) capable of executing the image analysis program according to embodiment 3. As shown in FIG. 17 , the image analysis device 3 includes an image recognition unit 10 and an undetection detection unit 40.

[0045] The undetected element detection unit 40 performs an analysis process to detect undetected elements, which are elements that have not been recognized by the image recognition unit 10, from the recognition result D10 obtained by the image recognition unit 10, using a reference graph as reference data stored in a knowledge DB (knowledge DB 43 shown in FIG. 18 described later), and outputs an analysis result D30 indicating the result of the analysis process. In the third embodiment, the drawing to be analyzed includes symbols and line segments as elements. The drawing to be analyzed is, for example, a sequence diagram including symbols and line segments as elements.

[0046] The image analysis device 3 and the HW configuration of the image analysis system including the image analysis device 3 are the same as those described in the first embodiment.

[0047] 18 is a functional block diagram showing the configuration of the undetected state detection unit 40 of the image analyzing device 3. The undetected state detection unit 40 has a determination unit 41 that makes a determination using reference data stored in a knowledge DB 43, and a notification unit 42 that outputs notification information. The knowledge DB 43 may be stored in a storage device of an external device, or may be stored in a storage device as part of the undetected state detection unit 40. The configuration of the undetected state detection unit 40 is not limited to that shown in FIG. 18 .

[0048] The reference data stored in the knowledge DB 43 includes graph conditions that must be satisfied by a graph constructed by regarding the symbols that make up the diagram as nodes and the line segments that make up the diagram as edges. The graph conditions are stored in the knowledge DB 43 as a reference graph list that includes a plurality of reference graphs.

[0049] When there is a symbol or line segment that does not satisfy the graph conditions, the undetected element detection unit 40 outputs notification information indicating that there is an undetected element (which may include information regarding the position and shape of the undetected element) as the analysis result D40. In other words, when the undetected element detection unit 40 determines that there is an "undetected element," it can present information regarding the location of the undetected element.

[0050] FIG. 19 is a diagram showing an example of image data D0 input to the image recognition unit 10 of the image analysis device 3. The image recognition unit 10 of the image analysis device 3 recognizes (also referred to as "extracting" or "detecting") the following from the input image data D0: symbol 201 (hatched rectangle), symbol 202 (triangle), symbol 203 (circle), symbol 204 (circle), symbol 205 (square), symbol 206 (circle), symbol 207 (circle), symbol 208 (triangle), line segment 211, line segment 212, line segment 213, line segment 214, line segment 215, line segment 216, line segment 217, and line segment 218. Symbols 201 to 207 and line segments 211 to 215 are the same as those in FIG. 13. In FIG. 19, line segment 216 connects symbol 203 (circle) and symbol 208 (triangle). A line segment 217 connects the symbol 208 (triangle) and the symbol 206 (circle). A line segment 218 connects the symbol 204 (circle) and the symbol 207 (circle). The recognition process executed by the image recognition unit 10 is the same as that in the first embodiment.

[0051] 20 is a diagram showing, in table form, an example of the recognition result D10 output from the image recognition unit 10 of the image analyzing device 3. The recognition result D10 is data including, for example, elements (e.g., symbols and line segments), start coordinates indicating the start position of each element in the image, the width W and height H of the element, and the characteristics (shape) of the symbol. The coordinates are shown as coordinates (x, y) in a coordinate system in which the upper left corner of the image in FIG. 19 is set as the origin (0, 0), the x-axis is positive in the right direction from the origin, and the y-axis is positive in the downward direction from the origin.

[0052] 21 is a diagram showing an example of graph conditions in the knowledge DB 43 referenced by the undetected detection unit 40 of the image analyzing device 3. The graph conditions are stored as a reference graph list including a plurality of reference graphs such as components 1, 2, ....

[0053] FIG. 22 is a diagram showing an example of a graph constructed when the symbol 208 and the line segments 216, 217, and 218 are not detected from the image data D0 in FIG.

[0054] Fig. 23 is a diagram showing an example of an analysis result D40 notified by the notification unit 42 of the undetected element detection unit 40. In Fig. 23, the undetected element detection unit 40 outputs display data and provides the user with text information such as "There is an undetected element" as an explanation in the display data.

[0055] Fig. 24 is a diagram showing another example of the analysis result D40 notified by the notification unit 42 of the undetected element detection unit 40. In Fig. 24, the undetected element detection unit 40 outputs display data, and provides the user with, as an explanation in the display data, text information such as "There is an undetected element," and an area where the undetected element exists (the area surrounded by a dashed line in Fig. 24).

[0056] 25 is a flowchart showing the operation of the undetected element detection unit 40 of the image analysis device 3. The undetected element detection unit 40 constructs a graph (e.g., FIG. 22) from elements of a drawing (step ST31), and refers to the graph conditions in FIG. 21. If an element of the drawing (such as a symbol or a line segment) does not match the graph condition, the undetected element detection unit 40 determines that an "undetected element exists" (step ST32). If the undetected element detection unit 40 determines that an "undetected element exists," the notification unit 42 notifies the user of the "undetected element exists" information for elements that meet the drawing condition. If there is no symbol that meets the graph condition, the undetected element detection unit 40 notifies the user of the condition, and if there is no "undetected element," the undetected element detection unit 40 outputs the recognition result of the image recognition unit alone (step ST33).

[0057] As described above, according to the third embodiment, elements included in the diagram to be analyzed are regarded as a model structure, and a reference model is used as reference data to determine the presence or absence of undetected elements and identify undetected locations, thereby enabling appropriate undetection processing to be performed according to the diagram, thereby improving the accuracy of image analysis and enhancing the quality of the analysis result D40.

[0058] It is also possible to combine the first and third embodiments. In this case, the reference data stored in the knowledge DB 43 includes a connection number condition that must be satisfied by the number of connections of line segments for each type of symbol, and a graph condition that must be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges, and when there is a symbol that does not satisfy the connection number condition and there is a symbol or line segment that does not satisfy the graph condition, the undetection detection unit 40 outputs notification information indicating the presence of an undetected element as the analysis result D40.

[0059] Furthermore, the second and third embodiments may be combined. In this case, the reference data stored in the knowledge DB 43 includes drawing conditions that symbols and line segments must satisfy, and graph conditions that must be satisfied by a graph constructed by regarding symbols as nodes and line segments as edges, and the undetection detection unit 40 outputs, as the analysis result D40, notification information indicating the presence of an undetected element when there is a symbol or line segment that does not satisfy the drawing conditions and there is also a symbol or line segment that does not satisfy the drawing conditions.

[0060] Furthermore, embodiments 1, 2, and 3 may be combined. In this case, the reference data stored in the knowledge DB 43 includes a connection number condition that must be satisfied by the number of line segment connections for each type of symbol, a drawing condition that must be satisfied by the symbols and line segments, and a graph condition that must be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges, and the undetection detection unit 40 outputs, as the analysis result D40, notification information indicating the presence of an undetected element when there is a symbol that does not satisfy the connection number condition, a symbol or line segment that does not satisfy the drawing condition, and a symbol or line segment that does not satisfy the graph condition.

[0061] In the above description, elements of a drawing have been described as symbols and line segments, but elements of a drawing may also include other elements such as character strings, etc. Even in such cases, by referring to the reference data in the knowledge DB, it is possible to determine the presence or absence of "missed detection elements," i.e., "undetected elements" (which may include locating the positions of the undetected elements).

[0062] 1 to 3: Image analysis device; 10: Image recognition unit; 20, 30, 40: Undetected detection unit; 23, 33, 43: Knowledge database; D0: Image data; D10: Recognition result; D20, D30, D40: Analysis result.

Claims

1. an image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes a connection number condition that the number of connections of the line segments for each of the symbols for each of the symbol types must satisfy; The undetected element detection unit outputs, as the analysis result, notification information indicating the presence of the undetected element when the number of connections of each of the line segments of the symbol in the drawing does not satisfy the connection number condition.

1. An image analysis device comprising:

2. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing, and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes drawing conditions, which are conditions on the shape, width, and height that the symbol and the line segment should satisfy; When the symbol or the line segment does not satisfy the drawing condition, the undetected element detection unit outputs, as the analysis result, notification information indicating that the undetected element exists.

1. An image analysis device comprising:

3. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes graph conditions that should be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges; When the symbol or the line segment does not satisfy the graph condition, the undetected element detection unit outputs, as the analysis result, notification information indicating that the undetected element exists.

1. An image analysis device comprising:

4. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing, and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes a connection number condition that the number of connections of the line segments for each of the symbols for each type of the symbols must satisfy, and drawing conditions that are conditions regarding the shape, width, and height that the symbols and the line segments must satisfy; The undetected element detection unit outputs, as the analysis result, notification information indicating the presence of the undetected element when the symbol does not satisfy the connection number condition and the symbol or the line segment does not satisfy the drawing condition.

1. An image analysis device comprising:

5. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes a connection number condition that the number of connections of the line segments for each of the symbols for each type of symbol must satisfy, and a graph condition that must be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges, The undetected element detection unit outputs, as the analysis result, notification information indicating the presence of the undetected element when the symbol does not satisfy the connection number condition and the symbol or the line segment does not satisfy the graph condition.

1. An image analysis device comprising:

6. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing, and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes drawing conditions, which are conditions on the shape, width, and height that the symbols and the line segments should satisfy, and graph conditions, which should be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges; The undetected element detection unit outputs, as the analysis result, notification information indicating the presence of the undetected element when the symbol or the line segment does not satisfy the drawing condition and the symbol or the line segment does not satisfy the drawing condition.

1. An image analysis device comprising:

7. An image recognition unit that performs a recognition process for recognizing elements constituting a drawing from image data of the drawing, and outputs a recognition result indicating the result of the recognition process; an undetected element detection unit that performs an analysis process using reference data stored in a knowledge database to detect undetected elements that are elements that have not been recognized by the image recognition unit from the recognition result, and outputs an analysis result that indicates the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes connection number conditions that the number of connections of the line segments for each of the symbols for each type of symbol must satisfy, drawing conditions that are conditions on the shape, width, and height that the symbols and the line segments must satisfy, and graph conditions that must be satisfied by a graph constructed by regarding the symbols as nodes and the line segments as edges, The undetection detection unit outputs, as the analysis result, notification information indicating the presence of the undetected element when there is a symbol in the drawing in which the number of connections of each of the line segments does not satisfy the connection number condition, there is a symbol or a line segment that does not satisfy the drawing condition, and there is a symbol or a line segment that does not satisfy the graph condition.

1. An image analysis device comprising:

8. An image analysis method executed by an image analysis device, comprising: performing a recognition process for recognizing elements constituting the drawing from image data of the drawing, and outputting a recognition result indicating a result of the recognition process; performing an analysis process using reference data stored in a knowledge database to detect undetected elements from the recognition result, which are elements that have not been recognized and are detection omissions, and outputting an analysis result indicating the result of the analysis process; having The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes a connection number condition that the number of connections of the line segments for each of the symbols for each of the symbol types must satisfy; In the step of outputting an analysis result indicating a result of the analysis process, when there is a symbol in which the number of connections of each of the line segments of the symbol in the drawing does not satisfy the connection number condition, notification information indicating that there is an undetected element is output as the analysis result.

1. An image analysis method comprising:

9. On the computer, performing a recognition process for recognizing elements constituting the drawing from image data of the drawing, and outputting a recognition result indicating a result of the recognition process; performing an analysis process using reference data stored in a knowledge database to detect undetected elements from the recognition result, which are elements that have not been recognized and are detection omissions, and outputting an analysis result indicating the result of the analysis process; Run the command, The drawing includes symbols and lines as the elements; the reference data stored in the knowledge database includes a connection number condition that the number of connections of the line segments for each of the symbols for each of the symbol types must satisfy; In the step of outputting an analysis result indicating a result of the analysis process, when there is a symbol in which the number of connections of each of the line segments of the symbol in the drawing does not satisfy the connection number condition, notification information indicating that there is an undetected element is output as the analysis result.

2. An image analysis program comprising: