Image Analysis System

Abstract

To perform an image analysis more accurately.SOLUTION: An image analysis system S comprises: a table detection unit 212; a frame detection unit 213; and an information acquisition unit 215. The table detection unit 212 is configured to detect an area corresponding to a table from an image including the table as a subject. The frame detection unit 213 is configured to detect a plurality of frames from the area corresponding to the table, and give each of the plurality of detected frames position information in the image. The information acquisition unit 215 is configured to acquire acquisition object information on the basis of a character recognition result about each of the plurality of frames, and the position information given to each of the plurality of frames.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to an image analysis system. To Regarding. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there are techniques for performing image analysis such as character recognition on paper media on which characters and tables are written. An example of such a technology related to image analysis is disclosed in Patent Document 1. In the technology disclosed in Patent Document 1, what information is written at what position is set in advance according to the format (i.e., appearance) of the document to be read. Then, based on this setting, image analysis such as detecting character strings from a position in the image that matches the format can be performed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-57311 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the format of the document to be read is not necessarily the same, and various formats may be mixed for each document. In this case, it is practically difficult to set the format in advance or to modify the format each time it is read, as in the technology disclosed in Patent Document 1.

[0005] In addition, in the general technology disclosed in Patent Document 1 and the like, it is assumed that an original is placed on a mechanism such as a document tray or an ADF (Auto Document Feeder) of a multifunction printer, and the original is read in an appropriate state with a predetermined distance between the scanner and the original, and an image is generated. However, the original is not necessarily read in an appropriate state and an image is generated. For example, when a user takes a picture of an original with a portable camera and generates an image, the original may be curved when photographed, or the positional relationship between the camera and the original may not be appropriate, resulting in distortion in the image. Thus, even when various formats are mixed or when distortion or the like occurs in the image, it is desirable to perform image analysis with high accuracy.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to perform image analysis with higher accuracy. [Means for solving the problem]

[0007] In order to solve the above problem, an image analysis system according to an embodiment of the present invention comprises: a table detection means for detecting an area corresponding to a table from an image including a table as a subject; a frame detection means for detecting a plurality of frames constituting the table from an area corresponding to the table and for assigning position information in the image to each of the detected frames; an information acquisition means for acquiring acquisition target information based on a character recognition result for each of the plurality of frames and position information assigned to each of the plurality of frames; The present invention is characterized by comprising: Effect of the Invention

[0008] According to the present invention, it is possible to perform image analysis with higher accuracy. [Brief description of the drawings]

[0009] [Figure 1]1 is a block diagram showing an example of the overall configuration of an image analysis system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing an example of a configuration of a terminal according to an embodiment of the present invention. [Diagram 3] 1 is a block diagram showing an example of a configuration of an image analysis device according to an embodiment of the present invention. [Figure 4] 11 is a flowchart illustrating a flow of a photographing process. [Diagram 5] 11 is a flowchart illustrating a flow of an image analysis process. [Figure 6] 13 is a schematic diagram showing how a frame line in a first direction and a frame line in a second direction are separately detected. FIG. [Figure 7] 1A and 1B are schematic diagrams showing a table structure included in a target image and a method for acquiring acquisition target information. [Figure 8] 11 is a flowchart illustrating the flow of a character detection process. [Figure 9] 11 is a schematic diagram showing the process of removing an outer frame area from a binarized image and then performing projective transformation. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0011] [System Configuration] Fig. 1 is a block diagram showing the overall configuration of an image analysis system S according to this embodiment. As shown in Fig. 1, the image analysis system S includes a terminal 1 and an image analysis device 2. Fig. 1 also shows a document 3 that is a target for image analysis processing in this embodiment. Here, it is assumed that the document 3 is a document that includes some kind of table.

[0012] The terminal 1 and the image analyzing device 2 are connected to each other so as to be able to communicate with each other directly or via a network (not shown). In this case, the network is realized by, for example, the Internet, a LAN (Local Area Network), a mobile phone network, or a network that combines these. Furthermore, this communication may be performed in accordance with any communication method, and the communication method is not particularly limited. Furthermore, the communication connection format may be a wired connection or a wireless connection.

[0013] The terminal 1 is a terminal equipped with a photographing function. The terminal 1 is realized, for example, by a smartphone, a tablet-type terminal, or a digital camera. The terminal 1 performs a photographing process. Here, the photographing process is a series of processes for photographing an original 3 to generate an image including the original 3 as a subject (hereinafter referred to as a "target image"). In the photographing process, the terminal 1 photographs an original 3 based on a user's operation to generate a target image including the original 3 as a subject. Then, the terminal 1 transmits the generated target image to the image analyzing device 2.

[0014] The image analysis device 2 is a device that performs image analysis processing. The image analysis device 2 is realized by, for example, a personal computer, a server device, or the like. The image analysis device 2 acquires a target image by receiving it transmitted from the terminal 1. Then, the image analysis device 2 performs image analysis processing on this target image. Here, the image analysis processing is a series of processes that perform image analysis such as character recognition to detect characters written in the target image.

[0015] In this image analysis process, image analysis device 2 detects an area corresponding to a table (e.g., a table) from an image that includes the table as a subject. Image analysis device 2 also detects multiple frames (e.g., cells) that make up the table from the area corresponding to the table, and assigns position information in the image to each of the detected multiple frames. Furthermore, image analysis device 2 acquires acquisition target information (e.g., a pair of test items and test values ​​in a medical checkup chart) based on the character recognition results for each of the multiple frames and the position information assigned to each of the multiple frames.

[0016] In this way, the image analysis system S including the terminal 1 and the image analysis device 2 detects the tables and frames step by step, and then acquires the desired target information by using the position information. Therefore, it is possible to perform image analysis with higher accuracy than an image analysis method that performs image analysis processing simply without detecting the tables and frames and without using the position information. Therefore, even if, for example, various formats are mixed and the arrangement of tables and frames differs depending on the manuscript, or if there is distortion in the image that would normally make image analysis difficult, it is possible to perform appropriate image analysis.

[0017] Next, each device included in the image analysis system S for realizing such image analysis processing will be described in more detail.

[0018] In the following, as an example for the purpose of explanation, it is assumed that the document 3 is a "health checkup chart" for a user who uses the image analysis system S. The reason for this will be explained. First, health checkups are extremely beneficial for early detection and treatment of illness. By undergoing regular health checkups, users can grasp changes in their own health conditions.

[0019] However, the format (i.e., appearance) of health checkup charts varies depending on the business entity that creates them and the institution that performs the tests. For example, the layout of tables and frames varies from one health checkup chart to another. In addition, the positions of test items and corresponding test values, as well as the positions of reference values ​​and previous test values, vary from one health checkup chart to another. Therefore, it is difficult to perform image analysis of health checkup charts using general technology. In addition, the medical examination chart is not shared as electronic data between each testing institution. Moreover, the user who is the patient only receives the medical examination chart printed on paper from each testing institution, and does not receive the electronic data in the first place. In this situation, it is difficult for the user to grasp the changes in his / her own health condition. Moreover, it is difficult for each testing institution, hospital, etc. to grasp the changes in the user's health condition.

[0020] Therefore, in this embodiment, the above-mentioned image analysis process is performed on a target image captured with such a medical checkup chart as a subject. As a result, in this embodiment, image analysis can be performed more accurately from the medical checkup chart. This also allows a user to convert their own medical checkup chart into electronic data by a simple operation such as taking a picture of the chart with the terminal 1. Furthermore, by using such electronic data, for example, it becomes easier for a user to understand changes in their own health condition over time. In addition, such electronic data can be used by each testing institution, hospital, etc. In this way, a medical checkup chart is a suitable target for the image analysis process in this embodiment, and therefore, the following description will be given assuming that the document 3 is a medical checkup chart.

[0021] However, this is merely an example for the purpose of explanation and does not limit the application of this embodiment. In other words, this embodiment can be applied to various cases where the document 3, which is the subject of the target image, includes some kind of table, other than the case where the document 3 is a medical examination chart.

[0022] [Device configuration] The configuration of the terminal 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the configuration of the terminal 1. As described above, the terminal 1 is realized by, for example, a smartphone or tablet terminal, or a digital camera.

[0023] 2, the terminal 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a communication unit 14, a drive 15, a storage unit 16, an input unit 17, an output unit 18, and an imaging unit 19. These units are connected by signal lines and transmit and receive signals between each other.

[0024] The CPU 11 executes various processes (for example, a process of acquiring a target image by photographing the original 3 as a subject) according to a program recorded in the ROM 12 or a program loaded from the storage unit 16 to the RAM 13. The RAM 13 also stores data and the like necessary for the CPU 11 to execute various processes.

[0025] The communication unit 14 performs communication control for the CPU 11 to communicate with other devices (for example, the image analyzing device 2).

[0026] Removable media (not shown) such as magnetic disks, optical disks, magneto-optical disks, or semiconductor memories are appropriately mounted in the drive 15. Programs read from the removable media by the drive 15 are installed in the storage unit 16 as required. In addition, various data read from the removable media by the drive 15 are used for arithmetic processing in the CPU 11 as required.

[0027] The storage unit 16 is configured with a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), and stores various data. The input unit 17 is composed of various buttons and a touch panel, or an external input device such as a mouse and a keyboard, and inputs various information in response to user instructions. The output unit 18 is composed of a display, a speaker, etc., and outputs images and sounds.

[0028] The imaging unit 19 is configured with a lens, an imaging element, etc., and captures an image of a subject (for example, the document 3). By capturing an image, the imaging unit 19 generates an image (for example, a target image).

[0029] In the terminal 1, these units cooperate with each other to execute various processes in this embodiment. When the various processes in this embodiment are executed, an imaging control unit 111 and a terminal side notifying unit 112 function in the CPU 11, as shown in FIG. In addition, in one area of ​​the storage unit 16, an image storage unit 161 and an analysis result storage unit 162 are set. Including cases not specifically mentioned below, data required to realize processing is transmitted and received between these functional blocks at appropriate times as appropriate.

[0030] The photographing control unit 111 controls photographing in the terminal 1. To this end, the photographing control unit 111 displays a user interface for the user to photograph the document 3 as a subject, a live view image acquired by the imaging unit 19, and the like, on a display included in the output unit 18. The photographing control unit 111 also generates and acquires an image by controlling the imaging unit 19 based on a photographing instruction operation from the user. As described above, the image to be acquired is assumed to be a target image including the document 3 (here, a medical examination chart) as a subject.

[0031] The imaging control unit 111 stores the acquired target image in the image storage unit 161. That is, the image storage unit 161 functions as a storage unit that stores the target image. Moreover, the imaging control unit 111 transmits the acquired target image to the image analysis device 2.

[0032] The terminal-side notifying unit 112 acquires the image analysis results by receiving them in the image analysis processing on the target image transmitted from the image analyzing device 2. Furthermore, the terminal-side notifying unit 112 stores the acquired image analysis results in the analysis result storage unit 162. That is, the analysis result storage unit 162 functions as a storage unit that stores the image analysis results.

[0033] The terminal-side notifying unit 112 also notifies the user of the acquired image analysis results. The user can understand the contents of the image analysis results (e.g., the test values ​​in the user's own health checkup chart) by referring to the notified image analysis results. This notification to the user can be realized, for example, by displaying on a display included in the output unit 18, or by printing on a paper medium from a printer (not shown).

[0034] The specific contents to be notified as the image analysis results will be described later together with the explanation of the operation with reference to the flowchart of FIG.

[0035] [Image analysis device configuration] Next, the configuration of the image analyzing device 2 will be described with reference to Fig. 3. Fig. 3 is a block diagram showing an example of the configuration of the image analyzing device 2. As described above, the image analyzing device 2 is realized by, for example, a personal computer or a server device.

[0036] As shown in Fig. 3, the image analysis device 2 includes a CPU 21, a ROM 22, a RAM 23, a communication unit 24, a drive 25, a storage unit 26, an input unit 27, and an output unit 28. These units are connected by signal lines and transmit and receive signals to and from each other. The hardware functions of these units are the same as those of the terminal 1 of the same name described above with reference to Fig. 2. Therefore, redundant explanations of the hardware functions will be omitted.

[0037] In the image analyzing device 2, these units work together to execute various processes in this embodiment. When the various processes in this embodiment are executed, an image acquiring unit 211, a table detecting unit 212, a frame detecting unit 213, a character recognizing unit 214, an information acquiring unit 215, and a device side notifying unit 216 function in the CPU 21, as shown in FIG. In addition, an image storage section 261 and an analysis result storage section 263 are set in one area of ​​the storage section 26. Including cases not specifically mentioned below, data required to realize processing is transmitted and received between these functional blocks at appropriate times as appropriate.

[0038] The image acquisition unit 211 acquires the target image by receiving the target image transmitted from the shooting control unit 111 included in the terminal 1. The image acquisition unit 211 also stores the acquired target image in the image storage unit 261. That is, the image storage unit 261 functions as a storage unit that stores the target image.

[0039] The shooting control unit 111 included in the terminal 1 may store the target image in a removable medium via the drive 15. Then, the image acquisition unit 211 may acquire the target image by reading, via the drive 25, the removable medium in which the target image is stored.

[0040] The table detection unit 212 detects areas (hereinafter referred to as "table areas") corresponding to each of a plurality of tables (or a single table may be included) included in the target image from the target image stored in the image storage unit 261. For example, the frame detection unit 213 detects table areas corresponding to each of a plurality of tables constituting a medical examination chart that is the subject of the target image.

[0041] The frame detection unit 213 detects a plurality of frames constituting a table from each table region detected by the table detection unit 212. For example, the frame detection unit 213 detects a plurality of frames constituting a medical examination table that is a subject in the target image. In this case, for example, the frame detection unit 213 detects a frame line in a first direction and a frame line in a second direction, which are frame lines of a plurality of frames constituting a table, from each table region, and detects a plurality of frames based on the intersection points of the detected frame lines in the first direction and the frame lines in the second direction. In this way, the frame detection unit 213 detects the frame lines in the first direction and the frame lines in the second direction separately, thereby being able to detect the frame lines and frames more accurately than when detection is performed without separating them.

[0042] Furthermore, the frame detection unit 213 divides and cuts out each of the detected frames, and assigns position information in the target image to each of the cut-out frames. For example, the frame detection unit 213 sets as position information a combination of table identification information (e.g., an identifier assigned to the table) assigned to each detected table region and frame identification information (e.g., an identifier assigned to the frame), and assigns this position information to each of the frames. Alternatively, for example, the frame detection unit 213 sets as position information coordinate values ​​in an image coordinate system used in the target image (e.g., values ​​on each coordinate axis corresponding to the center of the frame, etc.), and assigns this position information to each of the frames. As a result, the frame detection unit 213 generates an image (hereinafter referred to as a "frame image") in which each of the frames included in the target image is treated as a unit and in which position information is assigned to each of the frames. For example, in the case of a medical examination chart, a frame image is a unit in which a specific character string (for example, test item, test value, reference value, previous test value, attribute information such as the patient's name, etc.) is recorded, and can be handled as structured data having position information.

[0043] Character recognition unit 214 performs character recognition processing on a unit basis of each of the multiple box images generated by box detection unit 213. This character recognition processing can be realized, for example, by using an existing optical character recognition (OCR) technology. In this way, character recognition unit 214 can perform image analysis processing on a unit basis of a box including a specific character string based on the box image generated by box detection unit 213, and therefore can perform character recognition with higher accuracy than when image analysis processing is performed on the entire image. The character recognition unit 214 may not only use existing optical character recognition technology, but may also perform machine learning using character strings that may be included in the target image in advance (here, character strings such as words used in a medical examination chart).Then, the character recognition process may be performed using a learning model generated by this machine learning.

[0044] The information acquisition unit 215 acquires acquisition target information based on the character recognition results for each of the multiple frame images by the character recognition unit 214 and the position information assigned to each of the multiple frame images. Here, the acquisition target information is information that the user desires to acquire, such as a set of "examination item" and "examination value" in a medical checkup chart.

[0045] In order for the information acquisition unit 215 to acquire the acquisition target information, in this embodiment, dictionary data is prepared for the first information, which is a part of the acquisition target information. For example, when the first information is a "search item", dictionary data is prepared for the text of words (e.g., height, weight, blood pressure, etc.) that are expected to be used as the names of the test items. Here, there are variations in the notation of the names of the test items in the health checkup, and although the meaning indicates the same test item, the names of the test items may differ depending on the format of the health checkup chart. For example, even if the same test item is high density lipoprotein cholesterol (HDL), the name of the test item in the health checkup chart may be "HDL cholesterol", "HDL-C", or "HDL". Therefore, in the dictionary data, taking these variations in notation into consideration, text of multiple test item names is associated with the same test item.

[0046] Such dictionary data is, for example, created in advance by an administrator of the image analysis system S, and stored in the dictionary data storage unit 262. That is, the dictionary data storage unit 262 functions as a storage unit that stores the dictionary data.

[0047] Then, the information acquiring unit 215 scores the similarity between the character recognition result for each of the multiple frame images by the character recognizing unit 214 (i.e., the text of the character string included in each frame image) and each text of the word used as the test item name included in the dictionary data stored in the dictionary data storage unit 262. Furthermore, the information acquiring unit 215 marks a frame image that includes a character string with the highest similarity for a certain test item name as a frame image in which the certain test item (i.e., the first information) is described. For example, if a certain frame image includes the character string "HDL-C", the certain frame image is marked as a frame image in which the test item "high density lipoprotein" is described.

[0048] Next, the information acquiring unit 215 acquires second information, which is a part of the acquisition target information, based on the position information assigned to the marked frame image (i.e., the frame image from which the first information was acquired). For example, assume that the second information is a "search value" for a certain test item. In this case, the information acquiring unit 215 acquires position information of the marked frame image corresponding to the certain test item. Furthermore, the information acquiring unit 215 scans and searches each frame image existing in any predetermined direction (e.g., to the right) from this position information to see if the test value is included.

[0049] Here, the realistic upper and lower limits of the test value for each test item can be predicted from a medical perspective. Therefore, for example, a realistic upper and lower limit are associated with each test item in the dictionary data. Then, in the process of performing a scanning search for a certain test item, when a frame image containing a value equal to or lower than the upper limit value associated with the certain test item and equal to or higher than the lower limit value is detected, the information acquisition unit 215 specifies that the frame image is a frame image in which the test value for the certain test item is described. Then, the information acquisition unit 215 acquires the test value as the second information. As a result, even in a table such as a medical checkup chart in which the relative positional relationship between the frame in which the first information is described and the frame in which the second information is described differs depending on the format, a scanning search is performed, so that image analysis can be performed with high accuracy. In addition, since the second information is acquired based on the realistic upper and lower limits, it is possible to prevent the acquisition of erroneous second information, and from this viewpoint, image analysis can be performed with high accuracy.

[0050] In this way, the information acquisition unit 215 can acquire a set of first information (e.g., a certain test item) and corresponding second information (test value of the certain test item), which are acquisition target information. Note that such marking and scanning search based on the position information of the marked frame image may be performed each time marking is performed for one piece of first information (e.g., a certain test item). Alternatively, when there are multiple pieces of first information, marking may be performed for all the first information, and then scanning search may be performed.

[0051] The information acquiring unit 215 stores the thus acquired acquisition target information (for example, a set of an examination item name corresponding to an examination item and its examination value) in the analysis result storage unit 263 as an image analysis result. That is, the analysis result storage unit 263 functions as a storage unit that stores the image analysis result. The information acquiring unit 215 also transmits the image analysis result to the terminal 1 so that the terminal 1 can also notify the user of the image analysis result.

[0052] The device-side notifying unit 216 notifies the user of the image analysis results stored in the analysis result storage unit 263 by the information acquiring unit 215. The user can understand the contents of the image analysis results by referring to the notified image analysis results. This notification to the user is performed in the same manner as the terminal-side notifying unit 112 of the terminal 1. For example, this can be realized by displaying on a display included in the output unit 28, or printing on a paper medium from a printer (not shown), or the like.

[0053] The specific contents to be notified as the image analysis results will be described later together with the explanation of the operation with reference to the flowchart of FIG.

[0054] Above was a detailed explanation of the configuration of each device included in the image analysis system S. Next, the processing content of each process performed by each device included in the image analysis system S will be explained in more detail.

[0055] [Image analysis processing] 4 is a flowchart illustrating the flow of the photographing process performed by the terminal 1. The photographing process is executed when the user turns on the power to the terminal 1, for example.

[0056] In step S11, the photographing control unit 111 starts controlling photographing in the terminal 1. For example, the photographing control unit 111 displays, on a display included in the output unit 18, a user interface for the user to photograph the document 3 as a subject, a live view image acquired by the imaging unit 19, and the like.

[0057] In step S12, the photographing control unit 111 judges whether or not a photographing instruction operation from the user has been accepted. If a photographing instruction operation has been accepted, the result of step S11 is judged as Yes, and the process proceeds to step S12. On the other hand, if a photographing instruction operation has not been accepted, the result of step S11 is judged as No, and the process repeats the judgment of step S12.

[0058] In step S13, the shooting control unit 111 controls the imaging unit 19 to generate and acquire a target image.

[0059] In step S14, the imaging control unit 111 transmits the target image acquired in step S13 to the image analysis device 2. This ends the present process. By the photographing process described above, the image analyzing device 2 can obtain the target image.

[0060] [Image analysis processing] 5 is a flowchart explaining the flow of image analysis processing performed by image analysis device 2. The image analysis processing is executed when a target image is transmitted from terminal 1 or when an instruction operation to start image analysis processing by image analysis device 2 is received from a user.

[0061] In step S21, the image acquisition unit 211 acquires a target image.

[0062] In step S22, table detection unit 212 detects table regions corresponding to each of a plurality of tables (there may be only one table) included in the target image acquired in step S21. Specifically, the table detection unit 212 first performs preprocessing. For example, the table detection unit 212 converts the target image to grayscale and adjusts (i.e., resizes) the size to 1280×960. Next, the table detection unit 212 separates the background region from the frame and character region in the target image and performs binarization to express them as 0 or 1. For example, the table detection unit 212 performs binarization using retinex filtering. In binarization, for example, white in the image is expressed as 0 and black is expressed as 1. In addition, the table detection unit 212 performs filtering to remove the character region. As a premise, the frame of the table spreads over a wide area in the target image, and characters are included within the frame. Therefore, it is considered that the pixels are sparsely connected. For this reason, it is considered that the variance in both the x direction and the y direction of the frame is large.

[0063] Next, it is considered that the line segments in which the frame lines of a table are divided by noise are extremely long in either the x-direction or the y-direction, and extremely short in the other direction. From this, it is considered that the variance of the line segments is large in one of the x-direction and the y-direction, and small in the other direction. Based on this property, for example, the table detection unit 212 identifies the line segments and frame lines by threshold processing using the eigenvalues ​​of the covariance matrix in the x-direction and the y-direction of the image, and removes the areas that do not fit either the line segments or the frame lines as character areas. In addition, the table detection unit 212 performs thinning to thin the line segments and frame lines so that only one pixel at the center remains.

[0064] Then, the table detection unit 212 extracts from the target image a table corresponding to each of a plurality of tables (or a single table in some cases) contained in the image. TableFor example, when there are multiple tables in the target image, the table detection unit 212 divides the image so that there is only one table in one image. To this end, the table detection unit 212 creates accumulated energy maps in the x and y directions used in seam carving. The table detection unit 212 also performs backtracking in valleys in the energy map consisting of pixels whose energy difference with adjacent pixels is equal to or greater than a threshold. Then, the table detection unit 212 detects each table region by using the pixels traced by this backtracking as the boundaries of table or document regions, and divides each table region into an image corresponding to one table region.

[0065] In step S23, frame detection section 213 detects a plurality of frames that constitute a table from each of the table regions detected in step S22. Specifically, the frame detection unit 213 first performs frame line separation. In this case, as described above, the frame detection unit 213 detects the frame lines of the first direction and the frame lines of the second direction, which are the frame lines of the multiple frames constituting the table, separately. This point will be described with reference to FIG. 6. FIG. 6 is a schematic diagram showing the frame lines of the first direction and the frame lines of the second direction being detected separately. FIG. 6(a) shows the frame lines before separation in the image. In this way, the frame lines of the multiple frames constituting the table include the frame lines of the first direction (here, the vertical direction) and the frame lines of the second direction (here, the horizontal direction). The frame detection unit 213 detects these frame lines by separating them into the frame lines of the first direction (here, the vertical direction) as shown in FIG. 6(b-1) and the frame lines of the second direction (here, the horizontal direction) as shown in FIG. 6(b-2).

[0066] In this method, the width of the frame line is adjusted to 1 by the thinning performed in step S22 as a premise. Therefore, by deleting pixels with a width of 1 in the first direction (here, the vertical direction) of the image, the frame line in the second direction (here, the horizontal direction) is deleted. As a result, an image with only the frame line in the first direction (here, the vertical direction) is obtained as shown in FIG. 6(b-1). Similarly, by deleting pixels with a width of 1 in the second direction (here, the horizontal direction), the frame line in the first direction (here, the vertical direction) is deleted. As a result, an image with only the frame line in the second direction (here, the horizontal direction) is obtained as shown in FIG. 6(b-2). As a result, the frame lines in the first direction (here, the vertical direction) and the second direction (here, the horizontal direction) can be separated, and the frame lines can be accurately detected.

[0067] A case where such separation is not performed will be described. As a premise, when a paper medium such as a medical examination chart is photographed, distortion occurs in both the vertical and horizontal frame lines due to curling of the paper, etc. For this reason, if separation is not performed, the detection accuracy of the frame lines will decrease. In contrast, in this embodiment, the frame lines are separated in multiple directions by separating the frame lines in the vertical and horizontal directions and then detecting them. Then, each of the separated frame lines in multiple directions is detected as a curve with a limit on the curvature of the line, which simplifies the detection of the frame lines in multiple directions and allows the frame lines to be accurately detected in response to each distortion of the frame lines in both the vertical and horizontal directions.

[0068] Next, the frame detection unit 213 performs frame line detection. Since the frame lines in a plurality of directions are separated as described above, the separated images each contain only a frame line in the first direction (here, the vertical direction) or a frame line in the second direction (here, the horizontal direction). Therefore, the set of pixels obtained by looking at the connection relationship of eight neighbors for these images becomes a frame line in the vertical direction or the horizontal direction, respectively. In this embodiment, curve approximation is performed on this set of pixels by using the least squares method.

[0069] Specifically, pixel clusters are detected by looking at the connection relationships of the 8 neighbors of the separated image. The pixel clusters obtained in this way include not only the table borders, but also characters that were not deleted during filtering because they were adjacent to the borders. Therefore, characters are deleted by threshold processing using the number of pixels in the pixel cluster used during filtering and the eigenvalues ​​of the covariance matrix. In addition, curve approximation is performed starting from a pixel set with a large number of pixels, and the degree is gradually increased from 1 to 4 using a polynomial. A threshold judgment is performed as shown in the following equation (1), and an approximation curve is created only when the degree is below the threshold.

[0070]

number

[0071] Here, the threshold value (for example, 1.1 in formula (1) above) can be set appropriately. Then, when the approximation curve has been created, the separated frame lines are synthesized. To synthesize the frame lines, a pixel set is searched for within three pixels surrounding the created approximation curve. If a pixel set is found, an iterative method is used to perform curve approximation again using the least squares method, using both that pixel set and the pixel set used for curve approximation.

[0072] Once the approximation of the curve is completed for all pixel sets, the frame detection unit 213 deletes the intersecting lines. The frame lines of a table are composed of parallel lines, and vertical lines and horizontal lines do not intersect with each other. Therefore, when lines intersect within a frame line image separated vertically and horizontally, it is assumed that there is a high possibility that a line outside the frame of the table has been mixed in. In this embodiment, when vertical and horizontal frame lines intersect, the frame outer lines are deleted by deleting the intersecting lines with the fewest number of pixels from the pixel sets used for approximation.

[0073] Then, the frame detection unit 213 performs frame detection. In frame detection, a frame structure is detected using lines. When the above-mentioned frame line detection is completed, a group of vertical and horizontal approximation curves is generated. In the frame detection in this step, first, the position of the intersection of the vertical and horizontal approximation curves is specified. The coordinates of the intersection obtained in this way become candidate positions for the corners of the frame, and one vertical frame line can obtain intersections for the number of horizontal frame lines, and one horizontal frame line can obtain intersections for the number of vertical frame lines.

[0074] Then, one vertical or horizontal frame line is selected, and one pair of adjacent intersections on the approximation curve is selected.Then, the frame line image is searched between the intersections, and if a certain number of pixels exist, it is determined to be a frame line.By applying this process to adjacent pairs of intersections on all approximation curves, frame lines in the image can be detected.The area surrounded by the detected frame line is then detected as a frame.

[0075] Furthermore, the frame detection unit 213 divides and cuts out each of the detected multiple frames. As described above, the frame detection unit 213 detects a frame based on a frame line surrounding the frame. Therefore, the frame detection unit 213 searches for the outermost coordinates in each of the x and y coordinates included in the frame line surrounding the frame, and divides and cuts out each of the multiple frames using the coordinates obtained by this search. Furthermore, the frame detection unit 213 assigns position information in the target image to each of the cut-out multiple frames. For example, the frame detection unit 213 sets as position information a combination of table identification information (e.g., an identifier assigned to the table) assigned to each detected table area and frame identification information (e.g., an identifier assigned to the frame), and assigns this position information to each of the multiple frames. Alternatively, for example, the frame detection unit 213 sets as position information a coordinate value in an image coordinate system used in the target image (e.g., a value on each coordinate axis corresponding to the center of the frame, etc.), and assigns this position information to each of the multiple frames. As a result, frame detection section 213 generates frame images, which are images made up of each of a plurality of frames included in the target image as a unit, and to which position information has been added.

[0076] Returning to FIG. 5, in step S24, character recognition unit 214 performs character recognition processing on each of the plurality of frame images generated in step S23, with the frame image as a unit.

[0077] In step S25, the information acquisition unit 215 acquires acquisition target information based on the character recognition results for each of the multiple frame images character-recognized in step S24 and the position information assigned to each of the multiple frame images. This will be described with reference to Fig. 7. Fig. 7 is a schematic diagram showing a table structure included in the target image and a method for the information acquisition unit 215 to acquire the acquisition target information.

[0078] Fig. 7(a) shows a table structure included in a target image. In Fig. 7(a), the target image 5 includes a table 51 and a plurality of frames 52 (for convenience of illustration, only one frame is marked in the figure). Note that, for simplicity of explanation, only one table 51 is shown in the figure, but the target image 5 may include a plurality of tables 51. By performing the above-mentioned image analysis process on such a target image 5, the table detection unit 212 detects a table region corresponding to the table 51. Furthermore, the frame detection unit 213 generates a frame image for each of the plurality of frames 52 included in the table 51.

[0079] FIG. 7(b) shows a method of acquiring information to be acquired by the information acquiring unit 215. As described above, the information acquiring unit 215 scores the similarity between the character recognition result for each of the multiple box images by the character recognition unit 214 (i.e., the text of the character string included in each box image) and each text of the word used as the name of the examination item included in the dictionary data stored in the dictionary data storage unit 262. Furthermore, the information acquiring unit 215 marks the box image containing the character string with the highest similarity for a certain examination item name as a box image containing the examination item name as a result of the scoring, as a box image containing the examination item name. For example, it is assumed that the box 52a is marked this time. Then, the information acquiring unit 215 acquires the position information of the box image 52a marked in correspondence with this certain examination item. Furthermore, the information acquiring unit 215, as shown by an arrow in the figure, scans and searches for each box image present in any predetermined direction (for example, the right direction) from the position information of the box image 52a to see if the examination value is included.

[0080] In this way, the information acquiring unit 215 can acquire a set of first information (e.g., a certain test item) and corresponding second information (test value of this certain test item), which are acquisition target information. The acquisition target information acquired in this way (e.g., a set of the test item name corresponding to the test item and its test value) is used as the image analysis result.

[0081] In step S26, the terminal side notifying unit 112 or the device side notifying unit 216 notifies the user of the image analysis result in step S25. The user can understand the contents of the image analysis result by referring to the notified image analysis result. The notification may be performed by only either the terminal side notifying unit 112 or the device side notifying unit 216, or may be performed by both. This ends the process.

[0082] Here, the notification in step S26 may be, but is not limited to, simply by displaying the image analysis results. For example, when image analysis processing is performed on multiple medical examination forms corresponding to one user (for example, each of the medical examination forms for each year in the case where a medical examination is performed once a year), a change in the test value corresponding to a predetermined test item along a time series may be detected based on the image analysis result, and the detected change may be notified.

[0083] In this case, for example, the order in which each medical examination was performed is identified by detecting the implementation date written on each medical examination chart by character recognition or based on user operation, etc. Then, the results of the medical examinations (e.g., test values) obtained by character recognition from each medical examination chart are notified in correspondence with the order in which the medical examinations were performed. For example, notification is made by displaying test values ​​for the same test item in table format in correspondence with the order in which the medical examinations were performed. Alternatively, notification is made by displaying in graph format, for example, with time (e.g., the date the medical examination was performed) on the horizontal axis and test values ​​for the same test item on the vertical axis.

[0084] In this case, further, for example, a threshold value (e.g., an upper limit value or a lower limit value) indicating the appropriate range of the test value may be set, and if the test value falls outside this appropriate range (e.g., if it is equal to or greater than the upper limit value or equal to or less than the lower limit value), the test value may be displayed in a different color or together with a warning message to notify the user. In addition, for example, a threshold value for the amount of change in the test value from the previous health check may be set, and if the amount of change in the test value exceeds the threshold (e.g., if the test value has changed significantly), the test value may be displayed in a different color or together with a warning message to notify the user. This allows the user to not only grasp the changes in the test values ​​of the same test item over time, but also to grasp changes such as that the test value is outside the appropriate range or has changed significantly. Therefore, the user can more appropriately understand the results of the health check.

[0085] According to the photographing process and image analysis process described above, the table and frame are detected step by step, and then the desired information to be obtained is obtained using the position information. Therefore, it is possible to perform image analysis with higher accuracy than an image analysis method that performs image analysis simply without detecting the table and frame and without using the position information. Therefore, even if, for example, various formats are mixed and the arrangement of tables and frames differs depending on the document, or the image is distorted and image analysis is normally difficult to perform, it is possible to perform image analysis appropriately.

[0086] [Variations] Although the embodiment of the present invention has been described above, this embodiment is merely an example and does not limit the technical scope of the present invention. The present invention can take various other embodiments and can be modified in various ways, such as omissions and substitutions, without departing from the gist of the present invention. In this case, these embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents. As an example, the embodiment of the present invention described above may be modified as follows.

[0087] In the above-described embodiment, the image analysis system S is realized as a client-server type system, such as a combination of the terminal 1 and the image analysis device 2. However, the image analysis system S may be configured in a different manner from the example shown in the above-described embodiment.

[0088] For example, image analyzing device 2 may be realized by multiple computers instead of one computer. In this case, for example, some of the functional blocks of image analyzing device 2 (for example, character recognition unit 214 and information acquisition unit 215) may be realized by one computer, and other functional blocks may be realized by another computer. Also, for example, some or all of the functional blocks of image analyzing device 2 may be realized by distributed processing by multiple computers (for example, a cloud system).

[0089] Alternatively, for example, the functions of the terminal 1 and the image analyzing device 2 may be realized by a single computer by providing an imaging unit in the image analyzing device 2. In this case, the communication function may be omitted from this computer to make it a stand-alone device.

[0090] Furthermore, the person using terminal 1 may be the user who has undergone the medical examination, or another entity such as a business that employs the user who has undergone the medical examination.

[0091] Furthermore, in order to further improve the accuracy of the character recognition process in step S24, for example, character detection process may be performed on each of the multiple frame images generated in step S23. This process is performed, for example, by frame detection unit 213 that generated the multiple frame images, following the generation of the frame images. In this case, frame detection unit 213 detects characters from the target image for each region of the multiple frame images it has detected. This allows frame detection unit 213 to accurately detect a string of characters (for example, one word) from a predetermined region (for example, within a frame in which characters are written) based on the accurate frame detection result by itself.

[0092] FIG. 8 is a flowchart illustrating the flow of the character detection process. In step S31, frame detection section 213 acquires a frame image to be processed this time from a plurality of frame images generated by itself.

[0093] In step S32, the frame detection unit 213 performs binarization. In this modification, character detection is performed using a projection method in step S35 described later. When using the projection method, binarization is required to count black pixels in the character region and separate the character region from the background region. Any method can be used for binarization, but for example, Otsu's binarization is used. Otsu's binarization is also called discriminant analysis, and creates a histogram of luminance values ​​for an image, and performs binarization using a threshold value that maximizes the inter-class variance when the histogram is divided into two. Many medical checkup charts use a variety of colors, but if you look inside the frame, you will see that it is a binary image of the characters and background. Therefore, by using Otsu's binarization, it is possible to automatically calculate an optimal threshold value and extract the characters.

[0094] In step S33, the frame detection unit 213 deletes the outside-frame area. This will be described with reference to FIG. 9. FIG. 9 is a schematic diagram showing the deletion of the outside-frame area from the binarized image and then the projection transformation. In step S32, the frame image is cut out using the outermost coordinate of the four frame lines used to detect the frame. Therefore, if the frame line is tilted or distorted, characters and frame lines outside the frame are detected as shown in the binarized image in FIG. 9(a). These characters and frame lines become an obstacle to subsequent character detection. Therefore, by deleting the pixels outside the frame lines used in the frame detection, they can be deleted as shown in FIG. 9(b).

[0095] In step S34, the frame detection unit 213 performs projective transformation. The image of the target captured by the camera is distorted due to the curvature of the paper and the positional relationship between the camera and the target. In order to deal with this distortion, projective transformation is used. The transformation formula for projective transformation is expressed by the following formula (2), and can be executed when H is regular.

[0096]

number

[0097] The transformation matrix of this projective transformation includes eight transformation coefficients. Since two equations can be obtained from one pair of pre-transformation and post-transformation coordinates, the transformation coefficients can be determined by preparing four pairs of pre-transformation and post-transformation coordinates. In this modification example, the intersection points of the frame lines are used for these four coordinates. The post-transformation coordinates fix the upper-left coordinate of the frame, calculate the x-axis direction distance between the intersections of the upper-left and upper-right, and the lower-left and lower-right, and set the longer distance as the length of the horizontal frame after transformation. Similarly, calculate the y-axis direction distance between the intersections of the upper-left and lower-left, and the upper-right and lower-right, and set the longer distance as the length of the vertical frame after transformation to define the coordinates so that the frame after transformation becomes a rectangle. The intersection point coordinates are directly used for the pre-transformation coordinates, and the transformation coefficients are determined using these four coordinates, and a projective transformation is performed as shown in FIG. 9(c).

[0098] In step S35, the frame detection unit 213 performs character detection. For character detection, the projection method is used. In the projection method, for each of the y and x directions of the image, the number of pixels corresponding to black after binarization at each coordinate is counted, and the range where the number of pixels is not zero is detected as a line and a character. However, in this method, problems occur such as that components of katakana such as "リ" and "ル", and radicals of Chinese characters such as "検" and "接" are detected separately on the left and right. Therefore, in this modification example, character synthesis is performed after separate detection.

[0099] In this case, first, for the characters detected by the projection method, as represented by the following Equation (3), the horizontal length of each character is divided by the vertical length, and the aspect ratio is calculated for each detected character.

[0100]

Equation

[0101] This aspect ratio r i will be relatively large for full-width characters and relatively small for half-width characters. The medical examination chart that is the subject of image analysis processing has character areas such as test items and numerical areas for the test results, and the numerical areas are written in half-width characters. Therefore, if the only characters in the frame are half-width characters, it is considered that character composition is not necessary. Therefore, this aspect ratio r i The text and numeric regions of the table are distinguished using a threshold value. To do this, a threshold value is set, and if there is anything that exceeds the threshold, it is considered to be a text region, and the characters within the frame of that text region are candidates for character synthesis.

[0102] Whether or not to actually synthesize these character candidates is determined by examining adjacent characters that satisfy the condition expressed by the following formula (4).

[0103]

number

[0104] Then, it is considered whether to synthesize the characters in ascending order of distance, and the aspect ratio of the characters after the character synthesis is calculated using the above formula (3). If the calculated aspect ratio is equal to or greater than the threshold, the characters are actually synthesized as a single character detected separately in the character region. However, if the calculated aspect ratio is less than the threshold, the characters are actually synthesized as a single character detected without separation in the character region. This prevents, for example, a numerical value written in half-width characters from being erroneously synthesized, and on the other hand, when a radical and a character part of a kanji character are detected separately, these can be synthesized as a single character. This ends the process. The character recognition unit 214 can perform the character recognition process in step S24 with high accuracy on a group of character strings (for example, one word) accurately detected by this process.

[0105] [Configuration example] As described above, the image analysis system S according to this embodiment includes the table detection unit 212, the frame detection unit 213, and the information acquisition unit 215. Table detection section 212 detects an area corresponding to a table from an image that includes a table as a subject. The frame detection unit 213 detects a plurality of frames constituting a table from an area corresponding to the table, and assigns position information in the image to each of the detected frames. The information acquisition unit 215 acquires the acquisition target information based on the character recognition results for each of the plurality of boxes and the position information assigned to each of the plurality of boxes.

[0106] In this way, the image analysis system S detects the table and frame in stages, and then acquires the desired target information using the position information. Therefore, it is possible to perform image analysis with higher accuracy than an image analysis method that simply performs image analysis processing without detecting the table and frame and without using the position information. Therefore, even if, for example, various formats are mixed and the arrangement of tables and frames differs depending on the manuscript, or if there is distortion in the image that would normally make image analysis difficult, it is possible to perform appropriate image analysis.

[0107] The information acquisition unit 215 acquires the first information, which is a part of the acquisition target information, based on dictionary data for the first information and the character recognition results for each of the multiple boxes. The information acquisition unit 215 acquires second information, which is a part of the acquisition target information, based on position information corresponding to the frame from which the first information was acquired. This allows image analysis to be performed with higher accuracy by using dictionary data corresponding to the target information to be obtained.

[0108] The information acquisition unit 215 acquires the second information by using position information corresponding to the frame from which the first information was acquired as a reference, and by scanningly searching for other frames in a predetermined direction from the reference based on the position information of the other frames. This allows for more accurate image analysis to be performed, for example, in a table in which related information is arranged vertically or horizontally.

[0109] The frame detection unit 213 detects the frame lines of a plurality of frames, that is, the frame lines in a first direction and the frame lines in a second direction, separately from the area corresponding to the table, and detects a plurality of frames based on the intersections of the detected frame lines in the first direction and the frame lines in the second direction. In this way, by detecting the frame line in the first direction and the frame line in the second direction separately, the frame line and the frame can be detected more accurately than if they were detected without being separated.

[0110] The frame detection unit 213 cuts out each of the detected frames, The character recognition results are generated by performing image analysis processing on each of the extracted frames. This allows image analysis processing to be performed for each box containing a character string, thereby enabling character recognition with higher accuracy.

[0111] The image is an image generated by photographing any one of a plurality of tables having different formats on which the medical examination results are recorded, The information to be acquired includes at least the medical examination results. This makes it possible to obtain medical examination results recorded in various formats with high accuracy.

[0112] [Realization of functions through hardware and software] The function of executing the series of processes according to the above-mentioned embodiment can be realized by hardware, software, or a combination of these. In other words, it is sufficient that the function of executing the series of processes described above is realized in any one of the image analysis systems S, and there is no particular limitation on how this function is realized.

[0113] For example, when the function of executing the above-mentioned series of processes is realized by a processor that executes arithmetic processing, the processor that executes this arithmetic processing includes not only processors that are composed of various types of processing devices alone, such as single processors, multiprocessors, and multicore processors, but also processors that combine these various types of processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays).

[0114] Furthermore, for example, when the function of executing the above-mentioned series of processes is realized by software, the program constituting the software is installed in a computer via a network or a recording medium. In this case, the computer may be a computer in which dedicated hardware is built in, or may be a general-purpose computer (e.g., general electronic devices such as general-purpose personal computers) capable of executing a predetermined function by installing a program. Furthermore, the step of writing the program may include only processes that are performed chronologically according to the order, but may also include processes that are performed in parallel or individually. Furthermore, the step of writing the program may be performed in any order within the scope of the gist of the present invention.

[0115] A recording medium on which such a program is recorded may be provided to a user by being distributed separately from a computer main body, or may be provided to a user in a state in which the program is pre-installed in the computer main body. In this case, the recording medium distributed separately from the computer main body is, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark) Disc. The magneto-optical disk is, for example, an MD (Mini Disc). These recording media are, for example, mounted in the drive 15 in FIG. 2 and the drive 25 in FIG. 3 and installed in the computer main body. The recording medium provided to a user in a state in which the program is pre-installed in the computer main body is, for example, the ROM 12 in FIG. 2 and the ROM 22 in FIG. 3 in which the program is recorded, and the HDD or SSD included in the storage unit 16 in FIG. 2 and the storage unit 26 in FIG. 3. [Explanation of symbols]

[0116] 1 terminal, 2 image analysis device, 3 manuscript (health checkup chart), 11, 21 CPU, 12, 22 ROM, 13, 23 RAM, 14, 24 communication unit, 15, 25 drive, 16, 26 memory unit, 17, 27 input unit, 18, 28 output unit, 111 shooting control unit, 112 terminal side notification unit, 161, 261 image memory unit, 162, 263 analysis result memory unit, 211 image acquisition unit, 212 table detection unit, 213 frame detection unit, 214 character recognition unit, 215 information acquisition unit, 216 device side notification unit, 262 dictionary data memory unit, S image analysis system

Claims

1. A table detection means for detecting a plurality of table regions corresponding to a plurality of tables from a single target image containing a plurality of tables as subjects; a frame detection means for detecting a plurality of frames constituting the table for each of the plurality of table regions and assigning position information in the target image to each of the detected plurality of frames; an information acquisition means for acquiring acquisition target information for each of the plurality of tables based on a character recognition result for each of the plurality of frames and position information assigned to each of the plurality of frames; Equipped with the target image in which the table detection means detects a plurality of table regions is an image in which distortion occurs with respect to the plurality of tables that are objects; the frame detection means detects intersections of frame lines, and performs projective transformation for each frame using coordinates of the intersections, thereby correcting the distortion.

1. An image analysis system comprising:

2. A table detection means for detecting a plurality of table regions corresponding to a plurality of tables from a single target image including the plurality of tables as subjects; a frame detection means for detecting a plurality of frames constituting the table for each of the plurality of table regions and assigning position information in the target image to each of the detected plurality of frames; an information acquisition means for acquiring acquisition target information for each of the plurality of tables based on a character recognition result for each of the plurality of frames and position information assigned to each of the plurality of frames; Equipped with The target image includes a frame containing only half-width numbers and a frame containing full-width characters, the frame detection means detects characters from frames included in the target image, and regards a frame containing a character whose aspect ratio, which is a value obtained by dividing the width of one character by the height of the one character, is equal to or greater than a threshold value as a frame containing a full-width character, and sets the character within the frame as a candidate for processing to perform character synthesis.

1. An image analysis system comprising:

3. A table detection means for detecting a plurality of table regions corresponding to a plurality of tables from a single target image including the plurality of tables as subjects; a frame detection means for detecting a plurality of frames constituting the table for each of the plurality of table regions and assigning position information in the target image to each of the detected plurality of frames; an information acquisition means for acquiring acquisition target information for each of the plurality of tables based on a character recognition result for each of the plurality of frames and position information assigned to each of the plurality of frames; Equipped with The frame detection means generating an image by deleting pixels having a width in a first direction less than a predetermined value from the table region, and detecting a frame line in the first direction; generating an image by deleting pixels having a width less than a predetermined value in a second direction intersecting with the first direction from the table region, and detecting a frame line in the second direction; detecting the plurality of frames based on intersections of the detected frame lines in the first direction and the detected frame lines in the second direction; 1. An image analysis system comprising:

4. the table detection means divides the target image so that one image contains one table; the frame detection means further divides the target image divided by the table detection means so that one frame is included in one image; the information acquisition means acquires the acquisition target information for each of the target images in units of frames further divided by the frame detection means.

4. The image analysis system according to claim 1, wherein the image analysis system further comprises: a first section for detecting a first image;

5. the target image is an image in which distortion occurs in a plurality of tables having different formats on which medical examination results are recorded, the plurality of tables being generated by photographing the plurality of tables; The information to be acquired includes at least a medical examination result.

5. The image analysis system according to claim 1, wherein the image analysis system comprises: a first section;

Images