Document image analysis system and program

Abstract

To accomplish a technology capable of extracting text data in which the correct reading sequence among messages and the importance level are reflected from document images of an atypical layout structure.SOLUTION: A document image analyzing system 10 includes: a character recognizing unit 14 that recognizes characters in a document image file 32; a character attribute identifying unit 16 that identifies the size, center coordinates and inclination angle of each character; a line certifying unit 18 that certifies a line based on the size of each character and on the distance between characters; a block certifying unit 20 that certifies a text block formed of one or multiple lines based on the size of the character forming the line, the placement, and the distance between lines; an importance level setting unit 24 that sets an importance level for each text block in accordance with the size and inclination of the character forming the text block; and a scenario certifying unit 22 which certifies an area where each text block is placed in the document image based on the coordinates of the header character of the text block and the coordinates of the bottom character, and which gives an area label that represents the priority of the reading sequence of the text block belonging to each area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] This invention relates to a document image analysis system and program, and in particular to a technology for extracting text data by performing OCR processing on document images such as screen displays (web screens, etc.) for digital media announcements, or printed advertising flyers and direct mail (DM), and analyzing the structure of the extracted text data. [Background technology]

[0002] As a technique for capturing a printed document image into a computer and analyzing the structure of its written content (text data), for example, a document recognition device and a document recognition method are disclosed in Patent Document 1. [Patent Document 1] Japanese Patent Application Laid-Open No. 2002-108847

[0003] This technology performs multiple types of layout analysis on a document image input to a computer and associates the results of each analysis process, making it possible to extract accurate layout analysis results from document images that contain a mixture of vertical and horizontal writing. Furthermore, by applying multiple reading order determination methods to the analysis results, it is also possible to present the reading order between subregions in the document. DISCLOSURE OF THE INVENTION [Problem to be solved by the invention]

[0004] However, the technology in Patent Document 1 targets document images with relatively standard layouts, such as business cards, newspaper articles, books, office documents, and official documents, and the "analysis results" merely identify the distribution of vertical and horizontal writing areas within a document, or present the user with possible reading orders between partial areas.

[0005] For this reason, it was not useful for analyzing document images with atypical layouts, such as advertising flyers and direct mail, which pack a lot of information into a limited space and vary the size and angle of the text to emphasize specific messages or to emphasize the importance of one message over another, particularly for extracting a syntactically appropriate series of sentences from the laid-out design content.

[0006] In addition, digital media announcements generally use de facto standard data formats such as PDF, but because complex display modification data is added to text data, even if the character information itself can be easily extracted, it is difficult to analyze the document structure and extract a series of sentences that are appropriate in terms of syntax, or to extract the gist of the announcement. [Non-Patent Document 1] What is Adobe PDF? / Internet URL: https: / / www.adobe.com / jp / acrobat / about-adobe-pdf.html / Retrieved on: January 16, 2024

[0007] This invention was devised in light of the above-mentioned conventional situation, and aims to realize a technology that can analyze document images with non-standard layouts, such as advertising flyers and direct mail, and extract the reading order intended by the document creator and the level of importance between messages as analysis results. [Means for solving the problem]

[0008] In order to achieve the above object, the document image analysis system described in claim 1 comprises: a character recognition means for recognizing characters in a document image; a character attribute identification means for identifying the size, center coordinates and tilt angle of each character; a line identification means for identifying a line, which is a collection of multiple characters, based on the size of each character and the distance between characters; a block identification means for identifying a text block consisting of one or more lines, based on the size, arrangement and inter-line distance of the characters that make up the line; an importance setting means for setting an importance level to each text block according to at least the size of the characters that make up the text block; and a scenario identification means for identifying an area in the document image in which each text block is located, based on the coordinates of predetermined vertices of a circumscribing rectangle that surrounds the text block, and for assigning to each text block an area label that indicates the reading order priority of the text blocks belonging to that area, and is characterized in that the scenario identification means assigns the area label in accordance with the following rules. (a) An area label is assigned to each text block so that the closer the text block is to either the left or right edge of the document image, the higher the reading order priority, and the closer it is to the other of the left or right edges, the lower the reading order priority. (b) Each text block is assigned an area label so that the closer the text block is to the top of the document image, the higher the reading order priority, and the closer it is to the bottom, the lower the reading order priority.

[0009] The character attribute specification means generates, for example, a circumscribing rectangle surrounding each character, and calculates the size, center coordinates, and inclination angle of the character based on the coordinates of the vertices of the circumscribing rectangle.

[0010] The document image analysis system described in claim 2 is the system of claim 1, wherein the line recognition means recognizes characters where the distance between characters is within a predetermined range, the character sizes are similar within a predetermined range, and the characters are lined up horizontally as horizontal lines of horizontal writing, and sets a corresponding line type flag; recognizes characters where the distance between characters is within a predetermined range, the character sizes are similar within a predetermined range, and the characters are lined up at the same or a similar tilt angle to the horizontal as slant lines of horizontal writing, and sets a corresponding line type flag; recognizes characters where the distance between characters is within a predetermined range, the character sizes are similar within a predetermined range, and the characters are lined up vertically or at the same or a similar tilt angle to the vertical as vertical lines, and sets a corresponding line type flag; recognizes a character that has no adjacent characters within a predetermined distance as an independent character and sets a corresponding line type flag; and the block recognition means recognizes lines that satisfy the following conditions as belonging to the same text block, recognizes other lines as a text block consisting of one line, and recognizes an independent character as a text block consisting of one character. (a) Each row has the same row type flag. (b) The average size of the characters in each line is approximately within a specified range. (c) At least some of the characters in each line are arranged in parallel. (d) The spacing between each line must be within the specified range.

[0011] The document image analysis system described in claim 3 is the system of claim 1, characterized in that the importance setting means sets a higher importance to each text block the larger the average size of the characters constituting each text block, and sets a higher importance to each text block when the characters constituting each text block are slanted compared to when the characters are not slanted.

[0012] The document image analysis system according to claim 4 is the system according to claim 1, characterized in that the scenario determination means assigns an area label to each text block according to the following procedure. (1) Identify the coordinates (X0, Y0) of the upper left vertex and the coordinates (X1, Y1) of the lower right vertex of the circumscribing rectangle that encloses each text block. (2) The text block whose upper left vertex coordinate (X0) is the leftmost is extracted, and the coordinate (X1) of the lower right vertex of that text block is recognized as the provisional right edge of the area. (3) If there is a text block among the remaining text blocks whose upper left vertex coordinate (X0) is located to the left of the provisional right edge of the area and whose lower right vertex coordinate (X1) is located to the right of the provisional right edge of the area, the coordinate (X1) of the lower right vertex of that text block is updated as the provisional right edge of the new area. (4) After executing the process (3) for all text blocks to be processed, the coordinate (X1) of the latest provisional right edge is recognized as the right edge of the area. (5) A common area label is assigned to each text block whose upper left vertex coordinate (X0) is to the left of the right edge of the area. (6) Repeat steps (1) to (5) above for the remaining text blocks, and assign a common area label to each. (7) Text blocks with a common area label are extracted as processing targets. (8) The text block whose upper left vertex coordinate (Y0) is the topmost is extracted, and the lower right vertex coordinate (Y1) of that text block is recognized as the provisional bottom edge of the area. (9) If there is a text block among the remaining text blocks whose upper left vertex coordinate (Y0) is located above the provisional bottom edge of the area and whose lower right vertex coordinate (Y1) is located below the provisional bottom edge of the area, the coordinate (Y1) of the lower right vertex of that text block is updated as the provisional bottom edge of the new area. (10) After executing the process (9) for all text blocks to be processed, the coordinate (Y1) of the latest provisional bottom edge is recognized as the bottom edge of the area. (11) Add a common area label to each text block whose upper left vertex coordinate (Y0) is above the bottom edge of the area. (12) Repeat steps (7) to (11) above for the remaining text blocks, and add a common area label to each. (13) Repeat the above steps (2) to (12) for each text block having a common area label until the number of text blocks belonging to one area is equal to or less than a predetermined number.

[0013] The document image analysis system according to claim 5 is the system according to claim 1, characterized in that the scenario determination means assigns an area label to each text block according to the following procedure. (1) Identify the coordinates (X0, Y0) of the upper right vertex and the coordinates (X1, Y1) of the lower left vertex of the circumscribing rectangle that encloses each text block. (2) The text block whose upper right vertex coordinate (X0) is the rightmost is extracted, and the lower left vertex coordinate (X1) of that text block is recognized as the provisional left edge of the area. (3) If there is a text block among the remaining text blocks whose upper right vertex coordinate (X0) is located to the right of the provisional left edge of the area and whose lower left vertex coordinate (X1) is located to the left of the provisional left edge of the area, the lower left vertex coordinate (X1) of that text block is updated as the provisional left edge of the new area. (4) After executing the process (3) for all text blocks to be processed, the coordinate (X1) of the latest provisional left edge is recognized as the left edge of the area. (5) A common area label is assigned to each text block whose upper right vertex coordinate (X0) is to the right of the left edge of the area. (6) Repeat steps (1) to (5) above for the remaining text blocks, and assign a common area label to each. (7) Text blocks with a common area label are extracted as processing targets. (8) The text block whose upper right vertex coordinate (Y0) is the topmost is extracted, and the lower left vertex coordinate (Y1) of that text block is recognized as the provisional bottom edge of the area. (9) If there is a text block among the remaining text blocks whose upper right vertex coordinate (Y0) is located above the provisional bottom edge of the area and whose lower left vertex coordinate (Y1) is located below the provisional bottom edge of the area, the lower left vertex coordinate (Y1) of that text block is updated as the provisional bottom edge of the new area. (10) After executing the process (9) for all text blocks to be processed, the coordinate (Y1) of the latest provisional bottom edge is recognized as the bottom edge of the area. (11) Add a common area label to each text block whose upper right vertex coordinate (Y0) is above the bottom edge of the area. (12) Repeat steps (7) to (11) above for the remaining text blocks, and add a common area label to each. (13) Repeat the above steps (2) to (12) for each text block having a common area label until the number of text blocks belonging to one area is equal to or less than a predetermined number.

[0014] A document image analysis program according to claim 6 is characterized in that it causes a computer to function as the document image analysis system according to any one of claims 1 to 5. [Effects of the Invention]

[0015] In the case of the document image analysis system of this invention, lines, which are collections of characters, are identified based on the size and coordinates of characters extracted from digital media notification display data and document images, and text blocks, which are collections of lines, are identified, and a reading order can be derived based on the arrangement of each text block in the document image, making it possible to present a likely reading order even if the object to be analyzed has an atypical layout configuration. Furthermore, the importance of each text block can be derived based on the size of characters extracted from the document image. Furthermore, the structured text obtained as a result of the analysis can be easily converted into tagged text to be provided as training data for artificial intelligence. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram showing a functional configuration of a document image analysis system according to the present invention; [Figure 2] FIG. 10 is a diagram illustrating a sample document image. [Figure 3] 10 is a flowchart showing a procedure of processing by a document image receiving unit, a character recognition unit, and a character attribute identification unit. [Figure 4] 10A and 10B are diagrams illustrating results of processing by a character recognition unit and a character attribute identification unit. [Figure 5] 10 is a flowchart showing the procedure of horizontal line recognition processing by a line recognition unit; [Figure 6] 10A and 10B are diagrams illustrating an example of a group of characters to be processed by a line recognition unit. [Figure 7] 10A and 10B are diagrams illustrating a state in which a group of characters having a relatively small size is temporarily excluded from processing by the line recognition unit. [Figure 8] 10A and 10B are diagrams illustrating a state in which a part of a character is excluded from the line recognition process by the line recognition unit and is isolated in a reserved group. [Figure 9] 10A and 10B are diagrams showing how a line recognition unit temporarily excludes a group of characters with a relatively small size from processing, and how horizontally set lines are recognized. [Figure 10] 10A and 10B are diagrams illustrating a state in which characters recognized as lines in horizontal writing by a line recognition unit are excluded from processing targets. [Figure 11] 10 is a flowchart showing a procedure for vertically set line recognition processing and independent character recognition processing performed by a line recognition unit; [Figure 12] FIG. 10 is a diagram illustrating an example of a group of characters stored in a reserved group. [Figure 13] 10 is a flowchart showing the procedure of block recognition processing for horizontal lines in horizontal composition by a block recognition unit; [Figure 14] 10A and 10B are diagrams showing a specific example of block recognition processing for horizontal lines in horizontal writing by a block recognition unit. [Figure 15]10 is a flowchart showing the procedure of block recognition processing for horizontally set skewed lines by a block recognition unit; [Figure 16] 10 is a flowchart showing the procedure of block certification processing for vertically set lines by a block certification unit; [Figure 17] 10 is a flowchart showing a procedure for block recognition processing for an independent character by a block recognition unit; [Figure 18] 10 is a flowchart showing the procedure of vertical division processing by a scenario identification unit. [Figure 19] 10A and 10B are diagrams illustrating a specific example of vertical division processing by a scenario determination unit. [Figure 20] 10 is a flowchart showing the procedure of horizontal division processing by a scenario identification unit. [Figure 21] 10A and 10B are diagrams illustrating a specific example of horizontal division processing by a scenario determination unit. [Figure 22] 10A and 10B are diagrams showing the results of a second vertical division process and a second horizontal division process performed by the scenario determination unit. [Figure 23] 10A and 10B are diagrams illustrating results of vertical and horizontal division processes performed by a scenario identification unit on the entire area of ​​a document image. [Figure 24] FIG. 10 is a diagram showing the results of performing horizontal division processing on area [A] of the sample document image. [Figure 25] FIG. 10 is a diagram showing the results of performing vertical and horizontal division processes multiple times on an area [Ab] of a sample document image. [Figure 26] FIG. 10 is a diagram showing analysis result data of a sample document image. BEST MODE FOR CARRYING OUT THE INVENTION

[0017] As shown in FIG. 1, the document image analysis system 10 according to the present invention includes functional components, namely, a document image receiving unit 12, a character recognition unit 14, a character attribute identification unit 16, a line identification unit 18, a block identification unit 20, a scenario identification unit 22, an importance setting unit 24, and an analysis result output unit 26. These functional components are realized by the CPU of the server computer operating in accordance with the OS and dedicated application programs.

[0018] A client terminal 30 such as a PC operated by an operator 28 is connected to this system 10 via a network. When a document image file 32 to be analyzed is sent from a client terminal 30 to the system 10, the document image receiving unit 12 receives it, and various analysis processes are performed by the character recognition unit 14 to the importance setting unit 24, after which the analysis result data 34 is sent to the client terminal 30 via the analysis result output unit 26.

[0019] FIG. 2 shows an example (sample document image) of the document image file 32, which is a PDF file that forms one side of a sealed postcard folded in three. As shown in the figure, the document image file 32 has three postcard-sized advertising pages, with different background images and character strings on each page. Each character string has a different font, size, layout (horizontal / vertical / diagonal), and color, and the blocks of characters (text blocks) are arranged in an irregular and intricate manner. By arranging the characters irregularly in this way, it is possible to scatter many messages within a limited space and to create a sense of the importance of each message.

[0020] FIG. 3 is a flowchart showing the processing performed by the document image receiving unit 12, the character recognition unit 14, and the character attribute specification unit 16. First, when the document image file 32 is imported into the system 10 by the document image receiving unit 12 (S10), the character recognition unit 14 executes OCR (Optical Character Reader) processing (S11). Specifically, the system extracts the character portion from an image in which the background and characters are mixed together, identifies the corresponding character code, and determines the XY coordinates of the four vertices of the rectangle (circumscribed rectangle) surrounding each character.

[0021] Figure 4(a) shows an example, in which the horizontally written characters "Hokkaido" and "Freshly picked ingredients" are each enclosed in a separate rectangle. The four sides of the rectangle are positioned so that they are in close contact with the characters, so the dimensions of the rectangle vary depending on the size of the characters. Note that periods (.), commas (,), bullets (·), etc. included in the character string are excluded from this processing and are surrounded by a rectangle with the same height as the previous character. Figure 4(b) shows how each character of the word "healthy habits" is written diagonally horizontally and surrounded by a rectangle tilted at a specified angle. Figure 4(c) shows how each character of the vertically written phrase "Free Shipping Every Time" is surrounded by a rectangle. At this point, the characters are only recognized individually, and strictly speaking, the concepts of "horizontal writing" and "vertical writing" do not arise.

[0022] Next, the character attribute specification unit 16 assigns a character ID and a predetermined character attribute to each character (S12). The character attributes that are assigned here are (2) to (5) below. (1) Character ID (2) Characters (e.g., "kita") (3) Font size (height of the rectangle) (4) Tilt (angle of the rectangle) (5) Center coordinates of the rectangle (X, Y) The above (3) to (5) are calculated by the character attribute specification unit 16 based on the coordinates of the four vertices of the rectangle surrounding each character. At this time, characters whose character size is less than a predetermined threshold value are excluded by the character attribute specification unit 16 as noise that cannot be recognized by the naked eye (S13).

[0023] Next, the contents of the processing by the row validator 18 will be described. As described above, the processing by the character recognition unit 14 and the character attribute specification unit 16 is performed on a character-by-character basis, so as a first step in recognizing these as a sentence, it is necessary to group the individual characters into lines. Furthermore, character strings can be broadly divided into "horizontal writing" and "vertical writing," so the line recognition unit 18 first executes the line recognition process for horizontal writing shown in the flowchart of FIG.

[0024] First, the line recognition unit 18 extracts the character with the largest character size from among all the characters extracted from the document image, and recognizes this as "character A" (S20). For example, assuming that a group of characters as shown in FIG. 6 is to be processed, the line recognition unit 18 recognizes the largest character "Shaka" as "character A."

[0025] Next, the line recognition unit 18 temporarily extracts a group of characters whose size is similar to that of the character A ("shallow") as a processing target (S21). Here, "similar in size" corresponds to, for example, a character whose size is 80% or more of that of the character A. As a result of this processing, as shown in FIG. 7, the characters "This week," "Sanja Festival," and "Nakamise Street," which are relatively small in size, are temporarily excluded from the processing target.

[0026] Next, the line recognition unit 18 extracts all the characters present on the extension of the inclination direction of the character A, and executes a process of rearranging them in order from the left based on the X coordinate (S22). Here, "characters existing on the extension of the tilt direction" means "characters that intersect with a horizontal line extending from the center of the character A to the left and right" if the character A is a horizontal character rather than a tilted character. On the other hand, if the character A is a tilted character, it means "characters that intersect with a tilted line extending from the center of the character A to the left and right (a line parallel to the top or bottom side of the rectangle surrounding the character A)."

[0027] Next, the line recognition unit 18 executes a process of calculating the distance between each character (S23). If there is no next character within a certain distance from character A (for example, within 150% of the horizontal width of character A) (S24 / N), character A is excluded from the processing target and stored in the reservation group (S25). In this case, since there is no character to the right of character A (light) in the first place, the line recognition unit 18 excludes character A (light) from the target of the horizontal arrangement line recognition process and stores it in the reservation group.

[0028] Subsequently, for each of the characters "grass", "special", "collection", and "end" arranged vertically, the line recognition unit 18 identifies them as character A in descending order of size and then repeats the same process (S30 / N, S20 to S25). As a result, as shown in FIG. 8, the vertical string of "Asakusa Special Issue" and the single character "end" are excluded from the target of the horizontal arrangement line recognition process and isolated as a reservation group.

[0029] Continuing, the line recognition unit 18 identifies "thunder", which has the largest character size among the remaining characters, as "character A" (S20), and temporarily extracts a group of characters whose sizes are similar to character A ("thunder") as the processing target (S21). As a result, groups of characters such as "this week", "Sanja Matsuri", and "Nakamise Street", which have relatively small character sizes, are temporarily excluded from the processing target, and the characters shown in FIG. 9(a) remain as the processing target.

[0030] Next, the line recognition unit 18 extracts all the character groups existing on the extension line in the inclination direction (horizontal direction) of character A and executes a process of sorting them in order from left to right based on the X coordinate (S22). As a result, as shown in FIG. 9(b), the character "Kannon-sama", which is not located on the extension line in the inclination direction (horizontal direction) of character A, is excluded.

[0031] Next, the line recognition unit 18 calculates the distance between each character (S23), and after confirming that the next character "o" is within a certain distance from the character A "thunder" (S24 / Y), it checks the distance between the first character "asa" and the next character in order, searches for a dividing point that exceeds a certain distance (S26), and recognizes the line based on the search result (S27). In the example of Figure 9(b), since there is a relatively large gap between "ba" and "kaminari" (for example, a distance of more than 150% of the size of the character A), the string up to that point, "When you think of Asakusa," is recognized as one line, and the string that follows, "kaminariokoshi," is also recognized as a separate line. If no division point is found in S27, the line recognition unit 18 recognizes the entire horizontally arranged character string as one line without dividing it.

[0032] Next, the line recognition unit 18 assigns a line ID and a predetermined line attribute to each recognized line (S28), and then excludes the character string for which line recognition has been completed from the processing target (S29). The row attributes assigned here are (2) to (8) below. (1) Row ID (2) Line type flag (e.g., "F") (3) String (e.g., "When you think of Asakusa") (4) Center coordinates of the first character (X, Y) (5) Center coordinates of the last character (X, Y) (6) Average font size (7) Average slope (8) Number of characters (e.g., "6") Incidentally, the ID of the first character of each line is used as the line ID. The line type flag is a code that indicates the type of line, and is assigned "F (Flat)" for horizontal lines and "T (Tilt)" for inclined lines with an average inclination of ±3 degrees or more. Since "Speaking of Asakusa" and "Kaminariokoshi" are both horizontal lines, the line type flag for each is set to "F."

[0033] 10, the number of characters remaining to be processed is decreasing, but the line recognition unit 18 recognizes the largest remaining character, "kan," as the next character A (S20) and continues processing. Therefore, as the processing is repeated, the size of character A gradually decreases. Through this process, each character of the vertically arranged "this week" is also stored in a reserved group (S25). Then, when the processing is completed for all characters to be processed (S30 / Y), the line recognition unit 18 ends the horizontal line recognition processing.

[0034] Next, the line recognition unit 18 executes vertical line recognition processing and independent character recognition processing shown in the flowchart of FIG. First, as shown in FIG. 12, the line recognition unit 18 recognizes "Shaka" having the largest character size among the characters stored in the reserved group as "character B" (S40).

[0035] Next, the line recognition unit 18 temporarily extracts a group of characters whose size is similar to that of the character B ("shallow") as a processing target (S41). As a result of this processing, the group of characters "This week" which has a relatively small character size is temporarily excluded from the processing target.

[0036] Next, the line recognition unit 18 extracts all characters present on the extension of the slant direction of the character B, rearranges them from top to bottom based on the Y coordinate (S42), and then calculates the distance between each character (S43). The above "character group existing on the extension of the tilt direction" means "character group intersecting with a vertical line extending up and down from the center of character B" if character B is a vertical character rather than a tilted character. On the other hand, if character B is a tilted character, it means "character group intersecting with a tilted line extending up and down from the center of character B (a line parallel to the left or right side of the rectangle surrounding character B)."

[0037] Here, since the character B (light) and the next character "grass" are within a certain distance (S44 / Y), the line recognition unit 18 checks the distance to the next character in order from the beginning and searches for a split point exceeding the certain distance (S46).

[0038] Here, since there is a relatively large gap (for example, a distance of 150% or more of the vertical dimension of character B) between the character "collection" and the character "end", the line recognition unit 18 recognizes the character string "Asakusa Special Feature" up to that point as one line (S47), assigns a predetermined line attribute (S48), and excludes the recognized character string from the processing target (S49). The line attributes assigned here correspond to the following (2) to (8). (1) Line ID (2) Line type flag V (Vertical) (3) Character string ("Asakusa Special Feature") (4) Center coordinates (X, Y) of the first character (5) Center coordinates (X, Y) of the last character (6) Average character size (7) Average inclination (8) Number of characters (4)

[0039] Note that for horizontal lines, horizontal lines and inclined lines are distinguished, and different line type flags (F / T) are assigned to each. For vertical lines, a common line type flag V is assigned without distinguishing between vertical lines and inclined lines. This is because it is a rare case to use vertical text in an inclined manner. However, when analyzing a document image in which vertical inclined lines are frequently used, separate line type flags can be set for vertical and inclined vertical lines, and they can be distinguished and handled separately.

[0040] The remaining character here is only the single character "end" and does not form a character string. Therefore, it is not recognized as a line at this point and is recognized as "character B" in the next processing loop. Then, since there is no next character, the line recognition unit 18 determines that "there is no next character within a certain distance" (S44 / N) and recognizes it as an independent character (S45). The line recognition unit 18 assigns a predetermined line attribute to this independent character (S48), and then excludes the character for which line recognition has been completed from the processing target (S49). The row attributes assigned here are (2) to (7) below. (1) Row ID (2) Line type flag I (Isolate) (3) Character string (“end”) (4) Center coordinates (X, Y) (5) Font size (6) Tilt (7) Number of characters (1)

[0041] The line recognition unit 18 continues the same process for the remaining characters ("this week"), and ends the line recognition process when the process is completed for all characters to be processed (S50 / Y).

[0042] Next, the block recognition unit 20 performs a text block recognition process on each line. Here, a "text block" refers to a group of sentences or paragraphs that are organized in an easy-to-read format. Hereinafter, a text block will be abbreviated to "block." In some cases, a block is made up of only one line, but when a block is made up of multiple lines, it becomes necessary to combine the lines to form a block. The processing procedure for horizontal lines in horizontal composition by the block recognition unit 20 will be described below with reference to the flowchart in FIG.

[0043] First, the block recognition unit 20 extracts horizontal lines with flag F from all lines as targets for processing (S60). The reason why these lines are processed separately from flag T (slanted lines) is based on the empirical rule that horizontal lines and slanted lines do not form the same paragraph.

[0044] Next, the block recognition unit 20 identifies the line with the largest average character size among all horizontal lines (S61), and temporarily extracts a group of lines with average character sizes within a predetermined range (e.g., 80% or more) of this maximum average character size (S62), and arranges each line from top to bottom based on the Y coordinate of the center coordinate of the first character (S63). For example, if each line shown in Figure 14(a) is extracted as a horizontal line in S60, the block recognition unit 20 recognizes the line "Speaking of Asakusa" as the line with the largest average character size, and then, as shown in Figure 14(b), only lines with a similar average character size are retained, and lines with relatively small character sizes are temporarily excluded.

[0045] Next, the block recognition unit 20 recognizes the center coordinate X of the first character of the first line as the left end of the provisional paragraph, and recognizes the center coordinate X of the last character as the right end of the provisional paragraph (S64), and then excludes lines in which the last character is located to the left of the left end and lines in which the first character is located to the right of the right end as lines in which there are no characters parallel to each other (overlapping) and therefore do not constitute the same block (S65). As a result, as shown in FIG. 14(c), the line "thunder" in which the first character "thunder" is located to the right of the right end is temporarily excluded.

[0046] Next, the block recognition unit 20 checks the distance between each remaining line, starting from the first line and the next line, and when it detects a location where the distance is greater than a predetermined value (for example, greater than 150% of the average character size of the first line) (including locations where there is no next line) (S66), it recognizes the lines up to that point as belonging to the same block (S67). In the example of Figure 14(c), the lines "Speaking of Asakusa" and "Kannon-sama" are close to each other, and there is no line below the line "Kannon-sama," so the two lines are recognized as belonging to the same block.

[0047] Next, as shown in FIG. 14(d), the block authentication unit 20 sequentially concatenates the rows contained in the same block (S68), and assigns a block ID and a predetermined block attribute (S69). The block attributes that are assigned here are (2) to (7) below. (1) Block ID (2) Character string (When you think of Asakusa, you think of Kannon) (3) Coordinates of the first line (center coordinates of the first and last characters) (4) Coordinates of the last line (center coordinates of the first and last characters) (5) Average font size (6) Average slope (7) Number of lines (2) The block ID is the ID of the first character in the first line.

[0048] The block verification unit 20 excludes the rows for which the blocks have been verified from the processing target (S70), and then continues the block verification process for the remaining processing targets (S71). For the remaining rows that have not been combined with other rows, the block recognition unit 20 recognizes them as "blocks consisting of one row" and assigns them a block ID and block attributes.

[0049] Next, as shown in the flowchart of FIG. 15, the block recognition unit 20 extracts horizontally set slanted lines with flag T from all the line groups as processing targets (S80), and executes block recognition processing. However, since the block recognition process for this slanted row is substantially the same as the block recognition process for the horizontal row described above, a redundant explanation will be omitted.

[0050] Next, the block recognition unit 20 extracts vertically set lines with flag V from all the lines as processing targets (S100), and executes block recognition processing, as shown in the flowchart of FIG. The block recognition process for vertical rows is basically the same as the block recognition process for horizontal rows described above, but there are some differences, which will be briefly explained below.

[0051] That is, the block recognition unit 20 identifies the line with the largest average character size among all vertical lines (S101), and temporarily extracts a group of lines with average character sizes within a predetermined range (e.g., 80% or more) of this maximum average character size (S102), and arranges each line from right to left based on the X coordinate of the center coordinate of the first character (S103). This corresponds to the fact that lines in vertical writing are generally written from right to left.

[0052] Next, the block recognition unit 20 recognizes the center coordinate Y of the first character of the first line as the top end of the provisional paragraph, and recognizes the center coordinate Y of the last character as the bottom end of the provisional paragraph (S104), and then excludes lines in which the last character is located above the above top end and lines in which the first character is located below the above bottom end as not constituting the same block (S105).

[0053] Next, the block recognition unit 20 checks the distance between each remaining line, starting from the first line and the next line, and when it detects a location where the distance is greater than a predetermined value (for example, greater than 150% of the average character size of the first line) (including locations where there is no next line) (S106), it recognizes the lines up to that point as belonging to the same block (S107).

[0054] Next, the block authentication unit 20 sequentially connects the rows contained in the same block (S108), and then assigns a block ID and the same block attributes as above (S109). Here too, the block recognition unit 20 recognizes the remaining rows that have not been combined with other rows as "blocks consisting of one row" and assigns them a block ID and block attributes.

[0055] Next, the block recognition unit 20 extracts, as a processing target, independent characters having flag I from all the lines (S120), and executes block recognition processing, as shown in the flowchart of FIG.

[0056] However, the block recognition process for independent characters is extremely simple, and essentially consists of recognizing each independent character as a "block consisting of one character" and assigning a block ID and block attributes (S121). The block attributes that are assigned here are (2) to (6) below. (1) Block ID (2) Characters (3) Center coordinates of characters (4) Font size (5) Tilt (6) Number of lines (1) The block ID is the same as the ID of the individual character.

[0057] Next, a scenario determination process based on the arrangement of each block (the position in the document image) is executed by the scenario determination unit 22. Here, the scenario means the reading order of each block intended by the document creator. In other words, because a document image has a background image with a certain degree of cohesion in terms of design and color, even if the document image has a complex and intricate layout, a human can easily recognize the order in which each block should be read. For example, in the case of the sample document image shown in Figure 2, the combination of the background image and the text string makes it clear at a glance that the document is divided horizontally into three sections, so viewers can naturally first read the left page from top to bottom, then the center page from top to bottom, and finally the right page from top to bottom.

[0058] However, it is not easy for a computer to determine the reading order of each block based on such a background image, and even if the latest image recognition technology using AI (artificial intelligence) is used, a learning process based on a huge number of sample images is essential to achieve this. Therefore, the system 10 adopts an approach that determines the reading order (scenario) based on the arrangement of each block, without relying on background images.

[0059] The first vertical division process (horizontal area definition process) performed by the scenario determination unit 22 will be described below with reference to the flowchart in FIG. First, the scenario identification unit 22 generates a circumscribing rectangle surrounding each block, and identifies the coordinates (X0, Y0) of the upper left vertex and the coordinates (X1, Y1) of the lower right vertex of each rectangle (S130). For example, as shown in Figure 19(a), when a block 50 is placed in a document image, the scenario identification unit 22 generates a rectangle 50R whose sides touch the top, bottom, left, and right edges of the group of characters contained in the block, and then identifies the XY coordinates of the upper left vertex 50a and the XY coordinates of the lower right vertex 50b. Similarly, the scenario identification unit 22 generates circumscribing rectangles 52R to 66R for the other blocks 52 to 66, and then identifies the XY coordinates of the upper left vertices 52a to 66a and the XY coordinates of the lower right vertices 52b to 66b.

[0060] Next, the scenario identification unit 22 extracts the block whose upper left vertex coordinate (X0) is the leftmost, and identifies the lower right vertex coordinate (X1) of that block as the "provisional right edge of the area" (S131). In the case of FIG. 19(a), a block 52 is extracted, and its bottom right coordinate 52b is recognized as the "provisional right edge of the area."

[0061] Next, the scenario identification unit 22 searches for a block among the remaining blocks whose upper left vertex coordinate (X0) is located to the left of the provisional right edge of the area (S132), and if such a block exists, it checks whether its lower right vertex coordinate (X1) is located to the right of the provisional right edge (S133), and if YES, updates the lower right vertex coordinate (X1) of the block in question as the new "provisional right edge of the area" (S134). In the case of FIG. 19(a), since the block 62 meets the conditions of S132 and S133, the X coordinate (X1) of the lower right vertex coordinate 62b is recognized as the new provisional right edge.

[0062] In the example of Figure 19(a), since there are no other blocks that meet the conditions of S132 and S133, the scenario certification unit 22 certifies that processing has been completed for all blocks (S135 / Y) and officially certifies the coordinate (X1) of the lower right vertex 62b of block 62, which is the latest provisional right edge, as the right edge of the area (S136).

[0063] Next, the scenario identification unit 22 assigns a common area label to blocks whose upper left vertex coordinate (X0) is to the left of the right end of the area (S137). Specifically, as shown in FIG. 19(b), the area label of the capital letter "A" is assigned to blocks 52, 54, 56, 62, and 66 (for convenience of illustration, simply referred to as "label" in the drawing).

[0064] The scenario identification unit 22 repeats the same process for the remaining blocks (S138 / Y, S131 to S134) and assigns a common area label "B" to blocks 50, 58, 60, and 64 (S137).

[0065] As a result of the above, the group of blocks in Figure 19(a) is divided into area [A], which is a collection of blocks assigned the area label "A", and area [B], which is a collection of blocks assigned the area label "B", as shown in Figure 19(b).

[0066] Next, the first horizontal division process (vertical area definition process) performed by the scenario determination unit 22 will be described with reference to the flowchart of FIG. First, the scenario identification unit 22 extracts a group of blocks that have a common area label (S140). FIG. 21(a) shows that blocks 52, 54, 56, 62, and 66 with a common area label "A" are extracted.

[0067] Next, the scenario identification unit 22 extracts the block whose upper left vertex coordinate (Y0) is the topmost, and identifies the lower right vertex coordinate (Y1) of that block as the "provisional lower end of the area" (S141). In the case of FIG. 21(a), a block 52 is extracted, and its lower right vertex coordinate 52b is recognized as the "provisional lower end of the area."

[0068] Next, the scenario identification unit 22 searches for a block among the remaining blocks whose upper left vertex coordinate (Y0) is located above the provisional bottom edge of the area (S142), and if such a block exists, it checks whether its lower right vertex coordinate (Y1) is located below the provisional bottom edge (S143), and if the result is YES, it updates the lower right vertex coordinate (Y1) of the block in question as the new "provisional bottom edge of the area" (S144). In the case of FIG. 21(a), since the block 54 meets the conditions of S142 and S143, its lower right vertex coordinate 54b is recognized as the new provisional lower end.

[0069] Next, the scenario identification unit 22 repeats the same process (S145 / N), and since the block 56 meets the conditions of S142 and S143, it identifies its lower right vertex coordinate 56b as the new provisional lower end.

[0070] In the example of Figure 21(a), since there are no other blocks that meet the conditions of S142 and S143, the scenario certification unit 22 certifies that processing has been completed for all blocks (S145 / Y) and officially recognizes the bottom right vertex coordinate 56b of block 56, which is the latest provisional bottom edge, as the bottom edge of the area (S146).

[0071] Next, the scenario identification unit 22 assigns a common area label to blocks whose upper left vertex coordinate (Y0) is above the bottom edge of the area (S147). Specifically, as shown in FIG. 21(b), the lowercase alphabet "a" is added as an area label to blocks 52, 54, and 56.

[0072] The scenario identification unit 22 repeats the same process for the remaining blocks (S148 / Y, S141 to S147), and adds an area label "b" to block 62 and an area label "c" to block 66 (S147).

[0073] As a result of the above, the group of blocks belonging to area [A] is divided vertically into three areas: area [Aa] consisting of blocks with area label "Aa", area [Ab] consisting of blocks with area label "Ab", and area [Ac] consisting of blocks with area label "Ac", as shown in Figure 21(b).

[0074] At this point, only one block belongs to each of areas Ab and Ac, and further subdivision is not possible. However, since three blocks remain in area Aa, the scenario identification unit 22 attempts further subdivision by repeating vertical and horizontal division processes.

[0075] That is, by executing S131 and subsequent steps of the flowchart in Figure 18, the scenario identification unit 22 divides the area [Aa] horizontally into two areas, as shown in Figure 22(a), namely, area [AaA] to which blocks 52 and 56 belong, and area [AaB] to which only block 54 belongs.

[0076] Next, the scenario identification unit 22 executes S141 and subsequent steps of the flowchart in Figure 20, thereby dividing the area [AaA] vertically into two areas, area [AaAa] to which block 52 belongs and area [AaAb] to which block 56 belongs, as shown in Figure 22(b).

[0077] As a result of the above, area [A] has been subdivided into five blocks each containing only a single block, and each has been assigned a unique area label.

[0078] The scenario identification unit 22 then performs a similar area subdivision process on area [B] in Figure 19(b), thereby subdividing it into four areas, each consisting of a single block, as shown in Figure 23, and assigning each a unique area label. However, the area division process by the scenario identification unit 22 does not necessarily have to be repeated until there is only one block in each area, and may be rounded up to an appropriate number of times. As a result, there is a possibility that multiple blocks may exist in one area, in which case the reading order can be assigned according to a predetermined rule (for example, priority is given to the block with the coordinate of the upper left vertex further to the left and above).

[0079] Incidentally, in the case of the sample document image shown in Figure 2, the scenario identification unit 22 performed a similar vertical division process, resulting in division into four areas: [A], [B], [C], and [D]. From a human perspective, it should be divided into three areas, and the existence of area C, which only contains blocks of "cut lines," may seem unnecessary; however, as will be explained later, the reading order is determined in alphabetical order A → B → C → D, so this is not a particular problem. Although a background image is displayed in the figure for convenience, it goes without saying that in actual processing, the background image is ignored and the area is defined based only on the arrangement of blocks (the same applies hereinafter).

[0080] Figure 24 shows an extract of area [A] from the sample document image shown in Figure 2, which is divided vertically into four sections based on the arrangement of each block, with area labels "a" to "d" added to each area.

[0081] Figure 25 shows an extract of area [Ab] from the sample document image, which is subdivided into numerous areas, each consisting of a single block, and each with a unique area label.

[0082] As described above, each time the scenario identification unit 22 performs vertical and horizontal division processing, it assigns alphabetical area labels to each area in ascending order based on the rules of "left priority" and "top priority." Therefore, by sorting in ascending order based on the area labels as described below, it is possible to align each block in the same order as a human reading a document image. If the document image is in a Japanese layout that is read from right to left, the above "top left vertex coordinate" can be replaced with the "top right vertex coordinate" and the "bottom right vertex coordinate" can be replaced with the "bottom left vertex coordinate" and the above process can be performed to align each block in the correct order.

[0083] In the above, to make it easier for humans to understand, uppercase letters are assigned from the left to the results of the vertical division process, and lowercase letters are assigned from the left to the results of the horizontal division process, but this invention is not limited to this, and it is also possible to configure it so that the same type of letters are assigned from the beginning to each processing result in the processing order. Alternatively, instead of alphabets, it is of course possible to use sortable values ​​such as consecutive numbers in descending or ascending order or hiragana in the Japanese syllabary as components of area labels.

[0084] Next, the importance setting unit 24 sets an importance for each block. Here, importance refers to a numerical value derived based on the average character size for each block and whether or not it is tilted. The larger the character size, the higher the importance is assigned, and tilted characters are assigned a higher importance than non-tilted characters. This is based on the fact that in direct mail and flyers, the more important the message, the larger the text tends to be to make it stand out, or the text tends to be slanted.

[0085] Finally, the analysis result output unit 26 compiles the processing results up to this point as analysis result data 34 and transmits it to the client terminal 30 . Figure 26 shows an excerpt of the analysis result data for the sample document image shown in Figure 2, and is expressed as a table with the following data items: ID, text, upper left (X0), upper left (Y0), lower right (X1), lower right (Y1), size, angle, number of lines, flag, importance, and label. The meaning of each data item is as follows: "ID": Block ID (= ID of the first character) "Text": The concatenation of the strings contained in the block "Top Left (X0)": The X coordinate of the top left corner of the block "Top Left (Y0)": The Y coordinate of the top left corner of the block "Bottom Right (X1)": X coordinate of the bottom right vertex of the block "Bottom Right (Y1)": The Y coordinate of the bottom right corner of the block "Size": The average character size of the text contained in the block "Slant": The average slant angle of the text contained in the block "Number of lines": The number of lines in the document image of the text included in the block "Flag": Line type flag of the text included in the block "Importance": Importance of the block set by the importance setting unit 24 “Label”: Area label of the block set by the scenario identification unit 22

[0086] For example, in the figure, block "ID:1" (experience the health power of green juice!) is made up of horizontal lines with "Flag: F" and "Slope: 0", so the "Importance" value (93.3) and "Size" value (93.3) are equal. In contrast, in the case of the block "ID:71" (nutritional quality that producers care about), since it consists of a slanted row of "Flag: T", a larger value (85.4) is assigned to "importance" by multiplying the "size" value (71.2) by the coefficient "1.2". For other blocks with "flag: T", the "importance" values ​​are derived by multiplying the respective "size" values ​​by a common coefficient "1.2". This coefficient "1.2" is merely an example, and it is of course possible to use other values ​​as the coefficient.

[0087] By sorting based on the "importance" data item of this analysis result data 34, it is possible to arrange each block contained in the document image in order of importance, and as a result, the user can grasp the important message that the creator of the document image is trying to convey from a document image containing many character strings. Furthermore, when using text generation AI to create a summary of a document image, if messages are pre-arranged in order of importance in this way, it will be possible to obtain more accurate output results. Furthermore, this analysis result data 34 itself can be effectively used as training data (tagged text) to be supplied to artificial intelligence.

[0088] In addition, by sorting based on the data item "label," it is possible to arrange the blocks in a manner close to the scenario intended by the creator of the document image. As a result, even a visually impaired person can understand the written contents of a document image with a complex layout in the correct order by applying a computer's voice reading function to the analysis result data 34.

[0089] Incidentally, if only blocks are extracted from a sample document image and the scenario recognition unit 22 does not perform the area label assignment process, the computer will not be able to recognize that the document image is made up of multiple pages connected horizontally, and it is expected that a reading order will simply be assigned to each block from the top left to the bottom right of the entire area, ignoring the page structure.

[0090] In the above, an example has been shown in which horizontal lines and slanted lines are treated separately for horizontally written character strings only, and the latter is given a higher importance than the former, but the present invention is not limited to this. In other words, vertically-typed character strings can also be treated as "vertical lines" and "diagonal lines," with the latter being given greater importance than the former. [Explanation of symbols]

[0091] 10 Document Image Analysis System 12 Document image reception unit 14 Character recognition section 16 Character attribute identification part 18 line certification department 20 Block Certification Department 22 Scenario Certification Department 24 Importance setting section 26 Analysis result output section 28 Operator 30 client terminals 32 Document Image Files 34 Analysis result data 50 blocks 52 blocks 54 blocks 56 blocks 58 blocks 60 blocks 62 blocks 64 blocks 66 blocks

Claims

1. A system for analyzing document images in which horizontal lines, diagonal lines, and vertical lines are arranged on the same page, and the importance of each character string is expressed by the size and arrangement of the characters that make up each line, character recognition means for recognizing characters in a document image; character attribute specifying means for specifying the size, center coordinates and tilt angle of each character; a line identifying means for identifying a line that is a set of a plurality of characters; a block identifying means for identifying a text block consisting of one or more lines; an importance setting means for setting an importance level for each text block; a scenario determination means for determining the position of each text block in a document image based on the coordinates of predetermined vertices of a circumscribing rectangle surrounding the text block, and for assigning an area label to each text block indicating the reading order priority of the text block; a means for sorting the text blocks by the importance level to arrange the text blocks in order of importance; the line recognition means recognizes characters that are arranged horizontally and whose inter-character distances are within a predetermined range, whose character sizes are similar within a predetermined range, and whose characters are aligned horizontally as horizontal lines, and sets a corresponding line type flag; If the distance between slanted characters having a slant angle equal to or greater than a predetermined value falls within a predetermined range, the sizes of the slanted characters are similar within a predetermined range, and the slanted characters are arranged at a predetermined slant angle relative to the horizontal direction, the slanted characters are recognized as a horizontally typeset slanted line, and a corresponding line type flag is set; A line is determined to be a vertically set line when the distance between characters falls within a predetermined range, the size of each character is similar within a predetermined range, and the characters are aligned vertically, and a corresponding line type flag is set; the block recognition means recognizes as belonging to the same text block a plurality of lines that satisfy all of the following conditions: that each line has the same line type flag; that the average size of the characters contained in each line is similar within a predetermined range; that at least some of the characters contained in each line are arranged in parallel; and that the inter-line distance of each line is within a predetermined range; and recognizes other lines as being text blocks consisting of a single line; the scenario identification means assigns an area label to each text block so that the closer the text block is to either the left or right end of the document image, the higher the reading order priority, and the closer it is to either the right or left end, the lower the reading order priority, and so that the closer the text block is to the top of the document image, the higher the reading order priority, and the closer it is to the bottom end, the lower the reading order priority; The document image analysis system is characterized in that the importance setting means sets a higher importance for each text block the larger the average size of the characters constituting the text block, and for text blocks consisting of horizontally set slanted lines, sets a higher importance by multiplying the average size of the characters by a predetermined coefficient.

2. 2. The document image analysis system according to claim 1, wherein the scenario determination means assigns an area label to each text block according to the following procedure. (1) Identify the coordinates (X0, Y0) of the upper left vertex and the coordinates (X1, Y1) of the lower right vertex of the circumscribing rectangle that encloses each text block. (2) Extract the text block whose upper left vertex coordinate (X0) is the leftmost, and identify the lower right vertex coordinate (X1) of that text block as the provisional right edge of the area. (3) If there is a text block among the remaining text blocks whose upper left vertex coordinate (X0) is located to the left of the provisional right edge of the area and whose lower right vertex coordinate (X1) is located to the right of the provisional right edge of the area, the coordinate (X1) of the lower right vertex of that text block is updated as the provisional right edge of the new area. (4) After executing the process (3) for all text blocks to be processed, the coordinate (X1) of the latest provisional right edge is recognized as the right edge of the area. (5) A common area label is assigned to each text block whose upper left vertex coordinate (X0) is to the left of the right edge of the area. (6) Repeat steps (1) to (5) above for the remaining text blocks, and assign a common area label to each. (7) Text blocks with a common area label are extracted as processing targets. (8) The text block whose upper left vertex coordinate (Y0) is the topmost is extracted, and the lower right vertex coordinate (Y1) of that text block is recognized as the provisional bottom edge of the area. (9) If there is a text block among the remaining text blocks whose upper left vertex coordinate (Y0) is located above the provisional bottom edge of the area and whose lower right vertex coordinate (Y1) is located below the provisional bottom edge of the area, the coordinate (Y1) of the lower right vertex of that text block is updated as the provisional bottom edge of the new area. (10) After executing the process (9) for all text blocks to be processed, the coordinate (Y1) of the latest provisional bottom edge is recognized as the bottom edge of the area. (11) Add a common area label to each text block whose upper left vertex coordinate (Y0) is above the bottom edge of the area. (12) Repeat steps (7) to (11) above for the remaining text blocks, adding a common area label to each. (13) Repeat steps (2) to (12) above for each text block that has a common area label until the number of text blocks belonging to one area is equal to or less than a predetermined number.

3. 2. The document image analysis system according to claim 1, wherein the scenario determination means assigns an area label to each text block according to the following procedure. (1) Identify the coordinates (X0, Y0) of the upper right vertex and the coordinates (X1, Y1) of the lower left vertex of the circumscribing rectangle that encloses each text block. (2) Extract the text block whose upper right vertex coordinate (X0) is the rightmost, and identify the lower left vertex coordinate (X1) of that text block as the provisional left edge of the area. (3) If there is a text block among the remaining text blocks whose upper right vertex coordinate (X0) is located to the right of the provisional left edge of the area and whose lower left vertex coordinate (X1) is located to the left of the provisional left edge of the area, the lower left vertex coordinate (X1) of that text block is updated as the provisional left edge of the new area. (4) After executing the process (3) for all text blocks to be processed, the coordinate (X1) of the latest provisional left edge is recognized as the left edge of the area. (5) A common area label is assigned to each text block whose upper right vertex coordinate (X0) is to the right of the left edge of the area. (6) Repeat steps (1) to (5) above for the remaining text blocks, and assign a common area label to each. (7) Text blocks with a common area label are extracted as processing targets. (8) The text block whose upper right vertex coordinate (Y0) is the topmost is extracted, and the lower left vertex coordinate (Y1) of that text block is recognized as the provisional bottom edge of the area. (9) If there is a text block among the remaining text blocks whose upper right vertex coordinate (Y0) is located above the provisional bottom edge of the area and whose lower left vertex coordinate (Y1) is located below the provisional bottom edge of the area, the lower left vertex coordinate (Y1) of that text block is updated as the provisional bottom edge of the new area. (10) After executing the process (9) for all text blocks to be processed, the coordinate (Y1) of the latest provisional bottom edge is recognized as the bottom edge of the area. (11) Add a common area label to each text block whose upper right corner coordinate (Y0) is above the bottom edge of the area. (12) Repeat steps (7) to (11) above for the remaining text blocks, adding a common area label to each. (13) Repeat steps (2) to (12) above for each text block that has a common area label until the number of text blocks belonging to one area is equal to or less than a predetermined number.

4. A program that causes a computer to function as the document image analysis system according to any one of claims 1 to 3.

Images