Information processing device and program
The information processing apparatus improves document structuring by using layout features to extract and tag character strings, addressing the loss of hierarchical relationships in existing methods, enhancing readability and search efficiency.
Patent Information
- Application Number
- JP2022142922
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-09-08
- Publication Date
- 2026-11-11
- Estimated Expiration
- 2042-09-08
AI Technical Summary
Existing techniques for structuring unstructured documents using specific characters or symbols fail to capture the inclusion or hierarchical relationships between sentences, leading to a loss of information.
An information processing apparatus that extracts groups of character strings based on layout features, assigns different tags to lower-level structures based on their order in the document hierarchy, and determines subordinate structures using starting position differences, even with fractional shifts of less than one character.
Enhances the extraction of document structures, preserves inclusion relationships, improves readability, and increases search hit rates by accurately distinguishing and organizing text elements.
Smart Images

Figure 0007947842000001 
Figure 0007947842000002 
Figure 0007947842000003
Abstract
Description
TECHNICAL FIELD
[0001] The present invention relates to an information processing apparatus and a program.
BACKGROUND ART
[0002] There is a technique for assigning a logical structure to an unstructured document based on specific characters or symbols (hereinafter referred to as "heading information") that appear at the beginning of headings and bulleted items.
PRIOR ART DOCUMENTS
PATENT DOCUMENTS
[0003]
PATENT DOCUMENT 1
SUMMARY OF THE INVENTION
PROBLEMS TO BE SOLVED BY THE INVENTION
[0004] However, in the technique of classifying sentences into groups using heading information, information indicating the inclusion relationship or hierarchical relationship of sentences is lost.
[0005] The present invention aims to enable extraction of a greater number of structures compared to the case where an unstructured document is structured using specific characters, symbols, or the like.
MEANS FOR SOLVING THE PROBLEMS
[0006] The invention according to Claim 1 has a processor, wherein the processor extracts groups of character strings that give a superordinate structure of an unstructured document based on a first layout feature, and determines a subordinate structure of each character string included in each extracted group based on a second layout feature Furthermore, depending on the order in which the higher-level structure appears in the document, different tags are assigned to the corresponding lower-level structure even if the hierarchy is the same. The information processing apparatus is configured as described above. The invention according to Claim 2 is the information processing apparatus according to Claim 1, wherein the processor determines the subordinate structure to be assigned to each character string based on a difference in the start position of each character string in the character arrangement direction. The invention described in claim 3 is an information processing device according to claim 2, wherein if a fraction of less than one character is included between the starting position and the reference position of each string, the processor applies a predetermined rule to the fraction and determines a substructure to be assigned to each string. The invention described in claim 4 is an information processing apparatus according to claim 3, wherein the processor determines the substructure to be assigned to each string by rounding down the fractional part. The invention described in claim 5 is an information processing apparatus according to claim 3, wherein the processor determines the substructure to be assigned to each string by rounding the fractional part. The invention described in claim 6 is an information processing apparatus according to claim 1, wherein the processor determines the group based on the starting position of each string in the character arrangement direction. The invention described in claim 7 is an information processing apparatus according to claim 6, wherein the processor assigns multiple strings that are separated by a threshold or more within the same row to different groups. The invention described in claim 8 is an information processing apparatus according to claim 1, wherein the processor determines the group based on one of the character types, character frames, or border frames that describe the string, or a combination thereof. The invention described in claim 9 is an information processing device according to claim 1, wherein the processor outputs a set of multiple strings that are common to the corresponding groups. The invention described in claim 10 is an information processing device according to claim 9, wherein the processor determines the amount of character indentation of the string to be output as a set based on the difference in the underlying structure. The invention described in claim 11 is an information processing device according to claim 1, wherein the processor adds tags indicating the higher-level structure and the lower-level structure before and after each string. The invention described in claim 12 provides a computer with a function for extracting a group of strings that give a higher-level structure to an unstructured document based on a first layout feature, and a function for determining the substructure of each string included in each extracted group based on a second layout feature. A function that assigns different tags to corresponding subordinate structures, even if they have the same hierarchy, according to the order in which the higher-level structure appears in the document, This is a program designed to achieve that. [Effects of the Invention]
[0007] According to the invention described in claim 1, it is possible to extract more structure compared to structuring an unstructured document using specific characters or symbols. According to the invention described in claim 2, an inclusion relationship can be assigned to strings belonging to the same group. According to the invention described in claim 3, the underlying structure can be determined even if there is a starting position shift of less than one character. According to the invention described in claim 4, a starting position shift of less than one character can be ignored due to differences in lower levels. According to the invention described in claim 5, differences in the starting position of less than one character can be reflected in differences in lower levels. According to the invention described in claim 6, multiple groups can be extracted from an unstructured document based on the starting position of a string. According to the invention described in claim 7, even strings within the same line can be separated into different groups if certain conditions are met. According to the invention described in claim 8, multiple groups can be extracted from an unstructured document based on the character types. According to the invention described in claim 9, the readability of the output document can be improved. According to the invention described in claim 10, the inclusion relationship of multiple strings belonging to the same group can be expressed. According to the invention described in claim 11, the search hit rate can be improved. According to the invention described in claim 12, it is possible to extract more structure compared to structuring an unstructured document using specific characters or symbols. [Brief explanation of the drawing]
[0008] [Figure 1] This diagram illustrates an example of the configuration of the information processing system in Embodiment 1. [Figure 2] This figure shows an example of the server's hardware configuration. [Figure 3] This figure shows an example of the hardware configuration of a user terminal. [Figure 4] It is a flowchart illustrating an example of processing operation in Embodiment 1. [Figure 5] It is a diagram showing an example of a document to be processed. [Figure 6] It is a diagram illustrating an example of cluster extraction in Embodiment 1. [Figure 7] It is a table illustrating an example of a structure extraction result from a document. [Figure 8] It is a diagram showing an output example of a document using the layout structure extracted by the processing operation according to Embodiment 1. [Figure 9] It is a diagram showing an output example of a structured document obtained by comprehensively extracting character strings from the document. [Figure 10] It is a diagram showing another output example of a document using the layout structure extracted by the processing operation according to Embodiment 1. [Figure 11] It is a diagram illustrating the relationship of tags associated with a hierarchy extracted from a document. [Figure 12] It is a diagram showing another output example of a document using the layout structure extracted by the processing operation according to Embodiment 1. [Figure 13] It is a flowchart illustrating an example of processing operation in Embodiment 2. [Figure 14] It is a diagram showing an example of a document to be processed in Embodiment 2. [Figure 15] It is a diagram illustrating character frames set in a document. [Figure 16] It is a diagram illustrating an example of cluster extraction in Embodiment 2. [Figure 17] It is a flowchart illustrating an example of processing operation in Embodiment 3. [Figure 18] It is a diagram showing an example of a document to be processed in Embodiment 3. [Figure 19] It is a diagram illustrating character integration using character type information. [Figure 20] It is a flowchart illustrating an example of processing operation in Embodiment 4. [Figure 21]This figure shows an example of a document to be processed in Embodiment 4. [Figure 22] This diagram explains the outside and inside of the enclosed frame. [Figure 23] This figure shows an example of a document to be processed in another embodiment (2). [Modes for carrying out the invention]
[0009] Embodiments of the present invention will be described below with reference to the drawings. <Terminology> First, let's explain the terminology used in the embodiments. "Document" refers to, for example, printed materials, electronic documents, or digitized documents. The documents envisioned in this embodiment are written in text. The text may be arranged horizontally or vertically. Furthermore, the documents envisioned in this embodiment may include images such as photographs and illustrations. "Printed material" refers to paper, film, or other media on which text or images are fixed to the surface. An "electronic document" refers to a document in digital format created using information equipment and software. Examples of software include office suites. A "digitized document" refers to a document generated from a non-digitized document. This includes image files of printed materials scanned with a scanner, as well as image files of printed materials captured with a camera.
[0010] A "structured document" refers to an electronic document or digitized document that has a logical structure. This logical structure includes a hierarchical structure between strings of text. An "unstructured document" refers to an electronic document or digitized document that lacks a logical structure. "Layout features" refer to characteristics that appear in the layout of an unstructured document. Examples of these features include the starting position of a string, the spacing between two strings, character types, character boxes, and borders. "The spacing between two strings" refers to the length or width of the blank space between two strings that appear in the direction of the characters' arrangement.
[0011] A "string" refers to multiple characters that are consecutive in the direction of their arrangement. Periods and commas are treated as single characters. "Character type" refers to things like font, size, color, style, and text decoration. Styles include things like standard, italic, and bold. Text decorations include things like strikethrough, underline, superscript, and subscript. A "text box" is a frame that specifies the position and range for entering text. A text box is part of the data that describes an electronic document. Text boxes may or may not be visible.
[0012] A "border" is a type of decoration that enhances the readability of text. Borders are created independently of text boxes. Note that borders do not only enclose text as a border, but can also be lines or symbols that indicate boundaries with other texts. For example, symbols placed at the four corners of a text string serve the same function as a border. Similarly, lines placed between two text strings serve the function of distinguishing the two text strings. Note that if a particular text string can be distinguished from other text strings by differences in background color or pattern, it will be treated as similar to a border. Note that, like text boxes, borders may or may not be visible.
[0013] "Layout structure" refers to a structure that can be extracted from the layout's features. In this embodiment, two types of structures are dealt with: a higher-level structure and a lower-level structure. "Higher-level structure" refers to clusters (i.e., groups) of one or more strings that appear in an unstructured document. In other words, higher-level structure corresponds to clusters (i.e., groups) of strings that are distinguishable in terms of layout. A "substructure" refers to the layoutal structure of multiple strings belonging to a cluster (i.e., a group). In other words, a substructure is a layoutally distinct subcluster (i.e., subgroup) that defines, for example, a hierarchical relationship.
[0014] <Embodiment 1> <System Configuration> Figure 1 is a diagram illustrating an example of the configuration of the information processing system 1 in Embodiment 1. The information processing system 1 shown in Figure 1 consists of a server 10 and a user terminal 20 connected to a network N. For the sake of explanation, the information processing system 1 shown in Figure 1 represents only the user terminal 20 operated by user A and the user terminal 20 operated by user B. In the following, if we do not distinguish between User A and User B, we will refer to them simply as "User."
[0015] Server 10 is an example of an information processing device and has the function of extracting layout structure from an unstructured document. The server 10 described in this embodiment does not need to be specialized in the function of extracting the structure on the layout; it may also have functions for managing documents or controlling the execution of printing. A server that has the function of managing documents is also called a document management server. A server that has the function of controlling the execution of printing is also called a print server.
[0016] Server 10 can be either a cloud-based server or an on-premises server. In other words, network N can be either the internet or a LAN (=Local Area Network). Network N can also be 5G or other mobile communication systems. Incidentally, there is not limited to one server 10; there can be multiple servers.
[0017] The user terminal 20 is also an example of an information processing device. In this embodiment, the user terminal 20 is used for uploading documents to the server 10 and for viewing and downloading the results of information processing by the server 10 (i.e., processing results). The documents to be uploaded are, for example, unstructured documents. The processing results will include electronic documents with information about the layout structure. These electronic documents include, for example, HTML (HyperText Markup Language) documents and XML (Extensible Markup Language) documents. In the case of these documents, the structural information is embedded as tags. In this embodiment, the user terminal 20 can be a desktop computer, a notebook computer, a tablet computer, or a smartphone. The user terminal 20 may also be a wearable device.
[0018] <Device Configuration> Figure 2 shows an example of the hardware configuration of server 10. The server 10 shown in Figure 2 includes a processor 11 that controls the operation of the entire device, a ROM (Read Only Memory) 12 that stores the BIOS (Basic Input Output System), a RAM (Random Access Memory) 13 used as the work area for the processor 11, an auxiliary storage device 14, and a communication module 15. The processor 11 and other devices are connected via signal lines 16 such as a bus.
[0019] The processor 11, ROM 12, and RAM 13 function as a so-called computer. The processor 11 implements various functions through program execution. For example, the processor 11 performs the process of extracting the layout structure from an unstructured document. The auxiliary storage device 14 is, for example, a hard disk drive or semiconductor storage. The auxiliary storage device 14 stores programs such as the OS (Operating System), as well as electronic documents and digitized documents. The communication module 15 is a device that enables communication with user terminals 20 and other devices connected to the network N. The communication module 15 uses a module compliant with Ethernet®, Wi-Fi®, or any other communication standard.
[0020] Figure 3 shows an example of the hardware configuration of the user terminal 20. The user terminal 20 shown in Figure 3 includes a processor 21 that controls the operation of the entire device, a ROM 22 that stores the BIOS and the like, a RAM 23 used as the work area for the processor 21, an auxiliary storage device 24, a display 25, an I / O interface 26, and a communication module 27. The processor 21 and other devices are connected via signal lines 28 such as a bus.
[0021] The processor 21, ROM 22, and RAM 23 function as a so-called computer. The processor 21 implements various functions through program execution. The auxiliary storage device 24 is, for example, a hard disk drive or semiconductor storage. The auxiliary storage device 24 stores programs such as the operating system, as well as electronic documents and other data. The display 25 is, for example, a liquid crystal display or an organic EL (= Electroluminescent) display. The display 25 shows the electronic document or digitized document that is to be processed.
[0022] The I / O interface 26 is a device that accepts input from the user, such as using a keyboard or mouse. Specifically, the I / O interface 26 accepts input such as mouse cursor positioning and movement, and clicks. The I / O interface 26 is also a device that outputs data to an external terminal. The communication module 27 is a device that enables communication with servers 10 and other devices connected to the network N. The communication module 27 uses a module compliant with Ethernet®, Wi-Fi®, or any other communication standard.
[0023] <Example of processing operation> Figure 4 is a flowchart illustrating an example of processing operation in Embodiment 1. The symbol S represents a step. First, server 10 (see Figure 1) accepts the document to be processed (Step 1). As mentioned above, the document to be processed is uploaded from user terminal 20 (see Figure 1). However, if the document to be processed is managed by server 10, the document to be processed is identified by instructions from user terminal 20. Figure 5 shows an example of a document to be processed. The document shown in Figure 5 is an example where the text is arranged horizontally. The horizontal direction is defined as the X-axis, and the vertical direction as the Y-axis. In both cases, the upper left corner of the document is considered the origin.
[0024] Returning to the explanation of Figure 4. Next, server 10 identifies the rows to be processed (step 2). In this embodiment, the characters are arranged horizontally. In step 2, one row to be processed is identified from top to bottom. Here, "line" refers to a line in which text is written. Therefore, lines containing images such as photographs and illustrations are excluded from processing in step 2. However, if no text is written in a line identified as a processing target, server 10 may change the processing target to the next line. Incidentally, if the text is arranged vertically, the column to be processed is identified in step 2.
[0025] Next, server 10 merges consecutive characters within the same line into a string (step 3). Consecutive characters refer to a sequence of characters where there is no space equal to one or more characters of the font size between each character and the character to its right. Multiple consecutive characters may be found within a single line. Next, server 10 identifies the paired strings (step 4). The paired strings are defined as a combination of one string found in the line identified as the target for processing and a string found in the line immediately above it. Note that if the row to be processed is the first row (i.e., the top row), there is no row directly above it. In this case, no corresponding string is identified in step 4. In this case, a negative result is obtained in step 5, which will be described later.
[0026] If multiple strings are found in the row above, pairs are formed between each string. For example, if three strings A, B, and C are found in the row above, and two strings X and Y are found in the row to be processed, then six pairs will be identified: (string X, string A), (string X, string B), (string X, string C), (string Y, string A), (string Y, string B), and (string Y, string C).
[0027] Next, the server 10 determines whether the x-coordinates and y-coordinates of the starting points of the two pairs of strings are close (step 5). The x-coordinate is an example of information that identifies the lateral position of the starting point. In this embodiment, the origin of the x-coordinate is the left edge of the document, as explained in Figure 5. The x-coordinate is given, for example, by the number of pixels or dots from the origin to the right (i.e., in the x-axis direction) until the starting point is found. However, the x-coordinate may also be given as a length from the origin. Furthermore, if all font sizes used in the document are the same, the x-coordinate may be given as the number of characters from the left edge of the document.
[0028] The y-coordinate is an example of information that identifies the vertical position of the starting point. In this embodiment, the origin of the y-coordinate is the top edge of the document, as explained in Figure 5. The y-coordinate is given, for example, by the number of pixels or dots from the origin downwards (i.e., in the y-axis direction) until the starting point is found. However, the y-coordinate may also be given as a length from the origin. Furthermore, if all font sizes used in the document are the same, the Y-coordinate may be given as the number of characters from the top edge of the document.
[0029] In this embodiment, "the x-coordinates of the starting points are close" means that the difference between the x-coordinates of the two starting points is less than or equal to reference value #1. For reference value #1, for example, the number of pixels equivalent to two characters can be used. "The y-coordinates of the starting points are close" means that the difference between the y-coordinates of the two starting points is less than or equal to reference value #2. Reference value #2 can be, for example, the number of pixels corresponding to one row.
[0030] If the criteria for judgment are met, a positive result is obtained in step 5. In this case, server 10 merges the pairs of strings vertically (step 6). On the other hand, if the conditions for judgment are not met, a negative result is obtained in step 5. In this case, or after the execution of step 6, the server 10 determines whether or not all pairs have been processed (step 7).
[0031] If not all pairs have been processed, a negative result is obtained in step 7. In this case, server 10 returns to step 4 and identifies one of the remaining pairs to be processed. On the other hand, if all pairs have been processed, a positive result is obtained in step 7. In this case, server 10 determines whether or not all rows have been processed (step 8).
[0032] If not all rows have been processed, a negative result is obtained in step 8. In this case, server 10 returns to step 2. On the other hand, if all rows have been processed, a positive result is obtained in step 8. In this case, server 10 extracts a cluster (i.e., group) of strings that gives the higher-level structure of the document (step 9). Specifically, server 10 extracts a cluster of strings that gives the higher-level structure of the document by focusing on the X coordinate of the starting point of the merged string. The X coordinate of the starting point is an example of a "first feature on the layout".
[0033] Server 10 uses, for example, the following criteria to determine whether to merge or separate multiple strings that appear horizontally within the same line. Criterion 1: If the starting point of the second or subsequent strings appearing horizontally within the same line is less than 10 characters from the left edge, the target string is merged into the first string. However, if the starting point is more than 10 characters from the left edge, the target string is not merged into the first string. Criterion 2: If there are two or fewer spaces between two strings appearing on the same line, the two strings will be merged; however, if there are three or more spaces, the two strings will not be merged.
[0034] Incidentally, "to integrate" means to include in the same cluster (group), while "not to integrate" means not to include in the same cluster (group). Not including in the same cluster (group) means separating into a different cluster (group). This step 9 process merges or separates the clusters (groups) of strings extracted in steps 2 through 8.
[0035] Figure 6 illustrates an example of cluster extraction in Embodiment 1. The document shown in Figure 6 is based on the document illustrated in Figure 5. In Figure 6, four clusters #1 to #4 have been extracted. The extraction of these four clusters #1 to #4 constitutes the extraction of the higher-level structure. For example, cluster #1 contains 4 strings, cluster #2 contains 7 strings, cluster #3 contains 3 strings, and cluster #4 contains 2 strings.
[0036] Returning to the explanation in Figure 4. Once step 9 completes the extraction of clusters contained in the document, server 10 identifies the cluster to be processed (step 10). For example, cluster #1 in Figure 6 is identified as the target for processing. Next, server 10 extracts the substructure within the cluster by focusing on the X coordinate of the starting point of each string (step 11). In this embodiment, the server 10 determines the indentation amount for each string to be processed, based on the left edge of the "cluster". The X coordinate of the starting point of each string and the indentation amount are examples of the "second feature in the layout".
[0037] In Figure 6, the left edge of cluster #1 and cluster #2 coincides with the left edge of the document. Therefore, the indentation of the strings contained in cluster #1 and cluster #2 coincides with the X coordinate of the starting point. On the other hand, the left edge of cluster #3 and cluster #4 does not coincide with the left edge of the document. In this case, the indentation of the strings contained in cluster #3 and cluster #4 is given as the difference between the X coordinate of the left edge of the cluster and the X coordinate of the starting point.
[0038] Server 10 distributes the text to the next lower subcluster each time the indentation increases by one character. For example, a string with an indentation of 0 (zero) is classified into "Subcluster #1", a string with an indentation of 1 is classified into "Subcluster #2", a string with an indentation of 2 is classified into "Subcluster #3", and so on. Here, a subcluster is an example of a substructure of the strings contained within each cluster (i.e., group).
[0039] Incidentally, the indentation amount may sometimes result in fractional amounts of less than one character. Server 10 determines the subcluster by applying a predetermined rule to these fractional amounts. Examples of rules are shown below. ·Rule 1: After rounding down the fractional part, the hierarchy is determined by the amount of indentation after rounding down. For example, if the indentation is 2.4 characters, round down 0.4 to make the indentation 2 characters. Then, classify the corresponding string into "Subcluster #3". For example, if the indentation is 1.8 characters, round down 0.8 to make the indentation 1 character. Then, classify the corresponding string into "Subcluster #2".
[0040] ·Rule 2: After rounding off any fractional parts, the hierarchy is determined based on the indentation amount after rounding. For example, if the indentation is 2.4 characters, we change it to 2 characters. Then, we classify the corresponding string into "Subcluster #3". For example, if the indentation is 1.8 characters, change it to 2 characters. Then, classify the corresponding string into "Subcluster #3".
[0041] Once the substructure is determined, server 10 determines the hierarchy of each string based on the substructure (step 12). In this embodiment, the hierarchy of strings classified as "subcluster #1" is determined to be "hierarchy 1", the hierarchy of strings classified as "subcluster #2" is determined to be "hierarchy 2", and the hierarchy of strings classified as "subcluster #3" is determined to be "hierarchy 3".
[0042] Next, server 10 determines whether or not it has processed all clusters (step 13). If not all clusters have been processed, a negative result is obtained in step 13. In this case, server 10 returns to step 10 and identifies one of the remaining clusters to be processed. On the other hand, if all clusters have been processed, a positive result is obtained in step 13. In this case, server 10 terminates the structure extraction process. Figure 7 is a diagram illustrating an example of the results of extracting structure from a document. The extraction results shown in Figure 7 are based on the documents shown in Figures 5 and 6.
[0043] Figure 7 shows an example of determining the hierarchy for each string classified into the four clusters #1 to #4 described in Figure 6. As shown in Figure 7, the indentation of the strings belonging to cluster #3 and cluster #4 is 0 (zero). Therefore, all strings belonging to cluster #3 and cluster #4 are determined to be "Hierarchy 1".
[0044] <Output Example> The following describes an example of document output based on the extracted layout structure. <Example 1> Figure 8 shows an example of a document output using the layout structure extracted by the processing operation according to Embodiment 1. The output example in Figure 8 uses the document shown in Figure 5 as the source document.
[0045] In the example output shown in Figure 8, each string is grouped together in a cluster that provides a higher-level structure. That is, cluster #1 is placed at the first level, cluster #2 at the second level, cluster #3 at the third level, and cluster #4 at the fourth level. Although the document layout differs from that shown in Figure 6, readability is improved because information is arranged in units of meaning.
[0046] Furthermore, in the output example shown in Figure 8, the strings in hierarchy 1 are highlighted. That is, "Item 1:" in cluster #1, "Item 2:" in cluster #2, the three strings in cluster #3, and the two strings in cluster #4 are all displayed in the largest font size and in bold. Note that the highlighting method is not limited to this, and can be combined with other methods such as changing the font color or adding decorations.
[0047] Furthermore, strings at level 2 are positioned one character lower than strings at level 1, and strings at level 3 are positioned one character lower than strings at level 2. In other words, the hierarchical layout structure that existed in the original document is preserved in the structured document. Thus, by applying the processing operations described in this embodiment, it becomes possible to produce output that preserves both the cluster as a higher-level structure and the hierarchical structure within the cluster as a lower-level structure.
[0048] <Comparative Example> Figure 9 shows an example of a document output that has been structured by comprehensively extracting strings from within the document. In the case of Figure 9, the document shown in Figure 5 is used as the source document. In Figure 9, while the strings are comprehensively extracted, the layout structure is lost. For example, not only is the indentation at the beginning of each line completely lost, but the strings from the next line are placed consecutively in the order of "Item 1:" and "Item 2:". Furthermore, the strings that would be extracted as cluster #3 and cluster #4 in Figure 8 are arranged side by side in Figure 9. Thus, in the comparative example, the layout structure of the original document is lost. As a result, readability is significantly reduced compared to the output example of Embodiment 1.
[0049] <Example 2> Figure 10 shows another example of document output using the layout structure extracted by the processing operation according to Embodiment 1. The output example in Figure 10 also uses the document shown in Figure 5 as the source document. The output example shown in Figure 10 has tags attached to the string using the extracted higher-level and lower-level structures. Figure 11 illustrates the relationships between tags linked to the hierarchy extracted from the document. As shown in Figure 11, hierarchy 1 is represented as "Chapter>Text", hierarchy 2 as "Chapter>Paragraph>Text", and hierarchy 3 as "Chapter>Paragraph>List>Text".
[0050] For example, in Figure 10, the string "Item 1:" located at hierarchy level 1 of cluster #1 is, <chapter> <text> Item 1:< / text> It is represented as follows. Note that the string "1-3 subtotal:xxx yen" located at hierarchy level 1 of cluster #3 is, <chapter> <text> Subtotal for 1-3: xxx yen< / text> < / chapter> This is how it is represented. The same applies to other strings in cluster #3 and the strings in cluster #4. Outputting this tagged document allows the user terminal 20 (see Figure 1) to display a layout as shown in Figure 8.
[0051] <Example 3> In the aforementioned Example 1 (see Figure 8), it may seem strange that all the strings for cluster #3 and cluster #4 are displayed in the same manner as the titles for cluster #1 and cluster #2. This discrepancy stems from the fact that the strings of hierarchy 1 belonging to cluster #3 and cluster #4 in the original document have a different logical meaning from the strings of hierarchy 1 belonging to cluster #1 and cluster #2, etc. In other words, the information of hierarchy 1 belonging to cluster #1 and cluster #2 is the "title," whereas cluster # 3 or cluster # 4 This is because the information at level 1, which belongs to this category, consists of "annotations" and "supplementary explanations," etc.
[0052] While there are no rules that guarantee that all documents being processed will not cause any inconsistencies, it is possible to at least assume that the logical weight of clusters appearing for the third time or later in a document is smaller than the logical weight of clusters appearing for the first two times in the document. Figure 12 shows another example of document output using the layout structure extracted by the processing operation according to Embodiment 1. The output example in Figure 12 also uses the document shown in Figure 5 as the source document.
[0053] In Figure 12, the classification names associated with the third and subsequent clusters at level 1 have been changed from those associated with the first two clusters at level 1. Specifically, the Chapter tag has been changed to the Annotation tag. This makes it possible to display tagged documents in a way that differentiates the display of hierarchical 1 strings in clusters #3 and #4 from the display of hierarchical 1 strings in clusters #1 and #2.
[0054] In Example 3, the third and fourth clusters only contain strings of level 1, but if levels 2 and 3 are included, the tags may be modified to preserve the hierarchical structure of the tags. In Example 3, the tags applied to each level of the third and subsequent clusters within the document are changed to annotation tags, but other methods are also possible. For example, strings belonging to each level of the third and subsequent clusters within the document may be associated with the classification name of the level below. For example, strings belonging to level 1 of cluster #3 may be associated with "Chapter>Paragraph>Text". Subsequently, tags representing the position of each level below are added to the corresponding strings. This makes it possible to represent strings belonging to the third and subsequent clusters as belonging to a lower level than strings of the same level belonging to the first two clusters.
[0055] <Embodiment 2> In Embodiment 2, we also assume the information processing system 1 (see Figure 1). The difference lies in the content of the processing operations performed by the server 10 (see Figure 1). Figure 13 is a flowchart illustrating an example of processing operation in Embodiment 2. Figure 13 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 4. In the case of Figure 13, server 10 also accepts the document to be processed (Step 1).
[0056] Figure 14 shows an example of a document to be processed in Embodiment 2. In Figure 14, the text is arranged horizontally. The horizontal direction is defined as the X-axis, and the vertical direction as the Y-axis. In both cases, the upper left corner of the document is considered the origin. In Figure 14, the first and second lines of the document contain the string "ABCDEFG HIJK MNOPQ RSTUV", the third line contains the string "MNOPQ RSTUV", the fourth line contains the string "ABCD EFGHIJK", and the fifth line contains the string "ABCD EFGHIJK NM OPQRSTUV".
[0057] Returning to the explanation of Figure 13. Next, server 10 identifies the character frame to be processed (step 21). A character box is a frame that specifies the position and range for entering text, and is part of the data that describes an electronic document. Thus, the processing operation shown in Figure 13 assumes that the document being processed is an electronic document and that character boxes are placed within the document. Figure 15 illustrates the character boxes set in a document. The document shown in Figure 15 contains two character boxes, #1 and #2, each containing four strings of text.
[0058] Returning to the explanation of Figure 13. Once the character frame to be processed is identified, the server 10 merges the strings contained in the identified character frame vertically. The merging process is the same as in Embodiment 1; that is, steps 4 to 7 are executed. In the case of Figure 15, two strings, "ABCDEFG" and "HIJK", are extracted from the first line of character box #1. Two strings, "ABCDEFG" and "HIJK", are also extracted from the second line of character box #1. Then, the strings "ABCDEFG" from the first and second lines are merged vertically, and the strings "HIJK" from the first and second lines are merged vertically.
[0059] Returning to the explanation of Figure 13. Once the vertical integration of the first character frame #1 is complete, server 10 determines whether all character frames have been processed (step 22). If not all character frames have been processed, a negative result is obtained in step 22. In this case, server 10 returns to step 21. On the other hand, if all character frames have been processed, a positive result is obtained in step 22. In this case, server 10 extracts clusters (i.e., groups) of strings that give the higher-level structure of the document (step 9A). Specifically, server 10 extracts clusters of strings that give the higher-level structure of the document by focusing on the X coordinate of the starting point of the merged strings. In other words, it merges the strings horizontally.
[0060] For horizontal merging of strings, use criteria 1 and 2 described above. However, strings with different character frames will be excluded from merging. Figure 16 illustrates an example of cluster extraction in Embodiment 2. The document shown in Figure 16 is based on the document illustrated in Figure 14. Although the strings "ABCDEFG HIJK" in cluster #1 and "MNOPQ RSTUV" in cluster #3, as shown in Figure 16, are only two characters apart horizontally, they are extracted as separate clusters based on the criterion that strings in different character frames are not merged. The same applies to clusters #2 and #4. The following processing operations are the same as in Embodiment 1, so their explanation will be omitted. In the case of the document illustrated in Figure 14, since there is no indentation, each string belonging to each cluster is treated as hierarchy level 1.
[0061] <Embodiment 3> In Embodiment 3, we also assume the information processing system 1 (see Figure 1). The difference lies in the content of the processing operations performed by the server 10 (see Figure 1). Figure 17 is a flowchart illustrating an example of processing operation in Embodiment 3. Figure 17 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 4. In the case of Figure 17, server 10 also accepts the document to be processed (Step 1). Figure 18 shows an example of a document to be processed in Embodiment 3. In Figure 18, the text is arranged horizontally. The horizontal direction is defined as the X-axis, and the vertical direction as the Y-axis. In both cases, the upper left corner of the document is considered the origin.
[0062] In Figure 18, the first line of the document contains the string "[1]ABCDEFGHIJK (the font up to this point is Arial) [2]abcdefghijk (the font up to this point is Times New Roman)". In Figure 18, the second line of the document contains the string "[3]ABCDEFGHIJK (the font up to this point is Arial) [4]abcdefghijk (the font up to this point is Times New Roman)".
[0063] Returning to the explanation of Figure 17. Next, server 10 identifies the rows to be processed (step 2). Next, server 10 merges characters of the same character type within the same line into a string (step 31). Figure 19 illustrates the integration of characters using character type information. Figure 19 also assumes the document shown in Figure 18. First, server 10 extracts the string "[1]ABCDEFGHIJK [2]abcdefghijk" that appears on the first line of the document as two separate strings, string #1 and string #2, based on the difference in character types. The same applies to the second line of the document.
[0064] Returning to the explanation of Figure 17. Next, server 10 identifies the corresponding string (step 4). In the case of the document shown in Figure 18, one pair is identified from among four pairs: "[1]ABCDEFGHIJK" and "[3]ABCDEFGHIJK", "[1]ABCDEFGHIJK" and "[4]abcdefghijk", "[2]abcdefghijk" and "[3]ABCDEFGHIJK", and "[2]abcdefghijk" and "[4]abcdefghijk".
[0065] Next, server 10 performs steps 5 and 6 on the identified pairs of strings and attempts to merge the strings vertically. In the example in Figure 19, "[1]ABCDEFGHIJK" and "[3]ABCDEFGHIJK" are merged into a single string, and "[2]abcdefghijk" and "[4]abcdefghijk" are merged into a single string.
[0066] Returning to the explanation of Figure 17. If all rows have been processed, server 10 extracts clusters (i.e., groups) of strings that give the higher-level structure of the document (step 9B). Specifically, server 10 extracts clusters of strings that give the higher-level structure of the document by focusing on the X coordinate of the starting point of the merged strings. In other words, it merges the strings horizontally. In this case as well, server 10 uses criteria 1 and 2 described above to merge the strings horizontally.
[0067] When using only Criterion 1 and Criterion 2, the merged strings #1 and #2 shown in Figure 19 are merged into a single string. However, if, in addition to criteria 1 and 2, the criterion of not merging strings with different character types horizontally is applied, then merged string #1 becomes cluster #1 and merged string #2 becomes cluster #2. The following processing operations are the same as in Embodiment 1, so we will omit the explanation. In the case of the document illustrated in Figure 18, since there is no indentation, each string belonging to each cluster is treated as hierarchy level 1.
[0068] <Embodiment 4> In Embodiment 4, we also assume the information processing system 1 (see Figure 1). The difference lies in the content of the processing operations performed by the server 10 (see Figure 1). Figure 20 is a flowchart illustrating an example of processing operation in Embodiment 4. Figure 20 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 4. In the case of Figure 20, server 10 also accepts the document to be processed (Step 1).
[0069] Figure 21 shows an example of a document to be processed in Embodiment 4. In Figure 21, the text is arranged horizontally. The horizontal direction is defined as the X-axis, and the vertical direction as the Y-axis. In both cases, the upper left corner of the document is considered the origin. In Figure 21, the strings "ABCDEFG abcdefghijklm" are placed in lines 1 through 3 of the document, the string "ABCDEFG" is placed in line 4, and the string "ABCDEFGHIJKLMNOPQRSTUVWXYZ" is placed in line 5. In Figure 21, a border is drawn around the lowercase text string to distinguish it from the uppercase text string.
[0070] Returning to the explanation of Figure 20. Next, server 10 identifies the area outside and inside the enclosure (step 41). Figure 22 is a diagram illustrating the outside and inside of the enclosed frame. Once the areas outside and inside the enclosed frame are identified, steps 4-7 are executed for each area. The details of these steps are as explained in Figure 4. In the case of Figure 22, the text appearing outside the enclosed box is first merged vertically. Once the integration process is complete, server 10 determines whether it has processed both the area outside and inside the enclosed frame (step 42).
[0071] If neither process is completed, a negative result is obtained in step 42. In this case, server 10 returns to step 41. On the other hand, if both processes have been completed, a positive result is obtained in step 42. In this case, server 10 extracts clusters (i.e., groups) of strings that give the higher-level structure of the document (step 9C). Specifically, server 10 extracts clusters of strings that give the higher-level structure of the document by focusing on the X coordinate of the starting point of the merged strings. In other words, it merges the strings horizontally. In this case as well, server 10 uses criteria 1 and 2 described above to merge the strings horizontally.
[0072] When using only Criterion 1 and Criterion 2, the text outside the enclosed box and the text inside the enclosed box shown in Figure 22 are merged into a single text. However, if, in addition to criteria 1 and 2, a criterion is applied that does not merge the text outside the enclosed box with the text inside the enclosed box horizontally, the text outside the enclosed box and the text inside the enclosed box will not be merged. The following processing operations are the same as in Embodiment 1, so we will omit the explanation. In the case of the document illustrated in Figure 21, since there is no indentation, each string belonging to each cluster is treated as hierarchy level 1.
[0073] <Other Embodiments> (1) Although embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the embodiments described above. It is clear from the claims that embodiments with various modifications or improvements made to those described above are also included in the technical scope of the present invention.
[0074] (2) A combination of Embodiment 2 and Embodiment 4 described above is also possible. Figure 23 shows an example of a document to be processed in another embodiment (2). The document shown in Figure 23 and the document shown in Figure 21 are visually identical. However, the document shown in Figure 23 has a text box set, and the range of that text box also includes the surrounding frame. If this type of document is to be processed, the server 10 first identifies the character frame to be processed (step 21). Next, the server 10 determines whether the identified character frame contains a border frame, and if it does, separates the characters in the identified character frame into those outside and inside the border frame (step 41). After this, the same processing as in Embodiment 4 is performed.
[0075] (3) In the above-described embodiment, the server 10 performs the function of extracting the layout structure from an unstructured document, but the user terminal 20 may perform this function independently. In this case, the corresponding function is provided as a program executed on the user terminal 20.
[0076] (4) The processor in each of the embodiments described above refers to a processor in a broad sense, and includes not only general-purpose processors (e.g., CPUs) but also specialized processors (e.g., GPUs (=Graphical Processing Units), ASICs (=Application Specific Integrated Circuits), FPGAs (=Field Programmable Gate Arrays), programmable logic devices, etc.). Furthermore, the processor operations in each of the embodiments described above may be performed by a single processor alone, or by multiple processors located in physically separate locations working together. Also, the order in which each operation is executed by the processor is not limited to the order described in each of the embodiments described above, but may be changed individually.
[0077] <Note> (((1))) An information processing device having a processor, the processor extracts a set of strings that give a higher-level structure to an unstructured document based on a first feature of the layout, and determines the lower-level structure of each string contained in each extracted set based on a second feature of the layout. (((2))) The information processing apparatus described in (((1))) wherein the processor determines the substructure to be assigned to each string based on the difference in the starting position of each string in the character arrangement direction. (((3))) The information processing apparatus according to (((2))), wherein if a fraction of less than one character is included between the starting position and the reference position of each string, the processor applies a predetermined rule to the fraction and determines the substructure to be assigned to each string. (((4))) The information processing apparatus described in (((3))), wherein the processor determines the substructure to be assigned to each string by rounding down the fractional part. (((5))) The information processing apparatus described in (((3))) wherein the processor determines the substructure to be assigned to each string by rounding the fractional part. (((6))) The information processing apparatus according to any one of (((1))) to (((5))), wherein the processor determines the group based on the starting position of each string in the character arrangement direction. (((7))) The information processing apparatus according to (((6))), wherein the processor assigns multiple strings that are separated by a threshold or more within the same row to different groups. (((8))) The information processing apparatus according to any one of (((1))) to (((5))), wherein the processor determines the group based on one of the character types, character frames, or border frames, or a combination thereof, for describing a string. (((9))) The processor is an information processing device described in any one of (((1))) to (((8))) that outputs multiple strings common to the corresponding group as a single unit. (((10))) The information processing apparatus according to (((9))), wherein the processor determines the amount of character indentation of the string to be output as a set based on the difference in the underlying structure. (((11))) The information processing device according to any one of (((1))) to (((8))), wherein the processor adds tags indicating the upper-level structure and the lower-level structure before and after each string. (((12))) A program to enable a computer to perform the following functions: extract a group of strings that give a higher-level structure to an unstructured document based on a first layout feature, and determine the substructure of each string in each extracted group based on a second layout feature. [Explanation of symbols]
[0078] 1…Information processing system, 10…Server, 11, 21…Processor, 12, 22…ROM, 13, 23…RAM, 14, 24…Auxiliary storage device, 15, 27…Communication module, 16, 28…Signal line, 20…User terminal, 25…Display, 26…I / O interface< / chapter>
Claims
1. It has a processor, The aforementioned processor, Based on the first layout feature, extract a set of strings that give a higher-level structure to the unstructured document. Based on the second feature of the layout, the substructure of each string included in each extracted group is determined. Depending on the order in which the aforementioned higher-level structure appears in the document, different tags are assigned to the corresponding lower-level structures even if they have the same hierarchy. Information processing device.
2. The aforementioned processor, The substructure to be assigned to each string is determined by the difference in the starting position of each string in the direction of character arrangement. The information processing apparatus according to claim 1.
3. The aforementioned processor, If there is a fractional part of less than one character between the start position and the reference position of each string, a predetermined rule is applied to that fractional part to determine the substructure to be assigned to each string. The information processing apparatus according to claim 2.
4. The aforementioned processor, The aforementioned rounding down determines the substructure to be assigned to each string. The information processing apparatus according to claim 3.
5. The aforementioned processor, The substructure to be assigned to each string is determined by rounding the aforementioned fractional part. The information processing apparatus according to claim 3.
6. The aforementioned processor, The group is determined based on the starting position of each string in the direction of character arrangement. The information processing apparatus according to claim 1.
7. The aforementioned processor, If multiple strings exist within the same row that are separated by a threshold, assign those multiple strings to different groups. The information processing apparatus according to claim 6.
8. The aforementioned processor, The group is determined based on one of the following: the character type used to describe the string, the character frame, the border frame, or a combination thereof. The information processing apparatus according to claim 1.
9. The aforementioned processor, Output multiple strings common to the corresponding group as a single unit. The information processing apparatus according to claim 1.
10. The aforementioned processor, The amount of indentation for the string output as a single unit is determined by the difference in the substructure. The information processing apparatus according to claim 9.
11. The aforementioned processor, Tags indicating the higher-level structure and the lower-level structure are added before and after each string. The information processing apparatus according to claim 1.
12. On the computer, Based on the first layout feature, it has a function to extract a group of strings that give a higher-level structure to an unstructured document, Based on the second feature of the layout, a function is provided to determine the substructure of each string contained in each extracted group, A function that assigns different tags to corresponding subordinate structures, even if they have the same hierarchy, according to the order in which the higher-level structure appears in the document, A program to achieve this.
Citation Information
Patent Citations
Order deciding method for character area
JP1996147410A
Dividing method for area of document image and discriminating method for kind of multiple column
JP1997044594A
Image processing unit and control method thereof
JP2009021712A
Bulleted lists
JP2017134822A
Document processing device and document processing method
JP3940491B2
