Information processing device, information processing method, and information processing program

Abstract

To read subjects and amounts indicated in financial statements, fixed asset ledgers, and trial balance sheets and to streamline and speed up the process of sorting.SOLUTION: When no vertical lines are detected in image data, an information processing device 1 sets virtual vertical lines on the basis of the degree of existence of text along the vertical direction, and when no horizontal lines are detected in the image data, the information processing device 1 sets virtual horizontal lines on the basis of the degree of existence of text along the horizontal direction. The information processing device 1 then generates a two-dimensional array using the detected vertical and horizontal lines, as well as the virtual vertical and horizontal lines. The information processing device 1 has a two-dimensional array unit 115 as means for storing text in each element of the two-dimensional array on the basis of the coordinates of each text.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] A known prior art technique for creating a cash flow statement is a cash flow statement creation program that causes a computer to function as: a fund item master storage means that divides fund items, which are display items on the cash flow statement, into withdrawals and deposits; an association master storage means that associates fund items with the debit or credit items of all or some of the financial items; an input means for trial balance data created by another system; a storage means for storing, for that data, the fund items associated with each amount and financial item, and the withdrawal or deposit classifications established for the above-mentioned fund items; and a display means for aggregating the stored amounts for each fund item, and displaying the fund items, the aggregated amounts, and the amounts of cash and deposits as the cash flow statement for that single period (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2004-234063 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when preparing a balance sheet (B / S), profit and loss statement (P / L), cash flow statement (C / F), and the like, it is desirable to reduce and speed up the work of reading and organizing the items and amounts listed in, for example, printed financial statements, fixed asset ledgers, and trial balances.

[0005] Therefore, in one aspect, the present invention aims to provide technology that can reduce the labor and speed up the work of reading and organizing the items and amounts listed in financial statements, fixed asset ledgers, and trial balances. [Means for solving the problem]

[0006] In order to solve the above problem, an information processing device according to the present invention may have: means for performing image processing on imaging data obtained by imaging at least one of a financial statement, a fixed asset ledger, and a trial balance, to detect vertical and horizontal ruled lines, and acquiring coordinates of start and end points of the detected vertical and horizontal ruled lines; means for acquiring text and the coordinates of each of the text from the imaging data or digitized data obtained by digitizing at least one of the financial statement, the fixed asset ledger, and the trial balance; and means for setting virtual vertical ruled lines based on the degree of presence of the text along the vertical direction when vertical ruled lines are not detected in the imaging data, and setting virtual horizontal ruled lines based on the degree of presence of the text along the horizontal direction when horizontal ruled lines are not detected in the imaging data, generating a two-dimensional array from the detected vertical ruled lines and the virtual vertical ruled lines and the virtual horizontal ruled lines, and storing the text in each element of the two-dimensional array based on the coordinates of each of the text.

[0007] The information processing device of the present invention may further have a means for generating a two-dimensional array in which, when a single row of the two-dimensional array contains multiple subject names, each row is divided by subject name and each row contains only one subject name.

[0008] Furthermore, an information processing method according to the present invention may be an information processing method executed by an information processing device, and may include the steps of: applying image processing to imaging data in which at least one of a financial statement, a fixed asset ledger, and a trial balance has been imaged, to detect vertical and horizontal ruled lines, and acquiring coordinates of start and end points of the detected vertical and horizontal ruled lines; acquiring text and the coordinates of each of the text from the imaging data or the digitized data in which at least one of the financial statement, the fixed asset ledger, and the trial balance has been digitized; setting virtual vertical ruled lines based on the degree of presence of the text along the vertical direction when vertical ruled lines are not detected in the imaging data, and setting virtual horizontal ruled lines based on the degree of presence of the text along the horizontal direction when horizontal ruled lines are not detected in the imaging data; generating a two-dimensional array from the detected vertical ruled lines and the virtual vertical ruled lines and the virtual horizontal ruled lines; and storing the text in each element of the two-dimensional array based on the coordinates of each of the text.

[0009] The information processing method of the present invention may include a step of generating a two-dimensional array in which, when a single row of the two-dimensional array contains multiple subject names, each row is divided into separate rows for each subject name, and each row contains only one subject name.

[0010] In addition, the information processing program according to the present invention may cause a computer to execute the following processes: applying image processing to imaging data in which at least one of a financial statement, a fixed asset ledger, and a trial balance has been imaged, to detect vertical and horizontal ruled lines, and acquiring the coordinates of the start and end points of the detected vertical and horizontal ruled lines; acquiring text and the coordinates of each of the text from the imaging data or the digitized data in which at least one of the financial statement, the fixed asset ledger, and the trial balance has been digitized; setting virtual vertical ruled lines based on the degree of presence of the text along the vertical direction when no vertical ruled lines are detected in the imaging data, and setting virtual horizontal ruled lines based on the degree of presence of the text along the horizontal direction when no horizontal ruled lines are detected in the imaging data; generating a two-dimensional array from the detected vertical ruled lines and the virtual vertical ruled lines and the virtual horizontal ruled lines; and storing the text in each element of the two-dimensional array based on the coordinates of each of the text.

[0011] The information processing program of the present invention may further cause the computer to execute a process of generating a two-dimensional array in which, when a single row of the two-dimensional array contains multiple subject names, each row is divided by subject name and each row contains only one subject name. Effect of the Invention

[0012] According to one aspect of the present invention, it is possible to reduce the labor and speed up the work of reading and organizing the items and amounts listed in financial statements, fixed asset ledgers, and trial balance sheets. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a functional configuration of an information processing device according to an embodiment of the present invention. [Diagram 2] 2 is a flowchart showing the procedure of processing executed in the information processing device of FIG. 1, and showing the procedure of an information processing method according to an embodiment of the present invention. [Diagram 3]13 is a diagram for explaining the start and end points of vertical lines and the start and end points of horizontal lines when financial statements or the like include vertical and horizontal lines. FIG. [Figure 4] 13 is a diagram for explaining the start and end points of vertical lines when a financial statement or the like contains only vertical lines. FIG. [Diagram 5] 1. FIG. 4 is a diagram for explaining a range in which a two-dimensional array unit of the information processing device in FIG. [Figure 6] 1. FIG. 4 is a diagram for explaining how virtual horizontal lines are set by a two-dimensional array unit of the information processing device of FIG. [Figure 7] 1. FIG. 4 is a diagram for explaining how virtual vertical and horizontal ruled lines are set by a two-dimensional array unit of the information processing device of FIG. [Figure 8] FIG. 13 is a diagram illustrating a case where subject names exist in multiple columns of two-dimensional array data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. In the description of the present invention, at least one of a financial statement, a fixed asset ledger, and a trial balance will be referred to as a "financial statement, etc." Note that a financial statement is also called a financial statement, and includes a balance sheet (B / S) and a profit and loss statement (P / L).

[0015] (Overall composition) FIG. 1 is a block diagram showing a functional configuration of an information processing device 1 according to an embodiment, as an example of a specific configuration aspect of the information processing device according to the present invention.

[0016] The information processing device 1 is connected to a user terminal 2 so as to be capable of communicating with each other via a communication network 9. The information processing device 1 may be configured as, for example, an application server (AP server), or may be configured by a general-purpose computer.

[0017] The communication network 9 is a communication line network consisting of, for example, the Internet or a mobile communication network, and may include a Local Area Network (LAN), a Wide Area Network (WAN), an intranet, Ethernet (registered trademark), and the like.

[0018] The information processing device 1 has a control unit 11, a memory unit 12, and a communication unit 13, and these components are electrically connected within the information processing device 1 so that they can send and receive (transmit) information and data to each other via a communication bus 19.

[0019] The control unit 11 performs calculation processing and control of the overall operation related to the information processing device 1. The control unit 11 may be configured, for example, by a central processing unit (CPU) equivalent to a calculation unit. The control unit 11 realizes various functions related to the information processing device 1 by reading and executing a predetermined program (including an information processing program) stored in the storage unit 12. In other words, the various functions related to the information processing device 1 can be executed as each functional unit included in the control unit 11 by specifically realizing information processing by software (including an information processing program) stored in the storage unit 12 by the control unit 11, which is an example of hardware.

[0020] The storage unit 12 may be configured with any storage medium and stores various information and data. The storage unit 12 may be configured with, for example, a storage device such as a hard disk drive (HDD) or a solid state drive (SSD) that stores various programs related to the information processing device 1 executed by the control unit 11, or a memory such as a random access memory (RAM) that stores temporarily required information and data related to the program calculations, or may be a combination of these.

[0021] The communication unit 13 has a function as a communication interface for the communication network 9 that connects the information processing device 1 to the communication network 9, and performs communication processing with external devices such as the user terminal 2 connected via the communication network 9.

[0022] The user terminal 2 is, for example, a computer used by a corporation or the like that prepares financial statements, etc., and specifically, for example, a personal computer or a mobile information terminal. Note that, although there is one user terminal 2 connected to the information processing device 1 via the communication network 9 in Fig. 1, there may be multiple user terminals 2 connected to the information processing device 1 via the communication network 9.

[0023] The user terminal 2 has an input unit 21, a display unit 22, and a communication unit 23, and these components are electrically connected within the user terminal 2 so that they can send and receive (transmit) information and data to and from each other via a communication bus 29.

[0024] The input unit 21 is a mechanism for accepting operational inputs made by a user, and may be constituted by, for example, a keyboard.

[0025] The display unit 22 is a mechanism for displaying text, images, etc., and may be configured with, for example, a liquid crystal display. The display unit 22 may be configured with a touch panel and may also function as an input unit.

[0026] The communication unit 23 has a function as a communication interface for the communication network 9 that connects the user terminal 2 to the communication network 9, and performs communication processing with external devices such as the information processing device 1 connected via the communication network 9.

[0027] The information processing device 1 acquires electronic data such as financial statements transmitted from the user terminal 2 via the communication network 9, and executes a process of generating corresponding data between items and amounts related to the financial statements, etc., based on the acquired electronic data, and a calculation process using the generated data. The information processing device 1 then transmits the results of the processing via the communication network 9 to the user terminal 2 that has transmitted the electronic data such as financial statements, etc.

[0028] The types of electronic data such as financial statements include the following A and B. A) A PDF file that is data that has been converted into PDF format from financial statements, etc., edited and created using software implemented in the user terminal 2, and that allows for the selection of text (in other words, the extraction of character information). This PDF file is referred to as "digital PDF data."

[0029] A) A PDF file is data that has been converted from financial statements, etc., edited and created using software implemented in the user terminal 2, and in which text selection (in other words, extraction of text information) is not possible. It is also a PDF file obtained by scanning financial statements, etc. (documents, documents) printed on paper. These PDF files are called "scanned PDF data."

[0030] (Processing contents) Fig. 2 is a flow diagram showing the procedure of processing executed in information processing device 1 according to the embodiment. Fig. 2 is also a flow diagram showing the processing procedure of an information processing method according to the embodiment, as an example of a specific configuration mode of the information processing method according to the present invention.

[0031] The information processing device 1 according to the embodiment includes a ruled line extraction unit 113 as a means for detecting vertical and horizontal ruled lines by performing image processing on image data in which at least one of a financial statement, a fixed asset ledger, and a trial balance has been imaged, and acquiring the coordinates of the start and end points of the detected vertical and horizontal ruled lines, a character information generation unit 114 as a means for acquiring text and the coordinates of each of the text from the computerized data in which at least one of a financial statement, a fixed asset ledger, and a trial balance has been computerized, and a virtual ruled line extraction unit 113 as a means for extracting text and the coordinates of each of the text from the computerized data in which at least one of a financial statement, a fixed asset ledger, and a trial balance has been computerized, based on the degree of presence of text along the vertical direction when no vertical ruled lines are detected from the image data. the two-dimensional array unit 115 as a means for setting vertical rules that are easily detected, and setting virtual horizontal rules based on the degree of presence of text running horizontally when no horizontal rules are detected from the imaging data, generating a two-dimensional array using the detected vertical and horizontal rules and virtual vertical and horizontal rules, and storing text in each element of the two-dimensional array based on the coordinates of each piece of text; and a subject sorting unit 116 as a means for generating a two-dimensional array in which each line contains only one subject name by dividing the lines by subject name when one line of the two-dimensional array contains multiple subject names.

[0032] The information processing method according to the embodiment includes step S3 of detecting vertical and horizontal ruled lines by performing image processing on image data of at least one of a financial statement, a fixed asset ledger, and a trial balance, and acquiring coordinates of start and end points of the detected vertical and horizontal ruled lines; step S4 of acquiring text and the coordinates of each of the text from the digitized data or image data of at least one of a financial statement, a fixed asset ledger, and a trial balance, and step S5 of determining whether or not a vertical ruled line is detected based on the degree of presence of text along the vertical direction when no vertical ruled line is detected from the image data. The method includes step S5 of setting virtual vertical rules based on the degree of presence of text along the horizontal direction when no horizontal rules are detected from the imaging data, generating a two-dimensional array from the detected vertical and horizontal rules and the virtual vertical and horizontal rules, and storing the text in each element of the two-dimensional array based on the coordinates of each piece of text; and step S6 of, when one line of the two-dimensional array contains multiple subject names, creating a two-dimensional array in which each line contains only one subject name by dividing the line by subject name.

[0033] The control unit 11 of the information processing device 1 includes an acquisition unit 111, an image generation unit 112, a ruled line extraction unit 113, a character information generation unit 114, a two-dimensional arrangement unit 115, a subject organization unit 116, a subject name correction unit 117, a normalization unit 118, an output unit 119, and a calculation unit 120. Note that "transmission" in the description of the present invention includes exchange of information and data performed via storage in the memory unit 12 and reading out from the memory unit 12.

[0034] The acquisition unit 111 acquires, via the communication unit 13, digital PDF data or scanned PDF data as electronic data of financial statements or the like transmitted from the user terminal 2 via the communication network 9 (step S1).

[0035] In addition, the electronic data of financial statements, etc. (specifically, digital PDF data or scanned PDF data) sent from the user terminal 2 may be stored in a server separate from the information processing device 1 and the user terminal 2, and the acquisition unit 111 may acquire the electronic data of financial statements, etc. from the server.

[0036] Whether the digitized data sent from the user terminal 2 is digital PDF data or scanned PDF data may be input by the user via the input unit 21 of the user terminal 2 and transmitted to the information processing device 1 together with the digitized data, and recognized by the acquisition unit 111, or the acquisition unit 111 may determine this by reading internal information of the PDF data (PDF file), for example.

[0037] The type of content of the digitized data transmitted from the user terminal 2 (in other words, the type of table among financial statements, etc.) may be input by the user via the input unit 21 of the user terminal 2 and transmitted together with the digitized data to the information processing device 1. In this case, the type of content of the digitized data (the type of table among financial statements, etc.) is linked to the digitized data (including data created from the digitized data and data converted from the digitized data) and used in subsequent processing.

[0038] The digital PDF data or scanned PDF data as digitized data transmitted from the user terminal 2 to the information processing device 1 may be one page or multiple pages.

[0039] The acquisition unit 111 transmits the digitized data (specifically, digital PDF data or scanned PDF data) acquired via the communication unit 13 to the image generation unit 112 and the text information generation unit 114, along with information as to whether the digitized data is digital PDF data or scanned PDF data.

[0040] The image generating unit 112 generates imaging data of the electronic data transmitted from the acquiring unit 111 (step S2).

[0041] The image generation unit 112 images (in other words, converts into image files) the digital PDF data or scanned PDF data as digitized data on a page-by-page basis (in other words, on a page-by-page basis), and generates the same number of imaging data (specifically, image files) as the number of pages of the digital PDF data or scanned PDF data.

[0042] The data format of the imaging data (specifically, the image file) is not limited to a specific type as long as it is a data format that can be handled in the processing of the next step S3. Examples of the data format of the imaging data (image file) include the BMP format, the JPG format, the GIF format, and the PNG format.

[0043] The image generating unit 112 transmits image data (specifically, image files) for each page of the digital PDF data or scanned PDF data as electronic data to the ruled line extracting unit 113.

[0044] The ruled line extraction unit 113 extracts ruled lines from the imaging data transmitted from the image generation unit 112 (step S3). The processes of steps S3 to S9 are performed for each page of imaging data (in other words, for each image file).

[0045] The ruled line extraction unit 113 performs image processing on the imaging data to detect and extract ruled lines, and if vertical and horizontal ruled lines are detected, it obtains the information 1 below, and if vertical ruled lines are detected but horizontal ruled lines are not, it obtains the information 2 below, and assigns this information to the imaging data for each page to be processed (specifically, an image file). If neither vertical nor horizontal ruled lines are detected, the ruled line extraction unit 113 assigns the information 3 below (i.e., information that there are no ruled lines) to the imaging data for each page to be processed (an image file). The information 1 to 3 below is referred to as "ruled line information". 1) The coordinates of the start and end points of the vertical lines and the coordinates of the start and end points of the horizontal lines 2) The coordinates of the start and end points of the vertical line 3) No lines

[0046] The ruled line extraction unit 113 does not extract lines (that is, diagonal lines) whose inclination angles with respect to the vertical direction and the horizontal direction are greater than a predetermined angle.

[0047] Fig. 3 is a diagram for explaining the start and end points of vertical ruled lines and the start and end points of horizontal ruled lines when the image data (original financial statements, etc.) includes vertical ruled lines and horizontal ruled lines. Fig. 4 is a diagram for explaining the start and end points of vertical ruled lines when the image data (original financial statements, etc.) includes vertical ruled lines but does not include horizontal ruled lines.

[0048] The image processing method used in ruled line extraction section 113 for detecting and extracting ruled lines is not limited to a specific method or procedure, but may be, for example, a Hough transform.

[0049] The coordinates of the start point and end point of the ruled line may be expressed, for example, as coordinate values ​​in pixel units in a two-dimensional orthogonal coordinate system with an arbitrary point in the image as the origin.

[0050] The ruled line extraction unit 113 transmits the ruled line information for each page of imaging data (in other words, for each image file) to the two-dimensional arrangement unit 115.

[0051] The character information generating unit 114 generates character information regarding the digitized data transmitted from the acquiring unit 111 (step S4).

[0052] <If the digitized data is digital PDF data> If the type of digitized data transmitted from the acquisition unit 111 is digital PDF data, the character information generation unit 114 reads the internal information of the digital PDF data, extracts the text and the coordinates of each text, and acquires the text of the digitized data and the coordinates of each text.

[0053] The method of extracting text from digital PDF data and obtaining text coordinates is not limited to the use of a specific method, procedure or tool, but may include, for example, the use of PyMuPDF or PDFminer, which are external Python libraries.

[0054] <If the digitized data is scanned PDF data> If the type of digitized data transmitted from the acquisition unit 111 is scanned PDF data, the character information generation unit 114 performs optical character recognition (OCR) processing on the scanned PDF data to convert the image (in other words, image) to text (in other words, character, character data) and identify the coordinates of each text, thereby acquiring the text of the digitized data and the coordinates of each text.

[0055] The conversion of scanned PDF data into text through optical character recognition processing and the determination of text coordinates may be performed using, but is not limited to, any specific method, procedure, or tool, such as Google Vision API or Google Drive API.

[0056] The coordinates of the start and end points of the ruled lines used by ruled line extraction unit 113 in the previous processing of step S3 and the coordinates of the text used by character information generation unit 114 in the processing of this step S4 are either coordinates in the same coordinate system, or are coordinates that can be converted into each other (in other words, can be associated with each other).

[0057] The text and the coordinates of each piece of text grasped by the process in step S4 are referred to as "character information."

[0058] The character information generating unit 114 transmits the character information for each page of imaging data (in other words, for each image file) to the two-dimensional arrangement unit 115.

[0059] The two-dimensional array unit 115 generates two-dimensional array data whose elements are sequences of characters (specifically, subjects) and sequences of numbers (specifically, amounts) using the line information transmitted from the line extraction unit 113 and the character information transmitted from the character information generation unit 114 (step S5).

[0060] <When the rule information is the coordinates of the start and end points of vertical and horizontal ruled lines> When the coordinates of the start and end points of vertical ruled lines and the coordinates of the start and end points of horizontal ruled lines are acquired as ruled line information (see FIG. 3), the two-dimensional array unit 115 generates a two-dimensional array based on the coordinates of the start and end points of vertical ruled lines and the coordinates of the start and end points of horizontal ruled lines in the ruled line information. In other words, the two-dimensional array unit 115 generates a two-dimensional array by reproducing the vertical ruled lines and horizontal ruled lines in the original financial statements, etc., using the ruled line information.

[0061] Specifically, a two-dimensional array is a table frame in which vertical and horizontal lines divide the table into a grid, generating squares that are arranged vertically (in other words, in the column direction) and horizontally (in other words, in the row direction). Each square in the table frame as a two-dimensional array is also called an "element of the two-dimensional array."

[0062] When vertical and horizontal lines are reproduced based on the coordinates of the start and end points of the vertical lines and the coordinates of the start and end points of the horizontal lines in the line information, the squares formed by the vertical and horizontal lines dividing the lines into rectangles are incorporated into the two-dimensional array, while the parts that are not divided into rectangles by the vertical and horizontal lines (in other words, no squares are formed) are not incorporated into the two-dimensional array.

[0063] Each element of the two-dimensional array has coordinate information corresponding to the region (in other words, range) of each element based on the coordinates of the start and end points of the vertical ruled lines and the start and end points of the horizontal ruled lines used when the two-dimensional array was generated. This makes it possible to determine which element in the two-dimensional array a certain coordinate is located within (in other words, within the range), and to identify the row and column of the element in which the certain coordinate is located.

[0064] The two-dimensional array unit 115 further stores each piece of text in the generated two-dimensional array in the element (i.e., each square of the table frame) in which the coordinates of the text are located, based on the text in the character information and the coordinates of each piece of text. That is, the two-dimensional array unit 115 compares the coordinate information of each element of the generated two-dimensional array with the coordinates of each piece of text to identify the row and column of the element containing each piece of text.

[0065] The sequence of texts stored in each element of the two-dimensional array by the above process and sorted in order based on the coordinates of each text constitutes one account name or one amount.

[0066] <When the line information is the coordinates of the start and end points of a vertical line only> When the coordinates of the start and end points of only vertical ruled lines are acquired as the ruled line information (see FIG. 4), the two-dimensional array unit 115 virtually sets horizontal ruled lines based on the degree of presence of text along the horizontal direction (in other words, the row direction). In other words, the two-dimensional array unit 115 generates a two-dimensional array by reproducing the vertical ruled lines in the original financial statement or the like using the ruled line information and setting virtual horizontal ruled lines using character information.

[0067] In setting virtual horizontal rules, two-dimensional array section 115 first refers to the coordinates of each text in the character information, and assigns a score (called a "character presence score") to a predetermined range centered on the relevant coordinates.

[0068] The range for assigning a character presence score may be, for example, a square with a side length of approximately 0.5 to 2 times the vertical dimension of typical, average text (letters, numbers) expected in the financial statements etc. being processed, as shown in Figure 5, or a circle with a radius of approximately 0.5 to 2 times.

[0069] The two-dimensional array unit 115 then calculates the sum of the character presence scores along the horizontal direction. This generates a distribution of the sum of the character presence scores along the horizontal direction, which increases and decreases in the vertical direction. The distribution of the sum of the character presence scores may be generated as a distribution curve, or may be generated as a histogram with a predetermined range of classes.

[0070] Fig. 6 is a diagram for explaining how to set virtual horizontal lines. In Fig. 6, the distribution of the total value of the character presence score along the horizontal direction is generated as a distribution curve.

[0071] Next, the two-dimensional array unit 115 virtually sets horizontal rules at positions where the sum of the character presence scores along the horizontal direction is less than a predetermined threshold (referred to as a "character presence threshold").

[0072] Specifically, first, the uppermost position in the vertical direction among positions where the sum of the character presence scores along the horizontal direction is less than the character presence threshold is determined as the position of the top imaginary horizontal ruled line.

[0073] Next, the center position in the vertical direction of the range where the distribution of the sum of the character presence scores along the horizontal direction is less than the character presence threshold is determined as the position of the virtual horizontal ruled line. Alternatively, the position where the sum of the character presence scores along the horizontal direction is the smallest in the range where the distribution of the sum of the character presence scores along the horizontal direction is less than the character presence threshold is determined as the position of the virtual horizontal ruled line.

[0074] Finally, the lowest position in the vertical direction among the positions where the sum of the character presence scores along the horizontal direction is less than the character presence threshold is determined as the position of the bottom imaginary horizontal ruled line.

[0075] The character presence threshold is set to an appropriate value, for example, by taking into consideration the total character presence score that corresponds to (in other words, the expected total character presence score) when there are no or almost no letters or numbers lined up horizontally, or by taking into consideration the total character presence score that corresponds to (in other words, the expected total character presence score) when there are meaningful sequences of letters and numbers in financial statements, etc.

[0076] The virtual horizontal ruled line is assumed to exist between the leftmost vertical ruled line and the rightmost vertical ruled line in the horizontal direction, which exist at the determined position of the virtual horizontal ruled line (note that the vertical ruled line is based on the ruled line information). This identifies the coordinates of the start point and end point of the virtual horizontal ruled line. Note that if no vertical ruled line based on the ruled line information exists at the determined position of the virtual horizontal ruled line, the virtual horizontal ruled line at that position is excluded as it is not a ruled line that constitutes the table frame.

[0077] The two-dimensional array unit 115 then generates a two-dimensional array based on the coordinates of the start and end points of the vertical ruled lines in the ruled line information and the coordinates of the start and end points of the virtually set horizontal ruled lines.

[0078] When vertical and horizontal ruled lines are reproduced and set based on the coordinates of the start and end points of the vertical ruled lines in the ruled line information and the coordinates of the start and end points of the virtually set horizontal ruled lines, the squares formed by the vertical and horizontal ruled lines dividing the lines into rectangles are incorporated into the two-dimensional array, while the parts that are not divided into rectangles by the vertical and horizontal ruled lines (in other words, no squares are formed) are not incorporated into the two-dimensional array.

[0079] Each element of the two-dimensional array has coordinate information corresponding to the region (in other words, range) of each element based on the coordinates of the start and end points of the vertical ruled lines and the start and end points of the horizontal ruled lines used when the two-dimensional array was generated. This makes it possible to determine which element in the two-dimensional array a certain coordinate is located within (in other words, within the range), and to identify the row and column of the element in which the certain coordinate is located.

[0080] The two-dimensional array unit 115 further stores each piece of text in the generated two-dimensional array in the element (i.e., each square of the table frame) in which the coordinates of the text are located, based on the text in the character information and the coordinates of each piece of text. That is, the two-dimensional array unit 115 compares the coordinate information of each element of the generated two-dimensional array with the coordinates of each piece of text to identify the row and column of the element containing each piece of text.

[0081] The sequence of texts stored in each element of the two-dimensional array by the above process and sorted in order based on the coordinates of each text constitutes one account name or one amount.

[0082] <If the border information is no border> If the ruled line information indicates that there are no ruled lines, the two-dimensional array unit 115 virtually sets vertical ruled lines based on the degree of text presence along the vertical direction (in other words, the column direction), and virtually sets horizontal ruled lines based on the degree of text presence along the horizontal direction (in other words, the row direction). In other words, the two-dimensional array unit 115 generates a two-dimensional array by setting virtual vertical and horizontal ruled lines using character information.

[0083] In setting virtual vertical and horizontal ruled lines, two-dimensional array unit 115 first refers to the coordinates of each text in the character information, and assigns a score (that is, character presence score) to a predetermined range centered on the relevant coordinates.

[0084] The range for giving the character presence score may be, for example, a square with a side length of about 0.5 to 2 times the vertical dimension of general, average text (letters, numbers) expected in the financial statements, etc. to be processed, or may be a circle with a radius of about 0.5 to 2 times, as shown in Fig. 5. The range for giving the character presence score may also be a square with a side length of about 0.5 to 2 times the average value of the vertical dimension and horizontal dimension of general, average text (letters, numbers) expected in the financial statements, etc. to be processed, or may be a circle with a radius of about 0.5 to 2 times.

[0085] The two-dimensional array unit 115 then calculates the sum of the character presence scores along the vertical direction. This generates a distribution of the sum of the character presence scores along the vertical direction, which increases and decreases in the horizontal direction. The distribution of the sum of the character presence scores along the vertical direction may be generated as a distribution curve, or may be generated as a histogram with a predetermined range of classes.

[0086] The two-dimensional array unit 115 also calculates the sum of the character presence scores along the horizontal direction. This generates a distribution of the sum of the character presence scores along the horizontal direction, which increases and decreases in the vertical direction. The distribution of the sum of the character presence scores along the horizontal direction may be generated as a distribution curve, or may be generated as a histogram with a predetermined range of classes.

[0087] Fig. 7 is a diagram for explaining how to set virtual vertical and horizontal ruled lines. In Fig. 7, the distribution of the total value of the character presence score along the vertical direction and the distribution of the total value of the character presence score along the horizontal direction are generated as distribution curves.

[0088] Next, the two-dimensional array unit 115 virtually sets vertical rules at positions where the sum of the character presence scores along the vertical direction is less than a predetermined threshold (referred to as the "vertical character presence threshold").

[0089] Specifically, first, the leftmost position in the horizontal direction among positions where the sum of the character presence scores along the vertical direction is less than the vertical character presence threshold is determined as the position of the leftmost imaginary vertical ruled line.

[0090] Next, the center position in the horizontal direction of the range where the distribution of the sum of the character presence scores along the vertical direction is less than the vertical character presence threshold is determined as the position of the virtual vertical ruled line, or the position where the sum of the character presence scores along the vertical direction is the smallest in the range where the distribution of the sum of the character presence scores along the vertical direction is less than the vertical character presence threshold is determined as the position of the virtual vertical ruled line.

[0091] Finally, the rightmost position in the horizontal direction among the positions where the sum of the character presence scores along the vertical direction is less than the vertical character presence threshold is determined as the position of the rightmost imaginary vertical ruled line.

[0092] The vertical character presence threshold is set to an appropriate value, for example, taking into consideration the total character presence score that corresponds to (in other words, is expected to) the case where there are no or almost no letters or numbers lined up vertically.

[0093] The two-dimensional arrangement unit 115 also virtually sets horizontal rules at positions where the sum of the character presence scores along the horizontal direction is less than a predetermined threshold (referred to as the "horizontal character presence threshold").

[0094] Specifically, first, the uppermost position in the vertical direction among positions where the sum of the character presence scores along the horizontal direction is less than the horizontal character presence threshold is determined as the position of the top imaginary horizontal ruled line.

[0095] Next, the center position in the vertical direction of the range where the distribution of the sum of the character presence scores along the horizontal direction is less than the horizontal character presence threshold is determined as the position of the virtual horizontal ruled line. Alternatively, the position where the sum of the character presence scores along the horizontal direction is the smallest in the range where the distribution of the sum of the character presence scores along the horizontal direction is less than the horizontal character presence threshold is determined as the position of the virtual horizontal ruled line.

[0096] Finally, the lowest position in the vertical direction among the positions where the sum of the character presence scores along the horizontal direction is less than the horizontal character presence threshold is determined as the position of the bottom imaginary horizontal ruled line.

[0097] The horizontal character presence threshold is appropriately set to an appropriate value, for example, taking into consideration the total value of character presence scores corresponding to a case where there are no or almost no characters or numbers lined up horizontally (in other words, expected), or the total value of character presence scores corresponding to a sequence of characters and numbers that are meaningful in financial statements, etc. The vertical character presence threshold and the horizontal character presence threshold may be set to the same value, or may be set to different values.

[0098] The virtual horizontal ruled line is assumed to exist between the leftmost virtual vertical ruled line and the rightmost virtual vertical ruled line in the horizontal direction, which exist at the determined position of the virtual horizontal ruled line. Also, the virtual vertical ruled line is assumed to exist between the topmost virtual horizontal ruled line and the bottommost virtual horizontal ruled line in the vertical direction, which exist at the determined position of the virtual vertical ruled line. This allows the coordinates of the start and end points of the virtual horizontal ruled line to be specified, and also the coordinates of the start and end points of the virtual vertical ruled line to be specified.

[0099] Next, the two-dimensional array unit 115 generates a two-dimensional array based on the coordinates of the start and end points of the virtually set vertical ruled lines and the coordinates of the start and end points of the virtually set horizontal ruled lines.

[0100] When vertical and horizontal ruled lines are set based on the coordinates of the start and end points of the virtually set vertical ruled lines and the coordinates of the start and end points of the virtually set horizontal ruled lines, the squares formed by the vertical and horizontal ruled lines dividing the lines into rectangles are incorporated into the two-dimensional array, while the parts that are not divided into rectangles by the vertical and horizontal ruled lines (in other words, no squares are formed) are not incorporated into the two-dimensional array.

[0101] Each element of the two-dimensional array has coordinate information corresponding to the region (in other words, range) of each element based on the coordinates of the start and end points of the vertical ruled lines and the start and end points of the horizontal ruled lines used when the two-dimensional array was generated. This makes it possible to determine which element in the two-dimensional array a certain coordinate is located within (in other words, within the range), and to identify the row and column of the element in which the certain coordinate is located.

[0102] The two-dimensional array unit 115 further stores each piece of text in the generated two-dimensional array in the element (i.e., each square of the table frame) in which the coordinates of the text are located, based on the text in the character information and the coordinates of each piece of text. That is, the two-dimensional array unit 115 compares the coordinate information of each element of the generated two-dimensional array with the coordinates of each piece of text to identify the row and column of the element containing each piece of text.

[0103] The sequence of texts stored in each element of the two-dimensional array by the above process and sorted in order based on the coordinates of each text constitutes one account name or one amount.

[0104] The above-mentioned processing by the two-dimensional array unit 115 generates two-dimensional array data in which a sequence of characters (which constitutes the account name) or a sequence of numbers (which constitutes the amount) is stored in each element.

[0105] The two-dimensional arrangement unit 115 transmits the two-dimensional arrangement data for each page of image data (in other words, for each image file) to the subject arrangement unit 116.

[0106] The subject organizing unit 116 generates one-row, one-subject data using the two-dimensional array data transmitted from the two-dimensional array unit 115 (step S6).

[0107] Depending on the type of financial statement, the account names may exist in multiple columns of a two-dimensional array, in other words, multiple account names may be included in one row, as shown in Figure 8. In the example shown in Figure 8, for example, the account names exist in the first and third columns of the two-dimensional array, and the first row contains two account names, "Account Name 1A" and "Account Name 1B."

[0108] The subject sorting unit 116 checks the contents of the two-dimensional array data transmitted from the two-dimensional array unit 115, and generates data in which each row contains only one subject name (referred to as "one-subject-per-row data"). In other words, the subject sorting unit 116 generates, from the two-dimensional array data transmitted from the two-dimensional array unit 115, two-dimensional array data in which certain columns contain subject names and other columns contain items other than subject names (for example, a sequence of numbers representing amounts).

[0109] The subject organizing unit 116, for example, determines whether the sequence of text stored in each element of the two-dimensional array is a sequence of numbers, and determines for each row whether the row contains multiple subject names.

[0110] When multiple subject names are included in one row of a two-dimensional array, the subject sorting unit 116 separates the rows by subject name and generates a row for each subject name, thereby generating one-per-row subject data in which each row contains only one subject name. When multiple subject names are included in one row, the subject sorting unit 116 combines the subject name on the left side and the number on the right side included in the same row. At this time, if there is no number to the right of the subject name, there is no number corresponding to the subject name (empty: null).

[0111] The subject organizing unit 116 also identifies, for each row of the two-dimensional array, a column in which the subject name is stored (referred to as a "subject column").

[0112] The subject organizing unit 116 transmits the subject data, one line per page of image data (in other words, for each image file), to the subject name correcting unit 117 together with information on the subject column for each line.

[0113] The subject name correcting unit 117 corrects, as necessary, the subject name stored in each element of the subject column of the one-row-one-subject data transmitted from the subject organizing unit 116 (step S7).

[0114] The subject names stored in each element of the subject column of the one-row-one-subject data are based on the text extracted and converted in the processing of step S4, and it is possible that characters may be missing or converted incorrectly. Also, there may be variations in notation, typos, and omissions in the original financial statements, etc.

[0115] For this reason, the subject name correction unit 117 compares the subject name (referred to as the "original subject name") stored in each element of the subject column of the one-row-one-subject data transmitted from the subject sorting unit 116 with the subject name definition table, and corrects the original subject name so that it matches the subject name defined in the subject name definition table.

[0116] The account name definition table is a list of account names to be used in financial statements, etc., and is a list in which correct account names are defined and listed in order to correct incomplete account names that include typos, omissions, variations in notation, etc. into complete and unified account names to be used in financial statements, etc. The account names defined and listed in the account name definition table are called "defined account names."

[0117] The subject name correction unit 117 compares the original subject name with each of the defined subject names listed in the subject name definition table, and if there is a defined subject name that matches the original subject name, it keeps the original subject name as it is.On the other hand, if there is no defined subject name that matches the original subject name, it calculates the similarity between the original subject name and each defined subject name as a character string, and corrects the original subject name to the defined subject name with the highest similarity.

[0118] The method for calculating the similarity between the original subject name and the defined subject name as character strings is not limited to a specific method or procedure, but examples thereof include Gestalt pattern matching, the Levenshtein distance method, and the Jaro-Winkler distance method.

[0119] The subject name correcting unit 117 may further convert the Japanese calendar notation into the Western calendar notation.

[0120] The subject name correcting unit 117 transmits to the normalizing unit 118 one-line-one-subject data in which the subject name has been unified to the defined subject name for each page of image data (in other words, for each image file).

[0121] The normalization unit 118 deletes unnecessary rows and columns from the one-row, one-subject data transmitted from the subject name correction unit 117 (step S8).

[0122] First, the normalization unit 118 deletes rows in which all elements are empty (null).

[0123] The normalization unit 118 also compares the subject names (specifically, defined subject names) stored in each element of the subject column of the one-row-one-subject data with the normalization definition table, and keeps rows containing subject names defined as necessary subject names in the normalization definition table, while deleting rows containing subject names defined as unnecessary subject names. Note that when the subject names (specifically, defined subject names) stored in each element of the subject column of the one-row-one-subject data do not correspond to either the necessary subject names or the unnecessary subject names in the normalization definition table, the user may be asked to determine whether the subject name is a necessary subject name or an unnecessary subject name.

[0124] The normalization definition table is a list that defines and lists necessary subject names (in other words, subject names to be kept) and unnecessary subject names (in other words, subject names to be deleted) in order to ultimately create one subject data per row containing only the data that is ultimately required. The normalization definition table is a list in which, for example, subject names that are required in the processing of the later step S10 are listed as necessary subject names, and subject names that are not required in the processing of step S10 are listed as unnecessary subject names. The subject names listed in the normalization definition table as necessary and unnecessary are referred to as "normalization subject names."

[0125] The normalization definition table may be created as a list in which only the subject names required in the subsequent processing in step S10 are listed (that is, the normalized subject names are made up of only the necessary subject names).

[0126] The normalization definition table may be created as a list in which normalization account names are listed for each type of content (in other words, type of financial statements, etc.) of digitized data (here, one-row-one-item data created from digitized data).In this case, when the account names (specifically, defined account names) stored in each element of the item column of the one-row-one-item data are compared with the normalization account names listed in the normalization definition table, the normalization definition table corresponding to the type of content of the original digitized data of the one-row-one-item data is used.

[0127] The normalization unit 118 also deletes the following columns A to C (in other words, elements) for each row of the one-row-one-subject data. A) Empty (null) columns (elements) to the left of the Subject column (element) A) An empty (null) column (element) between the subject column (element) and the column (element) in which numbers are stored C) Empty (null) columns (elements) to the right of columns (elements) that contain numbers

[0128] The above process generates two-dimensional array data (called "final two-dimensional array data") that contains only the subject names that are ultimately required, and in which one of the two columns is the subject name (i.e., the subject column) and the other column is numbers.

[0129] The normalization unit 118 transmits the final two-dimensional array data for each page of imaging data (in other words, for each image file) to the output unit 119.

[0130] The output unit 119 outputs the final two-dimensional array data transmitted from the normalization unit 118 (step S9).

[0131] The output unit 119 outputs the final two-dimensional array data to the outside as a file in at least one of the following formats: TSV format, CSV format, and text format.

[0132] The output of the final two-dimensional array data by the output unit 119 is intended to enable use of the final two-dimensional array data other than the information processing device 1, and if use of the final two-dimensional array data other than the information processing device 1 is not planned or there is no need to consider such use, the final two-dimensional array data does not need to be output to the outside.

[0133] The output unit 119 also transmits the final two-dimensional array data transmitted from the normalization unit 118 to the calculation unit 120 .

[0134] The calculation unit 120 performs various calculations related to financial statements and the like using the final two-dimensional array data (specifically, for example, a TSV format file) transmitted from the output unit 119 (step S10).

[0135] The content of the calculation process in the calculation unit 120 is not limited to a specific type of calculation, and various calculations such as financial statements that can be performed using the final two-dimensional array data may be performed.

[0136] The calculation unit 120 may, for example, calculate and create at least one of a balance sheet (B / S), a profit and loss statement (P / L), a cash flow statement (C / F), a calculation of the depreciation shortfall in fixed assets, and an actual balance sheet (B / S) as an analysis of the current situation, and may also calculate and create at least one of a three-statement (i.e., B / S, P / L, C / F) plan, a capital investment plan, a calculation of the estimated depreciation amount, a loan repayment plan, a cash flow forecast, and an actual net asset trend as planning.

[0137] (Action and effect) According to the information processing device 1, information processing method, and information processing program of the embodiment, a two-dimensional array is generated using electronic data such as financial statements, and item names and amounts are automatically stored in the elements of the two-dimensional array, thereby making it possible to reduce the labor and speed up the work of reading and organizing the items and amounts listed in financial statements, etc.

[0138] The above describes an embodiment of the present invention, but the specific configuration of the present invention is not limited to the above embodiment, and the present invention also includes embodiments in which modifications and changes are made to the above embodiment without departing from the gist of the present invention.

[0139] For example, in the above embodiment, a predetermined program (including an information processing program) is executed in an information processing device 1 that is connected to a user terminal 2 so as to be able to communicate with each other via a communication network 9, and the processing of steps S1 to S10 is performed. However, a predetermined program (including an information processing program) may also be executed in the user terminal 2, and the processing of steps S1 to S10 may also be performed.

[0140] In addition, in the above embodiment, scanned PDF data (PDF file) is used as one type of electronic data of financial statements, etc., but image data (image file) of financial statements, etc. may be used as electronic data of financial statements, etc. In this case, the data format of the image data (image file) of financial statements, etc. may be, for example, BMP format, JPG format, GIF format, or PNG format. In this case, the process of step S2 is omitted (and the image generating unit 112 of the information processing device 1 is omitted), and the image data (image file) of financial statements, etc. is acquired in the process of step S1, and then the processes from step S3 onwards are performed.

[0141] In addition, the above embodiment describes the case where the coordinates of the start and end points of only vertical ruled lines are acquired as ruled line information, and does not describe the case where the coordinates of the start and end points of only horizontal ruled lines are acquired as ruled line information. However, when the coordinates of the start and end points of only horizontal ruled lines are acquired as ruled line information, vertical ruled lines may be virtually set by performing processing to swap vertical and horizontal in the description of the case where the coordinates of the start and end points of only vertical ruled lines are acquired as the above ruled line information. [Explanation of symbols]

[0142] 1. Information processing device 11 Control section 111 Acquisition Department 112 Image Generation Unit 113 Ruled line extraction section 114 Character information generation section 115 Two-dimensional array section 116 Subject Sorting Department 117 Subject name correction section 118 Normalization section 119 Output section 120 Calculation Department 12 Storage section 13. Communications Department 19 Communication Bus 2. User terminal 21 Input section 22 Display section 23 Communications Department 29 Communication Bus 9. Communication Networks

Claims

1. a means for performing image processing on image data of at least one of a financial statement, a fixed asset ledger, and a trial balance, to detect vertical and horizontal ruled lines, and to obtain coordinates of start and end points of the detected vertical and horizontal ruled lines; means for acquiring text and coordinates of each of the texts from the electronic data or the image data of at least one of the financial statements, the fixed asset ledger, and the trial balance; means for setting virtual vertical ruled lines based on the extent of the presence of the text along the vertical direction when no vertical ruled lines are detected from the imaging data, and for setting virtual horizontal ruled lines based on the extent of the presence of the text along the horizontal direction when no horizontal ruled lines are detected from the imaging data, generating a two-dimensional array using the detected vertical ruled lines and horizontal ruled lines and the virtual vertical ruled lines and virtual horizontal ruled lines, and storing the text in each element of the two-dimensional array based on the coordinates of each piece of the text.

23. An information processing apparatus comprising:

2. The method further comprises means for generating a two-dimensional array in which, when a plurality of subject names are included in one row of the two-dimensional array, each row is divided into rows for each subject name, and each row contains only one subject name.

2. The information processing apparatus according to claim 1,

3. An information processing method executed by an information processing device, a step of performing image processing on image data of at least one of the financial statements, the fixed asset ledger, and the trial balance to detect vertical and horizontal ruled lines and obtain coordinates of start and end points of the detected vertical and horizontal ruled lines; acquiring text and coordinates of each of the texts from the electronic data or the image data of at least one of the financial statements, the fixed asset register, and the trial balance; setting virtual vertical ruled lines based on the extent of the presence of the text along the vertical direction when no vertical ruled lines are detected from the imaging data, and setting virtual horizontal ruled lines based on the extent of the presence of the text along the horizontal direction when no horizontal ruled lines are detected from the imaging data, generating a two-dimensional array based on the detected vertical ruled lines and horizontal ruled lines and the virtual vertical ruled lines and virtual horizontal ruled lines, and storing the text in each element of the two-dimensional array based on the coordinates of each piece of the text.

23. An information processing method comprising:

4. a step of generating a two-dimensional array in which each row contains only one subject name by dividing the row by the subject name when each row of the two-dimensional array contains multiple subject names; 4. The information processing method according to claim 3.

5. a process of performing image processing on image data of at least one of a financial statement, a fixed asset ledger, and a trial balance, to detect vertical and horizontal ruled lines, and to obtain coordinates of start and end points of the detected vertical and horizontal ruled lines; A process of acquiring text and coordinates of each of the texts from the digitized data or the image data of at least one of the financial statements, the fixed asset ledger, and the trial balance; setting virtual vertical ruled lines based on the degree of presence of the text along the vertical direction when no vertical ruled lines are detected from the imaging data, and setting virtual horizontal ruled lines based on the degree of presence of the text along the horizontal direction when no horizontal ruled lines are detected from the imaging data, generating a two-dimensional array based on the detected vertical ruled lines and horizontal ruled lines and the virtual vertical ruled lines and virtual horizontal ruled lines, and storing the text in each element of the two-dimensional array based on the coordinates of each piece of the text.

2. An information processing program comprising:

6. and when a single row of the two-dimensional array contains multiple subject names, the computer is further caused to execute a process of dividing the rows by subject names and generating a two-dimensional array in which each row contains only one subject name.

6. The information processing program according to claim 5,

Images