Intelligence comparison device, intelligence comparison method, memory medium

Abstract

An information comparison device and the like are provided that can efficiently extract information to be compared from a document to be confirmed. [Solution] The information comparison device 1 comprises a prompt input unit 12 that inputs a prompt P to a generative artificial intelligence (generative AI) 10 to specify verification target items that may be included in a verification target text S and cause the generative AI to extract first comparison target information I1 related to the verification target items from the verification target text S, and an information comparison unit 15 that compares the first comparison target information I1 with second comparison target information I2 related to the verification target items extracted from a verification target table T that is different from the verification target text S. The information extraction unit 14 extracts second comparison target information I2 related to the verification target items that may be included in the verification target table T from the verification target table T expressed in a predetermined format, based on the format.

Description

[Technical Field]

[0001] The present disclosure relates to an information comparison device and the like. [Background technology]

[0002] Patent Document 1 discloses a financial analysis system that extracts factors from financial statement-related information. Specifically, it discloses that a sentence included in the financial statement-related information is broken down, and each sentence is subjected to morphological analysis, and factors identified through comparison with pre-stored factor patterns are linked to the subject and amount information included in each sentence. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-120284 Summary of the Invention [Problem to be solved by the invention]

[0004] In financial statement-related information such as that disclosed in Patent Document 1, information such as the amounts of account items included in financial statements may be written in the same or different formats (e.g., text and tables) in different locations across a vast amount of documents. Since strict accuracy is required for monetary information in financial statement-related information, it is particularly important to ensure consistency of amounts across the vast amount of financial statement-related information. Traditionally, ensuring such consistency has relied on manual confirmation by auditing firms and other entities responsible for auditing financial statement-related information, which is inefficient and prone to human error.

[0005] The present disclosure has been made in consideration of these circumstances, and aims to provide an information comparison device and the like that can efficiently extract information to be compared from a document to be verified. [Means for solving the problem]

[0006] In order to solve the above problem, an information comparison device of one embodiment of the present disclosure includes a prompt input unit that inputs a prompt to a generative artificial intelligence (generative AI) to specify items to be confirmed that may be included in a text to be confirmed and to extract first comparison information related to the items to be confirmed from the text to be confirmed, and an information comparison unit that compares the first comparison information with second comparison information related to the items to be confirmed extracted from a part to be confirmed that is different from the text to be confirmed.

[0007] In this embodiment, first comparison information (e.g., the amount of an account item) related to a desired item to be confirmed (e.g., an account item) can be efficiently extracted from the confirmation target text by a prompt input to the generation AI. Note that second comparison information to be compared with the first comparison information may be extracted from another text by a similar prompt input to the generation AI, or may be extracted from another confirmation target portion such as a table by any other method.

[0008] Another aspect of the present disclosure is an information comparison method that includes inputting a prompt to a generation AI to specify a verification target item that may be included in a verification target sentence and extract first comparison target information related to the verification target item from the verification target sentence, and comparing the first comparison target information with second comparison target information related to the verification target item extracted from a verification target portion different from the verification target sentence.

[0009] Yet another aspect of the present disclosure is a storage medium storing an information comparison program that causes a computer to input a prompt to a generation AI to specify a verification target item that may be included in the verification target sentence and extract first comparison target information related to the verification target item from the verification target sentence, and compare the first comparison target information with second comparison target information related to the verification target item extracted from a verification target portion different from the verification target sentence.

[0010] Any combination of the above components, or any conversion of these expressions into methods, devices, systems, recording media, computer programs, etc., are also encompassed within the present disclosure. [Effects of the Invention]

[0011] According to the present disclosure, information to be compared can be efficiently extracted from the text to be verified. [Brief explanation of the drawings]

[0012] [Figure 1] 1 shows a schematic overview of this embodiment. [Figure 2] FIG. 2 is a schematic functional block diagram of an information comparison device. [Figure 3] 10 is a flowchart schematically illustrating a specific example of a processing flow relating to generation or extraction of first comparison information. [Figure 4] An example of a consistency check rule is shown below. [Figure 5] The following is an example of a sentence specification rule. [Figure 6] Here is an example of a chapter extracted from the first document: [Figure 7] An example will be shown in which the text specifying unit extracts a paragraph from a chapter that includes a desired account title, accounting period, and some amount information. [Figure 8] 10 is a flowchart schematically illustrating an example of a specific processing flow regarding generation of second comparison information and comparison with first comparison information. [Figure 9] Here is an example of a table extracted from the second document: [Figure 10] 10A and 10B are diagrams illustrating typical examples of screens displayed on the display of a user device. [Figure 11] 10A and 10B are diagrams illustrating typical examples of screens displayed on the display of a user device. [Figure 12] 10 is a schematic diagram showing a modified example of an information comparison device that compares documents created at different times. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments for carrying out the present disclosure (hereinafter also referred to as embodiments) will be described in detail with reference to the drawings. In the description and / or drawings, identical or equivalent components, members, processes, etc. are designated by the same reference numerals, and redundant description will be omitted. The scale and shape of each part shown in the drawings are set for convenience to simplify the description and should not be construed as limiting unless otherwise specified. The embodiments are merely examples and do not limit the scope of the present disclosure in any way. Not all features and combinations thereof presented in the embodiments are necessarily essential to the present disclosure. For convenience, the embodiments are presented by breaking them down into components for each function and / or functional group that realize them. However, one component in an embodiment may actually be realized by a combination of multiple separate components, or multiple components in an embodiment may actually be realized by a single integrated component. Furthermore, although multiple embodiments and variants may be disclosed in parallel, any components of each embodiment and / or each variant may be combined in any manner as long as they do not interfere with each other's functions.

[0014] 1 shows a schematic overview of this embodiment. An information comparison device 1 according to this embodiment compares information relating to items to be confirmed that are commonly included in a first document D1 and a second document D2. Specifically, the information comparison device 1 compares first comparison information I1 extracted from the first document D1 with second comparison information I2 extracted from the second document D2.

[0015] In the example of Figure 1, first document D1 and second document D2 are documents subject to audits such as accounting audits. Furthermore, examples of items to be confirmed that are commonly included in first document D1 and second document D2 as documents subject to accounting audits include account items such as "Net assets (total)" listed in financial statements, and examples of the related first comparison information I1 and second comparison information I2 include the amount of the account item "Net assets."

[0016] In this case, the first comparison information I1 extracted from the first document D1 for the account item "Net assets" is information in the format of, for example, "Net assets amount (FY20XX): YYY yen," and the second comparison information I2 extracted from the second document D2 for the account item "Net assets" is information in the format of, for example, "Net assets amount (FY20XX): ZZZ yen." If the first document D1 and the second document D2 are audited documents for the same accounting period of the same company, the net assets amount "YYY yen" extracted as the first comparison information I1 and the net assets amount "ZZZ yen" extracted as the second comparison information I2 should match each other.

[0017] The information comparison device 1 according to this embodiment compares numerical information that should match and efficiently checks the consistency between them. For example, if the net asset value "YYY yen" in the first comparison information I1 and the net asset value "ZZZ yen" in the second comparison information I2 do not substantially match, the information comparison device 1 presents or notifies the accounting auditor or the like to that effect and urges them to take notice or make corrections.

[0018] The information comparison device 1 according to this embodiment may compare non-numerical information such as text information included in the first comparison information I1 and the second comparison information I2. For example, the information comparison device 1 according to this embodiment may check the consistency of text information that indicates the trend, attribute, state, status, etc. of an account item (item to be confirmed), such as "Net asset value has increased compared to the previous year," "Net asset value has decreased compared to the previous year," or "Net asset value has remained the same since the previous year," which is included in both the first comparison information I1 and the second comparison information I2.

[0019] Furthermore, the information comparison device 1 according to this embodiment is not limited to determining whether the first comparison information I1 and the second comparison information I2 match, but may also determine the logical consistency that should exist between the first comparison information I1 and the second comparison information I2. For example, as described below, if the first comparison information I1 extracted from the verification target document S by the generation AI 10 is "net assets have increased compared to the previous year," but the "net assets for the previous year" and "net assets for the current year" extracted from the verification target table T by the information extraction unit 14 are the same (no change), then the two pieces of information I1 and I2 are logically inconsistent, and at least one of them may be incorrect. This information is preferably presented or notified to an accounting auditor or other relevant person for warning or correction.

[0020] In the information comparison device 1 according to this embodiment, the extraction of the first comparison information I1 from the first document D1 and the extraction of the second comparison information I2 from the second document D2 may be performed by different methods. For example, the first comparison information I1 is extracted by the generation AI 10 from a sentence (confirmation target sentence) S included in the first document D1. The generation AI 10 analyzes the confirmation target sentence S in response to a prompt P, such as "What is the net asset value at the end of fiscal year 20XX?", input by the prompt input unit 12 (described later), and extracts the first comparison information I1, such as the net asset value "YYY yen," from the confirmation target sentence S.

[0021] Meanwhile, the second comparison information I2 is extracted by the information extraction unit 14 from a table (confirmation target table or confirmation target portion) T included in the second document D2. Because the confirmation target table T included in the second document D2 as a document to be audited conforms to a predetermined format, it is relatively easy to identify each account item, such as "Net Assets (Total)," that is a confirmation target item in the confirmation target table T. Therefore, an advanced artificial intelligence system based on machine learning or deep learning, such as the generation AI 10 described above, is not required to extract the second comparison information I2, such as the net asset amount "ZZZ yen," from the confirmation target table T. The format of the confirmation target table T included in the second document D2 as a document to be audited is typically determined according to the accounting standards adopted by the audited company. The information comparison device 1 according to this embodiment is applicable regardless of accounting standards, such as J-GAAP, US-GAAP, and IFRS.

[0022] The information extraction unit 14 may extract second comparison information I2, such as the net asset value "ZZZ yen," from the confirmation target table T based on an information extraction rule R manually created by an accounting auditor or the like (e.g., "The net asset value for the current fiscal year (20XX fiscal year) is in the nth cell to the right of the cell containing "Net Assets""). In this manner, the information extraction unit 14 may be implemented by a rule-based computer program or appropriate hardware. Note that the confirmation target from which the information extraction unit 14 extracts the second comparison information I2 is not limited to a table (confirmation target table T) and may have any format or description method as long as it is expressed in accordance with a predetermined format that allows for routine processing based on the given rule R.

[0023] In the example of Figure 1 (and the following examples), the extraction of the first comparison information I1 from the first document D1 is performed by the machine learning-based generation AI 10, and the extraction of the second comparison information I2 from the second document D2 is performed by the rule-based information extraction unit 14, but the extraction of the second comparison information I2 from the second document D2 may also be performed by a machine learning-based generation AI (which may be the same as or different from the above-mentioned generation AI 10). Note that the first document D1 containing the verification target sentence S and the second document D2 containing the verification target table T (the verification target portion) may be different documents or different portions of the same document.

[0024] Furthermore, the first document D1 and the second document D2 are not limited to documents subject to audits, such as financial audits. For example, the information comparison device 1 according to this embodiment can be applied to the analysis and / or comparison of legal documents, such as contracts, and other documents in any field or type. For example, as described above, the information comparison device 1 according to this embodiment extracts first comparison information I1 from a verification target document S (e.g., a contract) included in the first document D1 using the generation AI 10, and extracts second comparison information I2 from a verification target table T (e.g., a term sheet summarizing the contract) included in the second document D2 using the information extraction unit 14. In particular, the information comparison device 1 according to this embodiment is advantageous in that it can accurately extract first comparison information I1 from a verification target document S written in a free format using the generation AI 10.

[0025] FIG. 2 is a schematic functional block diagram of an information comparison device 1 according to this embodiment. The information comparison device 1 includes a generation AI 10, a sentence designation unit 11, a prompt input unit 12, a table designation unit 13, an information extraction unit 14, an information comparison unit 15, and an inconsistency presentation unit 16. Some of these functional blocks may be omitted as long as the information comparison device 1 can achieve at least some of the actions and / or effects described below (or above). These functional blocks are realized by the cooperation of hardware resources, such as a computer's central processing unit, memory, input devices, output devices, and peripheral devices connected to the computer, and software executed using these resources. Regardless of the type of computer or its location, each of the above functional blocks may be realized by the hardware resources of a single computer or by combining hardware resources distributed across multiple computers.

[0026] The text designation unit 11 designates a confirmation target text S in the first document D1 to be extracted from the first comparison information I1. As will be described later, the confirmation target text S may be a specific text of any length, such as a specific chapter, section, paragraph, or paragraph, contained in the first document D1. The designation of the confirmation target text S by the text designation unit 11 may be performed based on text designation rules (described later) manually created by an accounting auditor, an audited company, or the like via a user device D such as a personal computer.

[0027] The prompt input unit 12 inputs a prompt P to the generative artificial intelligence (generative AI) 10, specifying items to be verified that may be included in the verification target document S specified by the document designation unit 11, and causing the generative AI to extract first comparison information I1 related to the items to be verified from the verification target document S. As illustrated in FIG. 1 , a prompt P such as "What is the amount of net assets at the end of fiscal year 20XX?" is input to the generative AI 10 by the prompt input unit 12 to extract the amount (first comparison information I1) of the account item "net assets" (item to be verified) that may be included in the verification target document S from the verification target document S. The input or generation of such a prompt P may be autonomously performed by the prompt input unit 12 based on rules, etc., or may be manually performed or assisted (with the assistance of the prompt input unit 12) by an accounting auditor, an audited company, etc. via a user device D.

[0028] The generation AI 10 extracts first comparison information I1 (e.g., the amount of the account item) related to the desired item to be confirmed (e.g., an account item) from the confirmation target text S specified by the text specification unit 11 in response to the prompt P input by the prompt input unit 12. The first comparison information I1 generated or output by the generation AI 10 is preferably concise, short text information that is standardized (or follows a predetermined format), such as "Net asset value (FY20XX): YYY yen" in the example of Figure 1.

[0029] However, the first comparison information I1 may be in any format (for example, a table or an image) as long as the information comparison unit 15 (described later) can appropriately compare it with the second comparison information I2. Here, it is preferable that the first comparison information I1 and the second comparison information I2 are expressed in a similar format so that they can be easily compared with each other. For example, if the first comparison information I1 is in a table format, it is preferable that the second comparison information I2 be expressed as a similar table format that can be directly compared with the first comparison information I1.

[0030] The generation AI 10, which is composed of a neural network, is an artificial intelligence system based on a large language model (LLM) that has learned from a huge amount of text. The generation AI 10, which has such natural language processing capabilities, understands a prompt P given in natural language and analyzes a verification target text S written in natural language. Therefore, the generation AI 10 can accurately extract the desired verification target items and / or first comparison information I1 from the verification target text S in accordance with the given prompt P.

[0031] The table designation unit 13 designates a confirmation target table T (confirmation target portion) to be extracted from the second comparison target information I2 in a second document D2 that is different from (or the same as) the first document D1. The confirmation target table T is a specific table designated by the table designation unit 13 from among various tables that primarily represent financial and accounting status numerically and are included in, for example, financial statements, financial results summary reports, financial results briefing materials, quarterly reports, securities reports, etc., which are documents to be audited (second document D2). The designation of the confirmation target portion of the confirmation target table T, etc., by the table designation unit 13 may be performed based on table designation rules manually created by the accounting auditor, the audited company, etc., via the user device D. Furthermore, as described above, the confirmation target portion of the confirmation target table T, etc., is expressed according to a predetermined format. Therefore, the information extraction unit 14, which will be described next, can accurately extract the second comparison target information I2 based on rules created to match the format.

[0032] The information extraction unit 14 extracts, from the confirmation target table T as the confirmation target portion expressed in accordance with a predetermined format, second comparison target information I2 (e.g., the amount of the account item) related to a desired confirmation target item (e.g., an account item) that may be included in the confirmation target table T, based on the format. The second comparison target information I2 extracted by the information extraction unit 14 is preferably simple, short text information that is standardized (or follows a predetermined format), such as "Net asset value (FY20XX): ZZZ yen" in the example of FIG.

[0033] As mentioned above, the second comparison target information I2 does not have to be text information, but is preferably expressed in the same format as the first comparison target information I1 so that it can be easily compared with the first comparison target information I1. The information extraction unit 14 may extract information from the confirmation target portion, such as the confirmation target table T, based on an information extraction rule R (for example, "The net asset value for the current fiscal year (20XX fiscal year) is in the nth cell to the right from the cell for 'Net Assets'") manually created via the user device D by an accounting auditor, an audited company, or the like, as shown in FIG. 1.

[0034] The information comparison unit 15 compares first comparison information I1 extracted by the generation AI 10 from the confirmation target text S with second comparison information I2 extracted by the information extraction unit 14 from a confirmation target table T (confirmation target portion) different from the confirmation target text S. In the example of FIG. 1, the information comparison unit 15 compares "YYY yen," which is numerical information or monetary information included in the first comparison information I1, "Net asset value (FY20XX): YYY yen," with "ZZZ yen," which is numerical information or monetary information included in the second comparison information I2, "Net asset value (FY20XX): ZZZ yen." In a narrower sense, the information comparison unit 15 checks whether "YYY yen" derived from the first comparison information I1 and "ZZZ yen" derived from the second comparison information I2 substantially match. In a broad sense, the information comparison unit 15 checks whether there is logical consistency between the first comparison information I1, which may be non-numerical information, and the second comparison information I2, which may also be non-numerical information, as described above.

[0035] The inconsistency presentation unit 16 presents or notifies the accounting auditor, the audited company, etc., via the user device D, etc., that the numerical information and / or non-numerical information contained in the first comparison target information I1 and the second comparison target information I2 do not match as a result of the comparison by the information comparison unit 15. Note that in addition to or instead of the inconsistency presentation unit 16, an inconsistency presentation unit (not shown) may be provided that presents or notifies the accounting auditor, the audited company, etc., via the user device D, etc., that the first comparison target information I1 and the second comparison target information I2 do not match as a result of the comparison by the information comparison unit 15.

[0036] Next, an example of the information comparison device 1 according to this embodiment will be described based on a specific processing flow example and a screen example of the user device D. "S" in the flowchart indicates a step or process. Also, the content described above with reference to FIGS. 1 and 2 will not be described again.

[0037] 3 is a flowchart showing a specific example of a processing flow for generating or extracting first comparison information I1. In S1, a first document D1 is input or registered in the information comparison device 1. The first document D1 is, for example, one of the documents subject to an accounting audit included in a group of documents subject to an accounting audit, and may be an electronic document in any file format such as PDF (Portable Document Format).

[0038] In S2, the consistency check rule R is input or registered in the information comparison device 1. The consistency check rule R may be, for example, a spreadsheet file in any file format such as Excel (registered trademark). FIG. 4 shows an excerpt of an example of the consistency check rule R. This consistency check rule R is obtained by adding information about a chapter ID or chapter number (Chapter No.) that specifies a specific chapter (verification target sentence S) included in the first document D1 to the information extraction rule R (see FIG. 1) used for extracting the second comparison target information I2 from the verification target table T by the information extraction unit 14 described above.

[0039] The information in each column on the right side of FIG. 4 (from the second column onward, which is designated as "Source Marker Identifier") defines the information extraction rule R that is primarily used by the information extraction unit 14. Specifically, "No." defines the ID or number of the information extraction rule R, "x1" to "x3." specifies the confirmation target table T, "column." specifies the consolidated fiscal year (e.g., the current consolidated fiscal year or the previous consolidated fiscal year) in the confirmation target table T (as will be described later, in the confirmation target table T of a typical document to be confirmed by an accounting audit, information for each consolidated fiscal year is displayed side by side in each "column," hence the name "column"), and "row." specifies the account item (item to be confirmed) in the confirmation target table T (as will be described later, in the confirmation target table T of a typical document to be confirmed by an accounting audit, information for each account item is displayed side by side in each "row," hence the name "row").

[0040] Details will be given later, but for example, information extraction rule R for "P101" in the first row in Figure 4 sets the "Consolidated Balance Sheet" (x3) included in the section specified by the main heading "Regarding Audit Certification" (x1) and the subheading "Consolidated Financial Statements, etc." (x2) in the second document D2 as the confirmation target table T, and extracts numerical information (i.e., amount information) for the "Total Current Assets" (row) for the "Current Consolidated Fiscal Year" (column) from the confirmation target table T. For such information extraction rule R, if one or more chapter IDs are registered in the "Chapter No." in the first column in Figure 4, a chapter specification rule or sentence specification rule R11 is referenced in S4, which will be described later.

[0041] In S3 in Fig. 3, the text designation unit 11 or the like generates a list L of items to be checked in the first document D1 input in S1 for consistency with the second document D2, based on the consistency check rule R (Fig. 4) input in S2. This list L of items to be checked may be, for example, an extracted set of "row" information in Fig. 4, in which account items as items to be checked are listed, without duplication, or an extracted set of "column" information specifying the consolidated fiscal year and "row" information specifying the account items in Fig. 4. For example, starting from the first line in Fig. 4, "total current assets" (row) in the "current consolidated fiscal year" (column) is listed as an item to be checked in the first document D1 (text to be checked S).

[0042] In S4 in Fig. 3, the text designation unit 11 inputs or refers to the text designation rule R11 in accordance with the consistency check rule R (Fig. 4) input in S2. For example, in accordance with the chapter ID of "C05" being entered in the consistency check rule R in the first line in Fig. 4, the text designation unit 11 searches for or refers to the chapter ID of "C05" in the text designation rule R11 exemplified in Fig. 5.

[0043] The information in each column in Fig. 5 defines the text designation rule R11 used by the text designation unit 11. Specifically, the aforementioned chapter IDs, "Chapter No." and / or "Parent Chapter No.", define the ID or number of the text designation rule R11, and "x1" to "x4" specify information for identifying the chapter to be checked or the text S to be checked in the first document D1. For example, the text designation rule R11 for "C05" in the third line in Fig. 5, specified by the information extraction rule R for "P101" in the first line in Fig. 4, specifies the chapter in the first document D1 that is identified by the main heading "Place of public inspection" (x1) and the subheading "Business status" (x2).

[0044] In S5 in Fig. 3, the text designation unit 11 extracts a chapter C to be confirmed from the first document D1 based on the text designation rule R11 input or referenced in S4. For example, the text designation unit 11 extracts from the first document D1, from among the descriptions below the main heading "Place of public inspection" (x1), a chapter C with a subheading "Business status" (x2) as schematically shown in Fig. 6, based on the text designation rule R11 for "C05" on the third line in Fig. 5, which is specified by the information extraction rule R for "P101" on the first line in Fig. 4.

[0045] In S6 in Fig. 3, the text designation unit 11 extracts one or multiple consecutive paragraphs S that include descriptions of each of the items to be checked (each account item) in the list of items to be checked L generated in S3 from the chapter C (Fig. 6) extracted in S5. For example, the text designation unit 11 searches for each of the account items listed in the list of items to be checked L one by one in order, using the chapter C extracted in S5 as the search target, and identifies one or multiple paragraphs S that include each of the account items.

[0046] In this case, the text designation unit 11 may extract a paragraph S that includes a description of the fiscal year of each account item (the "row" information in FIG. 4) in addition to each account item (the "column" information in FIG. 4) listed in the confirmation item list L and / or the consistency check rule R (FIG. 4). Furthermore, the text designation unit 11 may extract a paragraph S that includes some amount information that can become the first comparison information I1 to be generated later by the generation AI 10.

[0047] 7 shows an example (different from that shown in FIG. 6) in which the text specification unit 11 extracts, from chapter C in S6, a paragraph S that includes desired account items and accounting periods based on the confirmation item list L and / or consistency check rules R, and some amount information. In this example, "cash flow from operating activities," "operating cash flow," "operating CF," etc. are specified as desired account items (confirmation items), and "current consolidated fiscal year," "current fiscal year," "current accounting period," etc. are specified as desired accounting periods in the confirmation item list L and / or consistency check rules R (the "column" and "row" information pairs in FIG. 4).

[0048] Among the multiple paragraphs included in Chapter C, paragraph S enclosed by a dotted line in Figure 7 includes both a description of "cash flow from operating activities" (dotted line) as the desired account item and a description of "current consolidated fiscal year" (solid line) as the desired accounting period. Furthermore, since paragraph S includes at least one amount information (dash-dotted line), the text designation unit 11 extracts paragraph S as a confirmation target text S for the desired account item and accounting period based on the confirmation target item list L and / or consistency check rule R. At this point, it is not necessary to identify which of the various amount information included in paragraph S represents the confirmation target item (cash flow from operating activities) (this will be identified by generation AI 10 in a later stage).

[0049] In the examples of Figures 3 and 7, the paragraph S extracted in S6 corresponds to the target sentence S in Figures 1 and 2. By selecting a relatively short paragraph S as the target sentence S in this way, the load on the generation AI 10 can be reduced and the accuracy of generating the first comparison information I1 can be improved. Furthermore, by adopting a stepwise approach in which one or more chapters C (first target sentences) are extracted as part of the first document D1 in the first sentence designation step (S5), and one or more paragraphs S (second target sentences) are extracted as part of the chapter C in the second sentence designation step (S6), the paragraph S can be efficiently extracted from the first document D1 while improving the extraction accuracy at each step. Note that the target sentence S is not limited to a paragraph S, and may be any position and length of sentence contained in the first document D1, such as the entire chapter C as shown in Figures 6 and 7, or the entire first document D1.

[0050] 3, the prompt input unit 12 inputs a prompt P to the generation AI 10, specifying items to be checked that may be included in the paragraph S (text to be checked) extracted in S6, and causing the generation AI 10 to extract first comparison information I1 related to the items to be checked from the paragraph S. As described above, the items to be checked in the first document D1 (e.g., accounts and / or their fiscal years) are listed in the consistency check rules R (FIG. 4) and / or the list of items to be checked L. The prompt input unit 12 then references the consistency check rules R and / or the list of items to be checked L, selects the accounts (the information in "row" in FIG. 4) and / or fiscal years (the information in "column" in FIG. 4) listed therein one by one, generates a prompt P related to each account and / or fiscal year, and inputs it to the generation AI 10.

[0051] For example, in the example of Fig. 7, the prompt input unit 12 generates a prompt P such as "What is the amount of cash flow from operating activities for the current consolidated fiscal year?" and inputs it to the generation AI 10 in order to extract monetary information for "the current consolidated fiscal year" (information recorded in "column" in Fig. 4) of the account item "cash flow from operating activities" (information recorded in "row" in Fig. 4) as the item to be confirmed from paragraph S. Such a prompt P can be generated autonomously by the prompt input unit 12 based on the consistency check rule R and / or the pair of accounting period (column) and account item (row) recorded in the list L of items to be confirmed.

[0052] In the following S8, the generation AI 10 analyzes paragraph S in Figure 7 in response to the prompt P entered in S7, and identifies or extracts "4.1 billion yen" (income) as the amount information for "cash flow from operating activities for the current consolidated fiscal year" related to the prompt P. As a result, the generation AI 10 generates first comparative information I1, such as "cash flow from operating activities for the current consolidated fiscal year is +4.1 billion yen," "operating CF (current consolidated fiscal year): +4.1 billion yen," etc.

[0053] As mentioned above, the first comparison information I1 generated by the generation AI 10 is required to be accurate because it will be compared with the second comparison information I2 later (if the first comparison information I1 is inaccurate, it will not be possible to make a useful comparison with the second comparison information I2 in the first place). On the other hand, the generation AI 10 tends to try to force an answer even when the information it is given is insufficient.

[0054] Therefore, it is preferable to devise a prompt P to be input to the generation AI 10 in S7 so that the generation AI 10 does not generate such an unreasonable and inaccurate answer (first comparison information I1). For example, by including a standard phrase at the beginning of the prompt P such as "Please output an answer to the question based on the given text. Please answer based on the information contained in the text, and if the information for the question is insufficient, please answer 'I don't know'," the generation AI 10 may be prompted to answer "I don't know" if the information in paragraph S is insufficient.

[0055] In addition, the information comparison unit 15, the information comparison device 1, or the like may check whether the first comparison information I1 generated by the generation AI 10 in S8 is appropriate. For example, the information comparison unit 15 may check whether the first comparison information I1 generated by the generation AI 10 contains only one amount information (whether multiple amount information is included), whether the amount information is actually written in the paragraph S from which it was extracted, etc. If no problems are found in this check, the information is provided to the information comparison unit 15 as legitimate first comparison information I1.

[0056] 8 is a flowchart showing a specific example of a processing flow for generating second comparison information I2 and comparing it with first comparison information I1 (S8 in FIG. 3). In S9, second document D2 is input or registered in information comparison device 1. Second document D2 is, for example, one of the documents subject to accounting audit included in a group of documents subject to accounting audit, and may be an electronic document in any file format such as PDF.

[0057] In S10, the table designation unit 13 designates or extracts a table T to be checked as a part of the second document D2 input in S9, whose consistency with the first document D1 should be checked, based on the consistency check rule R input or registered in S2 (FIG. 3). Specifically, in the consistency check rule R illustrated in FIG. 4, the table designation unit 13 designates a specific table T to be checked, which is included in the second document D2, based on the information extraction rule R for the first row "P101" in FIG. 4. For example, based on the information extraction rule R for "P101" in the first line in FIG. 4, the table designation unit 13 extracts from the second document D2, among the descriptions following the main heading "Regarding Audit Certification" (x1) in the second document D2, the table T to which the subheading "Consolidated Financial Statements, etc." (x2) and the subheading "Consolidated Balance Sheet" (x3) are attached, as schematically illustrated in FIG. 9.

[0058] In S11, the information extraction unit 14 extracts second comparison information I2 (e.g., the amount of the account item) related to a desired confirmation item (account item) included in the confirmation target table T from the confirmation target table T (FIG. 9) that is the confirmation target portion expressed in accordance with a predetermined format, based on the format and information extraction rule R. For example, based on information extraction rule R for "P101" in the first row in FIG. 4, the information extraction unit 14 extracts, as second comparison information I2, amount information (i.e., the amount of the total current assets for the current consolidated fiscal year) written at the intersection of the column for "Current Consolidated Fiscal Year" specified by "column" (the rightmost column in FIG. 9) and the row for "Total Current Assets" specified by "row" (the bottom row in FIG. 9).

[0059] In S12, the information comparison unit 15 compares the first comparison information I1 generated in S8 (FIG. 3) with the second comparison information I2 extracted from the confirmation target table T (FIG. 9) in S11. In S13, the inconsistency presentation unit 16 or an inconsistency presentation unit (not shown) presents or notifies the accounting auditor, the audited company, etc. via the user device D, etc., that the numerical information and / or non-numerical information included in the first comparison information I1 and the second comparison information I2 do not match as a result of the comparison in S12, or that there is no logical consistency between the first comparison information I1 and the second comparison information I2.

[0060] Note that the units of display for numerical information such as amounts may differ between the confirmation target text S in the first document D1 and the confirmation target table T (confirmation target portion) in the second document D2. In such cases, it is preferable for the information comparison unit 15 and the inconsistency presentation unit 16 to determine whether the substantial numerical values ​​are equivalent, regardless of the difference in the units of display for the numerical information. For example, if the amount of a certain account item is expressed as "41 (billion yen)" in the first comparison target information I1 and as "4,126 (million yen)" in the second comparison target information I2, it may be determined that the two are substantially identical.

[0061] 10 and 11 are schematic diagrams showing typical screen examples displayed on the display of a user device D used by an accounting auditor, an audited company, etc., in the series of processes shown in Fig. 3 and / or Fig. 8. For convenience, the diagrams are shown in two parts, Fig. 10 and Fig. 11, but in practice, it is preferable that these be displayed on a single screen. For example, Fig. 10 shows the left half of a single screen, and Fig. 11 shows the right half.

[0062] This screen has two document display areas DP1 and DP2. The first document display area DP1 displays a specific portion of a specific document selected in the first selection area SL1, and the second document display area DP2 displays a specific portion of a specific document selected in the second selection area SL2. In this embodiment, as shown in FIGS. 10 and 11, it is preferable to display a confirmation target sentence S of the first document D1 in one of the document display areas DP1 and DP2, and a confirmation target table T (confirmation target portion) of the second document D2 in the other document display area DP1 and DP2. However, the user can arbitrarily select what to display in each document display area DP1 and DP2 in the selection areas SL1 and SL2. For example, one portion of the first document D1 may be displayed in the first document display area DP1, and another portion of the first document D1 may be displayed in the second document display area DP2.

[0063] The comparison result display area RS displays a summary of the results of the comparison between the first comparison information I1 and the second comparison information I2 by the information comparison unit 15, in units of account items (items to be confirmed) for each fiscal year. For example, for account items that match or are consistent across the first and second comparison documents D1 and D2 (for example, "Cash flow from operating activities" (previous consolidated fiscal year) in FIG. 11), a mark such as "◎" or "◯" indicating "match" or "consistency" is displayed.

[0064] Furthermore, for account items that do not match or are inconsistent between the first document D1 and the second document D2 being compared (for example, "Cash flow from operating activities" (current consolidated fiscal year) in FIG. 11), a mark such as "x" (one embodiment of the inconsistency presentation unit 16) indicating "mismatch" or "inconsistency" is displayed. Furthermore, for account items for which it is not possible to clearly determine whether they match or are inconsistent between the first document D1 and the second document D2 being compared (for example, "Total net assets" in FIG. 11), a mark such as "▲" or "△" indicating "unknown" is displayed.

[0065] In the example of Figure 11, "Cash flow from operating activities" (current consolidated fiscal year), for which the comparison result is "x", is selected, and the detailed display area DT showing its details is expanded. In this detailed display area DT, all locations in the first document D1 and / or second document D2 where the amount of "Cash flow from operating activities" (current consolidated fiscal year) is stated (i.e., all locations from which the first comparative information I1 and / or the second comparative information I2 for the account item were extracted) can be selected using selection areas SL1 and SL2.

[0066] In the example of Figure 11, the "Business Status" section on page "17" of first document D1 is selected in first selection area SL1. In response to this selection, the first document display area DP1 (Figure 10) displays the confirmation target document S, which contains the amount "4,165 million yen" for "Cash Flow from Operating Activities" (current consolidated fiscal year) on that page and in that section. Then, the "4,165 million yen" extracted by generation AI10 as first comparison target information I1 is highlighted in first document display area DP1. Furthermore, this "4,165 million yen" as first comparison target information I1 is also displayed in summary in detail display area DT.

[0067] Meanwhile, the "Consolidated Cash Flow" section on page "50" of second document D2 is selected in second selection area SL2. In response to this selection, second document display area DP2 displays confirmation table T, which lists the amount "4,165,031" for "Cash Flow from Operating Activities" (current consolidated fiscal year) for that page and section. Then, "4,165,031" extracted by information extraction unit 14 as second comparison information I2 is highlighted in second document display area DP2. Furthermore, this "4,165,031" as second comparison information I2 is also displayed in summary in detail display area DT.

[0068] In the example of Figure 11, the first comparison information I1, "4,165,000 yen," and the second comparison information I2, "4,165,031" (4,165,030,000 yen), do not match, so they are judged as "X" (mismatch).

[0069] Through the above-described screen, users such as accounting auditors and audited companies can easily understand the comparison results between the first comparison information I1 and the second comparison information I2 while referring to the actual first document D1 (confirmation target document S) and second document D2 (confirmation target table T) on a single screen, and can efficiently correct any discrepancies or inconsistencies. Such correction work may be performed manually by users such as accounting auditors and audited companies, or may be performed automatically by the information comparison device 1.

[0070] The present disclosure has been described above based on the embodiments. Various modifications are possible to the combinations of the components and processes in the exemplary embodiments, and it will be obvious to those skilled in the art that such modifications are included within the scope of the present disclosure.

[0071] In the embodiment of Figure 1 etc., it was mainly assumed that the first document D1 and the second document D2 to be compared were documents created in the same fiscal year, but the information comparison device of the present disclosure may also compare documents created in different fiscal years or at different times.

[0072] For example, a first document D1 created earlier and a second document D2 created later may be compared using an information comparison device according to the present disclosure. Such a "newer" second document D2 may include descriptions of the "past" that are included in the "older" first document D1. If the two documents D1 and D2 describe the same subject, the first comparison information I1 and the second comparison information I2 contained in each document should match. The information comparison device according to the present disclosure can accurately compare descriptions of the same subject contained in the first document D1 and the second document D2 created at different times and check for consistency.

[0073] 12 is a schematic diagram showing a modified example of the information comparison device 1 that compares a first document D1 and a second document D2 created at different times. This modified example is a modification of the embodiment shown in FIG. 1, and components similar to those in FIG. 1 and other figures are designated by similar reference numerals, and redundant explanations will be omitted.

[0074] In this modified example, the first document D1 was created in fiscal year N-1, and the second document D2 was created in the following fiscal year, fiscal year N. Therefore, the current fiscal year (i.e., fiscal year N-1) in the first document D1 corresponds to the previous fiscal year (i.e., fiscal year N-1) in the second document D2.

[0075] The information comparison device 1 according to this modification compares information on items to be confirmed for fiscal year N-1 that is commonly contained in the first document D1 and the second document D2. Specifically, the information comparison device 1 compares first comparison information I1 for the current fiscal year extracted from the first document D1 with second comparison information I2 for the previous fiscal year extracted from the second document D2.

[0076] 12, first document D1 and second document D2 are documents subject to audits such as accounting audits. Furthermore, examples of items to be confirmed that are commonly included in first document D1 and second document D2 as documents subject to accounting audits include account items such as "Net assets (total)" listed in financial statements, and examples of the related first comparison information I1 and second comparison information I2 include the amount of the account item "Net assets."

[0077] In this case, the first comparison information I1 for the current year extracted from the first document D1 for the account item "Net assets" is information in the format of, for example, "Net assets amount (FYN-1): YYY yen," and the second comparison information I2 for the previous year extracted from the second document D2 for the account item "Net assets" is information in the format of, for example, "Net assets amount (FYN-1): ZZZ yen." The net assets amount "YYY yen" as the first comparison information I1 and the net assets amount "ZZZ yen" as the second comparison information I2 extracted for the same fiscal year (FYN-1) from the first document D1 and the second document D2 should match each other.

[0078] The information comparison device 1 according to this embodiment compares numerical information that should match, and efficiently checks the consistency. For example, if the net asset value "YYY yen" in the first comparison information I1 for the current year as seen from the first document D1 and the net asset value "ZZZ yen" in the second comparison information I2 for the previous year as seen from the second document D2 do not substantially match, the information comparison device 1 will present or notify this to an accounting auditor or other person, and urge them to take notice or make corrections.

[0079] The first comparison information I1 may be extracted from a sentence S1 included in the first document D1 by the generation AI 10. For example, the generation AI 10 extracts the first comparison information I1 from the sentence S1 included in the first document D1 based on a prompt P1 such as, "What is the net asset value at the end of the current fiscal year (fiscal year N-1 for the first document D1)?"

[0080] Similarly, the second comparison information I2 may be extracted by the generation AI10 from a sentence S2 included in the second document D2. For example, the generation AI10 extracts the second comparison information I2 from a sentence S2 included in the second document D2 based on a prompt P2 such as, "What was the net asset value at the end of the previous fiscal year (fiscal year N-1 for the second document D2)?" If the first document D1 and the second document D2 are the same type of document (created at different times), the first comparison information I1 and the second comparison information I2 can be extracted by a single, substantially identical generation AI10.

[0081] 1, at least one of the first comparison information I1 and the second comparison information I2 may be extracted from a standard table (corresponding to table T in FIG. 1) or a portion to be confirmed contained in the first document D1 or the second document D2, rather than from sentences S1 and S2 contained in the first document D1 or the second document D2. Information extraction from such a table or portion to be confirmed is preferably performed by the information extraction unit 14 described above.

[0082] The configuration, operation, and function of each device and method described in the embodiments can be realized by hardware resources, software resources, or a combination of hardware and software resources. Examples of hardware resources include processors, ROMs, RAMs, and various integrated circuits. Examples of software resources include operating systems, applications, and other programs. [Explanation of symbols]

[0083] 1 Information comparison device, 10 Generation AI, 11 Text specification unit, 12 Prompt input unit, 13 Table specification unit, 14 Information extraction unit, 15 Information comparison unit, 16 Inconsistency presentation unit, D1 First document, D2 Second document, I1 First comparison target information, I2 Second comparison target information.

Claims

1. a prompt input unit that inputs a prompt to the generative artificial intelligence (generative AI) to specify a verification target item that may be included in the verification target sentence and to extract first comparison target information related to the verification target item from the verification target sentence; an information comparison unit that compares second comparison information related to the confirmation target item extracted from a confirmation target portion different from the confirmation target sentence with the first comparison information; An information comparison device comprising:

2. 2. The information comparison device according to claim 1, further comprising an information extraction unit that extracts, from the confirmation target portion expressed in a predetermined format, the second comparison target information related to the confirmation target items that may be included in the confirmation target portion based on the format.

3. 3. The information comparison device according to claim 2, wherein the confirmation target portion is a table in accordance with a predetermined format.

4. The information comparison device according to claim 1 , wherein a first document including the confirmation target sentence and a second document including the confirmation target portion are different from each other.

5. The information comparison device according to claim 1 , wherein the confirmation target sentence and the confirmation target portion are included in the same or different documents to be inspected.

6. The information comparison device according to claim 5 , wherein the confirmation target item is an account item included in the document to be audited.

7. The information comparison device according to claim 6 , wherein the first comparison information and the second comparison information are amounts of the account items.

8. 4. The information comparison device according to claim 1, wherein the first comparison information and the second comparison information are numerical information that should match each other.

9. The information comparison device according to claim 1 , further comprising a mismatch notification unit that notifies the user that the first comparison target information and the second comparison target information do not match as a result of the comparison by the information comparison unit.

10. a sentence designation unit that designates the confirmation target sentence in the first document; The prompt input unit inputs a prompt to the generation AI to extract the first comparison information from the confirmation target sentence specified by the sentence specification unit.

4. The information comparison device according to claim 1.

11. inputting a prompt to the generation AI to specify a verification target item that may be included in the verification target sentence and to extract first comparison target information related to the verification target item from the verification target sentence; comparing second comparison information relating to the confirmation target item extracted from a confirmation target portion different from the confirmation target sentence with the first comparison information; Information comparison method to perform.

12. inputting a prompt to the generation AI to specify a verification target item that may be included in the verification target sentence and to extract first comparison target information related to the verification target item from the verification target sentence; comparing second comparison information relating to the confirmation target item extracted from a confirmation target portion different from the confirmation target sentence with the first comparison information; A storage medium storing an information comparison program that causes a computer to execute the above.

Images