Intelligence comparison device, intelligence comparison method, memory medium

Abstract

To provide an information comparison device or the like capable of efficiently extracting information to be compared from a confirmation object sentence.SOLUTION: The information comparison device 1 includes a prompt inputter 12 that specifies a check target item that can be included in a check target sentence S and inputs a prompt P that causes a generative artificial intelligence (AI) 10 to extract first comparison target information I1 related to the check target item from the check target sentence S, and an information comparator 15 that compares the first comparison target information I1 with second comparison target information I2 related to a check target item extracted from a check target table T different from the check target sentence S. This system is provided with an information extraction part 14 for extracting second comparison object information I2 related to a confirmation object item which can be included in a confirmation object table T from the confirmation object table T expressed according to a prescribed format on the basis of the format.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information comparison device and the like. [Background Art]

[0002] Patent Literature 1 discloses a financial statement analysis system that extracts factors from financial statement-related information. Specifically, it is disclosed that morphological analysis is performed on each sentence obtained by decomposing text included in financial statement-related information, and factors identified according to comparison with pre-stored factor patterns are associated with the subject and amount information included in each sentence. [Prior Art Literature] [Patent Literature]

[0003] [Patent Literature 1] Japanese Unexamined Patent Publication No. 2018-120284 [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] In financial statement-related information as disclosed in Patent Literature 1, information such as amounts of accounting items included in financial statements may be described in the same or different formats (e.g., text and tables) at different locations across a huge volume of documents. In particular, since strict accuracy is required for amount information in financial statement-related information, it is important to ensure the consistency of amounts throughout the entire huge volume of financial statement-related information. Conventionally, ensuring such consistency has relied on manual confirmation by auditing corporations and the like responsible for auditing financial statement-related information, which is inefficient and involves the risk of human error.

[0005] The present disclosure has been made in view of such circumstances, and an object of the present disclosure is to provide an information comparison device and the like that can efficiently extract information to be compared from a text to be checked. [Means for Solving the Problem]

[0006] To solve the above problems, an information comparison device in one aspect of the present disclosure includes a prompt input unit that inputs a prompt to a generative artificial intelligence (generative AI) to specify items to be checked that may be included in the document to be checked, and to extract first comparison information relating to the items to be checked from the document to be checked; and an information comparison unit that compares second comparison information relating to items to be checked extracted from a different document to be checked with the first comparison information.

[0007] In this embodiment, a prompt input to the generating AI allows for the efficient extraction of first comparison information (e.g., the amount of an account) related to a desired item to be verified (e.g., an account) from the document to be verified. The second comparison information to be compared with the first comparison information may be extracted from another document by a prompt input to the generating AI, or it may be extracted from another section to be verified, such as a table, by any other arbitrary method.

[0008] Another aspect of this disclosure is an information comparison method. This method performs the following actions: inputting a prompt to the generating AI to specify items to be checked that may be included in the document to be checked, and to extract first comparison information relating to those items from the document to be checked; and comparing second comparison information relating to items to be checked extracted from a different part of the document to be checked with the first comparison information.

[0009] Another aspect of this disclosure is a storage medium. This storage medium stores an information comparison program that causes a computer to perform the following actions: specify items to be checked that may be included in the document to be checked, input a prompt to extract first comparison information relating to those items from the document to be checked, and compare second comparison information relating to items to be checked extracted from a different part of the document to be checked with the first comparison information.

[0010] Furthermore, any combination of the above components, as well as any representations thereof converted into methods, apparatus, systems, recording media, computer programs, etc., are also included in this disclosure. [Effects of the Invention]

[0011] According to this disclosure, information to be compared can be efficiently extracted from the documents being reviewed. [Brief explanation of the drawing]

[0012] [Figure 1] A schematic overview of this embodiment is shown below. [Figure 2] This is a schematic functional block diagram of an information comparison device. [Figure 3] This flowchart schematically shows a specific processing flow example for generating or extracting the first comparison target information. [Figure 4] An example of consistency check rules is shown below. [Figure 5] Examples of text specification rules are given below. [Figure 6] Examples of chapters extracted from the first document are shown below. [Figure 7] This example shows how the text specification section extracts paragraphs from a chapter that contain the desired account title and accounting period, as well as some monetary information. [Figure 8] This flowchart schematically illustrates a specific processing flow example related to the generation of second comparison target information and comparison with the first comparison target information. [Figure 9] An example of a table extracted from the second document is shown. [Figure 10] A schematic diagram illustrates typical screens displayed on a user device's screen. [Figure 11] A schematic diagram illustrates typical screens displayed on a user device's screen. [Figure 12] A schematic diagram illustrates a modified version of an information comparison device that compares documents created at different times. [Modes for carrying out the invention]

[0013] Hereinafter, modes for carrying out the present disclosure (hereinafter also referred to as embodiments) will be described in detail with reference to the drawings. In the description and / or the drawings, identical or equivalent components, members, processes, and the like are denoted by the same reference numerals, and duplicate descriptions are omitted. The scales and shapes of the respective parts illustrated in the drawings are set for convenience in order to simplify the description, and are not to be construed as limiting unless otherwise stated. The embodiments are illustrative and do not limit the scope of the present disclosure in any way. Not all features and combinations thereof presented in the embodiments are necessarily essential to the present disclosure. For convenience, embodiments are presented by being decomposed into components for each function and / or each function group for realizing the embodiment. However, one component in an embodiment may actually be realized by a combination of a plurality of separate components, or a plurality of components in an embodiment may actually be realized by a single integrated component. In addition, a plurality of embodiments and modified examples may be disclosed in parallel, and any component of each embodiment and / or each modified example may be combined in any manner as long as their functions are not impaired mutually.

[0014] Figure 1 schematically shows an outline of the present embodiment. An information comparison apparatus 1 according to the present embodiment compares information related to confirmation target items commonly included in a first document D1 and a second document D2. Specifically, first comparison target information I1 extracted from the first document D1 and second comparison target information I2 extracted from the second document D2 are compared by the information comparison apparatus 1.

[0015] In the example of Figure 1, the first document D1 and the second document D2 are documents subject to audit such as accounting audit. Examples of confirmation target items commonly included in the first document D1 and the second document D2 as accounting audit target documents include account items such as "total net assets" published in financial statements and the like, and examples of the first comparison target information I1 and the second comparison target information I2 related thereto include the amount of the account item "net assets".

[0016] In this case, the first comparison target information I1 extracted for the account item "net assets" from the first document D1 is, for example, information in the format of "Net assets (FY20XX): YYY yen", and the second comparison target information I2 extracted for the account item "net assets" from the second document D2 is, for example, information in the format of "Net assets (FY20XX): ZZZ yen". When the first document D1 and the second document D2 are documents subject to accounting audit for the same accounting period of the same company, the net asset amount "YYY yen" extracted as the first comparison target information I1 and the net asset amount "ZZZ yen" extracted as the second comparison target information I2 should match each other.

[0017] The information comparison apparatus 1 according to the present embodiment compares, for example, numerical information that should match each other in this manner, and efficiently checks the consistency thereof. For example, if the net asset amount "YYY yen" in the first comparison target information I1 and the net asset amount "ZZZ yen" in the second comparison target information I2 do not substantially match, the information comparison apparatus 1 presents or notifies an accounting auditor or the like of this fact to prompt attention or correction.

[0018] Note that the information comparison apparatus 1 according to the present embodiment may compare non-numerical information such as character information included in the first comparison target information I1 and the second comparison target information I2. For example, the information comparison apparatus 1 according to the present embodiment may check the consistency of character information representing the tendency, attribute, state, or situation of an account item (item to be confirmed), such as "net asset amount increased compared to the previous fiscal year", "net asset amount decreased compared to the previous fiscal year", and "net asset amount remained unchanged from the previous fiscal year", which are commonly included in the first comparison target information I1 and the second comparison target information I2.

[0019] Furthermore, the information comparison device 1 according to this embodiment is not limited to determining whether the first comparison target information I1 and the second comparison target information I2 are identical, but may also determine whether there is logical consistency between the first comparison target information I1 and the second comparison target information I2. For example, as will be described later, if the first comparison target information I1 extracted by the generation AI 10 from the document to be checked S is "net asset amount has increased compared to the previous year," but the "net asset amount for the previous year" and the "net asset amount for the current year" extracted as the second comparison target information I2 by the information extraction unit 14 from the table to be checked T are the same (no change), then the two pieces of information I1 and I2 are not logically consistent, and at least one of them may be incorrect. It is preferable to present or notify accounting auditors, etc., of this fact in order to draw attention to it or to allow for correction.

[0020] In the information comparison device 1 according to this embodiment, the extraction of the first comparison target information I1 from the first document D1 and the extraction of the second comparison target information I2 from the second document D2 may be performed in different ways. For example, the first comparison target information I1 is extracted by the generating AI 10 from the text (text to be checked) S contained in the first document D1. The generating AI 10 analyzes the text to be checked S in response to a prompt P such as "What is the net asset amount at the end of fiscal year 20XX?" input by the prompt input unit 12 described later, and extracts the first comparison target information I1 such as the net asset amount "YYY yen" from the text to be checked S.

[0021] On the other hand, the second comparative information I2 is extracted by the information extraction unit 14 from the table (confirmation table or confirmation section) T included in the second document D2. Since the confirmation table T included in the second document D2 as an accounting audit document follows a predetermined format, it is relatively easy to identify each account title, such as "Net Assets (Total)," as a confirmation item in the confirmation table T. For this reason, an advanced artificial intelligence system based on machine learning or deep learning, such as the aforementioned generation AI 10, is not necessary to extract the second comparative information I2, such as the net asset amount "ZZZ yen," from the confirmation table T. The format of the confirmation table T included in the second document D2 as an accounting audit document is typically determined according to the accounting standards adopted by the audited company. The information comparison device 1 according to this embodiment is applicable regardless of accounting standards such as J-GAAP, US-GAAP, and IFRS.

[0022] The information extraction unit 14 may extract second comparison information I2, such as the net asset amount "ZZZ yen," from the verification target table T based on an information extraction rule R manually created by an accounting auditor or the like (for example, "The net asset amount for the current fiscal year (20XX) is in the nth cell to the right of the "Net Assets" cell."). Thus, the information extraction unit 14 may be implemented by a rule-based computer program or appropriate hardware. Note that the verification target from which the information extraction unit 14 extracts the second comparison information I2 is not limited to a table (verification target table T), and its format or description method is arbitrary as long as it is expressed in a predetermined format that allows for routine processing based on the given rule R.

[0023] In the example in Figure 1 (and subsequent examples), the extraction of the first comparison target information I1 from the first document D1 was performed by a machine learning-based generative AI 10, and the extraction of the second comparison target information I2 from the second document D2 was performed by a rule-based information extraction unit 14. However, the extraction of the second comparison target information I2 from the second document D2 may also be performed by a machine learning-based generative AI (which may be the same as the generative AI 10 mentioned above, or it may be a different one). Note that the first document D1 containing the text to be checked S and the second document D2 containing the table to be checked T (the part to be checked) may be different documents, or they may be different parts of the same document.

[0024] Furthermore, the first document D1 and the second document D2 are not limited to audited documents such as accounting audits. For example, the information comparison device 1 according to this embodiment can be applied to the analysis and / or comparison of legal documents such as contracts and other documents of any field or type. For example, as described above, the information comparison device 1 according to this embodiment extracts first comparison target information I1 from the document to be verified S (e.g., a contract) contained in the first document D1 using the generation AI 10, and extracts second comparison target information I2 from the document to be verified table T (e.g., a term sheet summarizing the contract) contained in the second document D2 using the information extraction unit 14. In particular, the information comparison device 1 according to this embodiment is advantageous in that it can accurately extract first comparison target information I1 from the document to be verified S written in a free format or free style using the generation AI 10.

[0025] Figure 2 is a schematic functional block diagram of the information comparison device 1 according to this embodiment. The information comparison device 1 comprises a generation AI 10, a text specification unit 11, a prompt input unit 12, a table specification unit 13, an information extraction unit 14, an information comparison unit 15, and a mismatch presentation unit 16. Some of these functional blocks can be omitted as long as the information comparison device 1 can realize at least some of the operations and / or effects described below (or previously mentioned). These functional blocks are realized through the cooperation of hardware resources such as the central processing unit, memory, input devices, output devices, and peripheral devices connected to the computer, and software executed using them. Regardless of the type or location of the computer, each of the above functional blocks may be realized with the hardware resources of a single computer, or it may be realized by combining hardware resources distributed across multiple computers.

[0026] The document specification unit 11 specifies the document to be verified S in the first document D1 for extraction of the first comparison information I1. As will be described later, the document to be verified S may be a specific document of any length, such as a specific chapter, section, article, or paragraph, included in the first document D1. Such specification of the document to be verified S by the document specification unit 11 may be performed based on document specification rules (described later) manually created by the accounting auditor or the audited company, etc., through a user device D such as a personal computer.

[0027] The prompt input unit 12 inputs a prompt P to the generative artificial intelligence (generative AI) 10, specifying items to be checked that may be included in the document S to be checked, which is specified by the document specification unit 11, and prompting the AI ​​to extract first comparison information I1 related to the items to be checked from the document S. As illustrated in Figure 1, the prompt input unit 12 inputs a prompt P such as "What is the net asset amount at the end of fiscal year 20XX?" to the generative AI 10 in order to extract the amount (first comparison information I1) of the account title "net assets" (item to be checked) that may be included in the document S from the document S. The input or generation of such prompt P may be performed autonomously by the prompt input unit 12 based on rules, etc., or it may be performed or assisted (assistance to the prompt input unit 12) manually by an accounting auditor or audited company, etc., through a user device D.

[0028] The generating AI 10 extracts first comparison information I1 (e.g., the amount of the account) related to a desired item to be checked (e.g., an account) from the document to be checked S specified by the document specification unit 11, in response to the prompt P entered by the prompt input unit 12. The first comparison information I1 generated or output by the generating AI 10 is preferably a standardized (or conforming to a predetermined format) concise and short text information, such as "Net Asset Amount (FY20XX): YYY yen" in the example in Figure 1.

[0029] However, the format of the first comparison target information I1 is arbitrary, as long as the information comparison unit 15, described later, can appropriately compare it with the second comparison target information I2 (for example, it may be in tabular format or image format). Here, it is preferable that the first comparison target information I1 and the second comparison target information I2 are expressed in similar formats so that they can be easily compared with each other. For example, if the first comparison target information I1 is in tabular format, it is preferable that the second comparison target information I2 be expressed as similar tabular format information that can be directly compared with it.

[0030] The generative AI 10, composed of a neural network, is an artificial intelligence system based on a large-scale language model (LLM) that has learned from a vast amount of text. This generative AI 10, possessing such natural language processing capabilities, understands prompts P given in natural language and analyzes the target text S written in natural language. Therefore, the generative AI 10 can accurately extract desired target items and / or first comparison information I1 from the target text S according to the given prompts P.

[0031] The table designation unit 13 designates the verification target table T (verification target section) to be extracted from the second comparison information I2 in the second document D2, which is different from (or the same as) the first document D1. The verification target table T is a specific table designated by the table designation unit 13 from among various tables that mainly represent the financial and accounting situation numerically, such as financial statements, financial results summaries, financial results briefing materials, quarterly reports, securities reports, etc., which are included in the audited documents (second document D2). The designation of verification target sections such as the verification target table T by the table designation unit 13 may be performed based on table designation rules manually created by the accounting auditor or the audited company via the user device D. Furthermore, as described above, since the verification target sections such as the verification target table T are expressed according to a predetermined format, the information extraction unit 14, which will be described next, can accurately extract the second comparison information I2 based on rules created in accordance with that format.

[0032] The information extraction unit 14 extracts second comparison information I2 (e.g., the amount of the account) related to desired items to be verified (e.g., account titles) that may be included in the verification target table T, which is expressed according to a predetermined format, based on that format. The second comparison information I2 extracted by the information extraction unit 14 is preferably a standardized (or according to a predetermined format) concise and short text information, such as "Net Asset Amount (FY20XX): ZZZ yen" in the example in Figure 1.

[0033] As mentioned above, the second comparison information I2 does not have to be text information, but it is preferable that it be expressed in the same format as the first comparison information I1 so that it can be easily compared with the first comparison information I1. Such information extraction from the verification target section, such as the verification target table T, by the information extraction unit 14 may be performed based on information extraction rules R (for example, "The net asset amount for the current fiscal year (20XX) is in the nth cell to the right of the "Net Assets" cell" shown in Figure 1) that are manually created by the accounting auditor or the audited company via the user device D.

[0034] The information comparison unit 15 compares the first comparison target information I1 extracted by the generation AI 10 from the document to be verified S with the second comparison target information I2 extracted by the information extraction unit 14 from a different document to be verified table T (verification target section). In the example in Figure 1, the information comparison unit 15 compares "YYY yen," which is numerical or monetary information contained in the first comparison target information I1, "Net asset amount (FY20XX): YYY yen," with "ZZZ yen," which is numerical or monetary information contained in the second comparison target information I2, "Net asset amount (FY20XX): ZZZ yen." More precisely, the information comparison unit 15 checks whether the "YYY yen" derived from the first comparison target information I1 and the "ZZZ yen" derived from the second comparison target information I2 are substantially the same. In a broader sense, the information comparison unit 15 checks whether there is logical consistency between the first comparison target information I1, which may be non-numerical information, and the second comparison target information I2, which may also be non-numerical information, as described above.

[0035] The discrepancy notification unit 16 presents or informs the accounting auditor or audited company, etc., via the user device D, etc., that, as a result of the comparison by the information comparison unit 15, the numerical information and / or non-numerical information contained in the first comparison target information I1 and the second comparison target information I2 do not match. In addition to or instead of the discrepancy notification unit 16, a discrepancy notification unit (not shown) may be provided that presents or informs the accounting auditor or audited company, etc., via the user device D, etc., that, as a result of the comparison by the information comparison unit 15, there is no logical consistency between the first comparison target information I1 and the second comparison target information I2.

[0036] Next, an embodiment of the information comparison device 1 according to this embodiment will be described based on specific processing flow examples and user device D screen examples. In the flowchart, "S" means a step or process. Also, the contents described above regarding Figures 1 and 2 will not be repeated.

[0037] Figure 3 is a flowchart schematically showing a specific processing flow example for the generation or extraction of the first comparison target information I1. In S1, the first document D1 is input or registered in the information comparison device 1. The first document D1 is, for example, one of the accounting audit target documents included in the accounting audit target document group, and may be an electronic document in any file format such as PDF (Portable Document Format).

[0038] In S2, the consistency check rule R is input to or registered in the information comparison device 1. The consistency check rule R may be a spreadsheet file in any file format, such as Excel®. Figure 4 shows an example of the consistency check rule R. This consistency check rule R is an information extraction rule R (see Figure 1) used by the aforementioned information extraction unit 14 to extract the second comparison target information I2 from the table T to be checked, with the addition of information such as a chapter ID or chapter number (Chapter No) that specifies a particular chapter (text to be checked S) contained in the first document D1.

[0039] The information in each column on the right side of Figure 4 (from the second column onwards, indicated as "Source Marker Identifier") defines the information extraction rule R that the information extraction unit 14 primarily uses. Specifically, "No" defines the ID or number of the information extraction rule R, "x1" to "x3" specify the table T to be checked, "column" specifies the consolidated fiscal year in the table T to be checked (for example, the current consolidated fiscal year or the previous consolidated fiscal year) (as described later, in the table T to be checked for a typical accounting audit document, the information for each consolidated fiscal year is displayed in each "column", hence the name "column"), and "row" specifies the account title (item to be checked) in the table T to be checked (as described later, in the table T to be checked for a typical accounting audit document, the information for each account title is displayed in each "row", hence the name "row").

[0040] As will be explained in more detail later, for example, the information extraction rule R for "P101" in the first row of Figure 4 extracts numerical information (i.e., monetary information) for "Total Current Assets" (row) for "Current Consolidated Fiscal Year" (column) from the "Consolidated Balance Sheet" (x3) included in the section identified by the main heading "About Audit Certificates" (x1) and the subheading "Consolidated Financial Statements, etc." (x2) in the second document D2, using the "Consolidated Balance Sheet" (x3) as the table to be checked. If one or more chapter IDs are registered in the "Chapter No." in the first column of Figure 4 for such an information extraction rule R, the chapter specification rule or document specification rule R11 will be referenced in S4, which will be explained later.

[0041] In S3 in Figure 3, the document specification unit 11, etc., generates a list L of items to be checked for consistency between the first document D1 (entered in S1) and the second document D2, based on the consistency check rule R (Figure 4) entered in S2. This list of items to be checked L may be, for example, extracted from the "row" information in Figure 4 where account titles as items to be checked are listed, without duplication, or it may be extracted from the pair of "column" information specifying the consolidated fiscal year and "row" information specifying the account titles in Figure 4. For example, from the first row in Figure 4, "Total Current Assets" (row) for "Current Consolidated Fiscal Year" (column) is listed as an item to be checked in the first document D1 (checked document S).

[0042] In S4 in Figure 3, the document specification unit 11 inputs or references document specification rule R11 according to the consistency check rule R (Figure 4) entered in S2. For example, in the consistency check rule R of the first line in Figure 4, the document specification unit 11 searches for or references the chapter ID "C05" in the document specification rule R11 exemplified in Figure 5.

[0043] The information in each column in Figure 5 defines the document specification rule R11 used by the document specification unit 11. Specifically, the aforementioned chapter ID, "Chapter No" and / or "Parent Chapter No," defines the ID or number of the document specification rule R11, and "x1" to "x4" specify information to identify the chapter or document S to be checked in the first document D1. For example, the document specification rule R11 "C05" in the third row of Figure 5, which is specified by the information extraction rule R "P101" in the first row of Figure 4, specifies the chapter identified in the first document D1 by the major heading "Place for Public Inspection" (x1) and the subheading "Status of the Business" (x2).

[0044] In S5 in Figure 3, the document specification unit 11 extracts chapter C to be reviewed in the first document D1 based on the document specification rule R11 entered or referenced in S4. For example, the document specification unit 11 extracts chapter C from the first document D1 that has the subheading "Business Status" (x2) as schematically shown in Figure 6, from the description following the main heading "Place for Public Review" (x1). This is based on the document specification rule R11 of "C05" on the third line in Figure 5, which was specified by the information extraction rule R of "P101" on the first line in Figure 4.

[0045] In S6 in Figure 3, the text specification unit 11 extracts one or more consecutive paragraphs S from the chapter C (Figure 6) extracted in S5 that contain descriptions of each item to be checked (each account) in the list of items to be checked L generated in S3. For example, the text specification unit 11 searches the chapter C extracted in S5 as the search target, sequentially searching for each account listed in the list of items to be checked L, and identifies one or more paragraphs S that contain each account.

[0046] In this case, the text specification unit 11 may extract paragraph S that contains a description of the fiscal year of each account (the "row" information in Figure 4) in addition to each account listed in the verification target item list L and / or consistency check rule R (Figure 4). Furthermore, the text specification unit 11 may extract paragraph S that contains some monetary information that may become the first comparison target information I1 to be subsequently generated by the generation AI 10.

[0047] Figure 7 shows an example (different from Figure 6) in which, in S6, the document specification unit 11 extracts paragraph S from chapter C containing desired account titles and accounting periods, and some monetary information, based on the verification target item list L and / or consistency check rule R. In this example, desired account titles (verification target items) include "Cash flows from operating activities," "Operating cash flow," "Operating CF," etc., and desired accounting periods include "Current consolidated fiscal year," "Current fiscal year," "Current accounting period," etc., as specified in the verification target item list L and / or consistency check rule R (the pair of "column" and "row" information in Figure 4).

[0048] Of the multiple paragraphs included in Chapter C, paragraph S, enclosed by a dotted line in Figure 7, contains both the description of "Cash Flows from Operating Activities" as the desired account (dotted line) and the description of "Current Consolidated Fiscal Year" as the desired accounting period (solid line). Furthermore, since paragraph S contains at least one monetary value (dotted-dotted line), the document specification unit 11 extracts paragraph S as the document to be verified for the desired account and accounting period based on the verification target item list L and / or consistency check rule R. At this point, it is not necessary to identify which of the various monetary values ​​contained in paragraph S represents the verification target item (Cash Flows from Operating Activities) (this will be identified by the generation AI 10 in a later stage).

[0049] In the examples in Figures 3 and 7, the paragraph S extracted in S6 becomes the text to be checked S in Figures 1 and 2. By using a relatively short paragraph S as the text to be checked S in this way, the load on the generating AI 10 can be reduced, and the generation accuracy of the first comparison target information I1 can be improved. Furthermore, by adopting a stepwise approach in which one or more chapters C (first text to be checked) are extracted from the first document D1 as part of the first text specification step (S5), and one or more paragraphs S (second text to be checked) are extracted from chapter C as part of the second text specification step (S6), paragraph S can be efficiently extracted from the first document D1 while improving the extraction accuracy at each step. Note that the text to be checked S is not limited to paragraph S, but may be any text of any location and length contained in the first document D1, and may be the entirety of chapter C as shown in Figures 6 and 7, or the entirety of the first document D1.

[0050] In S7 in Figure 3, the prompt input unit 12 inputs a prompt P to the generating AI 10, specifying the items to be checked that may be included in the paragraph S (text to be checked) extracted in S6, and prompting the AI ​​10 to extract the first comparison information I1 related to the items to be checked from paragraph S. As mentioned above, the items to be checked in the first document D1 (for example, account titles and / or their fiscal year) are listed in the consistency check rule R (Figure 4) and / or the list of items to be checked L. Therefore, the prompt input unit 12 refers to the consistency check rule R and / or the list of items to be checked L, selects the account titles (information labeled "row" in Figure 4) and / or fiscal year (information labeled "column" in Figure 4) listed therein one by one in order, generates a prompt P related to each account title and / or fiscal year, and inputs it to the generating AI 10.

[0051] For example, in the example in Figure 7, the prompt input unit 12 generates a prompt P such as "What is the amount of cash flow from operating activities for the current consolidated fiscal year?" and inputs it to the generation AI 10 in order to extract the amount information of "current consolidated fiscal year" (information recorded in "column" in Figure 4) for the account "cash flow from operating activities" (information recorded in "row" in Figure 4) as an item to be checked from paragraph S. Such prompts P can be autonomously generated by the prompt input unit 12 based on the accounting period (column) and account (row) pairs recorded in the consistency check rule R and / or the item to be checked list L.

[0052] In the subsequent S8, the generating AI10 analyzes paragraph S in Figure 7 in response to the prompt P entered in S7, and identifies or extracts "4.1 billion yen" (revenue) as the monetary information for "cash flow from operating activities for the current consolidated fiscal year" related to the prompt P. As a result, the generating AI10 generates the first comparison information I1, such as "Cash flow from operating activities for the current consolidated fiscal year is +4.1 billion yen," and "Operating CF (current consolidated fiscal year): +4.1 billion yen."

[0053] As mentioned above, the first comparison information I1 generated by the generating AI 10 is subsequently compared with the second comparison information I2, so it is required to be accurate (if the first comparison information I1 is inaccurate, a useful comparison with the second comparison information I2 cannot be made in the first place). On the other hand, the generating AI 10 tends to try to force an answer even when the given information is insufficient.

[0054] Therefore, it is preferable to devise a prompt P that is input to the generating AI 10 in S7 so that the generating AI 10 does not generate such unreasonable and inaccurate answers (first comparison information I1). For example, by including a standard phrase such as "Please output an answer to the question based on the given text. Answer based on the information contained in the text, and if the information for the question is insufficient, answer 'I don't know'." at the beginning of prompt P, the generating AI 10 may be prompted to answer "I don't know" if the information in paragraph S is insufficient.

[0055] Furthermore, the information comparison unit 15 or the information comparison device 1 may check whether the first comparison target information I1 generated by the generation AI 10 in S8 is appropriate. For example, the information comparison unit 15 may check whether the first comparison target information I1 generated by the generation AI 10 contains only one amount of information (and not multiple amounts of information), and whether the amount of information is actually described in the source paragraph S. If no problems are found in this check, the first comparison target information I1 is provided to the information comparison unit 15 as valid.

[0056] Figure 8 is a schematic flowchart illustrating a specific processing flow example related to the generation of the second comparison target information I2 and its comparison with the first comparison target information I1 (S8 in Figure 3). In S9, the second document D2 is input or registered in the information comparison device 1. The second document D2 is, for example, one of the audited documents included in the audited document group, and may be an electronic document in any file format such as PDF.

[0057] In S10, the table designation unit 13, based on the consistency check rule R entered or registered in S2 (Figure 3), designates or extracts a table T to be checked as the part to be checked for consistency with the first document D1 in the second document D2 entered in S9. Specifically, the consistency check rule R exemplified in Figure 4 designates a specific table T to be checked in the second document D2 in which the information in columns "x1" to "x3" is included. For example, based on the information extraction rule R "P101" in the first row of Figure 4, the table designation unit 13 extracts a table T from the second document D2 that has the subheadings "Consolidated Financial Statements, etc." (x2) and "Consolidated Balance Sheet" (x3), as schematically shown in Figure 9, from the description following the main heading "About Audit Certificates" (x1) in the second document D2.

[0058] In S11, the information extraction unit 14 extracts second comparison information I2 (for example, the amount of the account) related to the desired verification item (account) included in the verification target table T (Figure 9), which is expressed according to a predetermined format, based on the format and information extraction rule R. For example, based on the information extraction rule R of "P101" in the first row of Figure 4, the information extraction unit 14 extracts the amount information (i.e., the amount of the total current assets for the current consolidated fiscal year) as second comparison information I2, which is located at the intersection of the column for "Current Consolidated Fiscal Year" specified by "column" (the rightmost column in Figure 9) and the row for "Total Current Assets" specified by "row" (the bottommost row in Figure 9).

[0059] In S12, the information comparison unit 15 compares the first comparison target information I1 generated in S8 (Figure 3) with the second comparison target information I2 extracted from the verification target table T (Figure 9) in S11. In S13, the mismatch notification unit 16 and the inconsistency notification unit (not shown) present or inform the accounting auditor and the audited company, etc., via the user device D, etc., that, as a result of the comparison in S12, the numerical information and / or non-numerical information contained in the first comparison target information I1 and the second comparison target information I2 do not match, or that there is no logical consistency between the first comparison target information I1 and the second comparison target information I2.

[0060] Furthermore, the units of display for numerical information such as amounts may differ between the document S to be verified in the first document D1 and the table T (verification section) to be verified in the second document D2. In such cases, it is preferable for the information comparison unit 15 and the discrepancy notification unit 16 to determine whether the actual numerical values ​​are equivalent, regardless of the difference in the units of display for the numerical information. For example, if the amount of a certain account is expressed as "41 (billion yen)" in the first comparison information I1 and as "4,126 (million yen)" in the second comparison information I2, it may be determined that the two are substantially the same.

[0061] Figures 10 and 11 schematically show typical screen examples displayed on the user device D used by the accounting auditor and / or audited company in the series of processes shown in Figure 3 and / or Figure 8. For convenience, they are shown in two separate figures, 10 and 11, but in reality, it is preferable that they be displayed on a single screen. For example, Figure 10 shows the left half of a single screen, and Figure 11 shows the right half.

[0062] This screen is provided with two document display areas DP1 and DP2. The first document display area DP1 displays a specific part of a specific document selected in the first selection area SL1, and the second document display area DP2 displays a specific part of a specific document selected in the second selection area SL2. In this embodiment, as shown in Figures 10 and 11, it is preferable to display the confirmation target document S of the first document D1 in one of the document display areas DP1 and DP2, and to display the confirmation target table T (confirmation target section) of the second document D2 in the other of the document display areas DP1 and DP2. However, the user can arbitrarily select what to display in each document display area DP1 and DP2 in the selection areas SL1 and SL2. For example, one part of the first document D1 may be displayed in the first document display area DP1, and another part of the first document D1 may be displayed in the second document display area DP2.

[0063] The comparison results display area RS shows a summary of the comparison results between the first comparison target information I1 and the second comparison target information I2 by the information comparison unit 15, on a per-account (confirmation target item) basis for each fiscal year. For example, for accounts that match or are consistent throughout the entirety of the first comparison document D1 and the second comparison document D2 (for example, "Cash flows from operating activities" (previous consolidated fiscal year) in Figure 11), marks such as "◎" or "〇" indicating "match" or "consistency" are displayed.

[0064] Furthermore, for accounts that do not match or are inconsistent between the comparable first document D1 and second document D2 (for example, "Cash flows from operating activities" (for the current consolidated fiscal year) in Figure 11), a mark such as "×" (one form of the discrepancy indication section 16) indicating "discrepancy" or "inconsistency" is displayed. In addition, for accounts where it is not possible to clearly determine whether they match or are inconsistent between the comparable first document D1 and second document D2 (for example, "Total Net Assets" in Figure 11), a mark such as "▲" or "△" indicating "unknown" is displayed.

[0065] In the example in Figure 11, the "Cash Flows from Operating Activities" (Current Consolidated Fiscal Year), which has a comparison result of "×", is selected, and the detailed display area DT, which shows its details, is expanded. In this detailed display area DT, all locations in the first document D1 and / or the second document D2 in which the amount of "Cash Flows from Operating Activities" (Current Consolidated Fiscal Year) is listed (i.e., all locations in which the first comparable information I1 and / or the second comparable information I2 for that account are extracted) are selectable by the selection areas SL1 and SL2.

[0066] In the example in Figure 11, the "Business Overview" section on page "17" of the first document D1 is selected in the first selection area SL1. Accordingly, the first document display area DP1 (Figure 10) displays the document S to be verified, which contains the amount "4.165 billion yen" for "Cash Flows from Operating Activities" (for the current consolidated fiscal year) on that page and in that section. The "4.165 billion yen" extracted by the generation AI 10 as the first comparison information I1 is then highlighted within the first document display area DP1. Furthermore, this "4.165 billion yen" as the first comparison information I1 is also summarized and displayed in the detail display area DT.

[0067] Meanwhile, in the second selection area SL2, the "Consolidated CF" section on page "50" of the second document D2 is selected. In accordance with this selection, the second document display area DP2 displays the verification target table T, which contains the amount "4,165,031" for "Cash Flows from Operating Activities" (for the current consolidated fiscal year) on that page and in that section. The "4,165,031" extracted by the information extraction unit 14 as the second comparison target information I2 is then highlighted within the second document display area DP2. Furthermore, this "4,165,031" as the second comparison target information I2 is also summarized and displayed in the detail display area DT.

[0068] In the example in Figure 11, the first comparison information I1, "4.1065 billion yen," and the second comparison information I2, "4,165,031" (4.16503 billion yen), do not match, so it is judged as "×" (mismatch).

[0069] Through the screen described above, users such as accounting auditors and audited companies can easily grasp the comparison results of the first comparison information I1 and the second comparison information I2 while referring to the actual first document D1 (document to be checked S) and second document 2 (table to be checked T) on a single screen, and efficiently correct any discrepancies or inconsistencies. Such correction work may be performed manually by the users such as accounting auditors and audited companies, or it may be performed automatically by the information comparison device 1.

[0070] The present disclosure has been described above based on embodiments. Various modifications are possible for each component and each combination of processes in the exemplary embodiments, and it will be obvious to those skilled in the art that such modifications are included within the scope of the present disclosure.

[0071] In the embodiments shown in Figure 1, etc., it was primarily assumed that the first document D1 and the second document D2, which are the comparison subjects, were documents created in the same fiscal year. However, the information comparison device relating to this disclosure may also compare documents created in different fiscal years or at different times.

[0072] For example, a first document D1, created earlier in time, and a second document D2, created later in time, may be compared using the information comparison device relating to this disclosure. Such a "newer" second document D2 may include descriptions of the "past" described in the "older" first document D1. If the subject matter described in both documents D1 and D2 is the same, the first comparison information I1 and the second comparison information I2 contained in each should match. The information comparison device relating to this disclosure can accurately compare and check the consistency of descriptions of the same subject contained in the first document D1 and the second document D2, which were created at different times.

[0073] Figure 12 schematically shows a modified version of the information comparison device 1 for comparing the first document D1 and the second document D2, which were created at different times. This modified version is a variation of the embodiment shown in Figure 1, and components similar to those in Figure 1 and other figures are denoted by the same reference numerals, and redundant explanations are omitted.

[0074] In this modified example, Document 1 D1 was created in fiscal year N-1, and Document 2 D2 was created in the following fiscal year, fiscal year N. Therefore, the current fiscal year in Document 1 D1 (i.e., fiscal year N-1) corresponds to the previous fiscal year in Document 2 D2 (i.e., fiscal year N-1).

[0075] The information comparison device 1 in this modified example compares information regarding items to be verified in fiscal year N-1 that are included in common in the first document D1 and the second document D2. Specifically, the first comparison target information I1 for the current fiscal year, extracted from the first document D1, and the second comparison target information I2 for the previous fiscal year, extracted from the second document D2, are compared by the information comparison device 1.

[0076] In the example in Figure 12, Document 1 D1 and Document 2 D2 are the documents subject to audit, such as an accounting audit. Furthermore, as items to be checked that are common to Document 1 D1 and Document 2 D2 as documents subject to accounting audit, examples include account titles such as "Net Assets (Total)" that are listed in financial statements, and the amounts of the said account title "Net Assets" are exemplified as the first comparable information I1 and second comparable information I2.

[0077] In this case, the first comparative information I1 for the current year, extracted from the first document D1 regarding the account "Net Assets," is in the format of, for example, "Net Asset Amount (Year N-1): YYY yen," and the second comparative information I2 for the previous year, extracted from the second document D2 regarding the account "Net Assets," is in the format of, for example, "Net Asset Amount (Year N-1): ZZZ yen." The net asset amount "YYY yen" as the first comparative information I1 and the net asset amount "ZZZ yen" as the second comparative information I2, extracted from the first document D1 and the second document D2 for the same fiscal year (Year N-1), should match each other.

[0078] The information comparison device 1 according to this embodiment compares numerical information that should match each other, for example, and efficiently checks their consistency. For example, if the net asset amount "YYY yen" in the first comparison information I1 for the current year as seen from the first document D1 and the net asset amount "ZZZ yen" in the second comparison information I2 for the previous year as seen from the second document D2 do not substantially match, the device presents or notifies the accounting auditor or the like to that effect and prompts them to pay attention or make corrections.

[0079] The first comparison information I1 may be extracted by the generating AI 10 from the text S1 contained in the first document D1. For example, the generating AI 10 extracts the first comparison information I1 from the text S1 contained in the first document D1 based on the prompt P1, "What is the net asset amount at the end of the current fiscal year (fiscal year N-1 for the first document D1)?".

[0080] Similarly, the second comparative information I2 may be extracted by the generating AI 10 from the text S2 contained in the second document D2. For example, the generating AI 10 extracts the second comparative information I2 from the text S2 contained in the second document D2 based on the prompt P2, "What was the net asset amount at the end of the previous fiscal year (fiscal year N-1 for the second document D2)?". Note that if the first document D1 and the second document D2 are of the same type (but created at different times), the first comparative information I1 and the second comparative information I2 can be extracted by essentially the same single generating AI 10.

[0081] As shown in the example in Figure 1, at least one of the first comparison target information I1 and the second comparison target information I2 may be extracted from a standardized table (corresponding to table T in Figure 1) or a section to be verified included in the first document D1 and the second document D2, rather than from the texts S1 and S2 included in the first document D1 and the second document D2. It is preferable that such information extraction from a table or section to be verified is performed by the information extraction unit 14 described above.

[0082] The configuration, operation, and function of each device and method described in the embodiments can be realized by hardware resources or software resources, or by the cooperation of hardware resources and software resources. Hardware resources include, for example, processors, ROMs, RAMs, and various integrated circuits. Software resources include, for example, operating systems and application programs. [Explanation of Symbols]

[0083] 1 Information comparison device, 10 Generating AI, 11 Text specification unit, 12 Prompt input unit, 13 Table specification unit, 14 Information extraction unit, 15 Information comparison unit, 16 Inconsistency presentation unit, D1 First document, D2 Second document, I1 First comparison target information, I2 Second comparison target information.

Claims

1. A prompt input unit that inputs prompts to a generative artificial intelligence (generative AI) to specify items to be checked that may be included in the text to be checked, and to prompt the AI ​​to extract first comparison information related to those items from the text to be checked. An information comparison unit compares the first comparison information with second comparison information relating to the items to be checked, which is extracted from a different section of the document to be checked than the first comparison information. Equipped with, The aforementioned document and section subject to review are included in the same or different audited documents. The items to be checked are the account titles included in the audited document, The first and second comparison information are the amounts of the account titles in the information comparison device.

2. The information comparison device according to claim 1, further comprising an information extraction unit that extracts second comparison target information relating to the items to be confirmed that may be included in the item to be confirmed from the item to be confirmed, which is expressed in a predetermined format, based on the format.

3. The information comparison device according to claim 2, wherein the part to be checked is a table in a predetermined format.

4. The information comparison device according to any one of claims 1 to 3, wherein the first document containing the text to be verified and the second document containing the part to be verified are different from each other.

5. The information comparison device according to any one of claims 1 to 3, wherein the first comparison target information and the second comparison target information are numerical information that should match each other.

6. The information comparison device according to any one of claims 1 to 3, further comprising a discrepancy notification unit that indicates, as a result of the comparison by the information comparison unit, that the first comparison target information and the second comparison target information do not match.

7. The first document includes a document specification section for specifying the document to be confirmed, The prompt input unit inputs a prompt to the generating AI to extract the first comparison target information from the confirmation target text specified by the text specification unit. An information comparison device according to any one of claims 1 to 3.

8. The process involves inputting prompts to the generating AI to specify items to be checked that may be included in the document to be checked, and prompting it to extract first comparison information related to those items from the document to be checked. The process involves comparing the second comparison information, which pertains to the items to be checked and is extracted from a different section of the document to be checked than the first comparison information, This is done by a computer, The aforementioned document and section subject to review are included in the same or different audited documents. The items to be checked are the account titles included in the audited document, An information comparison method in which the first and second comparison information are amounts of the account titles.

9. The process involves inputting prompts to the generating AI to specify items to be checked that may be included in the document to be checked, and prompting it to extract first comparison information related to those items from the document to be checked. The process involves comparing the second comparison information, which pertains to the items to be checked and is extracted from a different section of the document to be checked than the first comparison information, Have the computer run it, The aforementioned document and section subject to review are included in the same or different audited documents. The items to be checked are the account titles included in the audited document, The first comparison information and the second comparison information are stored in a storage medium that stores an information comparison program in which the amounts of the account titles are stored.

Images