Document evaluation criteria extraction device and document evaluation criteria extraction method

Abstract

The present invention streamlines the collection of evaluation criteria for documents with configuration. [Solution] The document evaluation criterion extraction device (document processing device 100) comprises an evaluation criterion extraction unit 115 that creates a check content acquisition prompt that extracts check contents for each item from a checklist document that includes check contents for each document item, and an answer acquisition unit 118 that transmits the check content acquisition prompt to a language model server 210 and acquires the check contents directly or indirectly from the language model server 210. The evaluation criterion extraction unit 115 stores the check contents in a storage unit 130 (document evaluation report rule database 150) as evaluation criteria related to the items.

Description

[Technical Field]

[0001] The present invention relates to a document evaluation criterion extraction device and a document evaluation criterion extraction method for extracting evaluation criteria when evaluating a document. [Background technology]

[0002] Natural language processing using large-scale language models is becoming more common, and attention is being paid to using this technology to automate and streamline document processing such as evaluations / reviews, which have traditionally been done by humans. Evaluations / reviews require evaluation criteria, check items, and check rules, which must be prepared in advance.

[0003] Patent Document 1 describes a review support device as an example of a system related to evaluation / review. The review support device includes a storage device that stores location attribute management information that defines acquisition items for location information of the material to be reviewed. The review support device also includes an input processing unit that receives from the user a specification of a location in the material to be reviewed where a comment should be added and input of the comment. The review support device also includes a location information acquisition unit that acquires location information from a location specified by the user in accordance with the location attribute management information in the storage device, associates the location information with the comment, and stores the associated information in the storage device. The review support device also includes a search unit that searches the storage device for location information and comment related to the material to be reviewed. The review support device also includes a display processing unit that displays, together, location information and comment related to the material to be reviewed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-348310 Summary of the Invention [Problem to be solved by the invention]

[0005] Documents used in business are structured into chapters, items, etc., and are evaluated / reviewed according to the evaluation criteria, check items, and check rules that correspond to the chapters and items. The review support device described in Patent Document 1 is not suitable for collecting evaluation criteria, check items, and check rules that correspond to the structure of the document, such as chapters and items.

[0006] The present invention has been made in view of the above background, and an object of the present invention is to provide a document evaluation criterion extraction device and a document evaluation criterion extraction method that make it possible to efficiently collect evaluation criteria for documents having a structure. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems, a document evaluation criterion extraction device according to the present invention comprises: The system includes an evaluation criterion extraction unit that references minutes including the results of document review and requests extraction of evaluation criteria for each item of the document, merging the items of the document, and the minutes to create an evaluation criterion acquisition prompt, and an answer acquisition unit that sends the evaluation criterion acquisition prompt to a language model server and acquires the evaluation criteria directly or indirectly from the language model server, and the evaluation criterion extraction unit stores the evaluation criteria in a memory unit as evaluation criteria related to the items. Furthermore, the document evaluation criterion extraction device according to the present invention includes an evaluation criterion extraction unit that creates an evaluation criterion acquisition prompt by referencing minutes of a meeting including the results of reviewing the document and merging the minutes with text requesting the extraction of evaluation criteria for the document, and an answer acquisition unit that sends the evaluation criterion acquisition prompt to a language model server and acquires the evaluation criteria directly or indirectly from the language model server, and the evaluation criterion extraction unit stores the evaluation criteria in a memory unit. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide a document evaluation criteria extraction device and a document evaluation criteria extraction method that efficiently collect evaluation criteria for documents having a structure. Problems, configurations, and effects other than those described above will become clear from the description of the following embodiments. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a functional block diagram of a document processing apparatus according to an embodiment of the present invention; [Figure 2] FIG. 2 is a diagram showing an example of a tabular document according to the present embodiment. [Figure 3] FIG. 10 is a diagram showing a format information document of a table format document according to the present embodiment. [Figure 4] FIG. 10 is a diagram illustrating a configuration of an item value acquisition prompt according to the embodiment. [Figure 5] FIG. 10 is a diagram showing an item value acquisition result according to the embodiment. [Figure 6] 10 is a flowchart of a document analysis process according to the present embodiment. [Figure 7]10 is a flowchart of a document format acquisition process according to the present embodiment. [Figure 8] FIG. 10 is a diagram showing an item value acquisition result in text format according to a modified example of the present embodiment. [Figure 9] FIG. 10 is a diagram showing a format information document in which the item identification information is blank according to a modified example of the present embodiment. [Figure 10] 10 is a flowchart of a format information acquisition process according to the present embodiment. [Figure 11] 1 is a diagram illustrating an outline of a document evaluation reporting process performed by the document processing apparatus according to the present embodiment. [Figure 12] FIG. 10 is a diagram showing the configuration of an evaluation acquisition prompt according to the present embodiment. [Figure 13] FIG. 10 is a diagram showing evaluation results according to the present embodiment. [Figure 14] FIG. 10 is a diagram showing evaluation results according to the present embodiment. [Figure 15] FIG. 10 is a diagram showing the configuration of a report acquisition prompt according to the present embodiment. [Figure 16] FIG. 10 is a diagram showing the configuration of a report acquisition prompt according to the present embodiment. [Figure 17] FIG. 10 is a diagram showing a report according to the present embodiment. [Figure 18] FIG. 10 is a diagram illustrating a rating acquisition prompt according to the present embodiment. [Figure 19] FIG. 10 is a diagram illustrating a report acquisition prompt according to the present embodiment. [Figure 20] FIG. 10 is a diagram illustrating a rating acquisition prompt according to the present embodiment. [Figure 21] FIG. 10 is a diagram illustrating a report acquisition prompt according to the present embodiment. [Figure 22] FIG. 10 is a diagram illustrating a rating acquisition prompt according to the present embodiment. [Figure 23] FIG. 10 is a diagram illustrating a report acquisition prompt according to the present embodiment. [Figure 24] 10 is a flowchart of a document evaluation report process according to the present embodiment. [Figure 25] 10 is an example of a document including evaluation criteria according to the present embodiment. [Figure 26]FIG. 10 is a diagram showing the configuration of an evaluation criterion acquisition prompt according to the present embodiment. [Figure 27] FIG. 10 is a diagram showing acquired evaluation criteria according to the present embodiment. [Figure 28] FIG. 10 is a diagram illustrating an evaluation criterion acquisition prompt according to the present embodiment. [Figure 29] This is an evaluation criterion when the evaluation criterion acquisition prompt according to this embodiment is used. [Figure 30] FIG. 10 is a diagram showing an evaluation criterion acquisition prompt (check content acquisition prompt) according to the present embodiment. [Figure 31] 10 shows the evaluation criteria (check details) when the evaluation criteria acquisition prompt according to this embodiment is used. [Figure 32] 10 is a flowchart of an evaluation criterion extraction process according to the present embodiment. [Figure 33] 1 is a diagram illustrating an outline of a document proofreading process performed by a document processing apparatus according to an embodiment of the present invention. [Figure 34] FIG. 10 is a diagram showing the configuration of a synonym acquisition prompt according to the present embodiment. [Figure 35] FIG. 10 is a diagram showing an acquired synonym dictionary according to the present embodiment. [Figure 36] FIG. 10 is a diagram showing the configuration of a proof document acquisition prompt according to the present embodiment. [Figure 37] FIG. 10 is a diagram illustrating a synonym acquisition prompt according to the present embodiment. [Figure 38] FIG. 2 is a diagram showing a thesaurus containing meanings according to the present embodiment. [Figure 39] FIG. 10 is a diagram showing a proof document acquisition prompt according to the present embodiment. [Figure 40] 10 is a flowchart of a document proofreading process according to the present embodiment. [Figure 41] FIG. 10 is a functional block diagram of a document processing device according to a modified example of the present embodiment. [Figure 42] FIG. 2 is a hardware configuration diagram illustrating an example of a computer that realizes the functions of the document processing apparatus according to the above-described embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] <Document Processing Device Overview> An outline of a document processing device as a document analysis device, a document evaluation device, a document evaluation criterion extraction device, and a document proofreading device in a mode (embodiment) for carrying out the present invention will be described below. First, the document processing device functions as a document analysis device that extracts information from tabular documents such as spreadsheets, taking document structure into consideration. More specifically, the document processing device refers to the document format information, extracts item values corresponding to item identification information (item names), and outputs them as text.

[0011] Second, the document processing device functions as a document evaluation device that evaluates / reviews documents. In more detail, the document processing device evaluates the document by referring to the text extracted from the document and the document evaluation criteria, and outputs a report (evaluation report). Third, the document processing device functions as a document evaluation criteria extraction device that extracts document evaluation criteria (check contents) from document checklists and review meeting minutes. Fourth, the document processing device functions as a document proofreader that corrects spelling variations in a document. More specifically, the document processing device detects synonyms from the document and corrects spelling variations. The basic configuration of the document processing device will be described below, followed by a description of the functional configurations of the document analysis device, document evaluation device, document evaluation criterion extraction device, and document proofreading device in that order.

[0012] <Configuration of document processing device> 1 is a functional block diagram of a document processing device 100 according to this embodiment. The document processing device 100 is a computer, and includes a control unit 110, a storage unit 130, and an input / output unit 180. User interface devices such as a display, keyboard, and mouse are connected to the input / output unit 180. The input / output unit 180 includes a communication device, and is capable of transmitting and receiving data to and from a language model server 210 and other devices. The language model server 210 is a server for a service (dialogue-type artificial intelligence service) that accepts prompts including commands / instructions and returns answers to the prompts.

[0013] <Document processing device: memory unit> The storage unit 130 is configured to include storage devices such as a ROM (Read Only Memory), a RAM (Random Access Memory), and an SSD (Solid State Drive). The storage unit 130 stores a document database 131, a document format database 140, a document evaluation report rule database 150, an evaluation report database 160, and a program 138. Note that the various storage contents of the storage unit 130 may be stored in an external storage device such as a cloud server and read as needed.

[0014] The document database 131 stores documents to be processed by the document processing device 100 in association with document identification information. The program 138 includes a description of the processing to be executed by a functional unit included in the control unit 110, which will be described later. Other components of the storage unit 130 will be described later.

[0015] <Document processing device: control unit> The control unit 110 includes a CPU (Central Processing Unit) and is equipped with a document analysis unit 111, a document format generation unit 112, a document evaluation unit 113, an evaluation report unit 114, an evaluation criterion extraction unit 115, a dictionary generation unit 116, a document proofreading unit 117, and an answer acquisition unit 118. The answer acquisition unit 118 sends prompts generated by other functional units of the control unit 110 (described later) to the language model server 210 and receives answers directly or indirectly. The control unit 110 may include a GPU (Graphics Processing Unit), an NPU (Neural (network) Processing Unit), an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), etc.

[0016] <Document processing device: document analysis device> The following describes the document processing device 100 as a document analysis device. The document processing device 100 refers to format information (template information) of a tabular document and extracts pairs of item identification information (item name) and item value for items included in the tabular document.

[0017] ≪Document analysis device: Tabular document / format information≫ FIG. 2 is a diagram showing an example of a tabular document 410 according to this embodiment. The tabular document 410 is a spreadsheet-format document in which item values corresponding to a cell (column) containing an item name are entered / written in a cell adjacent to the cell containing the item name. In FIG. 2, the item name is written in a cell immediately to the right of the cell containing the item name. For example, the item value corresponding to the item name "requested delivery date" is "2020 / 10 / 10". Note that the item value may also be written in a cell below the item name. Note that an example of a tabular document / spreadsheet document is an Excel (registered trademark) document, but is not limited to this.

[0018] In a spreadsheet document, cells are identified by identification information that indicates their location. For example, the identification information for the cell containing the item name "Requested Delivery Date" is "B6:C7." The tabular document 410 is associated with document identification information and stored in the document database 131. The document database 131 may also store documents in text format.

[0019] FIG. 3 is a diagram showing a format information document 420 (template document) of a tabular document according to this embodiment. The cells for item names are the same as those in the tabular document 410. Cells in which item values are entered contain a character string with a "#" added to the beginning of the item name (item identification information). In the following, unless a distinction is particularly necessary, the item identification information and the item identification information with a "#" added to the beginning will not be distinguished. For example, there will be no distinction between "requested delivery date" and "#requested delivery date" as item identification information.

[0020] <Document analysis device: document format database> The document format database 140 stores the format information document 420 in association with format information identification information. Cell identification information (position information) and values extracted from the format information document 420 by the document analysis unit 111 (described later) are stored in association with the format information document 420 in the document format database 140. In other words, the document format database 140 stores format information identification, the format information document 420 (see FIG. 3), and the correspondence between the cell identification information and the value of the format information document 420 (see element 612 in FIG. 4 (described later)). Hereinafter, the correspondence between the cell identification information and the value of the format information document 420 will also be referred to as format information.

[0021] <Document analysis device: Document analysis unit> Before starting processing of the tabular document 410, the document analysis unit 111 extracts format information (correspondence between cell identification information and values) from the format information document 420 and stores it in the document format database 140. The document analysis unit 111 refers to the format information and extracts item values corresponding to item names (item identification information) from the tabular document 410. To explain in more detail, the document analysis unit 111 first extracts document information (see element 613 in FIG. 4 described later) that is correspondence between cell identification information (position information) and values from the tabular document 410.

[0022] Next, the document analysis unit 111 generates an item value acquisition prompt 610 (see FIG. 4, which will be described later) based on the extracted information. Subsequently, the document analysis unit 111 instructs the answer acquisition unit 118 to send the item value acquisition prompt 610 to the language model server 210, and receives an item value acquisition result 430 (see FIG. 5, which will be described later) as an answer.

[0023] <Document analysis device: item value acquisition prompt> 4 is a diagram showing the configuration of an item value acquisition prompt 610 according to this embodiment. Generally, a prompt is configured to include one or more elements. The item value acquisition prompt 610 includes elements 611 to 613. Element 611 indicates a command / instruction to extract an item value corresponding to item identification information beginning with "#" from the document indicated in element 613 by referring to the format (format information) indicated in element 612.

[0024] An element 612 is a correspondence between cell identification information and values (format information) extracted from the format information document 420 (see FIG. 3). Each row indicates a correspondence between identification information and a value. The element 613 is a correspondence (document information) between the identification information and the value of a cell extracted from the tabular document 410 (see FIG. 2). Each row indicates the correspondence between the identification information and the value.

[0025] <Document analysis device: Item value acquisition results> 5 is a diagram showing an item value acquisition result 430 according to this embodiment. The item value acquisition result 430 is JSON-formatted data specified by an element 611 of an item value acquisition prompt 610, and indicates the correspondence between item identification information (item name) and item value. JSON is an abbreviation for JavaScript Object Notation, and is the format of an object in the programming language JavaScript.

[0026] The correspondence will be explained using the item identification information "#Sales Order Probability" as an example. The identification information of the cell in which the document format information "#Sales Order Probability" is written is "E2:F3" (see element 612 in Figures 3 and 4). The language model server 210 obtains "A" as the value corresponding to "E2:F3" in element 613 and includes it in the item value obtainment result 430. The document analysis unit 111 associates the item value acquisition result 430 with the tabular document 410 and stores it in the document database 131 .

[0027] As described above, the document processing device 100 as a document analysis device includes a document analysis unit 111 that extracts document information (see element 613) indicating the correspondence between the identification information of each cell and the item value of the cell from a tabular document 410 that includes cells in which item values, which are the values of the items, are entered.

[0028] The document analysis unit 111 references the identification information of the cell in which the item value to be acquired is entered and format information (see element 612) indicating the correspondence between the item identification information of the item, as well as the document information, to create an item value acquisition prompt 610 (see Figure 4) that requests an item value acquisition result 430 (see Figure 5) indicating the correspondence between the item identification information and the item value entered in the cell corresponding to the item identification information. The document processing device 100 as a document analysis device includes a response acquisition unit 118 that transmits an item value acquisition prompt 610 to the language model server 210 and acquires the item value acquisition result 430 directly or indirectly from the language model server 210 .

[0029] <Document analysis device: document analysis processing> FIG. 6 is a flowchart of the document analysis process according to this embodiment. In step S11, the document analysis unit 111 extracts the identification information and values of the cells from the tabular document 410, which is the document to be processed (see element 613 in FIG. 4).

[0030] Step S12 is a document format acquisition process for acquiring format information corresponding to the table format document 410 of the document to be processed from the document format database 140. Details will be described later with reference to FIG. In step S13, the document analysis unit 111 generates an item value acquisition prompt 610 (see FIG. 4). Here, element 612 is the format information acquired in step S12. Element 613 is the correspondence between the cell identification information and the value acquired in step S11.

[0031] In step S14, the document analysis unit 111 instructs the answer acquisition unit 118 to send an item value acquisition prompt 610 to the language model server 210. Next, the document analysis unit 111 associates the answer, that is, the item value acquisition result 430 (see FIG. 5 ), with the tabular document 410 and stores it in the document database 131.

[0032] <Document analysis device: document format acquisition processing> 7 is a flowchart of the document format acquisition process according to this embodiment, which will be described with reference to FIG. In step S21, the document analysis unit 111 starts the process of repeating the process of step S22 for each piece of format information stored in the document format database 140. Note that the format information is the correspondence between the identification information and the value of the cell in the format information document 420 (see element 612 in FIG. 4).

[0033] In step S22, the document analysis unit 111 calculates the degree of match between the format information and the tabular document 410, which is the target of the document analysis process (see FIG. 6). The degree of match is the percentage or number of cell values and identification information in the format information that do not start with "#" that match the values and identification information of cells contained in the tabular document 410. In other words, it is the percentage or number of cells in the format information document 420 containing item names that do not start with "#" that match the cells in the tabular document 410 containing the same item names. Note that an item name cell is, for example, a cell for "(Sales) Order Probability" in "B2:D3" or a cell for "Contract Completion" in "F6:F7". In step S23, the document analysis unit 111 determines the format information with the highest degree of coincidence calculated in step S22 as the format information of the table format document 410 of the document to be processed.

[0034] As described above, the identification information of a cell is information that indicates the position of the cell in the tabular document 410 . The document analysis unit 111 creates an item value acquisition prompt 610 by referring to the format information in the document format database 140, which stores one or more pieces of format information, that has the largest number of cells whose cell identification information and cell values match between the cells included in the format information and the cells included in the document information (see element 613).

[0035] <Features of the document analysis device> The document processing device 100 as a document analysis device extracts an item value acquisition result 430 (see FIG. 5) from a tabular document 410 (see FIG. 2) by referring to format information (see element 612 in FIG. 4). Even for a complicated tabular document, an item value acquisition result, which is a correspondence between item identification information and item values required for document processing such as review / evaluation, can be acquired. The document format acquisition process (see Figure 7) acquires format information that most closely matches the format of the document being processed. This eliminates the need to specify a format (template document) for each document, allowing for efficient acquisition of field value results.

[0036] <<Modification of Document Analysis Device: Item Value Acquisition Results>> The item value acquisition result 430 (see FIG. 5) is in JSON format, but is not limited to this. It may be in text format. FIG. 8 is a diagram showing an item value acquisition result 430A in text format according to a modified example of this embodiment. The item value acquisition result 430A is text format data with the same content as the item value acquisition result 430.

[0037] ≪Document analysis device: document format generation unit≫ In the above-described embodiment, format information (see element 612 in FIG. 4) is extracted from format information document 420 (see FIG. 3) in which item identification information beginning with "#" is written. The document format generation unit 112 extracts format information from format information document 420A (see FIG. 9 described later) in which item identification information is blank. FIG. 9 is a diagram showing format information document 420A (template document) in which item identification information is blank according to a modified example of this embodiment. Compared to format information document 420, the item identification information is blank.

[0038] Fig. 10 is a flowchart of the format information acquisition process according to this embodiment. The process of acquiring format information from a format information document 420A in which the entry fields (cells) for item values are blank will be described with reference to Fig. 10. The document format acquisition process makes it possible to acquire format information without preparing a format information document 420 in which item identification information beginning with "#" is entered.

[0039] In step S31, the document format generation unit 112 extracts the correspondence between the cell identification information and the value (template document information) from the format information document 420A. In step S32, the document format generation unit 112 acquires a blank cell, the cell immediately above is not blank, and the cell immediately to the left is blank. Note that the cells immediately above, below, left, and right of a cell can be identified from identification information indicating their positions. Furthermore, the width of the cell immediately above is the same as that of the cell immediately above, and the height of the cell immediately to the left is the same as that of the cell immediately to the left.

[0040] In step S33, the document format generation unit 112 starts the process of repeating the process of step S34 for each cell acquired in step S32. In step S34, the document format generation unit 112 sets the item identification information obtained by adding "#" to the beginning of the value (character string) of the cell immediately above, and the identification information of the cell, as format information.

[0041] In step S35, the document format generation unit 112 acquires a blank cell that has a non-blank cell to the left and a blank cell above, where the width and height of the cell above are the same as those of the cell to the left. In step S36, the document format generation unit 112 starts the process of repeating the process of step S37 for each cell acquired in step S35.

[0042] In step S37, the document format generation unit 112 sets the item identification information obtained by adding "#" to the beginning of the value (character string) of the cell to the left and the identification information of the cell as the format information. In step S38, the document format generation unit 112 stores the format information acquired in steps S34 and S37 in the document format database 140 as format information of the format information document 420A.

[0043] As described above, the document processing device 100 as a document analysis device includes a document format generation unit 112 that extracts template document information indicating the correspondence between identification information indicating the position of each cell and the value entered in the cell from a template document (see format information document 420A shown in Figure 9) in which item values are left blank and which serves as a template for the tabular document 410.

[0044] When the cell immediately above a blank cell is not blank but the cell immediately to the left is blank, the document format generation unit 112 associates the identification information of the blank cell with the value of the cell immediately above and item identification information.Furthermore, when the cell immediately above a blank cell is not blank but the cell immediately to the left is blank, the document format generation unit 112 associates the identification information of the blank cell with the value of the cell immediately to the left and item identification information. The document format generating unit 112 defines the correspondence between the blank cell identification information and the item identification information as format information.

[0045] <Document processing device: document evaluation device> The document processing device 100 will be described below as a document evaluation device. The document processing device 100 evaluates / reviews documents that include items, such as a tabular document 410, in accordance with evaluation criteria for each item and reports the results. Note that even if the document is in text format, documents that include structures such as chapters, sections, and items can also be processed by the document evaluation device. In the following description, elements / components of document structures such as chapters and sections will also be referred to as items.

[0046] <Document evaluation device: Processing overview> Fig. 11 is a diagram for explaining an outline of the document evaluation reporting process (see Fig. 24 described later) by the document processing device 100 according to this embodiment. The document evaluation unit 113 generates an evaluation acquisition prompt 620 (see Fig. 12 described later) based on the evaluation criteria stored in the document evaluation reporting rule database 150 and the item value acquisition result 430 (see Fig. 5). The evaluation acquisition prompt 620 is sent to the language model server 210, which returns an evaluation result 450 (see Fig. 13 described later). The evaluation result 450 is the result of evaluation / review of the item value acquisition result 430 according to the evaluation criteria.

[0047] The evaluation reporting unit 114 generates a report acquisition prompt 630 (see FIG. 15 described later) based on the evaluation result 450 and the report example stored in the document evaluation reporting rule database 150. The report acquisition prompt 630 is sent to the language model server 210, and a report 460 (see FIG. 17 described later) is returned. The document processing device 100 as a document evaluation device will be described in detail below.

[0048] <Document evaluation device: Document evaluation report rule database> The document evaluation reporting rule database 150 stores evaluation criteria for evaluating / reviewing documents in association with format information (document format database 140, see element 612 in Figure 4). In the format information, not only the items (item identification information and item values) of a table-format document but also the components of a text-format document, such as chapters, sections, figures, and tables, are considered as items. Evaluation criteria (see element 622 in Figure 12, described later) are evaluation criteria related to each of these items. Since format information includes one or more items, one or more evaluation criteria are associated with one piece of format information.

[0049] The document evaluation report rule database 150 also stores questions corresponding to the evaluation criteria (see element 624 in FIG. 12, which will be described later). The questions are questions for obtaining evaluation results related to each item or evaluation criterion. Since format information includes one or more items, one or more questions are associated with one piece of format information.

[0050] Furthermore, the document evaluation report rule database 150 also stores report examples (see elements 632 to 635 in FIG. 15 described later) of the evaluation results evaluated / reviewed based on the evaluation criteria. The report examples are report examples of the evaluation results for each item. Since format information includes one or more items, one or more report examples are associated with one piece of format information. Additionally, the document evaluation / reporting rule database 150 may store documents (also referred to as checklists) that describe the viewpoints, check contents, and check rules for document evaluation / review.

[0051] <Document Evaluation Device: Evaluation Report Database> The evaluation report database 160 stores evaluation results of documents and reports summarizing the evaluation results in association with the document identification information of the documents to be evaluated. Furthermore, the evaluation report database 160 stores minutes of meetings in which documents were evaluated / reviewed with reference to the documents and reports in association with the document identification information of the documents to be evaluated.

[0052] <Document evaluation device: document evaluation unit> The document evaluation unit 113 evaluates / reviews a document by referring to evaluation criteria related to the items that make up the document. The document evaluation unit 113 generates an evaluation acquisition prompt 620 (see FIG. 12 described later) based on the evaluation criteria and document information. The document evaluation unit 113 then instructs the answer acquisition unit 118 to send the evaluation acquisition prompt 620 to the language model server 210, and receives the evaluation result (also simply referred to as the evaluation) as an answer.

[0053] <Document Evaluation Device: Evaluation Prompt> 12 is a diagram showing the configuration of the evaluation acquisition prompt 620 according to this embodiment. The evaluation acquisition prompt 620 includes elements 621-624. Element 621 indicates instructions / instructions for answering the question indicated in element 624 for the document to be evaluated indicated in element 623 .

[0054] The element 622 is an evaluation criterion for the items included in the document to be evaluated (see element 623). The evaluation criterion is stored in the document evaluation report rule database 150. Element 623 is document information to be evaluated (evaluation target document information). In Fig. 12, the document information in element 623 is the correspondence between item identification information and item values acquired by the document analysis unit 111 from a tabular document (see item value acquisition result 430 in Fig. 5). The document information in element 623 may be text-format document information created by a word processor. For example, it may be text-format item value acquisition result 430A (see Fig. 8).

[0055] Element 624 is a question for obtaining an evaluation result according to the evaluation criteria in element 622. The question may include procedures for obtaining the evaluation result. The question shown in FIG. 12 includes the following procedures: (1) correctly count the number of "high," "medium," and "low" in the "evaluation result," (2) calculate the project risk rank according to the evaluation criteria, and (3) compare the calculated project risk rank with the project risk rank included in the evaluation target. In this way, evaluation acquisition prompt 620 is a prompt for obtaining the evaluation result by referring to the evaluation criteria related to the "project risk rank" item.

[0056] <Document evaluation device: Evaluation results> Fig. 13 is a diagram showing an assessment result 450 according to this embodiment. Fig. 14 is a diagram showing an assessment result 450A according to this embodiment. The assessment results 450, 450A are answers that follow the procedure shown in the question of element 624 included in assessment acquisition prompt 620. Note that assessment results 450, 450A are different assessment results when the project risk ranks are compared, which is the final step, and when they match. In this way, the document evaluation unit 113 uses the evaluation acquisition prompt 620 to inquire of the language model server 210 about each evaluation criterion for each item included in the format information corresponding to the document to be evaluated, and obtains the evaluation results 450, 450A.

[0057] As described above, the document processing device 100 as a document evaluation device is equipped with a document evaluation unit 113 that creates an evaluation acquisition prompt 620 requesting evaluation results 450, 450A of the document information to be evaluated based on the evaluation criteria, by referring to the document information to be evaluated (see element 623) including correspondence between item identification information and item values for one or more items, and the evaluation criteria related to the items (see element 622). The document processing device 100 as a document evaluation device includes a response acquisition unit 118 that transmits an evaluation acquisition prompt 620 to the language model server 210 and acquires the evaluation results 450, 450A directly or indirectly from the language model server 210. The request for the evaluation result included in the evaluation acquisition prompt 620 is a question-type request (see element 624) for the evaluation result 450, 450A.

[0058] <Document Evaluation Device: Evaluation Reporting Unit> The evaluation report unit 114 generates an evaluation report by referring to the evaluation results 450, 450A. More specifically, the evaluation report unit 114 generates a report acquisition prompt 630 (see FIG. 15 described later) based on the evaluation results and report examples. The evaluation report unit 114 then instructs the answer acquisition unit 118 to send the report acquisition prompt 630 to the language model server 210, and receives a report as an answer. Similar to the document evaluation unit 113, the evaluation report unit 114 generates a report acquisition prompt 630 for each item that makes up the document and receives a report 460 (see FIG. 17 described later).

[0059] <Document Evaluation Device: Report Acquisition Prompt> 15 and 16 are diagrams showing the configuration of the report acquisition prompt 630 according to this embodiment. The report acquisition prompt 630 includes elements 631 to 637. Element 631 indicates an instruction / instruction to report the question indicated in element 636 and the answer (evaluation result) to the question indicated in element 637, with reference to the report examples indicated in elements 632 to 635.

[0060] Element 632 is an example of a question and answer where the determination result does not match. An element 633 is an example of a report when the judgment results do not match as shown in element 632. The report includes whether or not there are any findings, the findings, and a proposed correction. Element 634 is an example of a question and answer that results in a match. Element 635 is an example of a report when the judgment results match as shown in element 634. The report includes the presence or absence of findings, the findings themselves, and proposed corrections, but the report has no content.

[0061] Turning now to FIG. 16, the description of the get report prompt 630 continues. Element 636 is a question for obtaining the evaluation result in element 624 of evaluation acquisition prompt 620 (see FIG. 12). Element 637 is the evaluation result 450 (see FIG. 13) which is the answer to the question.

[0062] <Document Evaluation Device: Report> 17 is a diagram showing a report 460 according to this embodiment. The answer in element 637 is an evaluation result 450 with a mismatched judgment result, and corresponds to the question and answer in element 632. Therefore, report 460 is in line with the report in element 633. In this way, the evaluation report unit 114 obtains the report 460 by querying the language model server 210 using the report acquisition prompt 630 including the report example corresponding to the evaluation results 450, 450A for each item obtained by the document evaluation unit 113.

[0063] As described above, the document processing device 100 as a document evaluation device includes the evaluation report unit 114 that creates the report acquisition prompt 630 that requests the report 460 in a predetermined format based on the evaluation results 450, 450A. The answer acquirer 118 sends a report acquisition prompt 630 to the language model server 210 to acquire a report 460 directly or indirectly from the language model server 210 .

[0064] The report acquisition prompt 630 includes an example of a question included in a question-type request (see elements 632 and 634), an example of an evaluation result as an answer to the question (see elements 632 and 634), an example of a report in a predetermined format for the example question and the example answer (see elements 633 and 635), a question-type request included in the evaluation acquisition prompt 620 (see element 636), and an evaluation result acquired by the answer acquisition unit 118 (see element 637).

[0065] The document information to be evaluated (see element 623 in Figure 12) also includes an evaluation item (e.g., "project risk rank") whose item value is determined depending on the number of items included in a specified range (e.g., "high," "medium," or "low") in which the item value is one or more specified values. The evaluation criteria (see element 622) include the criteria that determine the value of the evaluation item. The evaluation acquisition prompt 620 includes a request to count the number of items whose item values fall within a specified range, a request to obtain the counted evaluation item value, which is the value of the evaluation item determined based on the counting result, and a request to report whether the counted evaluation item value matches the written evaluation item value, which is the value of the evaluation item included in the document information to be evaluated (see element 624).

[0066] The predetermined format for reporting (see element 633 in FIG. 15) is a format for reporting a mismatch between the counted and written endpoint values. The predetermined report (see element 633) is a format that includes a suggested correction to correct the written endpoint value to the count endpoint value if the count endpoint value and the written endpoint value do not match.

[0067] <Modifications of the evaluation prompt and report prompt> The evaluation criteria (see element 622) included in the evaluation acquisition prompt 620 (see FIG. 12) are evaluation criteria in which another item value ("AA", "A", "B", "C") is determined depending on the number of item values ("High", "Medium"). Some examples of evaluation criteria / check contents other than this format are shown below.

[0068] 18 is a diagram showing an evaluation acquisition prompt 620A according to this embodiment. The evaluation criteria in element 622A of the evaluation acquisition prompt 620A are evaluation criteria in which another item value ("final project risk rank") is determined based on two item values ("project risk rank by size" and "project risk rank by required quality"). In other words, the evaluation acquisition prompt 620A is a prompt for acquiring an evaluation result related to the "final project risk rank" item.

[0069] 19 is a diagram showing a report acquisition prompt 630A according to this embodiment. The report acquisition prompt 630A corresponds to the evaluation acquisition prompt 620A, and is a prompt for acquiring a report of the evaluation results related to the item "final project risk rank." Element 632A is an example where the determination result is not consistent. Element 633A is an example of a report when the determination results do not match as shown in element 632A. Element 634A is an example where the determination result is a match. Element 635A is an example of a report when the determination results in a match as shown in element 634A.

[0070] As described above, the document information to be evaluated (see element 623A) includes an evaluation item (e.g., "final project risk rank") whose item value is determined based on the maximum or minimum item value among multiple item values to be compared (e.g., "project risk rank by size" and "project risk rank by required quality").

[0071] The evaluation criteria include the criteria for determining the value of the evaluation item. The evaluation acquisition prompt 620A includes a request for a comparison evaluation item value, which is the value of the evaluation item determined based on the result of comparing the comparison item values, and a request to report whether the comparison evaluation item value matches the written evaluation item value, which is the value of the evaluation item included in the evaluation target document information (see element 624A).

[0072] The predetermined format for reporting (see element 633A) is a format for reporting a discrepancy between the comparison endpoint value and the recorded endpoint value. The predetermined report (see element 633A) is a format that includes a suggested correction to change the written endpoint value to the comparative endpoint value if the comparison endpoint value and the written endpoint value do not match.

[0073] 20 is a diagram showing an evaluation acquisition prompt 620B according to this embodiment. The evaluation criteria in element 622B of evaluation acquisition prompt 620B are judgment criteria for determining whether an item value is a predetermined value (for example, blank) and judging the appropriateness of the item value (presence or absence of a violation). FIG. 21 is a diagram showing a report acquisition prompt 630B according to this embodiment. Element 632B is an example where there is a violation. Element 633B is an example of a report in the event of a violation as shown in element 632B. Element 634B is an example of no violation. Element 635B is an example of a report when there is no violation as shown in element 634B.

[0074] As described above, the evaluation criteria (element 622B) includes prohibited item identification information, which is item identification information for which the item value is prohibited from being a predetermined value (for example, blank). The evaluation acquisition prompt 620B includes a request to acquire the item value of the prohibited item identification information included in the document information to be evaluated, and a request for prohibited value entry item identification information indicating item identification information whose acquired item value is a predetermined value. The predetermined format of the report (see element 633B) is a format that includes forbidden value entry identification information.

[0075] 22 is a diagram showing a prompt 620C for obtaining an evaluation according to this embodiment. The evaluation criteria in an element 622C of the prompt 620C for obtaining an evaluation are used to determine whether or not an item value is appropriate. FIG. 23 is a diagram showing a report acquisition prompt 630C according to this embodiment. Element 632C is an example where the item value is inappropriate (violates the evaluation criteria). Element 633C is an example of a report when the item value shown in element 632B is inappropriate. Element 634C is an example where the item value is appropriate. Element 635C is an example of a report when the item values shown in element 634B are appropriate.

[0076] <Document evaluation device: document evaluation report processing> 24 is a flowchart of the document evaluation report process according to this embodiment. The process of obtaining the evaluation result for a document and generating a report will be described with reference to FIG.

[0077] In step S41, the document evaluation unit 113 acquires document information to be the subject of an evaluation report. For example, for a tabular document, the document evaluation unit 113 acquires the item value acquisition results 430 (see FIG. 5) extracted by the document analysis unit 111 from the document database 131. Next, the document evaluation unit 113 acquires evaluation criteria, questions, and report examples from the document evaluation report rule database 150 based on the format information corresponding to the document.

[0078] In step S42, the document evaluation unit 113 starts the process of repeating steps S43 to S44 for each item having an evaluation criterion. In step S43, the document evaluation unit 113 generates an evaluation acquisition prompt 620 (see FIG. 12). Here, element 622 is an evaluation criterion corresponding to an item among the evaluation criteria acquired in step S41. Element 624 is a question corresponding to the evaluation criterion. Element 623 is the document information acquired in step S41. Element 624 is a question corresponding to the evaluation criterion (corresponding to an item) among the questions acquired in step S41.

[0079] In step S44, the document evaluation unit 113 instructs the answer acquisition unit 118 to send an evaluation acquisition prompt 620 to the language model server 210. Next, the document evaluation unit 113 stores the evaluation results 450, 450A (see FIGS. 13 and 14), which are the answers, in the evaluation report database 160 in association with the document identification information of the document.

[0080] In step S45, the evaluation report unit 114 starts the process of repeating steps S46 to S47 for each item having an evaluation criterion. In step S46, the evaluation report unit 114 generates a report acquisition prompt 630 (see FIGS. 15 and 16). Here, elements 632 to 635 are report examples corresponding to the items (evaluation criteria) among the report examples acquired in step S41. Element 636 is a question corresponding to the item, which is the question in element 624 of the evaluation acquisition prompt 620 generated in step S43. Element 637 is the evaluation result, which is the answer acquired in step S44.

[0081] In step S47, the evaluation reporting unit 114 instructs the answer acquisition unit 118 to send a report acquisition prompt 630 to the language model server 210. Next, the evaluation reporting unit 114 associates the answer, ie, the report 460 (see FIG. 17), with the document identification information of the document and stores it in the evaluation report database 160. In step S48, the evaluation report unit 114 creates a report of the evaluation of the document by combining the reports acquired in step S47, and stores the report in the evaluation report database 160.

[0082] <Features of the document evaluation device> The document processing device 100, which serves as a document evaluation device, acquires evaluation results 450 and 450A for each item by referring to evaluation criteria (see elements 622, 622A, 622B, and 622C in FIGS. 12, 18, 20, and 22). Furthermore, the document processing device 100 acquires a report 460 for each item by referring to report examples (see elements 632 to 635, 632A to 635A, 632B to 635B, and 632C to 635C in FIGS. 15, 19, 21, and 23). The document processing device 100 generates a report by evaluating and reviewing each item included in a document, thereby obtaining highly accurate evaluation results and reports according to the item. Furthermore, by preparing evaluation criteria and questions according to the item, it is possible to expect to obtain even more accurate evaluation results and reports.

[0083] <Document Processing Device: Document Evaluation Criteria Extraction Device> The document processing device 100 will be described below as a document evaluation criteria extraction device. The document processing device 100 collects evaluation criteria, check details, and check rules for each item of a document that includes items, such as a table-format document 410. The document processing device 100 extracts and collects these from documents that contain evaluation criteria and check details, such as minutes of evaluation / review meetings. Hereinafter, the evaluation criteria, check details, and check rules will be collectively referred to as evaluation criteria.

[0084] <Document evaluation criteria extraction device: evaluation criteria extraction unit> FIG. 25 shows an example of a document 510 including evaluation criteria according to this embodiment. The document 510 is, for example, the minutes of a meeting where a document was evaluated / reviewed. The minutes include evaluation criteria including review viewpoints and rules. The evaluation criteria extraction unit 115 extracts the evaluation criteria from the document including the evaluation criteria. More specifically, the evaluation criteria extraction unit 115 generates an evaluation criteria acquisition prompt 640 (see FIG. 26 described later) that commands / instructs extraction of the evaluation criteria from the document. Next, the evaluation criteria extraction unit 115 instructs the answer acquisition unit 118 to send the evaluation criteria acquisition prompt 640 to the language model server 210 and receives the evaluation criteria as a response. The evaluation criteria extraction unit 115 stores the received evaluation criteria in the document evaluation report rule database 150 in association with the format information of the document being evaluated / reviewed.

[0085] <Document Evaluation Criteria Extraction Device: Evaluation Criteria Acquisition Prompt> 26 is a diagram showing the configuration of the evaluation criterion acquisition prompt 640 according to this embodiment. The evaluation criterion acquisition prompt 640 includes elements 641-642. Element 641 shows instructions / instructions for generating and obtaining rating criteria with reference to the actual review findings shown in element 642 . Element 642 contains the textual information contained in document 510 .

[0086] <Document evaluation criteria extraction device: evaluation criteria> 27 is a diagram showing the acquired evaluation criteria 520 according to this embodiment. The evaluation criteria 520 are general evaluation criteria that exclude the specific content of individual documents from the content of review indications included in the element 642. In this way, the evaluation criteria extraction unit 115 extracts and acquires evaluation criteria from documents that include evaluation targets.

[0087] As described above, the document processing device 100 as a document evaluation criteria extraction device is equipped with an evaluation criteria extraction unit 115 that creates an evaluation criteria acquisition prompt 640 (see Figure 26) that extracts the evaluation criteria of the document by referring to the minutes including the document review results. The document processing device 100 as a document evaluation criterion extraction device also includes a response acquisition unit 118 that transmits an evaluation criterion acquisition prompt 640 to the language model server 210 and acquires evaluation criteria directly or indirectly from the language model server 210 . The evaluation criteria extraction unit 115 stores the evaluation criteria in the storage unit 130 (see the document evaluation report rule database 150).

[0088] <Document Evaluation Criteria Extraction Device: Variation of Evaluation Criteria Acquisition Prompt> The evaluation criteria acquired using the evaluation criteria acquisition prompt 640 (see FIG. 26) are evaluation criteria for the document that was the subject of review / evaluation, and further for documents of the same format as the document. The evaluation criteria extraction unit 115 may acquire evaluation criteria for each item determined by the document format (format information).

[0089] 28 is a diagram showing a criterion acquisition prompt 640A according to this embodiment. The criterion acquisition prompt 640A includes an element 643A that is not included in the criterion acquisition prompt 640. Element 641A indicates a command / instruction to generate and acquire criteria for each item in the document in element 643A. 29 shows the evaluation criteria 520A when the evaluation criteria acquisition prompt 640A according to this embodiment is used. Evaluation criteria are extracted for each item (chapter).

[0090] As described above, the evaluation criteria extraction unit 115 creates an evaluation criteria acquisition prompt 640A that extracts evaluation criteria for each item in the document (see element 643A in Figure 28) by referring to the items described in the document and the minutes including the results of reviewing the document. The evaluation criteria extraction unit 115 stores the evaluation criteria in the storage unit 130 (see the document evaluation report rule database 150) as evaluation criteria related to the item.

[0091] The source document is not limited to a text document such as minutes of a meeting. Text information may be extracted from a checklist document in a tabular format to extract evaluation criteria, which are the check details. 30 is a diagram showing an evaluation criterion acquisition prompt 640B (check content acquisition prompt) according to this embodiment. The evaluation criterion acquisition prompt 640B includes elements 641B and 642B.

[0092] Element 641B indicates a command / instruction to generate and acquire check contents (evaluation criteria) for each item (chapter) from the text information in element 642B. Element 642B shows text information extracted from the checklist document ("Checklist of Functional Specification" in FIG. 30). 31 shows the evaluation criteria 520B (check details) when the evaluation criteria acquisition prompt 640B according to this embodiment is used. Evaluation criteria are extracted collectively for each item (chapter).

[0093] As described above, the evaluation criterion extraction unit 115 creates a check content acquisition prompt (see evaluation criterion acquisition prompt 640B) that extracts check contents for each item (for example, chapter) from a checklist document that includes check contents for each item of the document. The answer acquisition unit 118 transmits a check content acquisition prompt to the language model server 210 and acquires the check content directly or indirectly from the language model server 210 . The evaluation criteria extraction unit 115 stores the check details as evaluation criteria related to the items in the storage unit 130 (document evaluation report rule database 150).

[0094] <Document evaluation criteria extraction device: evaluation criteria extraction process> FIG. 32 is a flowchart of the evaluation criterion extraction process according to this embodiment. In step S51, the evaluation criterion extraction unit 115 acquires from the evaluation report database 160 the minutes of the meeting in which the document was evaluated / reviewed. In step S52, the evaluation criterion extraction unit 115 starts the process of repeating steps S53 to S54 for each of the minutes.

[0095] In step S53, the evaluation criterion extraction unit 115 generates an evaluation criterion acquisition prompt 640A (see FIG. 28). Here, element 642A is text information included in the minutes. Element 643A is an item (item identification information) included in the format information of the document to be evaluated / reviewed in the minutes.

[0096] In step S54, the evaluation criterion extraction unit 115 instructs the answer acquisition unit 118 to send an evaluation criterion acquisition prompt 640A to the language model server 210. Next, the evaluation criterion extraction unit 115 associates the evaluation criterion 520A (see FIG. 29) that is the answer with the item of format information and stores it in the document evaluation reporting rule database 150. Note that before storing the evaluation criterion 520A in the document evaluation reporting rule database 150, the evaluation criterion extraction unit 115 may inquire of the user of the document processing device 100 about whether or not to adopt the evaluation criterion as an evaluation criterion. Steps S55 to S58 are the same as steps S51 to S54, except that the checklist is processed instead of the minutes.

[0097] <Features of the evaluation criteria extraction device> The document processing device 100, which serves as an evaluation criteria extraction device, extracts evaluation criteria for each document format (format information) and for each item that makes up a document from minutes and checklists. This makes it possible to obtain evaluation criteria without manual intervention. By collecting evaluation criteria, manually organizing them as needed, and referencing them in the document evaluation report process, it is expected that the quality of document evaluations by the document processing device 100 will improve.

[0098] <Document processing device: document proofreading device> The following describes the document processing device 100 as a document proofreading device. The document processing device 100 performs document proofreading including eliminating variations in word spelling corresponding to documents and tasks related to documents.

[0099] <Document Proofreader: Processing Overview> Fig. 33 is a diagram illustrating an outline of document proofreading processing (see Fig. 40 described later) by the document processing device 100 according to this embodiment. The dictionary generation unit 116 generates a synonym acquisition prompt 650 (see Fig. 34 described later) that references one or more related documents 540, such as documents relating to the same business. The synonym acquisition prompt 650 is sent to the language model server 210, which returns a synonym dictionary 550 (see Fig. 35 described later). The synonym dictionary 550 is a synonym dictionary specialized for the context contained in the document 540, the business related to the document 540, etc.

[0100] The document proofreading unit 117 generates a proofread document acquisition prompt 660 (see FIG. 36 described later) that references the synonym dictionary 550. The proofread document acquisition prompt 660 is sent to the language model server 210, which returns the proofread document 560. The document processing device 100 as a document proofreading device will be described in detail below.

[0101] <Document Proofreader: Dictionary Generation Unit> The dictionary generation unit 116 generates a synonym acquisition prompt 650 (see FIG. 34 described later) that commands / instructs the generation of a synonym dictionary. Then, the dictionary generation unit 116 instructs the answer acquisition unit 118 to send the synonym acquisition prompt 650 to the language model server 210, and receives the synonym dictionary 550 (see FIG. 35 described later) as the answer.

[0102] <Document Proofreader: Synonym Acquisition Prompt> 34 is a diagram showing the configuration of a synonym acquisition prompt 650 according to this embodiment. The synonym acquisition prompt 650 includes elements 651-652. Element 651 indicates a command / instruction to obtain a synonym dictionary (group of synonyms) by referencing the text information of document 540 indicated in element 652. The command / instruction includes criteria for selecting which word from the synonyms to use as a headword. Examples of criteria include katakana spelling priority, kanji spelling priority, or abbreviation priority. Element 652 contains the textual information contained in document 540 .

[0103] <Document proofreading device: synonym dictionary> 35 is a diagram showing an acquired synonym dictionary 550 according to this embodiment. The synonym dictionary 550 uses words that comply with the entry word selection criteria instructed / instructed in an element 651 as entry words (keys in JSON format).

[0104] <Document Proofreading Device: Document Proofreading Unit> The document proofreading unit 117 generates a proofread document acquisition prompt 660 (see FIG. 36 described later) that commands / instructs proofreading of the document. Next, the document proofreading unit 117 instructs the answer acquisition unit 118 to send the proofread document acquisition prompt 660 to the language model server 210, and receives the proofread document as the answer. The document proofreading unit 117 stores the proofread document in the document database 131 in association with the target document.

[0105] <Document Proofreader: Prompt for Proofread Document Acquisition> 36 is a diagram showing the configuration of a proofreading document acquisition prompt 660 according to this embodiment. The proofreading document acquisition prompt 660 includes elements 661-663. Element 661 indicates a command / instruction to correct and proofread the document indicated in element 663 by referring to the thesaurus 550 indicated in element 662. In addition to word variations, element 661 includes commands / instructions to correct tense variations and to correct sentence endings to the "desu / masu" style. Element 662 shows the thesaurus 550 . Element 663 indicates the text information of the document to be proofread.

[0106] As described above, the document processing device 100 as a document proofreading device includes a dictionary generation unit 116 that creates a synonym acquisition prompt 650 that requests a synonym group in which words contained in the text information (see element 652 in Figure 34) of a document group including one or more documents are grouped by synonym. The document processing device 100 as a document proofreading device includes an answer acquisition unit 118 that transmits a synonym acquisition prompt 650 to the language model server 210 and acquires a synonym group (see the synonym dictionary 550 shown in Figure 35) directly or indirectly from the language model server 210.

[0107] The document processing device 100 as a document proofreading device is equipped with a document proofreading unit 117 that references synonym groups and creates a proofread document acquisition prompt 660 (see Figure 36) that requests proofread documents in which variations in word spelling of documents included in a document group have been corrected. The answer acquisition unit 118 sends a proofread document acquisition prompt 660 to the language model server 210 to acquire the proofread document directly or indirectly from the language model server 210 .

[0108] The synonym retrieval prompt 650 includes a request for each synonym group to lemmaize words that meet predetermined criteria (see element 651). The predetermined criteria are either katakana notation priority, kanji notation priority, or abbreviation priority. The Get Proofread Document prompt 660 requests a proofread document in which the spelling variations have been corrected to favor lemmas (see element 661).

[0109] The proofread document acquisition prompt 660 requests correction of sentence-final spelling variations following correction of word spelling variations (see element 661). The variation in sentence-final spelling includes variation in tense and at least one of the variations in the desu / masu style and the da / dearu style.

[0110] <Document Proofreader: Variations of Synonym Acquisition Prompt and Proofread Document Acquisition Prompt> 37 is a diagram showing a synonym acquisition prompt 650A according to this embodiment. Compared to the synonym acquisition prompt 650 (see FIG. 34), emphasis is placed on grouping synonyms in consideration of the context. FIG. 38 is a diagram showing a meaning-containing synonym dictionary 550A according to this embodiment. 39 is a diagram showing a proofreading document acquisition prompt 660A according to this embodiment. Compared to the proofreading document acquisition prompt 660 (see FIG. 36), emphasis is placed on correcting word variations in light of context.

[0111] As explained above, the synonym get prompt 650A requests a synonym group that takes into account the meaning in context. The synonym group contains the meaning in the context. The proofread document acquisition prompt 660A requests a proofread document in which variations in word spelling have been corrected to take into account their meaning in context.

[0112] <Document proofreading device: document proofreading process> FIG. 40 is a flowchart of the document proofreading process according to this embodiment. In step S 61 , the dictionary generation unit 116 acquires the related documents 540 from the document database 131 .

[0113] In step S62, the dictionary generation unit 116 generates a synonym acquisition prompt 650 (see FIG. 34). Here, the element 652 is the entire text information contained in the document 540. In step S63, the dictionary generation unit 116 instructs the answer acquisition unit 118 to send a synonym acquisition prompt 650 to the language model server 210, and receives the synonym dictionary 550 as an answer.

[0114] In step S64, the document proofreading unit 117 starts the process of repeating steps S65 to S66 for each document acquired in step S61. In step S65, the document proofreading unit 117 generates a proofread document acquisition prompt 660 (see FIG. 36), where an element 662 is text information contained in the document.

[0115] In step S66, the document proofreading unit 117 instructs the answer acquisition unit 118 to send a proofread document acquisition prompt 660 to the language model server 210. The document proofreading unit 117 receives the constructed proofread document that is the answer, and stores it in the document database 131 in association with the document before proofreading.

[0116] <Features of document proofreading equipment> The document processing device 100, as a document proofreading device, performs document proofreading to correct variations in synonym spelling, tense, and sentence ending spelling. Synonyms are corrected by referring to a synonym dictionary 550 generated based on related documents 540, such as documents related to the same business (see element 662). This makes it possible to correct words according to meanings specific to the business or the context in which the words are included.

[0117] <<Modification of document proofreading device: proofread document acquisition prompt>> In the above-described embodiment, synonym variations, tense variations, and variations in the desu / masu / da / aru voice styles are corrected using a single proofread document acquisition prompt 660, 660A. However, corrections may be made by dividing the prompts into multiple prompts. By dividing the prompts into multiple prompts, it is expected that the accuracy of corrections will improve.

[0118] <<Variation: Language Model>> The document processing device 100 described above transmits a prompt to a language model server 210 that provides a language model service and acquires an answer. The document processing device 100 itself may be provided with a language model and make an inquiry to itself. FIG. 41 is a functional block diagram of a document processing device 100A according to a modified example of this embodiment. The document processing device 100A is provided with a language model 132. The answer acquisition unit 118A inputs a prompt and outputs an answer using the language model 132. By not using the language model server 210 of an external service, the risk of information leakage can be reduced. Furthermore, by using a language model 132 trained according to the business of the organization itself, the accuracy of answers can be improved, and it can be expected that the accuracy of document evaluation / review can be improved.

[0119] As described above, the answer acquisition unit 118A provided in the document processing device 100A as a document analysis device uses the language model 132 instead of the language model server 210 to acquire the item value acquisition result 430 in response to the item value acquisition prompt 610.

[0120] The answer acquisition unit 118A provided in the document processing device 100A as a document evaluation device uses the language model 132 instead of the language model server 210 to acquire the evaluation result 450 in response to the evaluation acquisition prompt 620 and acquire the report 460 in response to the report acquisition prompt 630.

[0121] The answer acquisition unit 118A provided in the document processing device 100 as a document evaluation criterion extraction device uses the language model 132 instead of the language model server 210 to acquire check contents in response to the check content acquisition prompt (see evaluation criterion acquisition prompt 640B).

[0122] The answer acquisition unit 118A provided in the document processing device 100 as a document proofreading device uses the language model 132 instead of the language model server 210 to acquire a synonym group (synonym dictionary 550) for the synonym acquisition prompt 650 and acquires a proofread document for the proofread document acquisition prompt 660.

[0123] Other variations Although several embodiments and modifications of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications are included within the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0124] <Hardware configuration> The document processing devices 100, 100A according to the above-described embodiments are realized by a computer 900 having a configuration as shown in FIG. 42, for example. FIG. 42 is a hardware configuration diagram showing an example of the computer 900 that realizes the functions of the document processing devices 100, 100A according to the above-described embodiments. The computer 900 includes a CPU 901, a ROM 902, a RAM 903, an SSD 904, and an input / output interface 905 (referred to as an input / output I / F (Interface) in FIG. 42). The computer 900 further includes a communication interface 906 (referred to as a communication I / F in FIG. 42) and a media interface 907 (referred to as a media I / F in FIG. 42). The computer 900 may include a hard disk drive (HDD) instead of the SSD 904, or may include a HDD in addition to the SSD 904.

[0125] The CPU 901 operates based on a program stored in the ROM 902 or the SSD 904, and performs control by the control unit 110 in Fig. 1. The ROM 902 stores a boot program executed by the CPU 901 when the computer 900 starts up, programs related to the hardware of the computer 900, and the like.

[0126] The CPU 901 controls an input device 910 such as a mouse or keyboard, and an output device 911 such as a display or printer, via an input / output interface 905. The CPU 901 acquires data from the input device 910 via the input / output interface 905, and outputs generated data to the output device 911.

[0127] The SSD 904 stores programs executed by the CPU 901 and data used by the programs. The communication interface 906 receives data from other devices (not shown) (e.g., the language model server 210) via a communication network and outputs the data to the CPU 901, and also transmits data generated by the CPU 901 to other devices via the communication network.

[0128] The media interface 907 reads a program or data stored in the recording medium 912 and outputs it to the CPU 901 via the RAM 903. The CPU 901 loads the program from the recording medium 912 onto the RAM 903 via the media interface 907 and executes the loaded program. The recording medium 912 is an optical recording medium such as a DVD (Digital Versatile Disk), a magneto-optical recording medium such as an MO (Magneto Optical disk), a magnetic recording medium, a conductive memory tape medium, a semiconductor memory, or the like.

[0129] For example, when the computer 900 functions as the document processing device 100, 100A according to the above-described embodiment, the CPU 901 of the computer 900 executes a program 138 (see FIG. 1) loaded onto the RAM 903, thereby realizing the functions of the document processing device 100, 100A. The CPU 901 reads the program from a recording medium 912 and executes it. Alternatively, the CPU 901 may read the program from another device via a communication network, or may install the program 138 from the recording medium 912 onto the SSD 904 and execute it. Note that the document processing device 100, 100A is not limited to the hardware computer 900, and may also be in the form of a virtual machine. [Explanation of symbols]

[0130] 100 Document processing device (document analysis device, document evaluation device, document evaluation criterion extraction device, document proofreading device) 111 Document Analysis Department 112 Document format generator 113 Document Evaluation Department 114 Evaluation and Reporting Department 115 Evaluation Criteria Extraction Unit 116 Dictionary Generation Unit 117 Document Proofreading Department 118,118A Answer acquisition part 131 Document Database 132 language models 140 Document Format Database 150 Document Evaluation Reporting Rules Database 160 Evaluation Report Database 210 Language Model Server 410 Tabular document 420,420A Format Information Document 430,430A Item value acquisition result 450,450A evaluation results 460 Report 510 documents 520,520A Evaluation Criteria 520B Evaluation Criteria (Check Contents) 540 documents 550,550A Synonym Dictionary (Synonym Group) 560 Proofreading Documents 610 Item Value Acquisition Prompt 612 elements (format information) 613 Element (Document Information) 620, 620A, 620B, 620C Evaluation Prompt 622 Elements (Evaluation Criteria) 623 Element (Document Information to be Evaluated) 624 Elements (Questions) 630,630A,630B,630C Report Acquisition Prompt 632~635 Elements (reporting example) 640,640A Evaluation Criteria Acquisition Prompt 640B Evaluation Criteria Acquisition Prompt (Check Content Acquisition Prompt) 650,650A Synonym retrieval prompt 660,660A Calibration document acquisition prompt

Claims

1. An evaluation criteria extraction unit that references minutes containing the results of document review and requests extraction of evaluation criteria for each item in the document, and combines the items in the document and the minutes to create an evaluation criteria acquisition prompt; an answer acquisition unit that transmits the evaluation criterion acquisition prompt to a language model server and acquires the evaluation criterion directly or indirectly from the language model server; The evaluation criterion extraction unit The evaluation criteria are stored in a storage unit as evaluation criteria related to the item. Document evaluation criteria extraction device.

2. A criterion extraction unit that generates a criterion acquisition prompt by combining text requesting extraction of evaluation criteria for a document with minutes containing review results of the document and the minutes; an answer acquisition unit that transmits the evaluation criterion acquisition prompt to a language model server and acquires the evaluation criterion directly or indirectly from the language model server; The evaluation criterion extraction unit The evaluation criteria are stored in a storage unit. Document evaluation criteria extraction device.

3. The answer acquisition unit Instead of the language model server, Using a language model, obtain the evaluation criteria in response to the evaluation criteria obtaining prompt.

3. The document evaluation criterion extraction device according to claim 1.

4. A document evaluation criterion extraction device, a step of creating a prompt for obtaining evaluation criteria by merging a text requesting extraction of evaluation criteria for each item of the document with reference to the minutes of the meeting including the review results of the document, the items of the document, and the minutes of the meeting; sending the metric acquisition prompt to a language model server to acquire the metric directly or indirectly from the language model server; and storing the evaluation criteria in a storage unit as evaluation criteria related to the item. Document evaluation criteria extraction method.

5. A document evaluation criterion extraction device, A step of creating a prompt for obtaining evaluation criteria by combining a text requesting extraction of evaluation criteria for a document with a minutes of a meeting including a review result of the document and the minutes of the meeting; sending the metric acquisition prompt to a language model server to acquire the metric directly or indirectly from the language model server; storing the evaluation criteria in a storage unit; Document evaluation criteria extraction method.

Images