Comment generation program, comment generation apparatus, and comment generation method

Abstract

To provide a comment generation program, a comment generation apparatus, and a comment generation method for generating appropriate comment samples corresponding to matters pointed out.SOLUTION: In a comment generation system for supporting creation of a document regarding an agreement between a plurality of parties concerned, a comment generation apparatus 1 includes an acquisition unit and a generation unit. The acquisition unit acquires matters pointed out, as a result of evaluating a draft regarding the agreement. The generation unit outputs, to a generation apparatus, an instruction to generate comment samples regarding the matters pointed out, based on the matters pointed out and a reference matter corresponding to the matters pointed out, the reference matter being specified from among a plurality of reference matters which can be referred for creating comments.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a comment generation program, a comment generation device, and a comment generation method. [Background technology]

[0002] Legal work involves the creation and revision of documents. For example, creating documents through repeated confirmation and revision between internal parties, or repeatedly revising draft documents with external parties when making agreements, can be a time-consuming process.

[0003] One of the most time-consuming tasks in document creation is writing comments for document reviewers. For example, in large companies, after sending a contract template from one company, the other party will make revisions, and if the revisions are not acceptable to the other party, the company will often have to provide a comment explaining the reasons, and writing such comments is a significant burden.

[0004] Conventionally, a technology that provides reference information for document revisions has been known as a document creation support technology. For example, Patent Document 1 discloses a technology that stores revisions and revision comments for each version of a past contract, and presents revised versions of similar clauses in the past contracts and comments on the revisions as revision candidates when revising a specific clause in an input contract. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2020 / 230346 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the technology described in Patent Document 1 merely presents past amendments and comments on similar contracts for reference. Therefore, it is insufficient as support for creating comments, and more effective support is needed. Therefore, the present invention aims to solve the above problems. [Means for solving the problem]

[0007] In order to solve the above problems, the present invention is a program for assisting in the creation of documents related to agreements made between multiple parties, which causes a computer to function as an acquisition unit and a generation unit, wherein the acquisition unit acquires comments indicating comments made by the other party to the agreement in a draft document related to the agreement, and the generation unit outputs to a generation device an instruction to generate a draft comment related to the comments based on the comments and a reference item corresponding to the comments, which is identified from multiple reference items that can be referenced to create a comment.

[0008] This configuration makes it possible to provide more effective support for creating comments. Specifically, it becomes possible to provide a draft comment based on the points raised by the other party, thereby more effectively supporting the creation of comments to be presented to the other party.

[0009] In order to solve the above problems, the present invention is a program for assisting in the creation of documents related to agreements exchanged between multiple parties, which causes a computer to function as an acquisition unit and a generation unit, wherein the acquisition unit acquires points of concern as a result of an evaluation process on a draft document related to the agreement, and the generation unit outputs to a generation device an instruction to generate a draft comment related to the points of concern based on the points of concern and a reference item corresponding to the points of concern that is identified from multiple reference items that can be referenced to create a comment.

[0010] This configuration allows for more effective support in creating comments. Specifically, it is possible to provide comment proposals based on findings obtained as a result of the evaluation process, thereby supporting efficient comment creation. Furthermore, generating comments in response to findings obtained as a result of the evaluation process can be useful, for example, in reducing the burden of in-house checking work.

[0011] In a preferred form of the present invention, the points of interest include changes made by the other party to the agreement in the draft from the original document, or comments made by the other party in the draft, and the generation unit sends an instruction to the generation device to generate the draft comment indicating a response to the points of interest based on the points of interest and the references.

[0012] In a preferred embodiment of the present invention, the draft is a document to which the other party to the agreement has added the points pointed out to a draft document submitted by the user party in the agreement.

[0013] In a preferred form of the present invention, when the comment indicates a response to a past comment that was made in the past during the negotiation process of the agreement, the generation unit outputs an instruction to the generation device to generate a draft comment regarding the comment based on the comment, the past comment related to the comment, and the reference.

[0014] This configuration makes it possible to generate comment proposals that take into account not only the most recent findings but also the past exchanges leading up to the findings, which is expected to prevent problems such as creating comments that contradict the series of past findings related to the subject of the findings, or recreating comments that are identical to comments made by the company in the past.

[0015] In a preferred embodiment of the present invention, the past comments include comments made by the other party in past versions of the draft.

[0016] In a preferred embodiment of the present invention, the past comments include comments made by the user party on a draft document submitted by the user party.

[0017] In a preferred form of the invention, the reference includes a reference text or a reference policy on which the comment is based.

[0018] In a preferred embodiment of the invention, the reference includes a reference statement or a reference policy that denies changes to the document.

[0019] When amending documents related to agreements, it is very important to comment on the reason for rejecting a change. By configuring in this way, it is possible to appropriately create comments when rejecting a change.

[0020] In a preferred embodiment of the present invention, the reference text includes example comments that show examples of reply comments.

[0021] With this configuration, it is possible to generate natural comment ideas that correspond to the points raised, based on the comment examples.

[0022] In a preferred form of the present invention, the computer further functions as an identification unit, the document includes a plurality of items, the reference items are registered in advance in correspondence with the items, the identification unit identifies the items corresponding to the pointed out matters and identifies the reference items corresponding to the items, and the generation unit outputs to a generation device an instruction to generate a draft comment regarding the pointed out matters based on the pointed out matters and the reference items identified by the identification unit.

[0023] With this configuration, it is possible to generate more appropriate comment proposals based on reference items corresponding to document items, such as contract clauses and points of contention.

[0024] In a preferred form of the invention, the item comprises a group of clauses that define a common set of legal aspects.

[0025] In a preferred form of the present invention, one or more reference items are registered for each item, and the identification unit identifies one of the reference items associated with the item corresponding to the pointed out issue based on background information regarding the agreement.

[0026] By configuring in this way, it is possible to generate appropriate comments that are in line with the context of the agreement or document in question.

[0027] In a preferred form of the present invention, the instructions include instructions to cause the generation device to identify a reference item from a plurality of reference items that corresponds to the issue, and to generate the draft comment using the reference item.

[0028] With this configuration, for example, candidate reference items can be stored in advance in the generating device, and a comment proposal can be generated using an appropriate reference item.

[0029] In a preferred form of the present invention, the computer further functions as an identification unit, wherein a plurality of the reference items are registered in a database, the reference items including standard reference items and user reference items, the identification unit refers to a setting indicating whether the standard reference items or user reference items are to be used in generating the proposed comment, and identifies the reference items in accordance with the setting, and the generation unit outputs an instruction to a generation device to generate a proposed comment regarding the indicated item based on the indicated item and the reference items identified by the identification unit.

[0030] This configuration allows the user to select the reference items to be used for generating comment proposals, either the standard reference items or the user reference items. In other words, the user can switch between using the standard content (standard reference items) or the content (user reference items) that the user has customized by adding or modifying information.

[0031] In a preferred embodiment of the present invention, the user references include historical document comments given in historical documents relating to past agreements.

[0032] This configuration allows for the generation of draft comments by referencing past document comments, which is expected to reflect the habits and response tendencies of the user or the company to which the user belongs, resulting in the generation of draft comments that are more in line with the intentions of the parties involved.

[0033] In a preferred form of the present invention, the document includes a plurality of items, the past document comments are registered in the database in correspondence with the items in the past document, and when the comment proposal is set to use the user reference items, the identification unit identifies the item corresponding to the pointed-out item and identifies the past document comment corresponding to that item as the reference item.

[0034] With this configuration, past document comments made on the same item as the item corresponding to the target pointed out issue in the past document can be used to generate a draft comment. This is expected to result in more appropriate draft comments being generated.

[0035] In a preferred form of the present invention, the pointed out items are associated with the pointed out locations in the draft, and the generation unit outputs an instruction to the generation device to generate the draft comment based on the pointed out items and reference items as well as the content of the pointed out locations in the draft.

[0036] With this configuration, it is possible to generate a comment proposal that corresponds to the description of the pointed out part.

[0037] In a preferred embodiment of the present invention, the generating unit outputs an instruction to the generating device to generate the proposed comment based on the findings and references as well as background information about the agreement.

[0038] By configuring in this way, it is possible to generate appropriate comments that are in line with the context of the agreement or document in question.

[0039] In a preferred form of the present invention, when a user rejects the changes indicated by the suggestions, the acquisition unit accepts a selection of the strength of the rejection, and the generation unit outputs an instruction to the generation device to generate the proposed comment based on the suggestions and references as well as the strength of the rejection.

[0040] With this configuration, it is possible to generate a comment proposal that reflects the nuance desired by the user.

[0041] In a preferred form of the present invention, the acquisition unit receives from the user a specification of the issues for which the proposed comments are to be generated, and the generation unit sends the command to the generation device to generate the proposed comments for the selected issues.

[0042] With this configuration, the user can view the evaluation results and obtain comment suggestions only for the parts they deem necessary. This makes it possible to provide suggestions that are easy for the user to understand without presenting unnecessary content. Furthermore, by generating a command to generate comment suggestions for the necessary parts, the amount of information to be transmitted and received and the amount of processing in the generation device are reduced. This is expected to improve processing speed and further improve the user experience.

[0043] In a preferred form of the present invention, one or more of the points of concern are associated with the draft, and the acquisition unit accepts from the user for each point of concern a choice as to whether or not to accept the changes indicated by each point of concern, and the generation unit transmits to the generation device the instruction to generate the draft comment for the point of concern for which the user has chosen to reject the change.

[0044] When revising documents related to agreements, it is very important to comment on the reasons for rejecting changes. By configuring in this way, it is possible to generate draft comments for the points that are pointed out for rejecting changes.

[0045] In a preferred embodiment of the present invention, the proposed document is associated with a plurality of the pointed out items, and the generation unit transmits the instruction to the generation device to generate the draft comments for the plurality of the pointed out items all at once.

[0046] In a preferred embodiment of the present invention, the acquisition section acquires, as the pointed out items, comments written in the body of the draft or in a file representing the draft.

[0047] Comments and corrections to documents can be made in various formats depending on the company. By configuring in this way, it is possible to acquire various comments as comments and generate draft comments for the comments.

[0048] In a preferred form of the present invention, the computer further functions as an identification unit, the document includes a plurality of items, the identification unit identifies the item corresponding to the pointed out matter and identifies an additional item corresponding to the item from among additional items that are registered in advance in association with the item, and the generation unit sends to the generation device an instruction to generate the proposed comment based on the pointed out matter and the reference matter as well as the additional items identified by the identification unit.

[0049] With this configuration, comment proposals can be generated by further considering additional information for each item. For example, if the generation device generates comment proposals using a sentence generation model that has learned information on the web, the model may have learned information based on old laws before they were revised, or incorrect information. In this case, there is a possibility that a comment proposal containing incorrect or unnecessary content will be generated. In response to this, the occurrence of the above-mentioned problems can be prevented by registering exceptions and supplementary information as additional information for each item in advance and generating comment proposals based on this information.

[0050] In order to solve the above problem, the present invention provides a comment generation device for assisting in the creation of documents related to agreements made between multiple parties, comprising an acquisition unit and a generation unit, wherein the acquisition unit acquires comments indicating comments made by the other party to the agreement in a draft document related to the agreement, and the generation unit outputs to the generation device an instruction to generate a draft comment related to the comments based on the comments and a reference item corresponding to the comments, which is identified from multiple reference items that can be referenced to create a comment.

[0051] In order to solve the above problem, the present invention provides a comment generation device for assisting in the creation of documents related to agreements exchanged between multiple parties, comprising an acquisition unit and a generation unit, wherein the acquisition unit acquires points of interest as a result of an evaluation process for a draft document related to the agreement, and the generation unit outputs to the generation device an instruction to generate a draft comment related to the points of interest based on the points of interest and a reference item corresponding to the points of interest that is identified from multiple reference items that can be referenced to create a comment.

[0052] In order to solve the above problem, the present invention is a computer-based method for assisting in the creation of documents related to agreements made between multiple parties, in which a computer acquires comments indicating comments made by the other parties to the agreement in a draft document related to the agreement, and outputs to a generation device an instruction to generate a draft comment related to the comments based on the comments and a reference item corresponding to the comments, which is identified from multiple reference items that can be referenced to create a comment.

[0053] In order to solve the above problem, the present invention is a computer-based comment generation method for assisting in the creation of documents related to agreements made between multiple parties, in which a computer obtains findings as a result of an evaluation process for a draft document related to the agreement, and outputs to a generation device an instruction to generate a draft comment related to the findings based on the findings and a reference item corresponding to the findings, which is identified from a plurality of reference items that can be referenced to create a comment. [Effects of the Invention]

[0054] According to the present invention, it is possible to generate appropriate comment proposals that correspond to the pointed out items. [Brief explanation of the drawings]

[0055] [Figure 1] FIG. 1 is a diagram showing the configuration of a system according to a first embodiment. [Figure 2] FIG. 2 is a hardware configuration diagram of an information processing apparatus and a terminal device according to the first embodiment. [Figure 3] FIG. 1 is a block diagram showing the functional configuration of a comment generation system according to a first embodiment. [Figure 4] FIG. 3 is a diagram showing an example of stored information in a database according to the first embodiment. [Figure 5] FIG. 3 is a sequence diagram showing the flow of processing related to generation of comment ideas according to the first embodiment. [Figure 6] 10 is a screen display example of the first embodiment. [Figure 7]10 is a flowchart showing the flow of a reference matter specification process according to the first embodiment. [Figure 8] 10 is a screen display example of the first embodiment. [Figure 9] FIG. 10 is a block diagram showing the functional configuration of a comment generation system according to a second embodiment. [Figure 10] FIG. 10 is a sequence diagram showing the flow of processing related to evaluation, generation of proposed revisions, and generation of proposed comments in the second embodiment. [Figure 11] 10 is a screen display example of the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0056] The present invention relates to a technology for supporting the creation of comments to be added to a draft document (document proposal, rough draft) when creating documents related to agreements between multiple parties. For example, contracts are often created through repeated revisions that are exchanged between the contracting parties. The present invention is a technology for generating draft comments that respond to comments made on a document based on the comments and references.

[0057] The present invention will now be described in more detail with reference to the accompanying drawings, in which preferred embodiments are shown, but which may be embodied in many different forms and are not limited to the embodiments set forth herein.

[0058] For example, in the following embodiment, the configuration, operation, etc. of a comment generation system will be described, but similar effects can also be achieved by a comment generation device, a comment generation system, or a method executed by a comment generation device, a comment generation program, etc. In this embodiment, the comment generation system takes the form of so-called cloud computing, in which the comment generation program is executed on a server (comment generation device) connected to the client terminal via a network in order to realize its functions on a client terminal (user terminal).

[0059] The comment generation program may be provided to a computer device as a computer-readable non-transitory storage medium or may be provided downloadably from an external server so that the computer device can function as a comment generation device. The comment generation program may also be an add-on (plug-in) program for document editing software such as Microsoft Word (registered trademark).

[0060] In the following embodiments, a "unit" may include, for example, a combination of hardware resources implemented by a broadly defined circuit and software information processing that can be specifically realized by these hardware resources. In this embodiment, "information" is represented, for example, by a physical signal value representing voltage or current, a high or low signal value as a binary bit set consisting of 0 or 1, or a quantum superposition (so-called quantum bit), and communication and calculation can be performed on the broadly defined circuit. A broadly defined circuit is a circuit realized by appropriately combining a circuit, a processor, a memory, etc. For example, it is a circuit including any of a central processing unit (CPU), a graphics processing unit (GPU), a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), etc.

[0061] <Definition> First, definitions of terms used in the present invention will be explained. In the present invention, an "agreement" refers to an agreement made between multiple parties, and includes terms of use, declarations to users, agreements with other parties, and the like.

[0062] Furthermore, "documents relating to agreements" refers to any document that indicates an agreement, especially legal documents that are somehow related to the law. Specific examples of documents relating to agreements include contracts, rules, regulations, notices, agreements, notices, and powers of attorney. Examples of rules include legal norms such as work regulations. Examples of notices include documents declaring policies such as privacy.

[0063] Some legal documents include units divided into articles, paragraphs, subsections, etc., and in this embodiment, each of these units is called a "clause." In the following embodiment, the document to be created is described assuming that it is a legal document, particularly a contract, that includes multiple clauses and relates to an agreement with a specific party.

[0064] In the present invention, an "item" refers to, for example, one of the viewpoints that are the subject of review or evaluation of a legal document. For example, a clause may be an item of the present invention, or an issue stipulated in a legal document may be an item of the present invention. In particular, it is preferable to define the necessary items for each type of agreement (contract type, etc.), and in the following embodiments, the term "item" refers to the items divided into types of agreement.

[0065] Legal documents, including contracts, contain matters that are open to discussion or negotiation, now or in the future, regarding their content, necessity, and other matters from a legal perspective. These are generally called "issues." Specifically, issues in this invention include the contract period, the subject matter of the contract, and the specific types of exceptions to confidential information in non-disclosure agreements. Furthermore, when a set of provisions is defined by multiple clauses, that set of provisions may be treated as a single issue. For example, the provisions defined for each of the five types of confidential information exceptions may be aggregated into a group called "confidential information exceptions," and that group may be treated as a single issue. In this embodiment, an example in which issues are defined as "items" is described.

[0066] In this invention, "draft" refers to a proposal or rough draft of a document to be created. For example, when a contract is concluded between a company and an external party, one party sends the first draft of the document to the other party, and then the parties mutually confirm and revise the document, often resulting in multiple versions being created before it is completed. In this invention, "draft" broadly includes document proposals that are in the process of being considered and created before the document is completed.

[0067] Furthermore, the "draft" of the present invention does not need to be the entire document proposal, but may include at least a portion of the proposal. For example, if revisions to a portion of a document are under consideration, only that portion may be treated as a draft.

[0068] When a problem is pointed out in such a draft, the present invention creates a comment regarding the problem. Information indicating the problem is called a "pointed matter" in the present invention. In this embodiment, information indicating a problem from the other party is assumed to be a pointed matter. Specifically, a comment (text) made by the other party and changes made from the original document in the draft are included in the pointed matter.

[0069] As mentioned above, document creation often involves multiple rounds of revisions. In such cases, the other party's responses to past criticisms made by the other party regarding the agreement may be included as criticisms. Furthermore, the criticisms made by the other party in the past may themselves be responses to criticisms made by the other party even earlier. In this way, the points raised in the draft in question may be based on a series of exchanges regarding past points.

[0070] In this invention, such past comments (comments made in the past during the negotiation process of the same agreement) are generally referred to as "past comments." Hereinafter, the term "past comments" refers to comments that are the premise of the particular comment in question. In other words, past comments include comments made by the other party in previous versions of the draft and comments made by the user party in the draft document submitted by the user party.

[0071] Obtaining past comments may be performed by any method, but for example, version (edition) management of the draft may be performed and past comments may be obtained from past versions of the same agreement. Alternatively, the date and time of comments included in the draft may be obtained and older comments may be obtained as past comments. For example, of the comments included in the draft, the most recent comment assigned with the other party's ID (or an ID other than your own company's) is obtained as a comment item, and comments made before that comment are obtained as past comments. Note that since comments may be divided into multiple parts, comments from other parties within a specified time period after the most recent comment from your company, or comments from other parties with dates and times after the most recent comment from your company, may also be obtained as comments.

[0072] In other embodiments, the issues raised may be, for example, information indicating comments made by a superior within the company or information indicating comments obtained as a result of a computer-based evaluation. By generating a proposed comment for such issues, it is possible to support, for example, the creation of a reply to the superior or a comment based on the evaluation results obtained by a check tool, thereby supporting document creation and checking work within the company. Note that the above-mentioned "evaluation by computer" can be performed in the same manner as the evaluation process of embodiment 2 described below.

[0073] The "reference items" of the present invention refer to any information that can be referenced to generate a comment on a pointed out item. Specifically, they include reference text and reference policies that serve as the basis for creating a comment. The reference text includes example comments that show examples of response comments to the pointed out item, and past comments may also be used as example comments. The reference policies refer to the purpose of the comment and the creation policy. For example, the reference policy can be whether the comment is accepted or rejected, or the strength of the rejection.

[0074] In this embodiment, the reference items are associated with items and registered in advance. As a result, in this embodiment, the reference items used to generate the proposed comments are selected according to the items corresponding to the issues. In the following embodiment, a case will be described in which a reference sentence is used as a reference item.

[0075] Explain specific examples of reference matters. For example, a reference sentence in a reference matter could be linked to an item regarding the amount of damages, and read, "Thank you for pointing that out. We are very sorry, but removing the upper limit on the damages we will pay would increase our risk too much, so we cannot delete the upper limit. Therefore, we have reverted to the original sentence." It is possible to register a sentence such as, "We appreciate your understanding."

[0076] By generating comments based on these references and the points raised, it is possible to generate comments that correspond to the points raised based on the references. For example, if the other party's comment says, "We have removed the cap on compensation for damages," it is expected that the comment will be generated using words that match the points raised, such as, "Thank you for your comment. We are very sorry, but removing the cap on the compensation for damages that we will have to pay would increase our risk too much, so we are unable to remove the cap. For that reason, we have reverted to the original text. We appreciate your understanding."

[0077] (Embodiment 1) <1. System Configuration> First, a first embodiment will be described. As shown in FIG. 1, a comment generation system 0 includes a comment generation device 1 and a user terminal 2. The comment generation device 1 and the user terminal 2 are configured to be able to communicate with each other via a network NW. The comment generation device 1 operates as a server, and the user terminal 2 operates as a client terminal. The comment generation device 1 is further connected to a generation device 3 via the network NW. The generation device 3 is an external server device that provides a sentence generation model. In this embodiment, the network NW is an IP (Internet Protocol) network, but there are no limitations on the type of communication protocol, the type of network, etc.

[0078] The user terminal 2 receives input of information relating to the draft and the pointed out matters from the user, and transmits and receives information to and from the comment generation device 1. The user terminal 2 also displays information relating to the draft comment received from the comment generation device 1. Based on the information received from the user terminal 2, the comment generation device 1 transmits to the generation device 3 a command to cause the generation device to generate a draft comment based on the indicated matters and reference matters. The generation device 3 is a device that provides a sentence generation model, receives commands from the comment generation device 1, inputs them into the sentence generation model, and transmits comment proposals as outputs of the sentence generation model to the comment generation device 1. The sentence generation model will be described later.

[0079] The comment generation device 1 may be configured to store a sentence generation model and generate comment ideas without using an external device, or a part or all of the processes executed by the comment generation device 1 in this embodiment may be executed by the user terminal 2. Also, a part of the processes executed by the comment generation device 1 in this embodiment may be executed by the generation device 3.

[0080] The comment generation device 1 transmits data in a predetermined format as various display information to the application of the user terminal 2 based on a request transmitted from the application of the user terminal 2. The data in the predetermined format is, for example, data in JSON (JavaScript (registered trademark) Object Notation) format or XML format. The application of the user terminal 2 renders display data based on the received data, and displays various information based on the proposed comment on a display (display device) of the user terminal 2.

[0081] For example, when a web browser application on the user terminal 2 executes a process for displaying various pieces of information, the web browser application first sends an HTTP (Hypertext Transfer Protocol) request to a web server and receives an HTTP response containing the file requested in the HTTP request. The received file is display information including proposed comments, indicates the content or instructions of the web page to be displayed, and is written using a markup language such as HTML (Hypertext Markup Language). A script, which is a program executed on the web browser application, may be attached to the received file. The web browser application renders a web page in the display area based on the received file. If a script is attached to the file, the browser executes the script and displays the web page according to the execution result.

[0082] <2. Hardware configuration> One or more information processing devices 10 (computer devices), such as a general-purpose server or a personal computer, can be used as the comment generation device 1 or the generation device 3. In this embodiment, the comment generation device 1 is an information processing device 10 in which a computer program (comment generation program) that executes a comment generation method is installed, but the comment generation program may be installed in a terminal device 90 so that the terminal device 90 functions as the comment generation device 1.

[0083] Furthermore, a terminal device 90 (computer device) such as a personal computer, smartphone, tablet terminal, etc. can be used as the user terminal 2. The user terminal 2 has an application (web browser application or dedicated application) for making requests to the comment generation device 1 and receiving responses, and uses the processing results of the comment generation program executed in the comment generation device 1 via this application.

[0084] Fig. 2(a) is a hardware configuration diagram of the information processing device 10. As shown in Fig. 2, the information processing device 10 has a control unit 101, a storage unit 102, and a communication unit 103, which are used to perform the functions of each unit and each process.

[0085] The control unit 101 has a processor such as a CPU that can execute an instruction set, and executes an OS and programs. The storage unit 102 includes a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS, a comment generation program, a DBMS (database server), and the like. The communication unit 103 has an interface for physically connecting to a network, and controls communication with the network NW to input and output information.

[0086] Fig. 2(b) is a hardware configuration diagram of the terminal device 90. As shown in Fig. 2, the terminal device 90 has a control unit 901, a storage unit 902, a communication unit 903, an input unit 904, and an output unit 905, which are used to perform the functions of each unit and each process.

[0087] The control unit 901 has a processor such as a CPU that can execute an instruction set, and executes an OS, programs, and the like. The storage unit 902 includes a volatile memory such as a RAM capable of storing an instruction set, and a non-volatile recording medium such as an HDD or SSD capable of recording an OS, a determination program, and the like. The communication unit 903 has an interface for physically connecting to a network, and controls communication with the network NW to input and output information. The input unit 904 includes an operation input device capable of input processing, such as a touch panel or keyboard, and an audio input device capable of audio input, such as a microphone. The output unit 905 includes a display device capable of display processing, such as a display, and an audio output device, such as a speaker.

[0088] <3. Functional configuration> Next, the functional configuration will be described with reference to Fig. 3, which is a block diagram showing the functional configuration of the comment generation system of this embodiment. The comment generation device 1 includes an acquisition unit 11, an identification unit 12, a generation unit 13, an output unit 14, and a database DB. This is a specific implementation of software-based information processing using hardware.

[0089] Each functional configuration will be described in detail below. First, the information stored in the database DB will be described. FIG. 4 is a diagram showing an example of information stored in the database DB in this embodiment. The database DB pre-registers multiple reference matters. In this embodiment, the reference matters are registered in association with respective items. In particular, in this embodiment, issues are registered as items for each contract type, and one or more reference matters are associated with and stored for each issue.

[0090] In this embodiment, one or more reference items are linked to and registered for all items, but reference items may be registered for only some items, and reference items that do not specify an item may be prepared separately. In this case, a comment draft for an item that does not have a reference item linked to it may be created using a reference item that does not specify an item.

[0091] In this embodiment, reference sentences showing example comments are registered as reference items, but reference policies may be registered instead of reference sentences. When registering reference policies in association with items, it is assumed that the strength of rejection is registered for each item. For example, it is conceivable to register a reference item indicating a strong rejection of the content (e.g., correction) indicated by the pointed out item for an important item, and a reference item indicating a weaker rejection for an item that is not so important.

[0092] In this embodiment, a plurality of data sets are prepared, each having one or more reference items registered for each item. Specifically, a standard data set consisting of standard reference items that are initially available to all users, and a user data set consisting of user reference items customized by the user through settings or continued use, are registered in the database DB.

[0093] User reference items are created by users editing or adding data to a data set of standard reference items. For example, a user can create a user data set by adding past document comments added in past documents related to past agreements to a standard data set as user reference items. The reference text in Figure 4 is an example of a user reference item when such past document comments are registered. The past document comments are linked to contract background information by the contract ID. In this case, the item corresponding to the past document comment is identified in the past document, and the past document comment is registered as a user reference item in association with that item.

[0094] Note that, in this embodiment, a configuration is assumed in which the database DB stores various information including reference matters in advance, but this information does not necessarily need to be stored in a medium accessible by the comment generation device 1. For example, the reference matters may be received from the user together with the pointed out matters and acquired by the acquisition unit 11 described below, or the generation device 3 may store the reference matters in advance, and the reference matters to be used for generating a draft comment may be identified in the generation device 3. For example, the generation device 3 may be made to learn the reference matters, or the reference matters may be input and specified to be used as reference targets when generating a draft comment.

[0095] In this embodiment, the database DB also stores additional information associated with some items. The additional information is additional information for each item that the generation unit 13 uses to generate commands. If the sentence generation model used in the generation device 3 has learned information on the Web, it may have learned information based on old, pre-amendment laws or incorrect information. In this case, there is a possibility that a draft comment containing incorrect or unnecessary content will be generated. In response to this, the above-mentioned problem can be solved by registering exceptions and supplementary information as additional information according to the item in advance and generating draft comments based on this.

[0096] Therefore, in this embodiment, additional items are registered in association with items that are particularly prone to such problems. Unlike reference items, additional items do not need to be registered for all items. It is preferable that additional items be configured so that the user can add, register, edit, or delete them as appropriate.

[0097] The database DB may also store document background information and contract background information for past documents in association with user reference items. Both document background information and contract background information are included in the "background information related to agreements" of the present invention. Document background information is information related to the background of a document, such as its identification, status, and responsibility in the agreement, and includes, for example, a document ID, a status tag indicating the stage in the adjustment of the contract (agreement) specified by each document, the document creation date and time, the person who entered the document into the database DB, and the legal officer responsible for the document. In addition, data bodies indicating the contents of the document may also be stored in association with the document background information.

[0098] The document background information is linked to the contract background information by a contract ID that identifies the contract background information related to the contract (agreement) that each document specifies. The legal officer information may be entered by the user when registering the document, or may be identified from the ID of the creator of the issues (revision history or comments).

[0099] Contract background information is information about the background of a contract, such as the type and relationship of the agreement, and includes, for example, information on the contract type, the contracting party (the other party to the contract), the company's position, and the subject of the contract. The contract type is an example of the "type of agreement" in this invention. Other types of agreement include atypical contracts, types of rules or regulations, etc. The company's position refers to the contractual position, such as information on the discloser or recipient of information in a confidentiality agreement. The subject of the contract is an example of the "subject of the agreement," and refers to the products, services, or business that are the subject of each contract (arrangement). For example, the subject product in a sales contract would be the subject of the contract.

[0100] As described above, multiple documents are often created through repeated revisions before a single contract is concluded. In this embodiment, the document background information for multiple documents created in the process of concluding the same contract is linked to the same contract background information in the database DB. This makes it possible to identify documents relating to the same agreement.

[0101] The data structure can be changed as desired. For example, in addition to document background information and contract background information, contract family information may be defined to manage groups of mutually related contracts. Specifically, the document background information may include a contract family ID that identifies the contract family information, and the document background information for multiple documents created in the process of concluding the same contract may be managed using the contract family ID. In this case, the contract family information may include a contract ID that identifies the contract background information, but the document background information may not include a contract ID.

[0102] Next, an overview of the processing performed by each unit of the comment generating device 1 will be described. The acquisition unit 11 acquires comments indicating comments made by the other party to the agreement in a draft of a document related to the agreement. In this embodiment, a document file representing the draft is received from the user terminal 2, and the acquisition unit 11 acquires comments written in the document file as comments. Specifically, messages written in the main text of the document file, comments added separately from the main text, records of change history, etc. can be acquired as comments. Note that the change history is information including the content of the changes, and may be information including the text before and after the change, or information including the content of the changed parts.

[0103] In addition, a file containing the findings may be obtained separately from the document file representing the draft. An example of obtaining findings from a separate file is a file containing the text of the identified section, the changes, and comments on the changes in a table format. In this case, the changes and comments are obtained as findings.

[0104] In this embodiment, in addition to the findings, the acquisition unit 11 acquires past findings related to each finding, the contents of the draft, and references corresponding to the findings. Furthermore, the acquisition unit 11 may acquire, for the target document, document background information indicating the background of the document and contract background information (agreement background information) indicating the background of the contract specified by the document. For the draft, only the contents of the part related to the findings may be acquired, but it is preferable to acquire the entire text. For the references, the acquisition unit 11 acquires the references identified by the identification unit 12 from multiple references registered in the database DB.

[0105] The identification unit 12 identifies an item corresponding to the pointed out matter and identifies a reference item corresponding to the item from among the reference items registered in the database DB. When multiple reference items corresponding to the item are registered, the identification unit 12 identifies the reference item to be used for generating a draft comment based on background information including document background information and contract background information. Note that the reference items may be learned in advance by the generation device 3, and the generation device 3 may be equipped with a configuration equivalent to the identification unit 12.

[0106] The generation unit 13 outputs to the generation device 3 a command to generate a draft comment regarding the pointed out matter based on the pointed out matter acquired by the acquisition unit 11 and a reference item corresponding to the pointed out matter that is identified from multiple reference items that can be referred to for creating a comment. The generation unit 13 of this embodiment generates a command statement to generate a draft comment indicating a response to the pointed out matter based on the pointed out matter and the reference item identified by the identification unit 12.

[0107] In this embodiment, the generation unit 13 generates a sentence including the pointed out matter and the reference matter as a command. If additional matters are registered in the database DB in association with the items corresponding to the pointed out matter, a command is further generated based on the additional matters. The command may also be generated using background information. Specifically, a sentence including the pointed out matter, the reference matters, the additional matters, and the background information is generated as a command. The generation unit 13 transmits the command thus generated to the generation device 3.

[0108] The output unit 14 transmits information about the proposed comment output by the generating device 3 in response to the command output by the generating unit 13 to the user terminal 2. Specifically, the output unit 14 performs display processing on information based on the proposed comment and transmits the processing results to the user terminal 2, thereby displaying the proposed comment on the user terminal 2. The output unit 14 also performs display processing on various other information and transmits the processing results, causing various screens to be displayed on the user terminal 2.

[0109] Next, the generating device 3 will be described. The generation device 3 is a device that provides a sentence generation model, receives commands from the comment generation device 1, inputs them into the sentence generation model, and sends comment proposals to the comment generation device 1 as outputs of the sentence generation model.

[0110] A sentence generation model refers to any model that can output a sentence. A model generated by machine learning can be typically used as a sentence generation model. In this embodiment, it is assumed that a conversational sentence generation model is used, which receives an input sentence and generates a sentence as a response to the input sentence.

[0111] More specifically, a sentence generation model is a machine learning model for natural language processing, and has the function of generating sentences based on input information. For example, a sentence generation model is a machine learning model based on a Transformer model with a self-attention mechanism. An example of a Transformer model is a GPT (Generative Pre-trained Transformer) model. By learning using a large dataset, a sentence generation model can perform not only general natural language generation tasks but also highly specialized sentence generation tasks.

[0112] For example, the sentence generation model may be a model trained on a large dataset using the GPT-3 or GPT-4 algorithm developed by OpenAI (for example, a ChatGPT model generated by combining training on a large dataset with reinforcement learning using a reward prediction model). In addition, although the present embodiment describes an example in which text information is input to the model, image information or audio information may also be input.

[0113] <4. Processing Procedure> Next, a processing procedure for generating a suggested comment in this embodiment will be described in detail with reference to Fig. 5. Fig. 5 is a sequence diagram showing the flow of generating a suggested comment. First, in step S501, the user terminal 2 transmits to the comment generating device 1 a document file expressing a draft on which the other party has made a comment (including the commented matter).

[0114] In the comment generation device 1, the acquisition unit 11 receives the document file in step S502 and recognizes the issues from the document file. Specifically, if the document file is in a format created by word processing software (for example, a format created by Microsoft Word (registered trademark) or a format created by word processing software executed on the cloud), and a comment is inserted in a data format different from that of the main text, or if a change history is recorded, the comment or change history is acquired as an issue. For example, each comment or change history, the ID of the person who made the comment or change, and the date and time of the comment or change are acquired, and the other party's latest comment or change history is acquired as an issue based on the ID and date and time.

[0115] Furthermore, when a document file is in a format created by spreadsheet software (for example, a format created by Microsoft Excel (registered trademark) or a format created by spreadsheet software executed on the cloud), and the contents of the columns containing the contents before and after the changes and comments are entered in each column of the table format, the contents of the columns containing the comments are acquired as issues to be pointed out. For example, the most recent comments or change history of the other party are determined based on the column title of a specific column in the table, and the contents of the determined column are acquired as issues to be pointed out.

[0116] Furthermore, if a comment is written directly in the main text, the entire text may be sent to the generation device 3, and the comment may be extracted from there. For example, the main text and the comment often have different writing styles, and are often enclosed in parentheses or symbols, so for a section written in a different style, format, or writing tendency from the main text, a command to extract the item where the relevant statement was found and the content of the statement, as well as information including the main text, is sent to the generation device 3, and the results output by the generation device 3 are obtained as findings.

[0117] The acquisition unit 11 also identifies the pointed-out portion in the text along with the pointed-out item. Specifically, for example, if a comment associated with the pointed-out portion is added to a document file, the acquisition unit 11 identifies the pointed-out portion. The acquisition unit 11 then acquires the content of the pointed-out portion (the content of a portion of the text). In addition to this, the acquisition unit 11 also acquires background information. For example, the acquisition unit 11 acquires an item to which a comment or a change history has been added as the pointed-out portion. For example, the acquisition unit 11 also acquires the content of the item to which a comment or a change history has been added (a portion of the text).

[0118] Next, in step S503, the output unit 14 performs display processing on the recognition result in step S502 and transmits the processing result to the user terminal 2. Fig. 6 is a diagram showing an example of how the recognition result is displayed on the user terminal 2.

[0119] The findings display screen W6 displays an outline of the document and the number of identified findings at the top of the screen. The left side of the screen displays the contents of the draft identified by the document file. There is also an "Edit read contents" button, and if there are any errors in the document contents, you can edit the contents by selecting this button.

[0120] The results of the findings are displayed on the right side of the screen. For each finding, the findings display area W61 is displayed along with the clause number corresponding to the finding, and a selection button W62 for the content of the finding, the date, and the comment creation policy is displayed.

[0121] In addition, a mark is attached to each issue and a serial number is displayed. The same mark is also displayed with the serial number in the corresponding part of the document on the left side of the screen, so that the user can recognize the position in the target document that corresponds to the issue, and the corresponding part in the target document is highlighted.

[0122] As in the pointed out item display section W61 of "Article 6" in FIG. 6, for pointed out items that are responses to past points, the content of the past points and the party who made the points are displayed along with the date.

[0123] The user can instruct the creation of a draft comment for each pointed out item by selecting the selection button W62. Then, when the user selects the comment creation button W63, a comment creation request is sent in step S505 in FIG.

[0124] Here, the user does not need to select the selection button W62 for all of the findings; if selections are made for only some of the findings, a comment creation request will be sent for one or more findings for which a comment creation policy has been selected using the selection button W62. That is, the user can use the selection button W62 to specify, from the findings added to the draft, those for which a comment proposal will be generated. Here, selecting a comment creation policy using the selection button W62 corresponds to selecting whether or not to accept the relevant findings and the strength of the rejection.

[0125] Here, when a comment creation request is made for some of the findings, the comment creation request includes only the items related to those findings. This results in shorter commands being generated by the generation unit 13 than when creating comment drafts for all the findings in the entire draft at once. Furthermore, processing time can be reduced by allowing the user to individually select the necessary parts and make a comment creation request. These actions are expected to have the effect of improving the user experience, so it is more preferable to generate comment proposals for some of the multiple issues, especially for the parts that correspond to the issues selected by the user.

[0126] In step S506, the identification unit 12 identifies reference items corresponding to each pointed out item for which a draft comment is to be generated. The process of identifying reference items will be described with reference to Fig. 7. First, in step S701, the identification unit 12 acquires the background information acquired by the acquisition unit 11 together with the draft in step S501.

[0127] Then, in steps S702 to S704, the identification unit 12 identifies the reference item. In this embodiment, first, in step S702, the identifying unit 12 identifies an item corresponding to the pointed out portion of the pointed out matter. For example, if the pointed out portion is a description portion of a definition clause, an item related to the definition clause is identified as the item corresponding to the pointed out matter.

[0128] Next, in step S703, the identification unit 12 identifies a reference sentence (reference item) registered in the database DB in association with the item. At this time, the identification unit 12 refers to a setting registered in advance that indicates whether a standard dataset or a user dataset is to be used to generate a draft comment. Then, the identification unit 12 identifies the reference sentence from the dataset corresponding to the setting among the datasets of reference items registered in the database DB.

[0129] If there are multiple reference sentences registered in association with the item, in step S703, the identification unit 12 identifies the reference sentence to be used to generate a comment proposal based on the background information acquired by the acquisition unit 11 in step S501.

[0130] Specifically, for example, the input background information is compared with the background information corresponding to each reference sentence registered in the database DB, and the reference sentence with the most matching information is identified as the reference sentence to be used for generating comment suggestions.

[0131] In addition, a comment creation policy may be registered for each reference sentence in the database DB, and the reference sentence may be identified based on the comment creation policy selected by the user. For example, reference sentences corresponding to each comment creation policy may be registered for each item, and the one corresponding to the comment creation policy selected by the user may be identified as the reference item to be used for generating a comment draft.

[0132] 5, when the identification of the reference matter in step S506 is completed as described above, in step S507, the generation unit 13 generates a command for the generation device 3. The generation unit 13 generates a comment command for causing the sentence generation model to generate a comment proposal indicating a response to the pointed out matter, based on the identified reference sentence and pointed out matter.

[0133] The sentence generation model of this embodiment is a conversational model that receives a sentence in a natural language as input and generates a sentence as a response, so the generation command is generated as a sentence in a natural language. More specifically, the generating unit 13 can generate a comment command by generating a sentence that combines a prepared instruction, a pointed out item, and a reference sentence.

[0134] Instructions such as "Please write a reply to the following comment that strongly rejects the comment based on the following reference text," "Please write a reply to the following comment that weakly rejects the comment based on the following reference text," and "Please write a reply to the following comment that accepts the comment," can be stored in advance in the database DB for each comment creation policy selected by the user.

[0135] Here, if additional information associated with the item corresponding to the identified issue is registered in the database DB, it is preferable to include the additional information in the comment command text. For example, to appropriately address the fact that the term "defect" is no longer used under the revised Civil Code, a rule that replaces the word "defect" with "contract non-conformity" in sales contract clauses regarding contract non-conformity may be used as an additional information. If the training data is outdated or incorrect, a sentence generation model may generate incorrect sentences (e.g., sentences containing words that should not be used or redundant sentences). For example, if the model is trained on contracts from before the legal revision, it may generate sentences containing words (e.g., "defect") that are no longer used depending on the type of clause due to the legal revision. Because the tendency to generate such incorrect sentences varies depending on the item (issue), it is preferable to adjust the text to prevent incorrect sentences for each item. Therefore, by including additional information, which is a rule that adjusts the text to prevent incorrect sentences corresponding to each pre-registered item, in the comment command text, more appropriate sentences can be generated.

[0136] In addition, it is preferable to include in the comment command text background information and the content of the pointed-out portion of the draft text that relates to the pointed-out item.

[0137] As a variant example, when the generation device 3 performs the process of identifying reference items, the comment command further includes a sentence instructing the device 3 to identify the reference items to be used in generating the comment proposal from among multiple reference items based on the identified items.

[0138] Once the generation command is generated, the process proceeds to step S508, where the generation unit 13 transmits a comment command to the generation device 3. Upon receiving the comment command, the generation device 3 inputs the comment command into the sentence generation model, which then outputs a comment proposal (step S509).

[0139] In step S510, the generation device 3 transmits the proposed comment output by the sentence generation model to the comment generation device 1. Then, in step S511, the output unit 14 transmits information based on the proposed comment output by the sentence generation model to the user terminal 2. Through the above process, the user can obtain a proposed comment by inputting a document file representing a draft.

[0140] FIG. 8 is an example of the display of the proposed comment display screen W8 output on the user terminal 2. As with the findings display screen W6, the top of the screen displays an outline of the document and the number of recognized findings. The left side of the screen displays the contents of the draft identified by the document file. There is also an "Edit read contents" button, and if there is an error in the document contents, the content can be edited by selecting this button.

[0141] Draft comments generated for each issue are displayed on the right side of the screen. For each issue, the content of the issue, the date, and a draft comment display area W81 are displayed along with the clause number corresponding to the issue. In the draft comment display area W81, a copy button is displayed along with the generated draft comment, and the user can copy the text by selecting the copy button. Furthermore, if the comment generation program of the present invention is provided as an add-on (add-in, plug-in) program for document editing software, an add comment button may be provided instead of the copy button. In this case, when the user operates the add comment button, a comment (reply) using the draft comment is added directly in the document editing software.

[0142] As with the findings display screen W6, a findings mark is attached to each finding and a serial number is displayed. The same findings mark is also displayed with the serial number in the corresponding part (point of findings) of the document on the left side of the screen, so that the user can recognize the position in the target document that corresponds to the findings, and the corresponding part in the target document is highlighted or otherwise emphasized.

[0143] As described above, the comment generation system according to the first embodiment can generate draft comments that indicate responses to points raised by the other party, thereby significantly reducing the workload of creating comments related to the exchange of contracts and the like.

[0144] Furthermore, not only the proposed comments but also the proposed revisions corresponding to the pointed out portions may be generated. Specifically, the generation unit 13 may output to the generation device 3 a command to generate a proposed revision for the pointed out portion based on the pointed out portion, reference matters, the description of the pointed out portion in the draft, background information, etc. Specifically, the generation unit 13 generates a command statement based on the pointed out portion, reference matters, the description of the pointed out portion in the draft, and background information, and transmits this to the generation device 3. In this way, revisions corresponding to the other party's points of criticism and accompanying proposed comments can be generated all at once.

[0145] Furthermore, a comment proposal may be generated that takes into account not only the pointed out matters but also the company's own proposed revisions. Specifically, the generation unit 13 may output to the generation device 3 a command to generate comments regarding the pointed out matters and the company's proposed revisions based on the pointed out matters, reference matters, the company's proposed revisions, etc. The company's proposed revisions may be proposed revisions generated by the generation device 3 or proposed revisions input by the user. This makes it possible to provide comments that address not only the other party's pointed out matters but also the company's own proposed revisions (for example, the intention of the revisions, etc.).

[0146] Reference items corresponding to the issues raised may be identified based on the company's proposed correction. For example, a configuration may be adopted in which the user selects the type of correction, and reference items associated with the type of correction are identified. For example, the identification unit 12 determines the type of company's proposed correction selected by the user. If the selected type of correction is a clerical error correction, the identification unit 12 identifies the reference sentence "This was a clerical error, so we will correct it" as a reference item associated with the clerical error correction. In this case, if the other party's comment is "What does 'damage compensation' mean?" and the company's proposed correction is "damage compensation," a comment based on the company's proposed correction and the other party's issues raised may be generated, such as "'damage compensation' was a clerical error, so we will correct it to 'damage compensation'." The method of identifying reference items based on the company's proposed correction is not limited to user selection; it may be any method that can identify reference items corresponding to the content of the company's proposed correction. For example, a sentence before correction and a sentence after correction by the company may be input into a sentence generation model, and the reference items to be referenced may be identified from multiple predetermined reference items.

[0147] (Embodiment 2) Next, a description will be given of a second embodiment. The second embodiment is similar to the first embodiment in that a proposed comment on a pointed out item is generated based on the pointed out item and a reference item. On the other hand, the comment generation device 1 of the second embodiment differs from the first embodiment in that it further includes an evaluation unit and a correction unit, performs evaluation processing and correction processing on the draft, and generates a comment proposal according to the content thereof.

[0148] In addition, in the first embodiment, an example was shown in which a draft comment was created in response to the points raised by the other party, but in the second embodiment, an example will be described in which a comment was created regarding points raised by one's own company, rather than points raised by the other party. For example, in a case where a template document prepared by the other party is used as a draft, which does not include the points raised, a form is assumed in which corrections are made and comments are sent to the other party regarding the corrections.

[0149] In the following, the description will be centered on the parts that are different from the first embodiment, and the description of the parts that are common to the first embodiment will be omitted. That is, the parts that are not described are the same as the first embodiment.

[0150] <1. Functional configuration> FIG. 9 is a block diagram showing the functional configuration of the comment generation system according to this embodiment. The comment generation device 1 includes an acquisition unit 11, an identification unit 12, a generation unit 13, an output unit 14, a database DB, an evaluation unit 15, and a correction unit 16.

[0151] The evaluation unit 15 executes an evaluation process on the draft and acquires evaluation information including findings associated with a part of the draft. The evaluation information may be acquired from an external device that evaluates the input document, but in this embodiment, the evaluation unit 15 executes an evaluation process on the draft to acquire the evaluation information.

[0152] The evaluation information includes, for each of one or more issues, the clause in the draft that corresponds to the issue and an evaluation comment. The evaluation comment is a statement that suggests corrections or points out problems with the contents of the input document, such as, "Is it necessary to specify a consultation clause?" or "Isn't the confidentiality obligation period set longer than necessary?" In addition, the evaluation comment may include information such as an explanatory text about the clause related to the issue, a template for the clause, and a risk level indicating the importance of the issue. The issues are legal evaluations, and are evaluations from the perspective of, for example, potential legal issues.

[0153] Specifically, the acquisition unit 11 acquires at least a portion of the draft together with background information, and the evaluation unit 15 acquires this information. The draft is then evaluated based on the background information. The evaluation may be performed by any method, such as evaluation using a machine learning model or rule-based evaluation. For example, when rule-based evaluation is applied, the evaluation unit 15 determines which evaluation comment to associate with each part of the draft (e.g., clause) from among multiple evaluation comments predetermined for each part of the draft (e.g., clause), based on keywords included in the part of the draft and predetermined keyword criteria (rule-based model).

[0154] Furthermore, when evaluation using machine learning is applied, the evaluation unit 15 determines the evaluation comment to be associated with the portion of the draft using a machine learning model that outputs an estimation result that estimates which evaluation comment should be associated from multiple predetermined evaluation comments based on the content of the portion of the input draft. Alternatively, a sentence generation model may be applied as a type of machine learning model, and the evaluation unit 15 may generate evaluation comments for the portion of the draft based on predetermined evaluation rules. The content of each clause and the relationship between multiple clauses can be evaluated using, for example, the contract type and contractual position, and the title of each clause in the draft as background information. The evaluation may be performed from the perspective of whether unnecessary clauses (or clauses that are disadvantageous to the party in the background information, given the relationship with the other party) are included, whether there are any clauses that should be added, and whether the content of the clauses is appropriate.

[0155] In this embodiment, evaluation processing is performed for each item on a rule basis, so evaluation comments (indications) are associated with the items, thereby identifying the items for each indication.

[0156] The correction unit 16 inputs the draft and evaluation information to the generation device 3, causes the generation device 3 to generate a revision proposal corresponding to the draft, and acquires the revision proposal. In this embodiment, an example will be described in which the correction unit 16 generates a revision command in natural language and transmits it to the generation device 3, and the generation device 3 generates the revision proposal using a sentence generation model. As the sentence generation model, a model similar to the sentence generation model for generating comments can be used.

[0157] The revision command includes the text of the draft and evaluation information, and is information indicating a command for causing the writing generation model to output a revision proposal for the draft. In this embodiment, since a conversational writing generation model that receives text as input is used, a sentence or phrase in natural language is generated and transmitted as the revision command, instructing the input document information to be revised using the draft and evaluation information. The revision unit 16 then receives the sentence or phrase output as an answer by the writing generation model and acquires the revision proposal.

[0158] Furthermore, in generating the revision proposal, related document information indicating the contents of related documents may be used instead of or in addition to the evaluation information. Related documents are documents related to the document to be created and are used as references for revising the document to be created. It is particularly preferable that related documents are documents that share background information with the document to be created, such as the type of agreement, the other party, the company's position, and the subject of the contract.

[0159] The related document information is information that indicates the content of a part of the related document that corresponds to the input document information. For example, if the input document information is information that indicates the content of a clause that defines a certain provision in the input document, the related document information is information that indicates the content of a clause that defines a similar provision in the related document.

[0160] Furthermore, the related document information of this embodiment includes information regarding corrections made to the portion indicated by the related document information when the related document was created. The information regarding the corrections may be, for example, information regarding the content before and after the correction, the correction location, the user who made the correction, comments regarding the correction, etc. The correction location may be identified based on the difference between the content before and after the correction, and is information that identifies the position within the document.

[0161] Here, the correction unit 16 of this embodiment generates a correction command for each pointed out item included in the evaluation information. At this time, if additional items are registered in the database DB in association with the items corresponding to the pointed out items, it is preferable to also include the additional items in the correction command. This is expected to have the effect of preventing a correction plan including incorrect or unnecessary content from being generated by a generation device 3 that has learned information based on old laws before amendment or incorrect information.

[0162] The acquisition unit 11 of this embodiment acquires, as the pointed out items, evaluation comments constituting the above-mentioned evaluation information and the details of corrections made by the correction unit, instead of the pointed out items indicating the points made by the other party acquired in embodiment 1. In addition, the acquisition unit 11 acquires other information similar to the information acquired in embodiment 1.

[0163] <2. Processing Procedure> Next, a processing procedure relating to evaluation, generation of a proposed revision, and generation of a proposed comment will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing the flow relating to evaluation, generation of a proposed revision, and generation of a proposed comment.

[0164] (1) Evaluation process First, in steps S1001 to S1004, evaluation processing is executed. This processing is for acquiring evaluation information indicating the evaluation of the draft. In this embodiment, as will be described below, the evaluation unit 15 evaluates the draft, but the evaluation information may be acquired by having an external device evaluate the draft.

[0165] In step S1001, the user terminal 2 sends a rating request together with the draft to the comment generation device 1. The rating request includes the draft and background information. In this embodiment, a document file containing the contract draft is uploaded. The background information is the same as in the first embodiment.

[0166] In the comment generation device 1, the acquisition unit 11 receives the evaluation request, and in step S1002, the evaluation unit 15 executes evaluation processing based on the draft and background information. The draft may be evaluated by any method, such as evaluation using a machine learning model or rule-based evaluation. As an example, in this embodiment, a method of evaluating points and clauses based on rules in accordance with background information is used.

[0167] In particular, in this embodiment, the content of each clause and the relationships between multiple clauses are evaluated using the contract type and contractual position from the background information, and the title of each clause in the draft. Here, multiple clauses refers to a group of multiple clauses that defines a set of regulations. For example, within a group, the presence or content of clause B may be determined based on the content of clause A, and then clause C may be confirmed based on the result. By specifying that the evaluation process branches depending on the content, it is possible to evaluate the relationships between multiple clauses in a rule-based manner.

[0168] When evaluating a draft, it is evaluated based on whether it contains unnecessary clauses (or clauses that would be disadvantageous to the party in relation to the other party in the background information), whether there are clauses that should be added, and whether the content of the clauses is appropriate.

[0169] In this embodiment, the issues raised in the draft are identified by linking them to the above rules and registering them in advance in a database DB. Specifically, for each issue raised, the above-mentioned evaluation rule, the type of the issue (deletion, addition, or content modification of a clause), evaluation comments, and a risk level are stored in advance in a storage unit.

[0170] Evaluation comments are statements that suggest revisions or point out problems with the draft content, such as, "Isn't it necessary to specify a consultation clause?" or "Isn't the confidentiality obligation period set longer than necessary?" In addition, evaluation comments may also include explanatory text about the clauses in question or template information for those clauses. Risk levels are information that indicate the degree of risk assumed for each issue. Risk levels may be fixed (unchangeable) by the system, or may be changeable by user input.

[0171] Examples of risk level settings are as follows. Issues that are likely to become issues after the contract is signed and involve disputes over the company's important rights and obligations can be assigned a risk level of "A." Examples include "intellectual property ownership," "scope of damages," and "definition of confidential information." Issues that are unlikely to become issues even if omitted can be assigned a risk level of "C." Examples include "confidential information exceptions" and "intermediate reporting obligations." Issues that do not fall into either A or C can be assigned a risk level of "B." Risk levels can be assigned for each issue, or they can be determined based on the wording of the clause. For example, risk levels can be set based on whether the clause's wording is favorable or unfavorable for the company.

[0172] As described above, the evaluation unit 15 executes the evaluation process, creates evaluation information indicating the evaluation of the draft, and the acquisition unit 11 acquires the evaluation information (step S1003). The evaluation information includes, for each of one or more findings identified in the above process, an item corresponding to the finding, an evaluation comment, and a risk level. Also, when acquiring evaluation information from an external device, evaluation information including, for each finding, a clause corresponding to the finding, an evaluation comment, and a risk level is acquired.

[0173] The draft may be evaluated using a sentence generation model. The sentence generation model may be the same as or different from the sentence generation model used for comment generation or revision. For example, the comment generation device 1 acquires the draft and background information using the acquisition unit 11, reads out pre-stored evaluation rules based on the background information using the evaluation unit 15, generates an evaluation command based on the draft and the evaluation rules corresponding to the background information, transmits the command to the generation device 3, and acquires the evaluation result (evaluation information) generated by the generation device 3.

[0174] At this time, the evaluation command is generated as a sentence in natural language. This is expected to reduce the impact on the evaluation due to variations in sentence spelling and differences in expression. In addition, it is expected that an appropriate evaluation will be performed by using an evaluation rule (e.g., an evaluation rule set for each contract type) read out from multiple evaluation rules based on background information (e.g., contract type).

[0175] Furthermore, evaluation using a sentence generation model and evaluation using a rule base may be performed simultaneously. In this case, if the evaluation using the sentence generation model and the evaluation using the rule base differ, it is preferable to prioritize the evaluation using the rule base. This ensures a minimum evaluation using the rule base even if there is uncertainty in the evaluation using machine learning. Note that the evaluation unit 15 may determine which evaluation rule to include in the evaluation command from among multiple evaluation rules corresponding to each piece of background information based on the background information, or the evaluation command may include multiple evaluation rules and a command to use an appropriate rule from the multiple rules based on the background information, thereby determining the evaluation rule to be used by the sentence generation model.

[0176] When the evaluation unit 15 finishes evaluating the draft, in step S1004 the output unit 14 performs display processing on the evaluation information and transmits the processing result to the user terminal 2. When the user terminal 2 receives the evaluation information, the evaluation information is displayed and the processing result is displayed on the display.

[0177] FIG. 11 shows an example of the evaluation screen displayed on the user terminal 2 as a result of the evaluation information display process. The evaluation screen W11 displays an outline of the document draft and an outline of the evaluation results at the top of the screen. The contents of the draft identified by the draft information are displayed on the left side of the screen. There is also a "Read Content Edit" button, and if there is an error in the content identified from the draft information, the content can be edited by selecting this button.

[0178] The right side of the screen displays the evaluation results based on the evaluation information. For each point of criticism, the type of criticism (deletion, addition, or revision of the clause) and an explanation are displayed as evaluation comments.

[0179] Furthermore, for each issue, a differently shaped mark is attached for each type of issue, and a serial number is displayed for each type.Similar marks are also displayed along with serial numbers in the corresponding parts of the draft on the left side of the screen, allowing the user to identify the positions in the draft that correspond to the issues, and the relevant parts in the draft are highlighted.In the example of Figure 11, circles indicate revisions to the content of clauses, and triangles indicate issues related to missing clauses.

[0180] Furthermore, in this embodiment, a correction input field W111 is displayed for each indicated item. In this embodiment, since both the evaluation information and the related contracts are used in the correction process, the related contracts are displayed in the correction input field W111. A correction button W112 is then provided, allowing corrections to be made using the displayed related contracts. Note that a configuration may also be adopted in which multiple candidate related contracts are displayed, allowing the user to select the related contract to use for correction.

[0181] The comment instruction section W113 is displayed in an operable manner when the edit button W112 is selected, and receives input from the user indicating that a comment proposal regarding the edit is desired to be generated, and a edit request is then sent in step S1006.

[0182] Here, it is preferable that the user does not need to select the Correct button W112 for all pointed out items, but selects the Correct button W112 only for pointed out items for which the user wishes to correct or for which a proposed comment regarding the correction is to be generated. This makes it possible to generate corrections or proposed comments regarding the correction only for pointed out items for which the user wishes to correct or for which a proposed comment regarding the correction is to be generated. For example, a correction request is sent for one or more pointed out items for which a correction instruction was issued using the Correct button W112 selected by the user.

[0183] Furthermore, when "Yes" is selected in the comment instruction section W113 for a problem for which the correction button W112 has been selected, a request for generating a comment regarding the correction of the problem is also sent along with the correction request. That is, the user can use the correction button W112 and the comment instruction section W113 to specify, among the problems identified as a result of the evaluation process, the problem to be corrected and the problem for which a proposed comment is to be generated. Note that the comment instruction section W113 may also accept a comment creation policy, such as a strong rejection or a weak rejection, as in the first embodiment. Alternatively, the correction button W112 and the comment instruction section W113 may be integrated, and the user may press the integrated correction button to simultaneously instruct the generation of a proposed revision and a comment.

[0184] (2) Correction process Next, the draft revision process will be described. The revision process is executed in steps S1005 to S1010. When a revision request is received as described above, the acquisition unit 11 acquires the draft, evaluation information, and related contracts in step S1006. The acquisition unit 11 of this embodiment also acquires the background information described above. In this embodiment, the draft and background information received in step S1001 and the evaluation information generated by the evaluation unit 15 in step S1002 are stored in a database DB, and the acquisition unit 11 acquires the draft, background information, evaluation information, and related contracts based on these. When the evaluation is performed by an external device, the acquisition unit 11 may acquire the evaluation information from the user terminal 2 or the external device.

[0185] The draft does not have to include the entire content of the document, and only the part related to the pointed out matter to be amended may be included in the draft. However, for example, if there is a group of multiple clauses that define a set of regulations and the pointed out matter for the clauses included in that group is to be amended, it is preferable to include the content of all the clauses in that group, rather than just the single clause that corresponds to the pointed out matter.

[0186] Next, in step S1007, the correction unit 16 generates a correction command to make the text generation model generate a proposed revision of the draft, based on the draft, background information, evaluation information, and related contracts acquired by the acquisition unit 11. Since the text generation model of this embodiment is a conversational model that receives a text in a natural language as input and generates a text that will be a response, the correction command is generated as a text in a natural language.

[0187] The correction unit 16 of this embodiment generates a sentence as a correction command, which describes instructions for correction for each clause corresponding to the pointed out item. In particular, if there are multiple pointed out items in a single clause or a group of clauses that defines a set of regulations, generating a correction command for each pointed out item may result in multiple contradictory correction proposals. Therefore, the correction unit 16 of this embodiment generates a sentence of correction command for each clause or group of clauses that corresponds to the multiple pointed out items as a correction command for a clause or group of clauses that includes multiple pointed out items.

[0188] The generation of the correction command will be specifically described below. First, for background information, when background information is input by selection, the words corresponding to the selected background information are identified. For example, by setting corresponding words in advance for each background information option, the words corresponding to the selected background information can be identified.

[0189] Then, the correction unit 16 generates a correction command based on the words, the draft, and a command separately registered in advance in the storage unit of the comment generation device 1. When the background information is input as free text, the correction command can be generated using the input background information and a command prepared in advance.

[0190] Here, it is preferable to store a character string describing a document creation rule corresponding to the background information as a word corresponding to the background information. For example, in a legal document, the legal meaning may differ between describing an exception within a sentence of a provision and describing a disclaimer separately after the sentence. By combining words specifying rules for such description methods according to the background information, an appropriate legal document can be created. Note that the comment generation device 1 may determine a rule to be included in a revision command based on the background information from among multiple rules corresponding to each piece of background information, or a configuration may be adopted in which multiple rules are included in the revision command and a statement to use an appropriate rule from the multiple rules based on the background information is included in the revision command, thereby determining the rule to be used by the sentence generation model.

[0191] Instructions such as "The following comments were made about the above draft as a result of the review. Please prepare a revised draft based on the following conditions, taking the comments into consideration," or "The following comments were made about the above draft as a result of the review. Please prepare an appropriate contract document based on the following rules, taking the comments into consideration," can be stored in advance in the database DB. Then, a sentence combining the contents of the draft, the instructions, and words based on the background information can be generated as a revision command to instruct the text generation model to revise the draft.

[0192] Here, if additional matters associated with the item corresponding to the pointed out matter are registered in the database DB, it is preferable to also include the additional matters in the text of the comment command. Alternatively, by making the above instruction statements different for each item, exceptions and supplementary matters for each item may be included in the instruction statements. This is expected to have the effect of preventing a generation device 3 that has learned information based on old laws before the amendment or erroneous information from generating an amendment proposal that includes incorrect or unnecessary content.

[0193] Once the correction command is generated, the process proceeds to step S1008, where the correction unit 16 transmits the generated correction command to the generation device 3. Note that the correction command generated as described above for each pointed out item, clause, or group of clauses may be transmitted in multiple batches. For example, it may be configured to first transmit the directive portion of the correction command, followed by words corresponding to the pointed out item or request. Also, it may be configured to determine whether to transmit the correction command all at once or in multiple batches depending on the number of characters in the correction command.

[0194] Upon receiving the correction command, the generation device 3 inputs the correction command into the sentence generation model. As a result, the sentence generation model outputs a correction proposal for the draft (step S1009), and in step S1010, the correction proposal is sent to the comment generation device 1. Note that the correction unit 16 may generate a correction command instructing to propose multiple correction proposals for one pointed out item in step S1008, and the generation device 3 may obtain multiple correction proposals for some or all of the pointed out items in step S1009.

[0195] (3) Comment generation process Next, the comment generation process will be described. In steps S1011 to S1016, the comment generation process is carried out. The processing flow from step S1011 to step S1016 is generally similar to that from step S506 to step S511 in the first embodiment.

[0196] However, in generating the command in step S1012, the information used and the purpose of the command are different from those in embodiment 1. Specifically, the generation unit 13 in this embodiment uses, as the findings, the above-mentioned evaluation information (evaluation comments) and the proposed revision received by the correction unit 16 from the generation device 3 in step S1010 or the content of revision based thereon. Then, based on this information, the generation unit 13 generates a command to generate a proposed comment to convey to the reviewer the purpose, intention, or reason of the findings.

[0197] Then, in steps S1013 to S1016, the generation of proposed comments and the output of information are similarly carried out, and in step S1016, a screen including the proposed comments is displayed to the user.

[0198] As described above, the comment generation system according to the second embodiment can make corrections in response to the findings obtained as a result of the evaluation process, and can also generate a comment draft indicating the purpose, intent, or reason for the corrections. This can significantly reduce the burden of creating comments for the other party, for example, when correcting a draft received from the other party. [Explanation of symbols]

[0199] 0: Comment generation system 1: Comment generator 11: Acquisition part 12: Specific part 13: Generation part 14: Output section 15: Evaluation section 16: Correction section 2: User terminal 3:Generation device 90: Terminal equipment 901: Control unit 902: Storage section 903: Communications Department 904: Input section 905: Output section 10: Information processing device 101: Control unit 102: Storage section 103: Communications Department W11: Evaluation screen W111: Correction input section W112: Correction button W113: Comment indicator W6:Identification display screen W61: Display of findings W62:Select button W63: Comment creation button W8: Comment proposal display screen W81: Comment proposal display section

Claims

1. A program for assisting in the preparation of documents relating to agreements between multiple parties, comprising: causing a computer to function as an acquisition unit and a generation unit; the acquisition unit acquires findings as a result of an evaluation process on the draft document related to the agreement; The generation unit outputs to a generation device an instruction to generate a draft comment regarding the indicated issue based on the indicated issue and a reference item corresponding to the indicated issue, the reference item being identified from a plurality of reference items that can be referred to for creating a comment. Comment generator.

2. The comment generating program according to claim 1 , wherein the reference items include a reference sentence or a reference policy that is a basis for creating a comment.

3. The comment generating program according to claim 2 , wherein the reference includes a reference statement or a reference policy that rejects changes to the document.

4. The comment generating program according to claim 2 , wherein the reference sentence includes a comment example showing an example of a reply comment.

5. The computer is further made to function as a specific part, The document includes a plurality of items; The reference matter is registered in advance in association with the item; the identifying unit identifies the item corresponding to the indicated matter and identifies the reference item corresponding to the item; The comment generation program according to claim 1 , wherein the generation unit outputs to a generation device an instruction to generate a draft comment regarding the pointed out matter based on the pointed out matter and the reference matter identified by the identification unit.

6. 6. The comment generating program according to claim 5, wherein the item includes a group of clauses that define a common set of provisions from a legal perspective.

7. One or more reference items are registered for each of the items, The comment generating program according to claim 5 , wherein the identifying unit identifies one of the reference items associated with the item corresponding to the pointed out matter based on background information about the agreement.

8. The comment generation program of claim 1, wherein the instructions include instructions to cause the generation device to identify a reference item corresponding to the pointed-out item from among a plurality of reference items and generate the comment proposal using the reference item.

9. The computer is further made to function as a specific part, a plurality of said references are registered in a database; The references include standard references and user references; the identifying unit refers to a setting indicating whether the standard reference items or the user reference items are to be used in generating the comment draft, and identifies the reference items according to the setting; The comment generation program according to claim 1 , wherein the generation unit outputs to a generation device an instruction to generate a draft comment regarding the pointed out matter based on the pointed out matter and the reference matter identified by the identification unit.

10. 10. The comment generating program according to claim 9, wherein the user reference items include past document comments added in past documents relating to past agreements.

11. The document includes a plurality of items; The past document comment is registered in the database in association with the item in the past document, The comment generation program of claim 10, wherein the identification unit, when the comment proposal is set to use the user reference item, identifies the item corresponding to the pointed-out matter and identifies the past document comment corresponding to the item as the reference item.

12. The pointed out matter is associated with a pointed out portion in the draft, The comment generation program according to claim 1 , wherein the generation unit outputs to the generation device an instruction to generate the comment proposal based on the content of the pointed-out section in the draft in addition to the pointed-out items and reference items.

13. The comment generating program according to claim 1 , wherein the generating unit outputs to the generating device an instruction to generate the draft comment based on the indicated matters and reference matters as well as background information about the agreement.

14. the acquisition unit accepts a selection of a strength of refusal when the user rejects the change indicated by the indicated item; The comment generating program according to claim 1 , wherein the generating unit outputs to the generating device an instruction to generate the proposed comment based on the indicated matters and reference matters as well as the strength of the rejection.

15. the acquisition unit receives, from a user, designation of a target of the pointed out items for which the proposed comment is to be generated; The comment generating program according to claim 1 , wherein the generating unit transmits the instruction to the generating device to generate the draft comment for the selected indication.

16. The draft is associated with one or more of the indicated matters; the acquisition unit receives from the user, for each of the specified items, a selection of whether or not to accept the change indicated by the specified item; The comment generating program according to claim 1 , wherein the generating unit transmits the instruction to the generating device to generate the draft comment for the indicated item for which the user has selected to reject the change.

17. A plurality of the pointed out matters are associated with the draft, The comment generation program according to claim 1 , wherein the generation unit transmits the instruction to the generation device to generate the comment drafts collectively for the plurality of pointed out items.

18. The comment generating program according to claim 1 , wherein the acquisition unit acquires, as the pointed out matter, a comment written in a body of the draft or a file representing the draft.

19. The computer is further made to function as a specific part, The document includes a plurality of items; the identifying unit identifies the item corresponding to the indicated matter, and identifies an additional item corresponding to the item from additional items registered in advance in association with the item; The comment generation program of claim 1, wherein the generation unit sends to the generation device an instruction to generate the comment proposal based on the additional items identified by the identification unit in addition to the pointed out items and the reference items.

20. 1. An apparatus for assisting in the creation of documents relating to agreements between multiple parties, comprising: An acquisition unit and a generation unit, the acquisition unit acquires findings as a result of an evaluation process on the draft document related to the agreement; The generation unit outputs to a generation device an instruction to generate a draft comment regarding the indicated issue based on the indicated issue and a reference item corresponding to the indicated issue, the reference item being identified from a plurality of reference items that can be referred to for creating a comment. Comment generator.

21. 1. A computer-implemented method for assisting in the preparation of documents relating to agreements between multiple parties, comprising: Obtaining findings as a result of an evaluation process on the draft document related to the agreement; outputting to a generating device instructions to generate a draft comment on the finding based on the finding and a reference item corresponding to the finding, the reference item being identified from a plurality of reference items that can be referenced to create a comment; Comment generation method.

Images