Generation support system, generation support method, and generation support program

Abstract

To clarify the presence / absence of a reply sentence to an item that should be confirmed by a postmortem.SOLUTION: A generation support system including a processor for executing a program and a storage device for storing the program can access a language model for interpreting a natural sentence and stochastically predicting a replay sentence to the natural sentence when the natural sentence being text data in the natural language is inputted. The processor outputs a generation source ticket in the natural language about an incident, a confirmation item group to be confirmed about the incident, and a first prompt being a natural sentence for requesting to generate a reply sentence about each confirmation item of the confirmation item group with reference to the generation source ticket to the language model. The language model acquires a postmortem including a first attempt result of generating the reply sentence of each confirmation item in the first prompt and outputs the postmortem as a result of outputting the generation source ticket, the confirmation item group, and the first prompt.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a generation support system, a generation support method, and a generation support program for supporting sentence generation. [Background technology]

[0002] A postmortem is a document written to record an incident, its impact, the actions taken to mitigate or eliminate the impact, the root causes of the incident, and any follow-up actions taken to prevent the incident from recurring.

[0003] Patent Document 1 discloses a technique for aggregating incident data for correlated incidents. In Patent Document 1, an incident service identifies an incident within an IT environment and determines correlations between the incident and other incidents within the IT environment. Once correlations are complete, the incident service aggregates the incident data for the incident with the incident data for the other incidents and generates a summary using the aggregated incident data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] U.S. Patent No. 1,165,863 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology in Patent Document 1 automatically detects correlations, which are relationships between incidents, and aggregates the detected correlations to create summaries. The method for detecting relationships between incidents is to determine the degree of match between keywords in items that make up the incident data (IP addresses, identification of device failures, identification of network vulnerabilities, identification of service interruptions, etc.).

[0006] However, in Patent Document 1, the information required for generating a postmortem (incident response process, time required for response, investigation information) is insufficient. In addition, in Patent Document 1, there is a possibility that an incorrect postmortem may be generated by correlating with an unrelated incident.

[0007] The present invention aims to clarify whether or not there is a response to an item to be confirmed in a postmortem. [Means for solving the problem]

[0008] A generation support system that is one aspect of the invention disclosed in this application is a generation support system having a processor that executes a program and a storage device that stores the program, and when a natural sentence that is text data in a natural language is input, it is capable of accessing a language model that interprets the natural sentence and probabilistically predicts a response sentence to the natural sentence, and the processor is characterized by executing the following: a first request process that outputs to the language model a source ticket in natural language regarding the incident, a group of check items to be checked for the incident, and a first prompt that is the natural sentence that references the source ticket and requests the generation of a response sentence for each check item in the group of check items; a first acquisition process that, as a result of outputting the source ticket, the group of check items, and the first prompt by the first request process, causes the language model to acquire a postmortem that includes the results of a first attempt to generate a response sentence for each of the check items in the first prompt; and a first output process that outputs the postmortem acquired by the first acquisition process. [Effects of the Invention]

[0009] According to a representative embodiment of the present invention, it is possible to clarify whether or not there is a response to an item to be confirmed in a postmortem. Problems, configurations, and effects other than those described above will become clear from the following description of the embodiment. [Brief explanation of the drawings]

[0010] [Figure 1]FIG. 1 is an explanatory diagram showing an example of generating a postmortem. [Figure 2] FIG. 2 is an explanatory diagram illustrating an example of the system configuration of the postmortem generation system. [Figure 3] FIG. 3 is a block diagram illustrating an example of the hardware configuration of a computer. [Figure 4] FIG. 4 is an explanatory diagram showing an example of an operation program executed by the ticket management device. [Figure 5] FIG. 5 is an explanatory diagram showing an example of an agent. [Figure 6] FIG. 6 is an explanatory diagram illustrating an example of the ticket DB. [Figure 7] FIG. 7 is an explanatory diagram showing an example of a check item group. [Figure 8] FIG. 8 is an explanatory diagram illustrating an example of the data structure of a postmortem. [Figure 9] FIG. 9 is an explanatory diagram illustrating an example of function definition information. [Figure 10] FIG. 10 is an explanatory diagram showing an example of related ticket search conditions. [Figure 11] FIG. 11 is an explanatory diagram showing an example of the related ticket search history. [Figure 12] FIG. 12 is a sequence diagram illustrating an example of a prompt analysis sequence. [Figure 13] FIG. 13 is a sequence diagram showing an example of a work memo summary generation sequence. [Figure 14] FIG. 14 is an explanatory diagram showing an example of the work memo summary text. [Figure 15] FIG. 15 is a sequence diagram illustrating an example of a ticket summary generation sequence. [Figure 16] FIG. 16 is an explanatory diagram showing an example of the ticket summary text. [Figure 17] FIG. 17 is a sequence diagram showing an example of a postmortem generation sequence. [Figure 18] FIG. 18 is an explanatory diagram showing an example of a postmortem. [Figure 19]FIG. 19 is an explanatory diagram showing an example of a generation support screen. DETAILED DESCRIPTION OF THE INVENTION

[0011] <Figure 1 Example of postmortem generation> 1 is an explanatory diagram showing an example of postmortem generation. The postmortem generation system 100 has a ticket DB 110, an agent 120, and a postmortem DB 130. The ticket DB 110 is a database that stores tickets 101 related to incidents. The tickets 101 are natural language text data (natural sentences) that summarize the details of the incident. Specifically, for example, the ticket 101 includes a type 111, a work memo 112, a work memo summary 113, and a ticket summary 114.

[0012] The type 111 indicates the type of ticket 101 (for example, incident, inquiry, task, or problem). In this example, the type 111 is an incident. The work memo 112 is a dialogue log written in natural language regarding the work performed between workers on the incident. The work note summary 113 is data summarizing the work note 112, and includes a work note summary text written in natural language that summarizes the work note 112, and a work note summary vector that vectorizes the work note 112. The ticket summary 114 is data summarizing the ticket 101, and includes a ticket summary text written in natural language that summarizes the ticket 101, and a ticket summary vector that vectorizes the ticket 101.

[0013] Agent 120 is software that causes a processor in postmortem generation system 100 to manage ticket DB 110 and postmortem DB 130 and to send and receive data to and from language model 140 in postmortem generation system 100 .

[0014] The language model 140 is a probabilistic model trained by natural language processing using a dataset, which is a collection of text data in natural language, and generates sentences according to prompt instructions. Natural language processing is a process in which a computer understands sentences written in natural language and executes them according to a purpose. Specific examples of natural language processing include morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and intent analysis.

[0015] Morphological analysis is an analysis that assigns parts of speech by breaking down natural language into morphemes, the smallest units that make up the language. Syntactic analysis is an analysis that clarifies the structure and meaning of a sentence by analyzing the grammatical structure of natural language. Semantic analysis is an analysis that understands the meaning of words and sentences by analyzing the meaning of natural language, and makes logical judgments and inferences. Contextual analysis is an analysis that understands natural language while taking into account the context before and after a sentence. Intention analysis is an analysis that extracts the intention of a speaker or writer from dialogues or sentences using natural language.

[0016] The language model 140 is a type of probabilistic model used in natural language processing, and is a model for probabilistically predicting how likely a given word or sentence is to occur in natural language. Specifically, in the field of natural language processing, it is a mathematical model for learning language patterns and grammatical rules, etc., and for generating and understanding natural language. For example, the language model 140 calculates the occurrence probability of a given word string or sentence, or compares the occurrence probabilities of multiple word strings or sentences, thereby automatically generating the most likely word or sentence based on the context when predicting the next word or sentence.

[0017] In this way, the language model 140 combines morphological analysis, syntactic analysis, semantic analysis, contextual analysis, and intention analysis to receive a query called a prompt and output a response to the query. The language model 140 includes large-scale language models such as ChatGPT.

[0018] In FIG. 1, language model 140 is implemented outside postmortem generation system 100, but it may also be implemented in postmortem generation system 100.

[0019] For the sake of convenience, the following description will focus on the agent 120 as the subject of the process executed by the processor. For the sake of convenience, the following description will also focus on the process executed by the processor for programs other than the agent 120 as the subject of the process.

[0020] (Step S101) When the agent 120 receives a user request to generate a postmortem 150-i (where i is an integer satisfying 1≦i≦n, and the initial value of i=0. n is an integer satisfying i≦n) for a certain ticket 101 (hereinafter, the generating ticket 101A), the agent 120 retrieves the generating ticket 101A from the ticket DB 110. The generating ticket 101A includes its work memo summary 113. The instruction information 121 is a prompt for the agent 120, and is, for example, text data in natural language that includes a check item group 740, which is a collection of check items required for the postmortem 150-i, and inquires about the check item group 740. The check item group 740 includes, for example, the incident name, impact, root cause, and cause of occurrence (described below in FIG. 7).

[0021] (Step S102) The agent 120 transmits a prompt 122 having the source ticket 101A and the instruction information 121 to the language model 140. The prompt 122 in step S102 is a query consisting of text data in a natural language that requests the generation of a sentence for each item in the confirmation item group 740 in the instruction information 121, with reference to the source ticket 101A.

[0022] (Step S103) The language model 140 attempts to generate an answer sentence for each check item in the check item group 740 in accordance with the instructions of the prompt 122 in step S102. An answer sentence is text data in natural language that is a written answer to a check item. For example, if the result of the attempt to generate an answer sentence is a sentence that refuses or denies the answer, such as "unclear," "I don't understand," or "I can't answer," or if there is no answer, the attempt does not correspond to an answer sentence. A check item that does not correspond to an answer sentence, i.e., a check item that is missing an answer sentence, is called a missing item.

[0023] (Step S104) In step S103, language model 140 transmits to agent 120 postmortem 150-0 including the results of the attempt to generate a response sentence for the confirmation item.

[0024] (Step S105-1) Agent 120 determines whether postmortem 150-0 is missing any items, and if not, proceeds to step S110.

[0025] (Step S106-1) If there are any missing items in step S105, the agent 120 searches the ticket 101 related to the generating ticket 101A (hereinafter referred to as related ticket 101B) from the ticket DB 110. Here, the ticket summary vector of the ticket summary 114 of the related ticket 101B is set to a vector whose distance from the ticket summary vector of the ticket summary 114 of the generating ticket 101A is within a predetermined distance, for example.

[0026] (Step S107-1) The agent 120 sends a prompt 123 having the related ticket 101B to the language model 140. The prompt 123 in step S107 is a query consisting of text data in a natural language that refers to the related ticket 101B and requests the generation of a sentence for a missing item.

[0027] (Step S108-1) The language model 140 attempts to generate answer sentences for the missing items as requested by the prompt 123 in step S107.

[0028] (Step S109-1) The language model 140 transmits to the agent 120 a postmortem 150-1 (or just the trial results) to which the trial results generated for the missing items in step S108 have been added.

[0029] Thereafter, the agent 120 and the language model 140 repeatedly execute steps S105 to S109 (S105-1 to S109-1, S105-2 to S109-2, . . . ) until there are no more missing items.

[0030] (Step S109-n) Steps S105 to S109 are repeated n times, and if it is determined that there are no missing items, agent 120 saves postmortem 150-n in postmortem DB 130. Steps S105 to S109 may be repeated until there are no missing items, or may end when a predetermined number of repetitions have been reached. When postmortems 150-0 to 150-n are not to be distinguished, they are referred to as postmortem 150.

[0031] <Figure 2 System configuration example> 2 is an explanatory diagram showing an example system configuration of postmortem generation system 100. Postmortem generation system 100 has a ticket management device 201, a generation support device 202, and a communication terminal 203. Ticket management device 201, generation support device 202, and communication terminal 203 are communicatively connected via a network 210 such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0032] The ticket management device 201 is a computer that manages the ticket DB 110 and the postmortem DB 130, as shown in FIG.

[0033] The generation support device 202 is a computer on which the agent 120 shown in Fig. 1 is implemented. The generation support device 202 is communicably connected to an external device 204 via a network 210. The generation support device 202 also has instruction information 121, function definition information 220, related ticket search conditions 221, and related ticket search history 222.

[0034] Function definition information 220 is information that defines functions to be executed by postmortem generation system 100. Related ticket search conditions 221 are conditions for searching related tickets 101B. Related ticket search history 222 is a history of searches for related tickets 101B.

[0035] In FIG. 2, the ticket management device 201 and the generation support device 202 are separate computers, but they may be one computer having the functions of the ticket management device 201 and the generation support device 202.

[0036] The external device 204 has a language model 140 , and upon receiving a query from the generation support device 202 , generates a response sentence using the language model 140 and returns it to the generation support device 202 .

[0037] The communication terminal 203 is a computer that allows the user 230 to input operations and displays output data from the ticket management device 201 and the generation support device 202. For example, the communication terminal 203 creates a ticket 101 in response to an operation input by the user 230 and transmits it to the ticket management device 201, and the ticket management device 201 registers the received ticket 101 in the ticket DB 110.

[0038] <Figure 3 Example of computer hardware configuration> Next, an example of the hardware configuration of the computers (ticket management device 201, generation support device 202, external device 204) will be described.

[0039] FIG. 3 is a block diagram showing an example of the hardware configuration of a computer. The computer 300 includes a processor 301, a storage device 302, an input device 303, an output device 304, and a communication interface (communication IF) 305. The processor 301, the storage device 302, the input device 303, the output device 304, and the communication IF 305 are connected via a bus 306. The processor 301 controls the computer 300. The storage device 302 serves as a working area for the processor 301. The storage device 302 is a non-transitory or temporary recording medium that stores various programs and data. Examples of the storage device 302 include a read-only memory (ROM), a random access memory (RAM), a hard disk drive (HDD), and a flash memory. The input device 303 inputs data. Examples of the input device 303 include a keyboard, a mouse, a touch panel, a numeric keypad, a scanner, a microphone, and a sensor. The output device 304 outputs data. The output device 304 may be, for example, a display, a printer, or a speaker. The communication IF 305 connects to a network and transmits and receives data.

[0040] <Figure 4 Operation program> 4 is an explanatory diagram showing an example of an operation program executed by the ticket management device 201. The operation program 400 is stored in the storage device 302 of the ticket management device 201. The operation program 400 is a program executed by the ticket management device 201 in response to an instruction from the generation support device 202. The operation program 400 has, as functions, a ticket search process 401, a ticket update process 402, a work memo acquisition process 403, a ticket acquisition process 404, and a postmortem update process 405.

[0041] The ticket search process 401 is a function that causes the ticket management device 201 to execute a process of searching for the generating ticket 101A from the ticket DB 110. The ticket update process 402 is a function that causes the ticket management device 201 to execute a process of registering and updating the ticket 101 in the ticket DB 110. The work memo acquisition process 403 is a function that causes the ticket management device 201 to execute a process of acquiring the work memo 112 from the ticket DB 110. The ticket acquisition process 404 is a function that causes the ticket management device 201 to execute a process of acquiring the ticket 101 from the ticket DB 110. The postmortem update process 405 is a function that causes the ticket management device 201 to execute a process of registering and updating the postmortem 150 in the postmortem DB 130.

[0042] <Figure 5 Agent 120> 5 is an explanatory diagram showing an example of the agent 120. The storage device 302 of the generation support device 202 stores, as the agent 120, a Chatbot program 500, a prompt analysis program 501, a work memo summarization program 502, a ticket summarization program 503, and a postmortem generation program 504.

[0043] The Chatbot program 500 is a program that the generation assistance device 202 uses to automatically conduct a conversation between the user 230 and the agent 120. The Chatbot program 500 may be a rule-based program or a machine learning program that uses the language model 140. Specifically, for example, the Chatbot program 500 outputs a conversational sentence input by the user 230 via the communication terminal 203 to the agent 120, generates a response sentence to the conversational sentence, and sends it to the communication terminal 203.

[0044] The prompt analysis program 501 is a program that causes the generation support device 202 to cause the language model 140 to analyze a prompt input via the communication terminal 203. Specifically, for example, the prompt analysis program 501 transmits a prompt and prompt analysis information to the external device 204. The prompt analysis information includes the instruction information 121 and the function definition information 220. The prompt analysis program 501 receives the prompt analysis result from the external device 204.

[0045] The work memo summarization program 502 is a program that causes the generation support device 202 to generate the work memo summary 113 in the language model 140. Specifically, for example, the work memo summarization program 502 transmits the work memo 112 to the external device 204. The external device 204 generates the work memo summary 113 from the work memo 112 using the language model 140, and transmits the work memo summary 113 to the generation support device 202.

[0046] The ticket summarization program 503 is a program that causes the generation support device 202 to generate the ticket summary 114 in the language model 140. Specifically, for example, the ticket summarization program 503 transmits the ticket 101 to the external device 204. The external device 204 generates the ticket summary 114 from the ticket 101 using the language model 140 and transmits the ticket summary 114 to the generation support device 202.

[0047] The postmortem generation program 504 is a program that the generation support device 202 uses to cause the language model 140 to generate the postmortem 150. Specifically, for example, the postmortem generation program 504 transmits the source ticket 101A and the instruction information 121 to the external device 204. The external device 204 generates the postmortem 150 from the source ticket 101A and the instruction information 121 in the language model 140, and transmits the postmortem 150 to the generation support device 202.

[0048] <Figure 6 Ticket DB110> 6 is an explanatory diagram showing an example of the ticket DB 110. The ticket DB 110 has a ticket detail information table 601, a work memo table 602, a timeline table 603, and a summary table 604. These tables 601 to 604 manage the ticket 101.

[0049] [Ticket detailed information table 601] The ticket detail information table 601 has the following data structures: ticket ID 600, type 111, status 610, ticket creation date and time 611, ticket creator 612, ticket update date and time 613, ticket updater 614, severity 615, impact 616, title 617, description 618, and conclusion 619.

[0050] The ticket ID 600 is identification information that uniquely identifies the ticket 101. The status 610 indicates the current state (in progress, completed, etc.) of the ticket 101. The ticket creation date and time 611 is the date and time when the ticket 101 was created by the communication terminal 203.

[0051] The ticket creator 612 is identification information that uniquely identifies the user who created the ticket 101. The ticket update date and time 613 is the date and time when the ticket 101 was updated on the communication terminal 203 and re-registered in the ticket DB 110. The ticket updater 614 is identification information that uniquely identifies the user who updated the ticket 101.

[0052] Severity 615 is natural language text data indicating the severity of type 111 (in this example, incident). Impact 616 is natural language text data indicating what has been affected and to what extent. Title 617 is the name of type 111 (in this example, incident). Description 618 is natural language text data that describes type 111 (in this example, incident) in detail. Conclusion 619 is natural language text data that indicates the final conclusion reached after discussing type 111 (in this example, incident).

[0053] [Work Memo Table 602] The work memo table 602 has a data structure including a ticket ID 600, a work ID 620, a work memo creator 621, a work memo creation date and time 622, and a work memo content 623. The work ID 620 is identification information that uniquely identifies the work for type 111 (in this example, an incident). The work memo creator 621 is identification information that uniquely identifies the creator of the work memo 112. The work memo creation date and time 622 is the date and time when the ticket 101 was created on the communication terminal 203. The work memo content 623 is a natural language dialogue log regarding the work performed between workers for type 111 (in this example, an incident).

[0054] The work memo summary text 624 is natural language text data that summarizes the work memo content 623. The work memo summary vector 625 is vector data obtained by vectorizing the work memo summary text 624.

[0055] [Timeline Table 603] The timeline table 603 has a data structure including a ticket ID 600, a task ID 620, a task date and time 630, a respondent 631, task content 632, and a task type 633. The task date and time 630 is the date and time when the task identified by the task ID 620 was performed. The respondent 631 is the user 230 who performed the task identified by the task ID 620. The task content 632 is text data in a natural language that describes in detail the task identified by the task ID 620. The task type 633 is the type of task identified by the task ID 620 (for example, investigation).

[0056] [Ticket Summary Table 604] The ticket summary table 604 has, as data structures, a ticket ID 600, a title 617, and a ticket summary 114. The ticket summary 114 has a ticket summary text 641 and a ticket summary vector 642.

[0057] The ticket summary text 461 is natural language text data summarizing the ticket 101 (status 610, ticket creation date and time 611, ticket creator 612, ticket update date and time 613, ticket updater 614, severity 615, impact 616, title 617, description 618, conclusion 619, and work note summary text 624). The ticket summary vector 642 is vector data obtained by vectorizing the ticket summary text 641.

[0058] <Figure 7 Instruction Information 121> 7 is an explanatory diagram showing an example of the instruction information 121. As described above, the instruction information 121 is text data in a natural language that can be interpreted by the language model 140. The language model 140 generates a response sentence describing a function to be called in accordance with the instruction information 121. The instruction information 121 includes a first instruction statement 701 to a fifth instruction statement 705 and a group of confirmation items 740.

[0059] The first directive 701 is a statement instructing the language model 140 to provide an answer. The second directive 702 is a statement instructing the language model 140 to generate a work memo summary 113. The third directive 703 is a statement instructing the language model 140 to generate a ticket summary 114. The fourth directive 704 is a statement instructing the language model 140 to generate a postmortem 150, i.e., an answer statement for the check item group 740. The supplemental statement 706 is a statement instructing the user to write "unknown" if the check item in the check item group 740 is unclear. The fifth directive 705 is a statement instructing the user not to call a function when the inquiry from the user 230 is not one of the second directive 702 to the fourth directive. For convenience, the first instruction statement 701 to the fifth instruction statement 705 (including the supplementary statement 706) obtained by excluding the check item group 740 from the instruction information 121 will be referred to as an instruction statement 710.

[0060] The confirmation item group 740 includes an occurrence date and time 741 , a responder 631 , a title 617 , a ticket creator 612 , a status 610 , a summary 742 , an impact 743 , a root cause 744 , and a cause of occurrence 745 .

[0061] The occurrence date and time 741 is the date and time when the incident occurred. The summary 742 is the general gist of the postmortem 150. The impact 743 shows what has affected others to what extent. The root cause 744 is the fundamental event that led to the incident occurring. The causative factor 745 is the main event that caused the incident to occur.

[0062] The language model 140 will generate sentences by referring to the source ticket 101A for the occurrence date and time 741, the person who responded 631, the title 617, the ticket creator 612, the status 610, the summary 742, the impact 743, the root cause 744, and the cause of occurrence 745. In accordance with the request 707 to confirm the existence of the work memo summary 113, the language model 140 will confirm whether the work memo summary 113 exists in the source ticket 101A.

[0063] The content of the instruction information 121 is valid in the language model 140 during the session from transmission of the instruction information 121 to the language model 140 (step S102) to reception of the postmortem 150-n from the language model 140 (step S109-n).

[0064] Therefore, after the instruction information 121 is sent to the language model 140, for example, a prompt such as "Summarize the work notes for ticket ID: 5" is sent to the language model 140. The language model 140 receives the prompt, references the ticket 101A that generated ticket ID: 5, and executes the process of summarizing the work notes for ticket ID: 5.

[0065] If the original ticket 101A with ticket ID: 5 does not have any work notes (in reality, the work note "filed" does exist), then since there are no work notes (except for the work note "filed"), the language model 140 will not be able to summarize the work notes for ticket ID: 5 and will return the response statement "Cannot be summarized."

[0066] <Figure 8 Postmortem 150> 8 is an explanatory diagram showing an example data structure of the postmortem 150. The postmortem 150 has, as its data structure, a title 617 (incident name), a postmortem creation date and time 743, a postmortem update date and time 744, a ticket creator 612, a status 610, a summary 742, an impact 743, a root cause 744, a contributing factor 745, and a related ticket ID 800.

[0067] The related ticket ID 800 is identification information that uniquely identifies the related ticket 101B. The related ticket ID 800 is associated with the postmortem 150-n when the postmortem 150-n is stored in the postmortem DB 130 (step S110).

[0068] <Figure 9 Function definition information 220> 9 is an explanatory diagram showing an example of the function definition information 220. The function definition information 220 has a function name 901 and a function definition 902. The function name 901 is the name of a function that the generation support device 202 (agent 120) calls to the ticket management device 201. The function definition 902 is text data in a natural language that defines the function of the function name 901. The language model 140 interprets the function definition 902 to specify the work memo acquisition process 403 as the function to be called.

[0069] Furthermore, the function definition 902 of the ticket summarization program 503 is a program that acquires the ticket 101 from the ticket management device 201 via an API and generates the ticket summary 114 using the acquired ticket 101. Therefore, for example, the language model 140 interprets the function definition 902 to specify the function name 901 to be called.

[0070] <Figure 10 Related ticket search conditions 221> FIG. 10 is an explanatory diagram showing an example of related ticket search conditions 221. The related ticket search conditions 221 are conditions for searching for related tickets 101B in the ticket search process 401. The related ticket search conditions 221 have a search item 1001 and a search query 1002. The search item 1001 is a check item among the check item group 740 for which the search query 1002 is set. The search query 1002 is a condition for specifying a search target from the ticket DB 110. In the search query 1002, "start" indicates the opening of the ticket 101, and "complete" indicates the closing of the ticket 101.

[0071] <Figure 11 Related ticket search history 222> 11 is an explanatory diagram showing an example of the related ticket search history 222. The related ticket search history 222 is the past related ticket search conditions 221 used to search for the related ticket 101B in the ticket search process 401 at the search date and time 1100.

[0072] <Figure 12 Prompt analysis sequence> FIG. 12 is a sequence diagram illustrating an example of a prompt analysis sequence.

[0073] (Step S1200) The communication terminal 203 creates a prompt 1200 in response to an operational input from the user 230 or by setting the ticket ID 600 in a prompt template, and transmits the created prompt 1200 to the generation support device 202. The contents of the prompt 1200 are, for example, text data in a natural language that describes at least one of the instruction statements 710 shown in Fig. 7, that is, a request to summarize the work memo 112 (first instruction statement 702), a request to summarize the ticket 101 (second instruction statement 703), and a request to generate the postmortem 150 (third instruction statement 704).

[0074] (Step S1201) The generation assistance device 202 receives a prompt 1200 from the communication terminal 203 using the Chatbot program 500 and passes it to the prompt analysis program 501.

[0075] (Step S1202) The generation support device 202 uses the prompt analysis program 501 to send a prompt 1200 and prompt analysis information 1201 to the external device 204. The prompt analysis information 1201 includes the instruction information 121 and the function definition information 220.

[0076] (Step S1203) The external device 204 inputs a prompt 1200 and prompt analysis information 1201 to the language model 140. The language model 140 interprets the prompt 1200 and refers to the prompt analysis information 1201 to identify the function and arguments that the generation assistance device 202 should execute.

[0077] (Step S1204) The external device 204 returns the function and arguments identified in step S1203 to the prompt analysis program 501 of the generation support device 202 as function specification information.

[0078] (Step S1205) The generation support device 202 uses the prompt analysis program 501 to refer to the function specification information, identify the process requested by the prompt 1200, and execute the identified process. For example, if the only function included in the function specification information is the work memo acquisition process 403, the work memo summary program 502 instructs the ticket management device 201 to execute the work memo acquisition process 403.

[0079] If the function specification information contains only the ticket acquisition process 404, the ticket summarization program 503 identifies the ticket acquisition process 404 to the ticket management device 201. If the functions contained in the function specification information are the ticket acquisition process 404, the ticket search process 401, and the postmortem update process 405, the postmortem generation program 504 identifies the ticket acquisition process 404, the ticket search process 401, and the postmortem update process 405.

[0080] (Step S1206) The generation support device 202 transmits to the ticket management device 201 an instruction to execute the process requested by the prompt 1200 identified in step S1205.

[0081] (Step S1207) The ticket management device 201 executes processing in accordance with the execution instruction in step S1206. Specifically, for example, the ticket management device 201 executes acquisition of the work memo 112, acquisition of the ticket 101, search for the related ticket 101B, and registration or update of the postmortem 150.

[0082] <Figure 13 Work memo summary generation sequence> 13 is a sequence diagram showing an example of a work memo summary generation sequence. In FIG. 13, the processing from steps S1300 to S1307 corresponds to the processing from steps S1200 to S1207 in FIG.

[0083] (Step S1300) The communication terminal 203 creates a prompt 1300 in response to an operational input from the user 230 or by setting the ticket ID 600 in a prompt template, and transmits the created prompt 1300 to the generation support device 202. The content of the prompt 1300 is, for example, an instruction statement 710, but it is sufficient if it includes at least the first instruction statement 702, i.e., text data in a natural language that describes a request to summarize the work memo 112.

[0084] (Step S1301) The generation assistance device 202 receives a prompt 1300 from the communication terminal 203 using the Chatbot program 500 and passes it to the prompt analysis program 501.

[0085] (Step S1302) The production support device 202 transmits the prompt 1300 and the prompt analysis information 1201 to the external device 204 using the prompt analysis program 501 .

[0086] (Step S1303) The external device 204 inputs a prompt 1300 and prompt analysis information 1201 to the language model 140. The language model 140 interprets the prompt 1300 and refers to the prompt analysis information 1201 to identify the function and argument to be executed by the generation assistance device 202. In this example, the function is the work memo acquisition process 403, and the argument is the ticket ID 600 (=5).

[0087] (Step S1304) The external device 204 returns the function and arguments identified in step S1303 to the prompt analysis program 501 of the generation support device 202 as function specification information.

[0088] (Step S1305) The generation support device 202 uses the prompt analysis program 501 to refer to the function specification information and identify the processing requested by the prompt 1300. In this example, since the only function included in the function specification information is the work memo acquisition processing 403, the work memo summarization program 502 identifies the work memo acquisition processing 403 to the ticket management device 201.

[0089] (Step S1306) The generation support device 202 sends, by the work memo summarization program 502, an instruction to execute the work memo acquisition process 403 for the ticket 101 with the ticket ID 600 of "5" requested by the prompt 1300, identified in step S1305, to the ticket management device 201.

[0090] (Step S1307) In accordance with the execution instruction in step S1306, the ticket management device 201 executes the work memo acquisition process 403 for the ticket 101 whose ticket ID 600 is "5" to acquire the work memo 112 to be summarized.

[0091] (Step S1308) The ticket management device 201 returns the work memo 112 acquired in step S1307 to the generation support device 202.

[0092] (Step S1309) The generation support device 202 transmits the work memo 112 acquired from the ticket management device 201 to the external device 204 .

[0093] (Step S1310) In step S 1309 , the external device 204 inputs the work memo 112 received from the generation support device 202 into the language model 140 , and generates the work memo summary text 624 in accordance with the contents of the prompt 1300 .

[0094] (Step S1311) The external device 204 transmits the work memo summary text 624 generated in step S1310 to the generation assistance device 202.

[0095] (Step S1312) The generation support device 202 passes the work memo summary text 624 sent in step S1311 to the Chatbot program 500 using the work memo summarization program 502.

[0096] (Step S1313) The generation assistance device 202 transmits the work memo summary text 624 to the communication terminal 203 using the Chatbot program 500. As a result, the work memo summary text 624 is displayed on the communication terminal 203.

[0097] (Step S1314) The generation support device 202 transmits a prompt 1301 describing a vectorization request for the work memo summary text 624 to the external device 204. The prompt 1301 is a prompt in which the work memo summary text 624 is embedded as a vectorization request target in a prompt template in which the vectorization request is described in text data in a natural language.

[0098] (Step S1315) The external device 204 receives the vectorization request for the work note summary text 624 in step S1314 and vectorizes the work note summary text 624. Specifically, for example, the language model 140 performs embedding, which is called embedding, and converts the work note summary text 624 into a work note summary vector 625 as a numerical vector representation.

[0099] (Step S1316) The external device 204 transmits the work memo summary vector 625 generated in step S1315 to the generation support device 202.

[0100] (Step S1317) The generation support device 202 transmits the work memo summary text 624 and the work memo summary vector 625 to the ticket management device 201 as the work memo summary 113 .

[0101] (Step S1318) The ticket management device 201 stores the work memo summary 113 transmitted in step S1317 in the ticket DB 201 in association with the work memo 112 acquired in step S1307.

[0102] <Figure 14 Work memo summary text 624> Fig. 14 is an explanatory diagram showing an example of the work memo summary text 624. Fig. 14 shows, as an example, the work memo summary text 624 for the ticket ID 600 "5".

[0103] <Figure 15 Ticket summary generation sequence> 15 is a sequence diagram showing an example of a ticket summary generation sequence. In FIG. 15, the processing from steps S1500 to S1507 corresponds to the processing from steps S1200 to S1207 in FIG.

[0104] (Step S1500) The communication terminal 203 creates a prompt 1500 in response to an operational input from the user 230 or by setting the ticket ID 600 in a prompt template, and transmits the created prompt 1500 to the generation support device 202. The content of the prompt 1500 is, for example, an instruction statement 710, but it is sufficient if it includes at least the second instruction statement 703, i.e., text data in a natural language describing a summary request for the ticket 101.

[0105] (Step S1501) The generation assistance device 202 receives a prompt 1500 from the communication terminal 203 using the Chatbot program 500 and passes it to the prompt analysis program 501.

[0106] (Step S1502) The production support device 202 transmits the prompt 1500 and the prompt analysis information 1201 to the external device 204 using the prompt analysis program 501 .

[0107] (Step S1503) The external device 204 inputs a prompt 1500 and prompt analysis information 1201 to the language model 140. The language model 140 interprets the prompt 1500 and refers to the prompt analysis information 1201 to identify the function and argument to be executed by the generation assistance device 202. In this example, the function is the ticket acquisition process 404, and the argument is the ticket ID 600 (=5).

[0108] (Step S1504) The external device 204 returns the function and arguments identified in step S1503 to the prompt analysis program 501 of the generation support device 202 as function specification information.

[0109] (Step S1505) The generation support device 202 uses the prompt analysis program 501 to refer to the function specification information and identify the processing requested by the prompt 1500. In this example, since the function specification information includes only the ticket acquisition processing 404, the ticket summary program 503 identifies the ticket acquisition processing 404 to the ticket management device 201.

[0110] (Step S1506) The generation support device 202 sends, by the ticket summarization program 503, to the ticket management device 201 an instruction to execute the ticket acquisition process 404 for the ticket 101 with the ticket ID 600 of "5" specified in step S1505 and requested by the prompt 1500.

[0111] (Step S1507) In accordance with the execution instruction in step S1506, the ticket management device 201 executes the ticket acquisition process 404 for the ticket 101 whose ticket ID 600 is "5" to acquire the ticket 101 to be summarized.

[0112] (Step S1508) The ticket management device 201 returns the ticket 101 acquired in step S1507 to the generation support device 202.

[0113] (Step S1509) The generation support device 202 transmits the ticket 101 acquired from the ticket management device 201 to the external device 204 .

[0114] (Step S1510) In step S 1509 , the external device 204 inputs the ticket 101 received from the generation support device 202 into the language model 140 , and generates a ticket summary text 641 in accordance with the contents of the prompt 1500 .

[0115] (Step S1511) The external device 204 transmits the ticket summary text 641 generated in step S1510 to the generation support device 202.

[0116] (Step S1512) The generation support device 202 passes the ticket summary text 641 sent in step S1511 to the Chatbot program 500 using the ticket summarization program 503.

[0117] (Step S1513) The generation assistance device 202 uses the Chatbot program 500 to send the ticket summary text 641 to the communication terminal 203. As a result, the ticket summary text 641 is displayed on the communication terminal 203.

[0118] (Step S1514) The generation support device 202 transmits a prompt 1501 describing a vectorization request for the ticket summary text 641 to the external device 204. The prompt 1501 is a prompt in which the ticket summary text 641 is embedded as the vectorization request target in a prompt template in which the vectorization request is described in text data in a natural language.

[0119] (Step S1515) The external device 204 receives the vectorization request for the ticket summary text 641 in step S1514 and vectorizes the ticket summary text 641. Specifically, for example, the language model 140 performs embedding, which is called embedding, and converts the ticket summary text 641 into a ticket summary vector 642 as a numerical vector representation.

[0120] (Step S1516) The external device 204 transmits the ticket digest vector 642 generated in step S1515 to the generation support device 202.

[0121] (Step S1517) The generation support device 202 transmits the ticket summary text 641 and the ticket summary vector 642 to the ticket management device 201 as the ticket summary 114 .

[0122] (Step S1518) The ticket management device 201 stores the ticket summary 114 transmitted in step S1517 in the ticket DB 201 in association with the ticket 101 acquired in step S1507.

[0123] If the work memo summary text 624 is not included in the ticket 101 from the ticket management device 201, the generation support device 202 acquires the work memo summary text 624 according to the work memo summary generation sequence shown in Fig. 13 and transmits the ticket 101 including the work memo summary text 624 to the external device 204 (step S159). This allows the language model 140 to generate a higher quality ticket summary text 641 than when the work memo summary text 624 is not included in the ticket 101.

[0124] <Figure 16 Ticket summary text 641> Fig. 16 is an explanatory diagram showing an example of a ticket summary text 641. Fig. 16 shows, as an example, the ticket summary text 641 for which the ticket ID 600 is "5".

[0125] <Figure 17 Postmortem generation sequence> 17 is a sequence diagram showing an example of a postmortem generation sequence. In FIG. 17, the processing from steps S1500 to S1507 corresponds to the processing from steps S1200 to S1207 in FIG.

[0126] (Step S1700) The communication terminal 203 creates a prompt 1700 in response to an operational input from the user 230 or by setting the ticket ID 600 in a prompt template, and transmits the created prompt 1700 to the generation support device 202. The description content of the prompt 1700 is, for example, an instruction statement 710, but is at least the third instruction statement 704, i.e., natural language text data describing a postmortem generation request for the ticket 101.

[0127] (Step S1701) The generation assistance device 202 receives a prompt 1700 from the communication terminal 203 using the Chatbot program 500 and passes it to the prompt analysis program 501.

[0128] (Step S1702) The production support device 202 transmits the prompt 1700 and prompt analysis information 1201 to the external device 204 using the prompt analysis program 501 .

[0129] (Step S1703) The external device 204 inputs a prompt 1700 and prompt analysis information 1201 to the language model 140. The language model 140 interprets the prompt 1700 and refers to the prompt analysis information 1201 to identify the function and arguments that the generation assistance device 202 should execute.

[0130] In this example, for generating postmortem 150-0, the function is ticket acquisition process 404 and the argument is ticket ID 600 (=5). If there are missing items, the function is ticket search process 401 and the argument is search query 1002 corresponding to the missing items. For acquiring related ticket 101B, the function is ticket acquisition process 404 and the argument is ticket ID 600 of ticket 101 searched for by search query 1002. For registering or updating postmortem 150-n, the function is postmortem update process 405 and the argument is postmortem 150-n to be registered or updated.

[0131] (Step S1704) The external device 204 returns the function and arguments identified in step S1703 to the prompt analysis program 501 of the generation support device 202 as function specification information.

[0132] (Step S1705) The generation support device 202 uses the prompt analysis program 501 to refer to the function specification information and identify the processing requested by the prompt 1700. In this example, the functions included in the function specification information are the ticket acquisition processing 404, ticket search processing 401, and postmortem update processing 405, so the postmortem generation program 504 identifies the ticket acquisition processing 404, ticket search processing 401, and postmortem update processing 405.

[0133] (Step S1706) The generation support device 202 sends, by the postmortem generation program 504, to the ticket management device 201 an instruction to execute the ticket acquisition process 404 for the ticket 101 with the ticket ID 600 of "5" specified in step S1505 and requested by the prompt 1500.

[0134] (Step S1707) In accordance with the execution instruction in step S1706, the ticket management device 201 executes the ticket acquisition process 404 for the ticket 101 whose ticket ID 600 is "5" to acquire the generating ticket 101A.

[0135] (Step S1708) The ticket management device 201 returns the source ticket 101A acquired in step S1707 to the generation support device 202.

[0136] (Step S1709) The generation support device 202 transmits the source ticket 101A obtained from the ticket management device 201 to the external device 204 using the postmortem generation program 504. If the source ticket 101A does not include a work memo summary 113, the generation support device 202 also transmits a prompt to the external device 204 requesting that the external device 204 generate a work memo summary 113 for the work memo 112 in the source ticket 101A.

[0137] (Step S1710-n (n=0)) In step S1709, the external device 204 inputs the source ticket 101A and the confirmation item group 740 received from the generation support device 202 into the language model 140, and generates the postmortem 150-0 in accordance with the contents of the prompt 1700. If a prompt requesting the generation of a work memo summary 113 has been received, the external device 204 generates the work memo summary 113 before generating the ticket summary text 641, and inputs the source ticket 101A including the generated work memo summary 113 into the language model 140 to generate the postmortem 150-0.

[0138] (Step S1711-n(n=0)) The external device 204 transmits the postmortem 150-n (n=0) generated in step S1710 to the generation assistance device 202.

[0139] (Step S1712-n(n=0)) The generation support device 202 determines whether or not there are any missing items in the postmortem 150-n (n=0) transmitted in step S1711-n (n=0) using the postmortem generation program 504. If there are no missing items (step S1712-n (n=0): No), the process proceeds to steps S1713 and S1715. If there are missing items (step S1712-n (n=0): Yes), the process proceeds to step S1717-n (n=0).

[0140] (Step S1713) The generation assistance device 202 passes the postmortem 150-n and its display instruction to the Chatbot program 500.

[0141] (Step S1714) The Chatbot program 500 transmits the postmortem 150-n to the communication terminal 203. As a result, the postmortem 150-n is displayed on the communication terminal 203.

[0142] (Step S1715) The generation support device 202 sends a save instruction for the postmortem 150-n to the ticket management device 201.

[0143] (Step S1716) The ticket management device 201 stores the postmortem 150-n in the postmortem DB 130.

[0144] (Step S1717-n(n=0)) The generation support device 202 refers to the related ticket search conditions 221 and specifies a search query 1002 for search items 1001 that match the missing items. The generation support device 202 may also send the related ticket search history 222 to the communication terminal 203 via the Chatbot program 500. In this case, the related ticket search history 222 is displayed on the communication terminal 203. The communication terminal 203 accepts the specification of the search query 1002 from the related ticket search history 222 through a user operation, and sends it to the postmortem generation program 504 via the Chatbot program 500. In this way, the search query 1002 is specified.

[0145] (Step S1718-n(n=0)) The generation support device 202 sends a search query 1002 to the ticket management device 201 using the postmortem generation program 504 .

[0146] (Step S1719-n(n=0)) The ticket management device 201 acquires, from the ticket DB 201, the related ticket 101B that matches the search query 1002. Specifically, for example, the ticket management device 201 acquires, from the ticket DB 201, as the related ticket 101B, the ticket 101 that matches the search query 1002 and has a ticket digest vector 642 that is within a predetermined distance from the ticket digest vector 642 of the generating ticket 101A.

[0147] (Step S1720-n (n=0)) The ticket management device 201 transmits the associated ticket 101B to the generation support device 202.

[0148] (Step S1721) The generation support device 202 transmits to the external device 204 the related ticket 101B and a prompt 1701 for regenerating the postmortem 150-n using the related ticket 101B.

[0149] (Step S1710-n (n=1)) In step S1721-n (n=0), the external device 204 inputs the prompt 1701 and related ticket 101B received from the generation support device 202 into the language model 140, attempts to generate an answer sentence for the missing item, and generates a postmortem 150-n (n=1).

[0150] (Step S1712-n (n=1)) The generation support device 202 determines whether or not there are any missing items in the postmortem 150-n (n=1) transmitted in step S1711-n (n=1) using the postmortem generation program 504. If there are no missing items (step S1712-n (n=0): No), the process proceeds to steps S1713 and S1715. If there are missing items (step S1712-n (n=1): Yes), steps S1717-n (n=1) to S1721-n (n=1) are executed.

[0151] In this manner, steps S1710-n to S1712-n and steps S1717-n to S1721-n are repeatedly executed until there are no more missing items.

[0152] If there are no more missing items (step S1712-n: No), steps S1713 to S1716 are executed, and the postmortem generation sequence ends. In this way, in the postmortem generation sequence, postmortem 150-n is generated until there are no more missing items, and the more steps S1710-n to S1712-n and steps S1717-n to S1721-n are repeated, the more the quality of postmortem 150-n improves.

[0153] Furthermore, if language model 140 is used to generate postmortem 150 for source ticket 101A that does not have work memo summary text 624, postmortem 150 will be generated using only ticket details (ticket ID 600, type 111, status 610, ticket creation date and time 611, ticket creator 612, ticket update date and time 613, severity 615, impact 616, title 617, description 618, and conclusion 619), which means that the process leading up to resolution tends to be unclear.

[0154] On the other hand, when language model 140 is caused to generate postmortem 150-n for source ticket 101A that has work memo summary text 624, postmortem 150-n is generated that includes the process leading up to the resolution described in work memo summary text 624. Therefore, the quality of postmortem 150-n is improved for source ticket 101A that has work memo summary text 624 compared to when there is no work memo summary text 624.

[0155] Furthermore, if step S1712-n has been executed a predetermined number of times, the postmortem generation sequence may be terminated even if there are missing items. For example, if the same missing item has not been composed a predetermined number of times in a row, the generation assistance device 202 may terminate the repetition of steps S1710-n to S1712-n and steps S1717-n to S1721-n.

[0156] The postmortem generation sequence may also be terminated if the remaining number of missing items falls below a predetermined number.

[0157] Furthermore, in the postmortem generation sequence, if there are no missing items (step S1712-n: No), postmortem 150-n is displayed on communication terminal 203 in steps S1713 and S1714. However, if there are missing items (step S1712-n: Yes), postmortem 150-n may also be displayed on communication terminal 203 as in steps S1713 and S1714.

[0158] In this case, the communication terminal 203 receives an instruction to continue or end the postmortem generation sequence through operational input from the user 230, and sends it to the postmortem generation program 504 via the chatbot program 500. If an instruction to continue is received, the generation assistance device 202 specifies the search query 1002 (step S1717-n). Meanwhile, the postmortem generation sequence ends.

[0159] Also, in the above example, in step S1710-n (n≧1), the external device 204 uses the language model 140 to generate a postmortem 150-n (n≧1) including the results of an attempt to generate an answer sentence for the missing item, and transmits the postmortem 150-n (n≧1) to the generation assistance device 202; however, it is also possible to transmit the results of an attempt to generate an answer sentence for the missing item to the generation assistance device 202 instead of the postmortem 150-n (n≧1).

[0160] <Figure 18 Postmortem 150> 18 is an explanatory diagram showing an example of postmortem 150. As an example, Fig. 18 shows postmortem 150-0 and 150-1 with ticket ID 600 of "5." In postmortem 150-0, cause of occurrence 745 is a missing item, but in postmortem 150-1, "omitted restart after deployment" has been added to cause of occurrence 745, which is a missing item, generated by language model 140 by referencing related ticket 101B.

[0161] <Figure 19 Generation support screen> 19 is an explanatory diagram showing an example of a generation support screen 1900. The generation support screen 1900 is displayed on a display, which is an example of the output device 304 of the communication terminal 203, for example.

[0162] The generation support screen 1900 includes a ticket management screen 1901 , a chat screen 1902 , a prompt analysis information setting screen 1903 , and an execution log screen 1904 .

[0163] [Ticket Management Screen 1901] The ticket management screen 1901 has a ticket list display section 1910. The ticket list display section 1910 displays a list 1911 of tickets 101. The communication terminal 203 sets a filter condition 1912 in response to an operation input by the user 230, and displays a list 1911 of tickets 101 narrowed down by the filter condition 1912. The filter condition 1912 is, for example, a type 111.

[0164] The ticket management screen 1901 has a ticket detail information display section 1913. The ticket detail information display section 1913 displays the ticket ID 600, title 617 (incident name), description 618, and related ticket ID 800. The title 617 (incident name) is the title 617 (incident name) of the ticket 101 selected by the user 230 from the list 1911 of tickets 101 narrowed down by the filter conditions 1912.

[0165] The ticket detailed information display section 1913 has a work memo summary display section 1914, a postmortem display section 1915, and a timeline information display section 1916. The work memo summary display section 1914 displays the work memo summary 113.

[0166] The work memo summary display area 1914 displays the work memo summary text 624. When the generate button "Generate" in the work memo summary display area 1914 is pressed, the work memo summary sequence shown in Fig. 13 is executed, and the work memo summary text 624 is displayed in the work memo summary display area 1914. When the save button "Save" in the work memo summary display area 1914 is pressed (step S1317), the work memo summary text 624 is saved in the ticket DB 201 (step S1318).

[0167] Postmortem display section 1915 displays postmortem 150. Pressing the generate button "Generate" in postmortem display section 1915 executes the postmortem generation sequence shown in Fig. 17, and postmortem 150 is displayed in postmortem display section 1915. Pressing the save button "Save" in postmortem display section 1915 (step S1715) saves postmortem 150 in postmortem DB 202 (step S1716).

[0168] The timeline information display unit 1916 displays the timeline information (ticket ID 600, task ID 620, task date and time 630, respondent 631, task content 632, task type 633) of the timeline table 603. By pressing the generate button "Generate" in the timeline information display unit 1916, the timeline information (ticket ID 600, task ID 620, task date and time 630, respondent 631, task content 632, task type 633) is rearranged in chronological order and displayed in the timeline information display unit 1916. The communication terminal 203 can add timeline information by operation input from the user 230. By pressing the save button "Save" in the timeline information display unit 1916, the timeline information is saved in the timeline table 603 (step S1318).

[0169] The ticket management screen 1901 displays the work memo content 623 used in the work memo summary sequence shown in FIG.

[0170] [Chat screen 1902] The chat screen 1902 displays prompts 1921 (1300, 1500, 1700) that are input text from the communication terminal 203, and response texts 1922 (work memo summary text 624, ticket summary text 641, postmortem 150) from the agent 120. When the send button 1920 is pressed, the prompt 1921 is sent to the generation support device 202, and the response text 1922 from the generation support device 202 is displayed. In this way, the work memo summary text 624 and postmortem 150 can be generated and displayed on either the ticket management screen 1901 or the chat screen 1902.

[0171] [Prompt Analysis Information Setting Screen 1903] The prompt analysis information setting screen 1903 has an upload area 1930 to which a prompt file 1931 can be uploaded. The prompt file 1931 contains, for example, instruction information 121 and function definition information 220. Therefore, the communication terminal 203 edits the contents of the instruction information 121 and function definition information 220 in response to an editing operation by the user 230, and uploads them to the generation support device 202 as a prompt file 1931. The generation support device 202 updates the instruction information 121 and function definition information 220 in accordance with the uploaded prompt file 1931.

[0172] [Execution log screen 1904] The execution log screen 1904 displays data transmitted from the ticket management device 201 and the generation support device 202 as an execution log 1940. By checking the execution log 1940, the user 230 can understand what the ticket management device 201 and the generation support device have executed and what sentences have been obtained from the language model 140.

[0173] As described above, according to this embodiment, it is possible to check whether there are any missing items in postmortem 150. Furthermore, by resolving the missing items, it is possible to improve the quality of postmortem 150. Furthermore, by including work memo summary text 624 in generating ticket 101A, it is possible to improve the quality of postmortem 150.

[0174] The present invention is not limited to the above-described embodiments, and includes various modifications and equivalent configurations within the spirit and scope of the appended claims. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to configurations including all of the described configurations. Furthermore, part of the configuration of one embodiment may be replaced with the configuration of another embodiment. Furthermore, the configuration of another embodiment may be added to the configuration of one embodiment. Furthermore, part of the configuration of each embodiment may be added to, deleted from, or replaced with other configurations.

[0175] Furthermore, the aforementioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole in hardware, for example by designing them as integrated circuits, or may be realized in software by having a processor interpret and execute a program that realizes each function.

[0176] Information such as programs, tables, files, etc. that realize each function can be stored in storage devices such as memory, hard disks, SSDs (Solid State Drives), or recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0177] In addition, the control lines and information lines shown are those that are considered necessary for explanation, and do not necessarily represent all the control lines and information lines that are necessary for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0178] 100 Postmortem Generation System 101 tickets 101A Original ticket 101B related tickets 112 Work Notes 113 Work Memo Summary 114 Ticket Summary 120 Agent 121 Instruction information 140 language models 150 Postmortem 201 Ticket Management Device 202 Generation support device 203 Communication terminal 204 External device 210 Network 220 Function definition information 221 Related ticket search conditions 222 Related ticket search history 301 processor 302 Storage Devices 400 Operation Program 401 Ticket Search Process 402 Ticket Update Process 403 Work Note Acquisition Process 404 Ticket Acquisition Process 405 Postmortem Update Processing 461 Ticket Summary Text 500 Chatbot Program 501 Prompt Analysis Program 502 Work Memo Summary Program 503 Ticket Summary Program 504 Postmortem Generator 624 Work Memo Summary Text 625 Work Memo Summary Vector 641 Ticket Summary Text 642 Ticket Abstract Vector 740 Confirmation items group 1201 Prompt Analysis Information

Claims

1. A generation support system having a processor that executes a program and a storage device that stores the program, When a natural sentence, which is text data in a natural language, is input, a language model can be accessed that interprets the natural sentence and probabilistically predicts a response sentence to the natural sentence, The processor: a first request process that outputs to the language model a source ticket of the natural sentence related to the incident, a group of check items to be checked for the incident, and a first prompt that is the natural sentence requesting the generation of an answer sentence for each check item in the group of check items by referencing the source ticket; a first acquisition process in which, as a result of outputting the source ticket, the group of confirmation items, and the first prompt by the first request process, the language model acquires a postmortem including a first attempt result of generating an answer sentence for each of the confirmation items in the first prompt; a first output process for outputting the postmortem acquired by the first acquisition process; A generation support system characterized by executing the above.

2. The generation support system according to claim 1, The processor: a first determination process is executed on the first postmortem acquired by the first acquisition process to determine whether or not there is a missing item among the group of confirmation items, the first trial result of which is not a response sentence that can confirm the confirmation item; In the first output process, the processor outputs the first post-mortem in which the presence or absence of the missing item has been determined by the first determination process. A generation support system characterized by:

3. The generation support system according to claim 2, The processor: a search process for searching for related tickets related to the generating ticket from a group of tickets based on a search query for the missing item; a second request process for outputting to the language model the related tickets searched by the search process and a second prompt for requesting generation of the answer sentence for the missing item in the related ticket; a second acquisition process for acquiring a second attempt result in which the language model attempts to generate an answer sentence for the missing item in the second prompt as a result of outputting the related ticket and the second prompt through the second request process; a second output process for outputting the second trial result acquired by the second acquisition process; A generation support system characterized by executing the above.

4. The generation support system according to claim 3, Each ticket in the group of tickets includes a ticket summary vector, which is an embedded representation obtained by vectorizing a ticket summary text that summarizes the ticket through embedding; In the search process, the processor searches the group of tickets for tickets that match the search query for the missing item and whose ticket summary vectors have a distance within a predetermined distance as the related tickets. A generation support system characterized by:

5. The generation support system according to claim 3, In the second acquisition process, the processor acquires a second postmortem including the group of check items, the first trial result, and the second trial result; In the second output process, the processor outputs the second postmortem acquired by the second acquisition process. A generation support system characterized by executing the above.

6. The generation support system according to claim 3, The processor: executing a second determination process for determining whether or not the missing item exists based on the second trial result acquired by the second acquisition process; A generation support system characterized by:

7. The generation support system according to claim 6, The processor: A generation support system characterized in that the search process, the second request process, the second acquisition process, and the second output process are repeatedly executed until the missing items are eliminated by the second determination process.

8. The generation support system according to claim 1, The source ticket includes a work note summary text summarizing a work note describing the work content for the incident. A generation support system characterized by:

9. A generation support method executed by a generation support system having a processor that executes a program and a storage device that stores the program, When a natural sentence, which is text data in a natural language, is input, a language model can be accessed that interprets the natural sentence and probabilistically predicts a response sentence to the natural sentence, The processor: a first request process that outputs to the language model a source ticket of the natural sentence related to the incident, a group of check items to be checked for the incident, and a first prompt that is the natural sentence requesting the generation of an answer sentence for each check item in the group of check items by referencing the source ticket; a first acquisition process in which, as a result of outputting the source ticket, the group of confirmation items, and the first prompt by the first request process, the language model acquires a postmortem including a first attempt result of generating an answer sentence for each of the confirmation items in the first prompt; a first output process for outputting the postmortem acquired by the first acquisition process; A generation support method characterized by executing the above.

10. When a natural sentence, which is text data in a natural language, is input, a processor that can access a language model that interprets the natural sentence and probabilistically predicts a response sentence to the natural sentence, a first request process that outputs to the language model a source ticket of the natural sentence related to the incident, a group of check items to be checked for the incident, and a first prompt that is the natural sentence requesting the generation of an answer sentence for each check item in the group of check items by referencing the source ticket; a first acquisition process in which, as a result of outputting the source ticket, the group of confirmation items, and the first prompt by the first request process, the language model acquires a postmortem including a first attempt result of generating an answer sentence for each of the confirmation items in the first prompt; a first output process for outputting the postmortem acquired by the first acquisition process; A generation support program that causes the program to execute the above steps.

Images