Dialogue data output method, dialogue data output program and dialogue data output device, and question data output method, question data output program and question data output device

Abstract

To provide a dialogue data output method, a dialogue output program and a dialogue data output device that do not require supervised learning, and secondarily provide a question data output method, a question data output program and a question data output device that output question data based on dialogue data.SOLUTION: A dialogue data output method executes processing of: acquiring historical data in which conversational content regarding inquiries on products or services has been transcribed, or historical data of text-based conversations; inputting the acquired historical data into a language model; acquiring dialogue data that associates questions with answers to the questions from the language model; and outputting the acquired dialogue data.SELECTED DRAWING: Figure 9

Description

[Technical Field]

[0001] The present invention relates to a question-and-answer data output method, a question-and-answer data output program, and a question-and-answer data output device for outputting question-and-answer data, as well as a question-and-answer data output method, a question-and-answer data output program, and a question-and-answer data output device for outputting question data based on question-and-answer data. [Background technology]

[0002] One of the measures to improve operational efficiency at contact centers is to reduce the number of calls received. To achieve this, the company aims to reduce the number of calls received by expanding its online FAQs (Frequency Asking Questions) and allowing users to solve problems and resolve questions themselves. FAQs should be carefully selected and posted in the "Frequently Asked Questions" section, and the process of deciding what to post itself is both a heavy workload and difficult.

[0003] In response to this situation, Patent Document 1 proposes an extraction method for extracting question and answer data consisting of candidate questions and candidate answers that can be registered in an FAQ from a dialogue history obtained by performing voice recognition on recorded data of responses generated at a contact center or by saving chat records. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6743108 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in conventional technologies, when building an extraction system, it is necessary to carry out everything from preparing training data to model learning based on the collected dialogue history, which is costly.

[0006] The present invention has been made in light of these circumstances. Its primary object is to provide a question-and-answer data output method, a question-and-answer data output program, and a question-and-answer data output device that do not require supervised learning. A secondary object is to provide a question-and-answer data output method, a question-and-answer data output program, and a question-and-answer data output device that output question data based on question-and-answer data. [Means for solving the problem]

[0007] A question-and-answer data output method according to one aspect of the present application acquires historical data in which conversational content regarding product or service inquiries has been transcribed, or historical data of text-based conversations, inputs the acquired historical data into a language model, acquires question-and-answer data from the language model that associates questions with answers to those questions, and executes a process of outputting the acquired question-and-answer data. [Effects of the Invention]

[0008] In one aspect of the present application, question and answer data can be output without performing supervised learning. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram illustrating an example of the configuration of an FAQ creation support system. [Figure 2] FIG. 2 is a block diagram showing an example of the hardware configuration of a creation server 1. [Figure 3] 2 is a block diagram showing an example of the hardware configuration of a user terminal 2. FIG. [Figure 4] FIG. 2 is an explanatory diagram showing an example of a call log DB 131. [Figure 5] FIG. 10 is an explanatory diagram showing an example of a QA pair DB 132. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a vector DB 133. [Figure 7] FIG. 2 is an explanatory diagram showing an example of an FAQ DB 134. [Figure 8] 10 is a flowchart illustrating an example of a procedure for business processing. [Figure 9]10 is a flowchart illustrating an example of an extraction process procedure. [Figure 10] FIG. 10 is an explanatory diagram showing a specific example of extraction processing. [Figure 11] FIG. 10 is an explanatory diagram showing another specific example of the extraction process. [Figure 12] FIG. 10 is an explanatory diagram showing another specific example of the extraction process. [Figure 13] 10 is a flowchart illustrating an example of a procedure for vectorization processing. [Figure 14] 10 is a flowchart illustrating an example of a procedure for a selection process. [Figure 15] FIG. 10 is a conceptual diagram illustrating an example of a clustering result. [Figure 16] FIG. 10 is an explanatory diagram showing an example of a determination result screen. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment will be described below with reference to the drawings. FIG. 1 is an explanatory diagram showing an example of the configuration of an FAQ creation support system. The FAQ creation support system 100 includes a creation server 1, a user terminal 2, and a generation AI (Artificial Intelligence). The creation server 1, the user terminal 2, and the generation AI are connected to each other via a network N so that they can communicate with each other. The generation AI may also be implemented in the creation server 1. Also, in FIG. 1, the FAQ creation support system 100 is a client-server system including the creation server 1 and the user terminal 2, but it may also be a standalone system in which the creation server 1 and the user terminal 2 are integrated into a single computer.

[0011] The generation AI mainly generates and outputs sentences in natural language. The generation AI includes a generation server 3 that provides services and a language model 4 that has undergone language learning. The generation AI may be built on the creation server 1.

[0012] The contact center is shown merely as a data source for the call log, and is not an essential component of the FAQ creation support system 100. Although only one user terminal 2 is shown in Fig. 1, two or more terminals may be used.

[0013] FIG. 2 is a block diagram showing an example of the hardware configuration of the creation server 1. The creation server 1 includes a control unit 11, a main memory unit 12, an auxiliary memory unit 13, a communication unit 14, and a reading unit 15. The control unit 11, the main memory unit 12, the auxiliary memory unit 13, the communication unit 14, and the reading unit 15 are connected via a bus B. The creation server 1 is configured by a server computer, a workstation, a PC (Personal Computer), or the like. The creation server 1 may also be configured by a multicomputer consisting of multiple computers, a virtual machine virtually constructed by software, or a quantum computer. Furthermore, the functions of the creation server 1 may be realized by a cloud service.

[0014] The control unit 11 has one or more arithmetic processing devices such as a CPU (Central Processing Unit), an MPU (Micro-Processing Unit), a GPU (Graphics Processing Unit), etc. The control unit 11 reads and executes a control program 1P (program, program product) stored in the auxiliary storage unit 13, thereby performing various information processing, control processing, etc., and realizing various functional units.

[0015] The main memory unit 12 is a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, etc. The main memory unit 12 mainly temporarily stores data required for the control unit 11 to execute arithmetic processing.

[0016] The auxiliary storage unit 13 is a hard disk or an SSD (Solid State Drive) or the like, and stores the control program 1P and various DBs (Databases) required for the control unit 11 to execute processing. The auxiliary storage unit 13 stores a call log DB 131, a QA pair DB 132, a vector DB 133, and an FAQ DB 134. The auxiliary storage unit 13 may be separate from the creating server 1 and may be an external storage device connected externally. The various DBs and the like stored in the auxiliary storage unit 13 may be stored in a database server or cloud storage different from the creating server 1.

[0017] The call log DB 131 stores call logs. Call logs are text data (historical data) obtained by transcribing the contents of conversations between an operator and a customer at a contact center. In the following explanation, the call log is assumed to be a transcript of a voice call at a contact center, but this is not limited to this. The call log is not limited to a text version of a voice call, but may also be a log of text chat. Furthermore, multiple emails containing questions and answers exchanged between a support desk and a customer may also be treated as a call log.

[0018] The QA pair DB 132 stores QA pairs. The QA pairs are data extracted from call logs. The QA pairs are text data that associates questions from customers with answers from operators. The QA pairs are an example of question and answer data.

[0019] The vector DB 133 stores vector data of QA pairs. Each QA pair consisting of a question and an answer is vectorized and stored in the vector DB 133.

[0020] The FAQ DB 134 stores vector data of FAQs. The FAQ vector data is created by vectorizing questions and answers contained in FAQs that have already been published or are in use. Each set of data consisting of a question and an answer is vectorized and stored in the FAQ DB 134.

[0021] The communication unit 14 communicates with the user terminal 2 via the network N. In addition, the control unit 11 may use the communication unit 14 to download the control program 1P from another computer via the network N or the like and store it in the auxiliary storage unit 13.

[0022] The reading unit 15 reads the portable storage medium 1a including a CD (Compact Disc)-ROM and a DVD (Digital Versatile Disc)-ROM. The control unit 11 may read the control program 1P from the portable storage medium 1a via the reading unit 15 and store it in the auxiliary storage unit 13. The control unit 11 may also read the control program 1P from the semiconductor memory 1b.

[0023] 3 is a block diagram showing an example of the hardware configuration of the user terminal 2. The user terminal 2 is configured from a notebook computer, a panel computer, a tablet computer, a smartphone, etc. The user terminal 2 includes a control unit 21, a main memory unit 22, an auxiliary memory unit 23, a communication unit 24, an input unit 25, and a display unit 26. Each component is connected by a bus B.

[0024] The control unit 21 has one or more arithmetic processing units such as a CPU, an MPU, a GPU, etc. The control unit 21 provides various functions by reading and executing a control program 2P (program, program product) stored in the auxiliary storage unit 23.

[0025] The main memory unit 22 is an SRAM, a DRAM, a flash memory, etc. The main memory unit 22 mainly temporarily stores data necessary for the control unit 21 to execute arithmetic processing.

[0026] The auxiliary storage unit 23 is a hard disk or SSD, etc., and stores various data necessary for the control unit 21 to execute processing. The auxiliary storage unit 23 may be a separate unit from the user terminal 2 and may be an external storage device connected externally. The various DBs, etc. stored in the auxiliary storage unit 23 may be stored in a database server or cloud storage.

[0027] The communication unit 24 communicates with the creation server 1 via the network N. In addition, the control unit 21 may use the communication unit 24 to download the control program 2P from another computer via the network N or the like, and store it in the auxiliary storage unit 23.

[0028] The input unit 25 is a keyboard and a mouse. The display unit 26 includes a liquid crystal display panel or an organic EL (electroluminescence) display panel, etc. The display unit 26 displays the QA pairs output by the creation server 1. The input unit 25 and the display unit 26 may be integrated to form a touch panel display. The user terminal 2 may also display on an external display device.

[0029] The database will now be described. Figure 4 is an explanatory diagram showing an example of the call log DB 131. The call log DB 131 stores call logs in which the voice of a call at a contact center has been converted into text. The call log DB 131 includes a log ID, a start date and time column, an end date and time column, and a text column. The log ID column stores a log ID that can uniquely identify a call log. The start date and time column stores the date and time when the call started. The end date and time column stores the date and time when the call ended. The text column stores the call log in which the voice of the call has been converted into text. The call log DB 131 does not necessarily have to have the start date and time column and the end date and time column.

[0030] FIG. 5 is an explanatory diagram showing an example of the QA pair DB 132. The QA pair DB 132 stores QA pairs (first question and answer data) that are combinations of questions and answers extracted from call logs. The QA pair DB 132 includes a QA ID column, a log ID column, a question column, and an answer column. The QA ID column stores a QA ID that can uniquely identify a QA pair. The log ID column stores the log ID of a call log that includes the QA pair. The question column stores question text (first question data). The answer column stores answer text.

[0031] FIG. 6 is an explanatory diagram showing an example of the vector DB 133. The vector DB 133 stores candidate vectors (features, first feature vectors) obtained by vectorizing QA pairs. The vector DB 133 includes a vector ID column, a QA ID column, a log ID column, and a candidate vector column. The vector ID column stores a vector ID that can uniquely identify a candidate vector. The QA ID column stores a QA ID assigned to the QA pair that is the basis of the candidate vector. The log ID column stores the log ID of the call log that includes the QA pair. The candidate vector column stores candidate vectors obtained by vectorizing QA pairs. The QA ID column and the log ID column may be omitted from the vector DB 133.

[0032] FIG. 7 is an explanatory diagram showing an example of the FAQ DB 134. The FAQ DB 134 stores FAQ vectors (features, second feature vectors) obtained by vectorizing each pair of a question (second question data) and an answer (second question and answer data) included in an existing FAQ. The FAQ DB 134 includes an FAQ ID column, a target column, a theme column, a question column, an answer column, and an FAQ vector column. The FAQ ID column stores an FAQ ID that can uniquely identify an FAQ vector. The target column stores information about the product or service that is the subject of the FAQ. The theme column stores the theme of the FAQ. The question column stores the text of the question. The answer column stores the text of the answer. The FAQ vector column stores FAQ vectors that vectorize questions and answers. The FAQ DB 134 does not necessarily have to have a target column, a theme column, a question column, or an answer column.

[0033] In the selection process described later, it is necessary to distinguish between candidate vectors and FAQ vectors. This can be achieved by using different numbering systems for the vector ID of a candidate vector and the FAQ ID of an FAQ vector.

[0034] (Business processing) Next, a description will be given of the business processing performed by the FAQ creation support system 100. Fig. 8 is a flowchart showing an example of the procedure of the business processing. The control unit 11 of the creation server 1 collects call logs (step S1).

[0035] The control unit 11 acquires voice data of the conversation from the contact center via the network N or by using a storage medium. The control unit 11 passes the acquired voice data to the generation AI. The generation AI converts the voice data into text. The control unit 11 may convert the voice data into text using known voice recognition technology. The control unit 11 stores the text data from the generation AI in the call log DB 131. The voice data may be converted into text at the contact center, and the control unit 11 may acquire the text data via the network N or by using a storage medium. A call log basically captures the conversation from the time the call is received until it is hung up as one piece of data (one item).

[0036] The call log is processed by the language model 4. If the language model 4 has an upper limit on the number of tokens it can accept at one time, an upper limit on the number of characters or words is set accordingly. If the number of characters or words in the collected call log exceeds the upper limit, it is split into multiple call logs, each of which does not exceed the upper limit on the number of characters or words. The split points are determined by pattern matching or the like to capture question marks and prevent the question and its answer from being separated.

[0037] The control unit 11 extracts QA pairs from the voice data (step S2). Details of QA pair extraction will be described later. It is assumed that multiple QA pairs are extracted from one call log.

[0038] The control unit 11 vectorizes the QA pairs (step S3). Vectorization is a process of converting text data into vector data. Vectorization is also called embedding or filling. The control unit 11 converts the text data constituting the QA pairs into vector data using a known method such as Word2Vec. The control unit 11 may pass the QA pairs to the generation AI, which then converts the QA pairs into vector data. Vectorization is performed for each QA pair. The control unit 11 stores the vector data of the QA pairs in the vector DB 133.

[0039] The control unit 11 performs clustering (step S4). The clustering targets are the vector data of the QA pairs and the vector data of the FAQs.

[0040] The control unit 11 identifies clusters that contain vectors of QA pairs that could become new FAQs based on the number of elements (number of vectors) in each cluster and the type of element (whether it is a vector of a QA pair or a vector of an FAQ) (step S5).

[0041] The control unit 11 restores the vectors of the QA pairs included in the identified cluster to sentences and outputs them (step S6). The control unit 11 then ends the business process. The user considers the content of the new FAQ to be added based on the output sentences.

[0042] (Extraction of QA pairs) The extraction process of QA pairs will now be described. The extraction process corresponds to step S2 in FIG. 8. FIG. 9 is a flowchart showing an example procedure of the extraction process. The control unit 11 of the creation server 1 acquires one call log from the call log DB 131 (step S11). The control unit 11 creates a prompt for extracting QA pairs (step S12). The control unit 11 inputs the prompt to the language model 4 via the generation server 3 (step S13). The control unit 11 acquires the output (QA pair) of the language model 4 via the generation server 3 (step S14). The control unit 11 stores the acquired QA pair in the QA pair DB 132 (step S15). The control unit 11 determines whether there are any unprocessed call logs (step S16). If the control unit 11 determines that there are any unprocessed call logs (YES in step S16), the process returns to step S11 and processes the unprocessed call logs. When the control unit 11 determines that there is no unprocessed call log (NO in step S16), the process ends.

[0043] 10 is an explanatory diagram showing an example of extraction processing. The prompt 6 includes an instruction 61 of the language model 4 and a call log 62 to be processed. Text 63 is an example of a QA pair output by the language model 4.

[0044] FIG. 11 is an explanatory diagram showing another specific example of extraction processing. The prompt 7 shown in FIG. 11 includes, as instructions to the language model 4, an instruction 71 to extract a QA pair, a call log 73, and an instruction 72 to extract supplemental information. The supplemental information may be the service, product, function, contract plan (contract information), or matters heard from the customer by the operator, which are the subject of the inquiry. When the language model 4 processes the call log using such a prompt, text 74 is obtained. The text 74 includes service information 741, plan information 742, and a QA pair 743.

[0045] FIG. 12 is an explanatory diagram showing another specific example of extraction processing. Prompt 8 shown in FIG. 12 describes instructions based on a call log in which the customer did not ask a clear interrogative question. When a customer does not ask a clear interrogative question because the customer's desired information is vague or the customer lacks prior knowledge of the product or service, the operator makes an effort to clarify the specifics of what the customer wants to know by interviewing the customer about various matters, such as the customer's difficulties and the circumstances leading up to the inquiry. Prompt 8 is created on the assumption that a call log with such content will be processed. Prompt 8 includes a key point extraction instruction 81, a customer information extraction instruction 82, an inquiry content extraction instruction 83, an inquiry content extraction instruction 84, an answer content extraction instruction 85, and a call log 86.

[0046] Assuming that the call log 86 does not contain any questions in the form of clear questions, a gist extraction instruction 81 is issued to summarize the gist of the entire call. A customer information extraction instruction 82 is issued to extract customer information. This is because, if the inquiry is specific to a customer or the industry to which the customer belongs, customer information will help understand the content. An inquiry content extraction instruction 83 is issued to extract the content of the customer's inquiry. By using "inquiry content," it is expected that the language model 4 will accurately extract what the customer wanted to know. A hearing matter extraction instruction 84 is issued to have the language model 4 extract the questions the operator asked the customer in order to understand what the customer wanted to know. An answer content extraction instruction 85 is an instruction to extract and output the customer's answer. The call log 86 is the call log to be processed.

[0047] As described above, the extraction process makes it possible to extract QA pairs to be added as new FAQs. Furthermore, by using prompt 8 shown in FIG. 12, it is possible to extract content equivalent to a QA pair even if no clear QA pair is included. This content can essentially be considered a QA pair. In the following explanation, clear QA pairs and substantial QA pairs will be collectively referred to simply as "QA pairs."

[0048] By executing the extraction process, the FAQ creation support system 100 achieves the following effects: It is possible to output QA pairs from call logs without performing supervised learning. It is possible for users to determine the content of new FAQs to add from among the QA pairs.

[0049] In the task of considering what to add to an FAQ from call logs, extracting QA pairs using the extraction and repair method described above can reduce the user's workload. However, if the extracted QA pairs are diverse, it is not efficient to add all of them to the FAQ. Also, the content of an extracted QA pair may already be included in the FAQ. Furthermore, an increase in the number of FAQs is undesirable, as it increases the difficulty of finding the desired FAQ. Below, we will explain the process of selecting QA pairs containing content to be added to the FAQ from the many extracted QA pairs. Note that the QA pairs used in this selection process are not limited to those created by the extraction and repair method described above, and can be any data consisting of a question and its answer.

[0050] FIG. 13 is a flowchart showing an example procedure for vectorization processing. The vectorization processing corresponds to step S3 in FIG. 8. The vectorization processing is a process that is executed prior to the selection processing. The control unit 11 of the creation server 1 acquires one record of a QA pair from the QA pair DB 132 (step S21). The control unit 11 vectorizes the QA pair (step S22). The control unit 11 stores the vector of the QA pair in the vector DB 133 (step S23). The control unit 11 determines whether or not there is an unprocessed QA pair (step S24). If the control unit 11 determines that there is an unprocessed QA pair (YES in step S24), it returns the process to step S21 and processes the unprocessed QA pair. If the control unit 11 determines that there is no unprocessed QA pair (NO in step S24), it ends the process.

[0051] The QA pairs to be vectorized in step S22 are assumed to be those shown in Fig. 10. The data to be vectorized is not limited to QA pairs, and may be text containing information other than QA pairs (supplementary information). For example, data containing QA pairs 743, such as text 74 shown in Fig. 11, or data containing customer information, introductions, additional questions, and answers output by the prompt shown in Fig. 12 may be vectorized.

[0052] (Selection processing) The selection process will now be described. The selection process corresponds to steps S4 to S6 in Fig. 8. The assumptions for executing the selection process are as follows: FAQ vectors obtained by vectorizing FAQs that are already in use are stored in the FAQ DB 134. Also, it is assumed that candidate vectors obtained by vectorizing QA pairs or text containing QA pairs through vectorization processing or the like are stored in the vector DB 133.

[0053] FIG. 14 is a flowchart showing an example of the selection process. The control unit 11 of the creation server 1 acquires candidate vectors from the vector DB 133 (step S31). The control unit 11 acquires FAQ vectors from the FAQ DB 134 (step S32). The control unit 11 performs clustering on the candidate vectors and FAQ vectors as a universal set (step S33). Each vector is assigned an attribute or label indicating whether it is a candidate vector or an FAQ vector. Any known clustering method can be used, but x-means or g-means, which automatically determines the number of clusters, is employed. Note that any method that can handle vector data and automatically determine the number of clusters is not limited to these methods and can be employed. From the clusters obtained by clustering, the control unit 11 extracts clusters that do not contain FAQ vectors as elements but only candidate vectors (step S34). From the extracted clusters, the control unit 11 selects clusters (specific targets) whose number of elements is equal to or greater than a predetermined threshold (step S35). The control unit 11 may rank the clusters in descending order of the number of elements and select clusters from first to a predetermined rank. The control unit 11 restores the candidate vectors that make up the selected cluster to a sentence (step S36). If the original sentence can be obtained from the QA pair DB 132 using the QAID, restoration is not necessary. The control unit 11 outputs the question contained in the restored sentence (step S37). The output destination is the user terminal 2, etc. It may also be stored in the auxiliary memory unit 13. The control unit 11 ends the processing. Note that the control unit 11 may output the restored sentence, i.e., the question and the answer to the question (answer data).

[0054] FIG. 15 is a conceptual diagram showing an example of a clustering result. Two clusters are shown in FIG. 15. Cluster 111 is a cluster that includes only candidate vectors. Cluster 113 is a cluster that includes candidate vectors and FAQ vectors. Each cluster represents a question obtained by restoring the element vectors. New QA candidates 112 are derived from the question sentences included in cluster 111. The user considers what to add to the FAQ by referring to the new QA candidates 112. Existing QAs 114 can be identified from the question sentences included in cluster 113. The user updates the content of the identified FAQ as necessary.

[0055] FIG. 16 is an explanatory diagram showing an example of a determination result screen. The determination result screen d01 is a screen that presents the results of the selection process to the user. The example in FIG. 16 shows a case where one cluster has been selected. The determination result screen d01 includes a theme d011, a number of items d012, and example questions d013. The theme d011 displays the common theme among the QA pairs included in the selected cluster (hereinafter referred to as the "selected cluster"). The theme is created, for example, by providing the QA pairs included in the selected cluster to LLM5. The number of items d012 indicates the number of elements in the selected cluster = the number of QA pairs included in the selected cluster. The example questions d013 indicate questions extracted from the QA pairs included in the selected cluster. The determination result screen d01 allows the user to determine new QAs that should be added to the FAQ.

[0056] New QAs were considered from the clustering results, but it is also possible to consider maintenance of existing QAs from the clustering results. If a cluster containing existing FAQs contains many QA pairs obtained from call logs, it is possible that the existing FAQs are not being used by end users. If the number of end user accesses is low, methods to increase access should be considered. If the number of end user accesses is high, it is likely that the content is not being conveyed, so consideration should be given to updating the content. Existing FAQs that have few end user accesses and belong to isolated clusters that are far from other clusters may have already fulfilled their role, so deletion should be considered.

[0057] The technical features (constituent elements) described in each embodiment can be combined with each other, and by combining them, new technical features can be formed. The embodiments disclosed herein are to be considered as illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. In addition, the claims are written in a format in which a claim cites two or more other claims (multiple claim format), but this is not limited to this. Multiple claims that cite at least one other claim (multi-multi claim format) may also be written. [Explanation of symbols]

[0058] 100: FAQ creation support system 1: Creation server 11: Control section 12: Main memory 13: Auxiliary storage section 131: Call log database 132: QA Pair DB 133: Vector DB 14: Communications Department 15: Reading unit 1P: Control program 1a: Portable storage medium 1b: Semiconductor memory 2: User terminal 21: Control unit 22: Main memory 23: Auxiliary storage section 24: Communications Department 25: Input section 26:Display section 2P: Control program 3: Generation server B: Bus N: Network

Claims

1. Obtain historical data of conversations about products or services that have been transcribed, or historical data of conversations in writing, The acquired historical data is input into a language model, acquiring question and answer data in which questions and answers to the questions are associated with each other from the language model; Output the acquired question and answer data A method for outputting question and answer data to be processed.

2. inputting a prompt into the language model along with the historical data to organize the question and the answer; The question and answer data output method according to claim 1 .

3. inputting an instruction to the language model to extract supplemental information including product or service functions, user attributes, or contract information from the history data; Supplementary information including the function of the product or service that is the subject of the inquiry, the attributes of the user who made the inquiry, or contract information, which is acquired from the language model, is output together with the question and answer data.

3. The question and answer data output method according to claim 1 or 2.

4. acquiring first question and answer data including first question data; acquiring second question and answer data including second question data; performing clustering based on the feature amounts of the first question-and-answer data and the feature amounts of the second question-and-answer data; identifying a cluster that does not include the second question data from among the clusters obtained by the clustering; outputting the first question data included in the identified cluster; The question data output method to perform the processing.

5. determining the size of each cluster; Clusters whose size is equal to or larger than a predetermined value are identified as targets. The question data output method according to claim 4.

6. The feature amount of the first question-and-answer data is a first feature vector obtained by vectorizing the first question-and-answer data, and the feature amount of the second question-and-answer data is a second feature vector obtained by vectorizing the second question-and-answer data, and clustering is performed based on the first feature vector and the second feature vector.

6. The question data output method according to claim 4 or 5.

7. The first question data and the answer data associated with the first question data are output in association with each other.

6. The question data output method according to claim 4 or 5.

8. Obtain historical data of conversations about products or services that have been transcribed, or historical data of conversations in writing, The acquired historical data is input into a language model, acquiring question and answer data in which questions and answers to the questions are associated with each other from the language model; The question and answer data is processed as the first question and answer data.

6. The question data output method according to claim 4 or 5.

9. Obtain historical data of conversations about products or services that have been transcribed, or historical data of conversations in writing, The acquired historical data is input into a language model, acquiring question and answer data in which questions and answers to the questions are associated with each other from the language model; Output the acquired question and answer data A question and answer data output program that executes processing.

10. A question and answer data output device including a control unit, The control unit Obtain historical data of conversations about products or services that have been transcribed, or historical data of conversations in writing, The acquired historical data is input into a language model, acquiring question and answer data in which questions and answers to the questions are associated with each other from the language model; Output the acquired question and answer data A question and answer data output device that executes processing.

11. acquiring first question and answer data including first question data; acquiring second question and answer data including second question data; performing clustering based on the feature amounts of the first question-and-answer data and the feature amounts of the second question-and-answer data; identifying a cluster that does not include the second question data from among the clusters obtained by the clustering; outputting the first question data included in the identified cluster; A query data output program that executes processing.

12. A question data output device including a control unit, The control unit acquiring first question and answer data including first question data; acquiring second question and answer data including second question data; performing clustering based on the feature amounts of the first question-and-answer data and the feature amounts of the second question-and-answer data; identifying a cluster that does not include the second question data from among the clusters obtained by the clustering; outputting the first question data included in the identified cluster; A query data output device that executes the processing.

Images