FAQ acquisition methods, FAQ acquisition systems, and FAQ acquisition programs

Abstract

This approach reduces the workload of FAQ creators while providing convenient access to a collection of similar FAQs. [Solution] The FAQ acquisition method involves inputting information about the response to be processed into the LLM service 4, acquiring FAQs related to the response as the FAQ to be processed based on the output of the LLM service 4, registering the FAQ to be processed into the FAQ search database 16, extracting one or more FAQs containing questions similar to the questions in the FAQ to be processed from among multiple existing FAQs related to the response registered in the FAQ search database 16 as similar FAQs, and inputting the similar FAQs into a large-scale language model to acquire aggregated FAQs based on the output of the large-scale language model.

Description

Technical Field

[0001] The present disclosure relates to a FAQ (Frequently Asked Questions) acquisition method, a FAQ acquisition system, and a FAQ acquisition program for acquiring FAQs including combinations of questions and answers thereto.

Background Art

[0002] Conventionally, in call centers, service desks and the like, a plurality of operators handle telephone inquiries from users (i.e., customers, clients, etc.). Since quick and appropriate answers are required for questions from users, FAQs including pre-assumed questions and corresponding answers are utilized. In addition, FAQs are also useful for transferring operations from experienced operators to new operators (i.e., new employee training).

[0003] On the other hand, creating FAQs requires a great deal of labor. In addition, since FAQs need to be updated (i.e., adding or modifying questions and answers) every time the trend of inquiries from users changes, the management burden is also large.

[0004] Accordingly, techniques for reducing the burden of creating FAQs have been developed. For example, an apparatus that supports FAQ creation performs processing of deleting unnecessary utterance sentences other than questions from a plurality of interaction histories including input questions, or processing of extracting utterance sentences related to the questions from the interaction histories, groups the processed results into a plurality of question groups based on a predetermined classification model, and receives input from a creator for the question group selected on the question group creation screen to generate FAQs. A technique for displaying a corresponding screen on a display device is known (see Patent Document 1).

[0005] Furthermore, regarding a response system that receives requests from users and presents appropriate answers to those requests, a technique is known that utilizes RAG (Retrieval-Augmented Generation) to search for external information in text generation by a language model and uses the search results to generate answers (see Patent Document 2). [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2021-157534 [Patent Document 2] Patent No. 7588752 [Overview of the Initiative] [Problems that the invention aims to solve]

[0007] The prior art described in Patent Document 1 above involves the creation of FAQs based on input operations by an operator such as an administrator (i.e., an FAQ creator), and aims to reduce the burden of FAQ creation on the operator.

[0008] However, the human operation involved in creating FAQs using such conventional technologies requires a certain level of skill from the operator, and also places a heavy workload on the operator. Therefore, it is desirable to reduce (or eliminate) the operator's operation in creating FAQs.

[0009] Furthermore, the RAG technology described in Patent Document 2 above could also be used to create FAQs. However, when creating FAQs using RAG technology with Large Language Models (LLMs), the usability of the FAQ decreases if the created FAQ contains many similar questions (i.e., duplicate questions).

[0010] Therefore, the main purpose of this disclosure is to provide an FAQ acquisition method, an FAQ acquisition system, and an FAQ acquisition program that enable the acquisition of highly convenient FAQs, which are aggregated similar FAQs, while reducing the workload of FAQ creators in the creation of FAQs. [Means for solving the problem]

[0011] The FAQ acquisition method of this disclosure is an FAQ acquisition method using an FAQ acquisition system that acquires FAQs including combinations of questions and answers, wherein information on the response to be processed is input into a large-scale language model, and based on the output of the large-scale language model, FAQs related to the response are acquired as FAQs to be processed, the FAQs to be processed are registered in an FAQ search database, one or more FAQs containing questions similar to the questions in the FAQs to be processed are extracted as similar FAQs from among multiple existing FAQs related to the response registered in the FAQ search database, and the similar FAQs are input into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model.

[0012] Furthermore, the FAQ acquisition system of this disclosure includes a processor that performs a process to acquire FAQs that include combinations of questions and answers, wherein the processor inputs information about the response to be processed into a large-scale language model, and based on the output of the large-scale language model, acquires FAQs related to the response as FAQs to be processed, registers the FAQs to be processed into an FAQ search database, extracts one or more FAQs from among multiple existing FAQs related to the response registered in the FAQ search database that include questions similar to the questions in the FAQs to be processed as similar FAQs, and inputs the similar FAQs into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model.

[0013] Furthermore, the FAQ acquisition program of this disclosure is an FAQ acquisition program for performing a process to acquire FAQs that include combinations of questions and answers, and is configured such that a computer inputs information about the response to be processed into a large-scale language model, and based on the output of the large-scale language model, acquires FAQs related to the response as FAQs to be processed, registers the FAQs to be processed into an FAQ search database, extracts one or more FAQs from among multiple existing FAQs related to the response registered in the FAQ search database that include questions similar to the questions in the FAQs to be processed as similar FAQs, and inputs the similar FAQs into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model. [Effects of the Invention]

[0014] According to this disclosure, when creating FAQs, it is possible to reduce the workload of FAQ creators while obtaining highly convenient FAQs that aggregate similar FAQs. [Brief explanation of the drawing]

[0015] [Figure 1] Overall configuration diagram of the FAQ acquisition system according to the first embodiment [Figure 2] Figure 1 shows a flowchart illustrating the flow of the FAQ acquisition process using the FAQ acquisition system. [Figure 3] Figure 1 is an explanatory diagram showing an example of speech text generated by the speech recognition service. [Figure 4] Figure 1 is an explanatory diagram showing an example of a text extraction prompt sent from the text extraction unit to the LLM service. [Figure 5] Figure 4 is an explanatory diagram showing an example of response information (selected text) to the text extraction prompt. [Figure 6] Figure 1 shows an example of an FAQ creation prompt sent from the FAQ creation unit to the LLM service. [Figure 7] Figure 1 shows an example of an FAQ created by the FAQ creation unit. [Figure 8] Explanatory diagram showing an example of a FAQ group created by a FAQ creation unit [Figure 9] Explanatory diagram showing an example of question information of a processing target FAQ sent from the query search unit shown in Figure 1 to a search service [Figure 10] Explanatory diagram showing an example of FAQ similarity information acquired by the query search unit shown in Figure 1 [Figure 11] Explanatory diagram showing an example of a FAQ aggregation prompt sent from the FAQ aggregation unit shown in Figure 1 to an LLM service [Figure 12] Explanatory diagram showing an example of similar FAQs sent from the FAQ aggregation unit to the LLM service [Figure 13] Explanatory diagram showing an example of similar FAQs sent from the FAQ aggregation unit to the LLM service [Figure 14] Explanatory diagram showing an example of similar FAQs sent from the FAQ aggregation unit to the LLM service [Figure 15] Explanatory diagram showing an example of aggregated FAQs output by the LLM service [Figure 16] Explanatory diagram showing an example of processing by the FAQ aggregation unit 27 [Figure 17] Explanatory diagram showing an example of aggregation result information displayed by the display unit shown in Figure 1 [Figure 18] Overall configuration diagram of a FAQ acquisition system according to the second embodiment [Figure 19] Flowchart showing the flow of FAQ acquisition processing by the FAQ acquisition system according to the second embodiment [Figure 20] Explanatory diagram showing a method for selecting a processing target FAQ according to the second embodiment [Figure 21] Explanatory diagram showing an example of a method for calculating a difference between an existing FAQ and a result of FAQ aggregation processing according to the second embodiment [Figure 22] Explanatory diagram showing an example of aggregation result information displayed by a display unit according to the second embodiment [Figure 23] Explanatory diagram showing an example of a FAQ aggregation prompt sent from a FAQ aggregation unit according to the second embodiment to an LLM service [Figure 24] This diagram illustrates an example of similar FAQs sent from the FAQ aggregation unit to the LLM service according to the second embodiment. [Figure 25] This diagram illustrates an example of similar FAQs sent from the FAQ aggregation unit to the LLM service according to the second embodiment. [Figure 26] This diagram illustrates an example of similar FAQs sent from the FAQ aggregation unit to the LLM service according to the second embodiment. [Modes for carrying out the invention]

[0016] The first invention made to solve the aforementioned problem is an FAQ acquisition method by an FAQ acquisition system that acquires FAQs including combinations of questions and answers, wherein information on the response to be processed is input into a large-scale language model, and based on the output of the large-scale language model, FAQs related to the response are acquired as FAQs to be processed, the FAQs to be processed are registered in an FAQ search database, one or more FAQs containing questions similar to the questions in the FAQs to be processed are extracted as similar FAQs from among multiple existing FAQs related to the response registered in the FAQ search database, and the similar FAQs are input into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model.

[0017] According to this, when creating FAQs, it becomes possible to reduce the workload of FAQ creators while obtaining highly convenient FAQs (i.e., aggregated FAQs) that consolidate similar FAQs.

[0018] Furthermore, the second invention is configured such that the multiple FAQs registered in the FAQ search database are managed by an index, and similar FAQs are extracted from the multiple FAQs based on the index.

[0019] This makes it easier to search for similar FAQs in the FAQ search database.

[0020] Furthermore, the third invention is configured such that the index is associated with a vector obtained by transforming the corresponding FAQ question, and questions similar to the FAQ question to be processed are determined based on the similarity between the vector obtained by transforming the FAQ question to be processed and the vectors obtained by transforming each of the multiple FAQs.

[0021] According to this, similar FAQs containing questions similar to the FAQ being processed can be easily obtained.

[0022] Furthermore, the fourth invention is configured such that questions similar to the questions in the FAQ to be processed are determined based on an evaluation of the semantic relationship between the questions in the FAQ to be processed and the plurality of FAQs.

[0023] According to this, similar FAQs containing questions similar to the FAQ being processed can be easily obtained.

[0024] Furthermore, the fifth invention is configured such that the aggregated FAQs are obtained by deleting some of the similar FAQs so that they do not overlap with each other.

[0025] According to this, it is possible to easily obtain highly convenient FAQs that aggregate similar FAQs.

[0026] Furthermore, the sixth invention is that the aggregated FAQ is obtained by combining at least some of the similar FAQs with one another.

[0027] According to this method, it is possible to obtain a highly convenient FAQ that aggregates similar FAQs while suppressing the loss of information contained in the FAQ being processed and similar FAQs.

[0028] Furthermore, the seventh invention is configured to display aggregated result information regarding the aggregated FAQs on a display, and the aggregated result information includes the number of similar FAQs that were deleted during the acquisition of the aggregated FAQs.

[0029] According to this, FAQ creators will be able to easily grasp the status of FAQ aggregation through aggregated results information.

[0030] Furthermore, the eighth invention is configured such that the information regarding the response to be processed, which is input to the large-scale language model, includes text data obtained by converting the voice data of the response or text data input in the response.

[0031] According to this, information regarding customer service can be easily obtained.

[0032] Furthermore, the ninth invention is configured such that if the handling of the subject to processing includes a telephone conversation, and the telephone conversation includes at least one of the following: a conversation by a new operator, a wrong number, and a prank call, then the FAQs of the subject to processing related to that conversation are excluded from the subject for obtaining the aggregated FAQs.

[0033] According to this, information regarding customer service can be appropriately obtained.

[0034] Furthermore, the tenth invention is an FAQ acquisition system including a processor that performs a process to acquire FAQs including combinations of questions and answers, wherein the processor inputs information about the response to be processed into a large-scale language model, and based on the output of the large-scale language model, acquires FAQs related to the response as FAQs to be processed, registers the FAQs to be processed into an FAQ search database, extracts one or more FAQs from among a plurality of existing FAQs related to the response registered in the FAQ search database that include questions similar to the questions in the FAQs to be processed as similar FAQs, and inputs the similar FAQs into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model.

[0035] According to this, when creating FAQs, it becomes possible to reduce the workload of FAQ creators while obtaining highly convenient FAQs (i.e., aggregated FAQs) that consolidate similar FAQs.

[0036] Furthermore, the 11th invention is an FAQ acquisition program for performing a process to acquire FAQs including combinations of questions and answers, wherein the program inputs information about the response to be processed into a large-scale language model, and based on the output of the large-scale language model, the program acquires FAQs related to the response as FAQs to be processed, registers the FAQs to be processed into an FAQ search database, extracts one or more FAQs from among multiple existing FAQs related to the response registered in the FAQ search database that contain questions similar to the questions in the FAQs to be processed as similar FAQs, and inputs the similar FAQs into the large-scale language model, thereby acquiring aggregated FAQs based on the output of the large-scale language model.

[0037] According to this, when creating FAQs, it becomes possible to reduce the workload of FAQ creators while obtaining highly convenient FAQs (i.e., aggregated FAQs) that consolidate similar FAQs.

[0038] The embodiments of this disclosure will be described below with reference to the drawings.

[0039] (First Embodiment) As shown in Figure 1, the FAQ (Frequently Asked Questions) acquisition system 1 according to the first embodiment includes a PBX (Private Branch Exchange) 2, a speech recognition service 3, an LLM (Large Language Models) service 4, an embedded model service 5, a search service 6, a main control unit 7, and an aggregated FAQ database (DB) 8.

[0040] The FAQ acquisition system 1 acquires (or generates) FAQs from audio recordings of conversations containing questions and their answers. In this embodiment, the FAQ acquisition system 1 can acquire FAQs based on audio recordings of operator interactions in call centers, service desks, etc. Operator interactions include conversations exchanged between users (i.e., customers, clients, etc.) and operators during telephone inquiries.

[0041] PBX2 is a cloud PBX, for example, installed on the cloud, and can manage and control telephone calls between users and operators. For example, PBX2 acquires call audio and related information (hereinafter referred to as "call information") from operator interactions and provides them to the main control unit 7 as appropriate.

[0042] PBX2 has a call audio storage unit 11 and a call information storage unit 12, which consist of storage and other components. The call audio storage unit 11 sequentially stores call audio data for each interaction by multiple operators. The call audio data includes audio data in a known file format (e.g., WAVE format). The call information storage unit 12 sequentially stores call information associated with each call audio data. The call information includes the call duration (conversation time) and operator name (or operator identification information) for each interaction.

[0043] Furthermore, the acquisition of FAQs by FAQ Acquisition System 1 is not limited to conversations via telephone using PBX2, IP lines, or fiber optic lines (i.e., call audio). For example, FAQ Acquisition System 1 may acquire FAQs based on the results of interactions including direct dialogue between the user and the operator (e.g., recorded dialogue audio, call audio other than telephone). FAQs may also be acquired based on conversation text such as chat or email between the user and the operator. In this case, since the text has already been transcribed, it does not need to be input into the speech recognition service.

[0044] The speech recognition service 3 can convert call audio data into text data based on a known speech recognition model. For example, the speech recognition service 3 performs the process of converting call audio into speech-to-text in response to a request from the main control unit 7. The speech recognition service 3 can be implemented using, for example, cloud services utilizing Whisper®, Amazon Transcribe, Deepgram, etc., or a computer equipped with speech recognition software such as the open-source software Whisper®.

[0045] LLM Service 4 can perform natural language analysis and generation by utilizing large datasets and language models built using deep learning. For example, LLM Service 4 performs processes such as text extraction (or summarization), FAQ creation, and aggregation of the created FAQs in response to requests from the main control unit 7. LLM Service 4 can be implemented using cloud services that utilize well-known large-scale language models such as ChatGPT®, Google Gemini®, and Claude, or computers equipped with large-scale language models such as Llama.

[0046] The embedded model service 5 can vectorize text data using known embedding techniques. For example, in response to a request from the main control unit 7, the embedded model service 5 performs the process of converting the text of questions and answers that constitute the FAQ into vectors. The embedded model service 5 can be implemented, for example, by a cloud service using text-embedding-3-large or by a computer equipped with an embedded model such as Multilingual-E5-large.

[0047] The search service 6 utilizes RAG (Retrieval-Augmented Generation) as an information retrieval technology to search for relevant text and data based on queries consisting of questions and search statements for information retrieval. For example, the search service 6 searches for FAQs (text) similar to the FAQ (text) being processed in response to a request from the main control unit 7. The search service 6 is implemented by a computer equipped with cloud services such as Azure AI Search or software such as LlamaIndex or Elasticsearch.

[0048] The search service 6 includes an indexing / search engine 15 and an FAQ search database (DB) 16. The indexing / search engine 15 can create an index for RAG using vectors (hereinafter referred to as "embedded vectors") generated by the embedded model service 5. The embedded vectors are used to enable efficient searching of FAQs and are stored in the index of the FAQ search database 16. In other words, each FAQ created in the FAQ acquisition system 1 is managed by the index of the FAQ search database 16.

[0049] The main control unit 7 includes a voice acquisition unit 21, a voice recognition unit 22, a text extraction unit 23, an FAQ creation unit 24, an index creation unit 25, a query search unit 26, an FAQ aggregation unit 27, and a display unit 28. The main control unit 7 is configured to communicate with the PBX 2, the voice recognition service 3, the LLM service 4, the embedded model service 5, the search service 6, and the aggregated FAQ database 8 via a known communication network (e.g., the Internet or a local network).

[0050] The voice acquisition unit 21 sequentially receives call voice data input from the PBX2. The input call voice data is call voice data stored in the call voice storage unit 11. Each call voice data corresponds to the voice in a single call between the user and the operator (for example, one call communication between the user's telephone and the operator's terminal).

[0051] The speech recognition unit 22 uses the speech recognition service 3 to perform speech recognition processing on the call audio data (related to a single call) input to the voice acquisition unit 21.

[0052] For example, when a request for speech recognition is made to the speech recognition service 3, the speech recognition unit 22 can send dictionary data useful for speech recognition processing to the speech recognition service 3 in addition to the call audio data to be processed. The dictionary data includes data on frequently used terms, such as the names of service counters that operators introduce themselves to (e.g., "○○ Support Desk," "○○ Repair Reception"). As a result of its speech recognition processing, the speech recognition unit 22 can obtain the call audio converted into text (hereinafter referred to as "voice text").

[0053] The text extraction unit 23 uses the LLM service 4 to perform a process (hereinafter referred to as "text extraction process") to extract text suitable for creating FAQs from the speech text acquired by the speech recognition unit 22. The text extraction unit 23 can perform this text extraction process using call information acquired from the PBX 2. As a result of the text extraction process, the text extraction unit 23 can acquire only text suitable for creating FAQs (hereinafter referred to as "selected text").

[0054] The FAQ creation unit 24 uses the LLM service 4 to perform a process (hereinafter referred to as the "FAQ creation process") to create an FAQ from the selected text (an example of information related to the response to be processed) obtained by the text extraction unit 23. As a result of the FAQ creation process, the text extraction unit 23 can obtain FAQ text data that includes the question and its answer. The FAQ data created by the FAQ creation unit 24 is sequentially stored in the FAQ storage unit 30 and used for subsequent processing.

[0055] The index creation unit 25 uses the embedding model service 5 to perform a process (hereinafter referred to as the "embedded information generation process") that generates embedded information from the FAQ (text) created by the FAQ creation unit 24. The embedded information includes embedded vectors and their additional information. As a result of the embedded information generation process, the index creation unit 25 can obtain the FAQ and its corresponding embedded information.

[0056] Furthermore, the index creation unit 25 uses the search service 6 to create an index for RAG using its FAQs and embedded information. As a result, the embedded vectors associated with each FAQ are stored in the index of the FAQ search database 16. Alternatively, the search service 6 may directly use the embedded model service 5 to create an index for RAG.

[0057] The query search unit 26 retrieves the FAQ generated by the FAQ creation unit 24. Furthermore, the query search unit 26 performs a process to extract FAQs similar to the generated FAQ (hereinafter referred to as "similar FAQs") (hereinafter referred to as "similar FAQ extraction process") for each generated FAQ. Here, the generated FAQ is, for example, an FAQ generated from selected text or voice text (an example of information about the interaction to be processed) based on call audio data obtained from one or more calls (interactions). As will be described later, the FAQ selection unit 31 can sequentially select the FAQs to be processed for the similar FAQ extraction process from the FAQs generated by the FAQ creation unit 24. The FAQ to be processed may be a single FAQ selected by the administrator of the FAQ acquisition system 1, or it may be any single FAQ.

[0058] The process of extracting similar FAQs is performed, for example, as follows:

[0059] First, the FAQ selection unit 31 selects an FAQ to be processed from among the FAQs stored in the FAQ storage unit 30.

[0060] The query search unit 26 generates embedded information using the FAQ (text) to be processed obtained from the FAQ creation unit 24 by performing the same embedded information generation process as described above. Note that the embedded information generation process may use only the text corresponding to the questions included in the FAQ to be processed.

[0061] Subsequently, the query search unit 26 uses the search service 6 to search the FAQ search database 16 for similar FAQs to the FAQ to be processed. This search for similar FAQs uses the embedding vector of the FAQ to be processed and the embedding vector associated with each FAQ stored in the FAQ search database 16. The query search unit 26 can obtain the extracted similar FAQs as a result of the search. Typically, the similarity to the FAQ to be processed is the highest, so the extracted similar FAQs will also include the FAQ to be processed itself that exists in the FAQ search database 16.

[0062] In the example above, the similar FAQ extraction process was shown to be performed using embedding vectors, but the similar FAQ extraction process is not limited to using such vectors. For example, a semantic ranking method (an example of semantic relevance evaluation) may be used to score the semantic relevance between FAQs (or between the questions contained within them) from embedding vectors or keywords, and similar FAQs may be extracted based on the obtained scores. In this case, only the top N FAQs with scores above a threshold may be extracted as similar FAQs. This allows for the exclusion of FAQs that are not very similar in advance, thereby reducing the computational cost.

[0063] Furthermore, the aforementioned keywords refer to "words" that, when extracted, are judged to have a high degree of similarity if they are found in similar FAQs. For example, product names and model numbers. For example, the text extraction unit 23 obtains product names and model numbers by instructing the LLM service 4 to extract them from the audio text. The index creation unit 25 uses these product names and model numbers as keywords along with embedded information to create an index for RAG through the search service 6.

[0064] Furthermore, if a user's inquiry concerns an electrical device they have purchased, the model number of that device could serve as such a keyword. For example, if a user registers their phone number, email address, and the model number of an electrical device with the manufacturer when they purchase it, the manufacturer and the call center can share this information, allowing it to be registered in the customer database. If the PBX2 shown in Figure 1 includes a customer database, when a call from a user is received, the model number of the electrical device owned by the user can be identified from the user's phone number. Thus, keywords can include information that is not contained in the audio of the call.

[0065] The FAQ aggregation unit 27 uses the LLM service 4 to perform a process (hereinafter referred to as "FAQ aggregation process") to aggregate one or more similar FAQs for the FAQ to be processed obtained from the query search unit 26. However, if there is only one similar FAQ, including the FAQ to be processed, that is, if there are no other FAQs similar to the FAQ to be processed, aggregation is not necessary, and that single similar FAQ is output as is.

[0066] The aggregated FAQs (i.e., one or more aggregated FAQs) are sequentially stored in the aggregated FAQ database 8. The FAQ aggregation unit 27 may also transmit the FAQ information stored in the aggregated FAQ database 8 to the index creation unit 25 and the search service 6. This allows the search service 6 to be used as part of a chatbot that searches for FAQs registered in the FAQ search database 16.

[0067] The display unit 28 can output the results of the FAQ aggregation process performed by the FAQ aggregation unit 27 as tables or graphs using known visualization methods. For example, the administrator of the FAQ acquisition system 1 can view the results of the FAQ aggregation process from an external terminal using a general-purpose web browser based on the output from the display unit 28 (for example, by displaying them on a screen).

[0068] At least some of the functions of the main control unit 7 described above can be realized, for example, in a cloud environment or on-premises environment where multiple processing units work in cooperation. In a cloud environment, the functions of the main control unit 7 can be realized by a predetermined control program (an example being an FAQ acquisition program). Alternatively, the main control unit 7 may be realized by a distributed processing system or a virtual server. In such a case, the main control unit 7 may be composed of a distributed system including servers, data centers, storage, and network equipment, each connected by a known communication network.

[0069] Alternatively, at least some of the functions of each section 21-28 in the main control unit 7 can also be realized, for example, by one or more processors in one or more servers (e.g., edge servers) executing a predetermined control program (an example of an FAQ acquisition program).

[0070] Next, referring to Figure 2, the FAQ acquisition process by the FAQ acquisition system 1 shown in Figure 1 will be explained.

[0071] In the main control unit 7 of the FAQ acquisition system 1, first, the voice acquisition unit 21 acquires call voice data for one interaction (i.e., one call) from the PBX 2 (ST101).

[0072] Next, the speech recognition unit 22 requests the speech recognition service 3 to perform speech recognition processing on the call audio data (ST102). At this time, the speech recognition unit 22 transmits one call audio data to the speech recognition service 3.

[0073] When the speech recognition service 3 receives a request for speech recognition processing, it generates speech text as shown in Figure 3 by converting the call audio data, and transmits it to the speech recognition unit 22.

[0074] Next, the text extraction unit 23 requests the LLM service 4 to perform text extraction processing on the audio text (ST103). At this time, the text extraction unit 23 sends prompts (instructions) regarding the text extraction processing to the LLM service 4 along with the audio text data. These prompts may include instructions such as, "Please extract the name of the operator's contact person," "Please determine whether this is a wrong number or a call to the wrong department," or "Please determine whether this is a prank call." For example, the prompt shown in Figure 4 is sent to the LLM service 4 along with the audio text data shown in Figure 3.

[0075] When the LLM service 4 receives a request for text extraction processing, it generates a response based on the prompt, which will be information for extracting selected text suitable for creating FAQs from the audio text, and sends it to the text extraction unit 23. For example, if it receives a prompt like the one in Figure 4 along with the audio text data in Figure 3, it generates response information (analysis information) like the one in Figure 5.

[0076] Furthermore, the text extraction unit 23 can exclude text related to conversations unsuitable for creating FAQs from the text extraction process, based on the corresponding call information obtained from the PBX2 and the analysis information generated by the LLM service 4.

[0077] Inappropriate conversations include, for example, conversations by new operators, wrong numbers, prank calls, and conversations that are relatively short (e.g., less than 10 seconds). The text extraction unit 23 can determine, for example, whether a conversation was made by a new operator based on the operator identification information included in the call information. The text extraction unit 23 can also determine, for example, whether a conversation was made by a wrong number (e.g., LLM service 4 responds "wrong number": "yes") based on information generated by LLM service 4.

[0078] Furthermore, the text extraction unit 23 can also select to extract only the voice text related to the call audio of an operator with a specific service counter name (e.g., "○○ Support Desk") based on the corresponding call information or analysis information generated by the LLM service 4. In this case, service counter names other than that (e.g., "○○ Repair Reception") are excluded from the extraction.

[0079] Next, the FAQ creation unit 24 requests the LLM service 4 to execute the FAQ creation process for the selected text (ST104). At this time, the FAQ creation unit 24 sends the data for the selected text along with a prompt for the FAQ creation process (hereinafter referred to as the "FAQ creation prompt") to the LLM service 4.

[0080] The prompt for creating an FAQ, as shown in Figure 6 for example, includes instructions for creating the FAQ, as well as information such as the origin of the selected text (in this case, that it is a transcript from a call center) and the purpose of creating the FAQ (in this case, to hand over duties to new operators).

[0081] When the LLM service 4 receives a request for FAQ creation, it creates an FAQ from the selected text and sends it to the FAQ creation unit 24. The created FAQ includes text related to the user's question and text related to the operator's answer, as shown in Figure 7, for example. Figure 7 shows an example where an operator responds to a user's question about how to set up the phone with an answer that includes the procedure for checking the phone's settings.

[0082] Next, the index creation unit 25 executes the process of creating an index related to the FAQ (hereinafter referred to as the "index creation process") (ST105).

[0083] In the index creation process, first, the index creation unit 25 requests the embedding model service 5 to execute the embedding information generation process for the FAQ. Upon receiving this request, the embedding model service 5 generates embedding information, including embedding vectors, from the FAQ and sends it to the index creation unit 25. The index creation unit 25 then requests the search service 6 to create an index for the FAQ and the embedding information. Upon receiving this request, the search service 6 creates an index for RAG and stores the embedding vectors in the index of the FAQ search database 16.

[0084] Note that multiple FAQs may be generated for a single call audio file. In that case, embedded information will be generated for each FAQ. In other words, embedded information will be generated for each FAQ.

[0085] As a result, the FAQ search database 16 registers the FAQs that are subject to FAQ aggregation processing, as well as search information related to those FAQs. In addition, the FAQ search database 16 may register information for searching FAQs, such as the identification number of the FAQ and information about keywords contained in the FAQ, along with the embedding vector.

[0086] After that, the process returns to step ST101 and repeats until the indexing process for FAQs is completed for all voice messages received in PBX2 (which may be filtered by category and time period) (Yes in ST106).

[0087] The FAQ selection unit 31 selects the FAQs to be processed (ST107) using the FAQ group created by the FAQ creation unit 24 and stored in the FAQ storage unit 30, and the aggregated FAQ list output by the FAQ aggregation unit 27. At this time, from the QA numbers of the FAQ group created by the FAQ creation unit 24, the QA numbers that are not included in the aggregated FAQ list are selected in order to be processed.

[0088] For example, suppose the only QA number in the aggregated FAQ list is 20240604_125826. As shown in Figure 8, in the FAQ group created by the FAQ creation unit 24, based on the QA numbers (20240604_125826, 3612895_20240808091227, 20240716_142702), the QA numbers (3612895_20240808091227, 20240716_142702) that are not included in the aggregated FAQ list (20240604_125826) are selected as FAQs to be processed. In this case, the FAQ with the higher-ranking QA number (3612895_20240808091227) in Figure 8 is selected. If a consolidated FAQ list has not yet been created, any QA number may be selected from the FAQ group created by the FAQ creation unit 24. The ranking refers to, for example, the order in which the FAQs were created, and FAQs originating from more recent audio data may be given priority in selection.

[0089] Next, the query search unit 26 executes a similar FAQ extraction process (search for similar FAQs) related to the FAQ to be processed selected by the FAQ to be processed unit 31 (ST108). In the similar FAQ extraction process, as described above, the embedding model service 5 is requested to execute an embedding information generation process, thereby generating embedding information including embedding vectors from the questions included in the FAQ to be processed.

[0090] Therefore, the query search unit 26 requests the search service 6 to search for similar FAQs. At this time, the query search unit 26 sends the search service 6 an instruction to search for similar FAQs, along with information about the questions included in the FAQ to be processed (see Figure 9) and data such as its embedding vector.

[0091] When the search service 6 receives a request to search for similar FAQs, it extracts similar FAQs through the search and sends them to the query search unit 26. At this time, the search service 6 can also transmit information on the similarity (for example, cosine similarity between vectors) used to determine their similarity along with the similar FAQs. This similarity information includes, for example, the QA numbers of the similar FAQs, including the FAQ being processed, along with a score indicating the degree of similarity, as shown in Figure 10. As mentioned above, since the embedded information is created from the questions of the FAQ being processed, the degree of similarity with the FAQ being processed itself is usually the highest. Therefore, the extracted similar FAQs will also include the FAQ being processed itself, which exists in the FAQ search database 16. However, the similarity determination may be performed in a manner in which the FAQ being processed and similar FAQs are distinguished.

[0092] Next, the FAQ aggregation unit 27 requests the LLM service 4 to perform FAQ aggregation processing on similar FAQs, including the FAQs to be processed (ST109). At this time, the FAQ aggregation unit 27 sends a prompt for FAQ aggregation processing (hereinafter referred to as the "FAQ aggregation prompt") to the LLM service 4 along with the data of similar FAQs, including the FAQs to be processed. Furthermore, the FAQ aggregation unit 27 may also send the similarity information obtained by the query search unit 26 to the LLM service 4. Note that if there is only one similar FAQ, that is, if no similar FAQs other than the FAQ to be processed are found, there is no need to perform FAQ aggregation processing, and the single corresponding FAQ is output directly to the display unit 28.

[0093] The prompt for FAQ aggregation includes instructions on how to aggregate FAQs, as shown in Figure 11, for example, as well as the purpose for obtaining the aggregated FAQs (e.g., handing over tasks to new operators) and instructions for listing the source QA numbers (excluding QA numbers unrelated to aggregation).

[0094] The data of similar FAQs sent from the FAQ aggregation unit 27 to the LLM service 4 along with the FAQ aggregation prompt includes multiple combinations of QA numbers, questions, and answers, as shown in Figures 12, 13, and 14, for example.

[0095] The LLM service 4 aggregates similar FAQs that overlap (i.e., are roughly identical) with each other and outputs a list of the combined FAQs and the original QA numbers (QA numbers unrelated to the aggregation are not included) to the FAQ aggregation unit 27. For example, when similar FAQs like those in Figures 12-14 are input, the LLM service 4 outputs the aggregated FAQs along with a list of the original QA numbers (3612895_20240808091227, 20240716_142702) as shown in Figure 15. These are output to the display unit 28 and also to the aggregated FAQ database 8.

[0096] This allows the FAQ aggregation unit 27 to synthesize questions and answers from at least two similar FAQs. In the example in Figure 15, two FAQs are synthesized to produce one synthesized FAQ, but three or more FAQs may be synthesized to produce one FAQ.

[0097] Note that there are cases where the list of QA numbers used for aggregation does not include the QA number of the FAQ being processed. In such cases, FAQs with content different from the FAQ being processed are aggregated. In other words, the FAQ being processed was not aggregated. In this case, the aggregation result output by LLM Service 4 is discarded, and the FAQ being processed is output as is. At this time, the list of QA numbers used for aggregation will only contain the QA number of the FAQ being processed.

[0098] The FAQ aggregation unit 27 adds the list of source QA numbers output by the LLM service 4 to the aggregated FAQ list and outputs it to the FAQ selection unit for processing. For example, as shown in Figure 16, the list of source QA numbers (3612895_20240808091227, 20240716_142702) is added to the already existing aggregated FAQ list (20240604_125826, 3612895_20240808091227, 20240716_142702) and output as an aggregated FAQ list (20240604_125826, 3612895_20240808091227, 20240716_142702). This makes it possible to delete duplicate FAQs.

[0099] Subsequently, if there are still candidates for FAQs to be processed that the FAQ selection unit can select (the FAQ group created by the FAQ creation unit 24 excluding the aggregated FAQ list) remaining (Yes in ST110), the process returns to ST107 and the aggregation process is repeated. If there are no candidates remaining, the process proceeds to ST111.

[0100] Next, the display unit 28 outputs the results of the FAQ aggregation process performed by the FAQ aggregation unit 27 (ST111). The results of the FAQ aggregation process output by the display unit 28 include aggregation result information, for example, as shown in Figure 17. For example, the aggregation result information includes, for each aggregated FAQ, the QA number, question (text), answer (text), the QA numbers of duplicate FAQs (a list of QA numbers that were used as the source for aggregation, output by the LLM service 4), and the aggregation count. The aggregation count indicates the number of QA numbers in the QA number column for duplicate FAQs. Note that FAQs with a small aggregation count may be hidden if it is determined that they are not frequently asked by phone.

[0101] This type of FAQ retrieval process (or method) reduces (or eliminates) the input operations required by FAQ creators, thereby reducing their workload while enabling the acquisition of highly convenient FAQs that aggregate similar FAQs.

[0102] Furthermore, the LLM output may contain AI-specific hallucinations. Also, incorrect operator instructions may result in errors in the generated FAQs. Therefore, the FAQ output by the display unit 28 may be modified manually by the FAQ creator. The manually modified FAQs are output to the aggregated FAQ database 8, updating the database. This reduces the workload of the FAQ creator while enabling the creation of higher-quality FAQs.

[0103] (Second Embodiment) Next, the FAQ acquisition system 1 according to the second embodiment will be described with reference to Figures 18 to 24. Elements similar to those in the first embodiment are denoted by the same reference numerals and detailed descriptions are omitted. Furthermore, matters not specifically mentioned below regarding the FAQ acquisition system 1 according to the second embodiment are the same as those in the first embodiment.

[0104] The first embodiment described a configuration for creating a new FAQ when no existing FAQs exist. On the other hand, there are cases where, after creating an FAQ as in the first embodiment, the accumulated call audio data increases and it becomes necessary to update the FAQ. There are also cases where it is necessary to update an FAQ created manually or by another system using call audio data.

[0105] In the FAQ acquisition system 1 according to the second embodiment, in such cases, a highly convenient FAQ can be created by aggregating existing FAQs and newly created FAQs from new call audio data.

[0106] As shown in Figure 19, in the FAQ acquisition process according to the second embodiment, the index creation unit 25 executes the process of creating an index (hereinafter referred to as the "existing index creation process") with respect to the existing FAQs stored in the existing FAQ storage unit 32 (ST201).

[0107] Furthermore, if the existing FAQs are FAQs created according to the first embodiment (i.e., FAQs obtained from the FAQ storage unit 30), the existing index may be reused for FAQs for which an index has already been created, and the process of creating an existing index may be skipped.

[0108] In the existing index creation process, first, the index creation unit 25 requests the embedding model service 5 to execute an embedding information generation process for each existing FAQ. Upon receiving this request, the embedding model service 5 generates embedding information, including embedding vectors, from that FAQ and sends it to the index creation unit 25. The index creation unit 25 then requests the search service 6 to create an index for that FAQ and embedding information. Upon receiving this request, the search service 6 creates an index for RAG and stores the embedding vectors in the index of the FAQ search database 16.

[0109] As a result, the FAQ search database 16 registers existing FAQs that are subject to FAQ aggregation processing, as well as search information related to those FAQs. In addition, the FAQ search database 16 may register information for searching FAQs, such as the identification number of the FAQ and information about keywords contained in the FAQ, along with the embedding vector.

[0110] After that, the process returns to step ST201 and repeats until the existing index creation process is completed for all existing FAQs (Yes in ST202).

[0111] The FAQ selection unit 31 selects the FAQs to be processed (ST203) using the existing FAQs stored in the existing FAQ storage unit 32, the new FAQs created by the FAQ creation unit 24 and stored in the FAQ storage unit 30, and the aggregated FAQ list output by the FAQ aggregation unit 27. The FAQ selection unit 31 sequentially selects the QA numbers from the existing FAQs stored in the existing FAQ storage unit 32 and the FAQs created by the FAQ creation unit 24 that are not included in the aggregated FAQ list, as the FAQs to be processed. In this case, existing FAQs are selected preferentially. This allows for the aggregation of high-quality existing FAQs that have been manually created or verified.

[0112] For example, suppose the only QA number in the aggregated FAQ list is EX00001. The FAQ selection unit 31, for example as shown in Figure 20, selects an FAQ to be processed from among the QA numbers (EX00002, 2025_0101_001, 2025_0101_002) that are not included in the aggregated FAQ list (EX00001), based on (A) the QA numbers of the existing FAQ group stored in the existing FAQ storage unit (EX00001, EX00002) and (B) the QA numbers of the FAQ group created by the FAQ creation unit 24 (2025_0101_001, 2025_0101_002). In this FAQ selection, existing FAQs take precedence. In this case, the FAQ with the highest priority QA number, EX00002, is selected.

[0113] If a consolidated FAQ list has not yet been created, you may select any QA number from the existing FAQs stored in the existing FAQ storage unit and the FAQs created by the FAQ creation unit 24, prioritizing the existing FAQs.

[0114] Next, steps ST204 and ST205, which are similar to steps ST107 and ST108 shown in Figure 2, are executed. The FAQ aggregation unit 27 requests the LLM service 4 to perform FAQ aggregation processing using similar FAQs for the FAQs to be processed. At this time, the FAQ aggregation unit 27 sends data of similar FAQs (data of the FAQs to be processed and similar FAQs other than the target) to the LLM service 4 along with an FAQ aggregation prompt.

[0115] Subsequently, if there are still candidates for FAQs to be processed that the FAQ selection unit can select (existing FAQs stored in the existing FAQ storage unit and FAQs created by the FAQ creation unit 24, excluding the aggregated FAQ list) remaining (Yes in ST206), the process returns to ST203 and repeats the aggregation process. If there are no candidates remaining, the process proceeds to ST207.

[0116] Existing FAQs are of high quality because they are created or reviewed manually and are actually used. Therefore, highlighting changes from existing FAQs by changing the font color or making them bold can reduce the workload on FAQ creators in reviewing AI hallucination.

[0117] Therefore, the display unit 28 calculates the difference (i.e., the changes) between the existing FAQs stored in the existing FAQ storage unit and the results of the FAQ aggregation process performed by the FAQ aggregation unit 27 (ST207). In the example shown in Figure 21, the difference between (A) the FAQ (QA number EX00002) in the FAQ group stored in the existing FAQ storage unit and (B) the corresponding FAQ in the results of the FAQ aggregation process performed by the FAQ aggregation unit 27 is calculated. As a result, the display unit detects the text difference "Please update if your version is ABC01." in the existing FAQ stored in the existing FAQ storage unit and displays it with emphasis.

[0118] Next, the display unit 28 outputs the result of the FAQ aggregation process performed by the FAQ aggregation unit 27 (ST208). At this time, the difference between the existing FAQs stored in the existing FAQ storage unit and the result of the FAQ aggregation process performed by the FAQ aggregation unit 27 is highlighted by changing the font color, changing from thin to bold, adding an underline, etc.

[0119] The results of the FAQ aggregation process output by the display unit 28 include aggregation result information, such as that shown in Figure 22. For example, the aggregation result information includes, for each aggregated FAQ, the QA number, question (text), answer (text), the QA number of the duplicate FAQ (a list of the QA numbers that served as the source for aggregation output by the LLM service 4), and the aggregation count. The aggregation count indicates the number of QA numbers in the QA number column for the duplicate FAQ. If the existing FAQ was created in the first embodiment, the aggregation count may be increased by the aggregation count in the first embodiment. Note that FAQs with a small aggregation count may be hidden if it is determined that they are not frequently asked about by phone.

[0120] This type of FAQ acquisition process (or method) allows for the addition and updating of FAQs based on existing ones, thus reducing (or eliminating) the input work required from FAQ creators. Therefore, it becomes possible to acquire highly convenient FAQs that aggregate similar FAQs while reducing the workload of FAQ creators.

[0121] Furthermore, the FAQ aggregation unit 27 may transmit the similarity information obtained by the query search unit 26 to the LLM service 4. Note that if the number of similar FAQs is 1, that is, if no similar FAQs are found other than the FAQ being processed, there is no need to perform the FAQ aggregation process, and the single corresponding FAQ is output directly to the display unit 28. (Modified version of the second embodiment)

[0122] Next, we will explain an example of aggregating other FAQs based on the FAQ being processed.

[0123] As mentioned above, existing FAQs are of high quality because they are created or reviewed manually and are actually used. Therefore, minimizing changes from existing FAQs will reduce the workload on FAQ creators. Alternatively, other FAQs can be consolidated based on FAQs that are prioritized for processing, which will further reduce the amount of changes from existing FAQs.

[0124] Here, we will omit the explanation of the configuration which is the same as in the second embodiment. The FAQ aggregation prompt includes, for example, instructions for aggregating FAQs based on the FAQ to be processed, as shown in Figure 23, as well as instructions for obtaining the aggregated FAQs (for handing over tasks to new operators) and instructions for listing the source QA numbers (excluding QA numbers unrelated to aggregation).

[0125] The LLM service 4 receives a prompt for FAQ aggregation, along with data for the FAQs to be processed and data for similar FAQs that are not being processed, distinguishing between them. For example, as shown in Figures 24, 25, and 26, it includes multiple combinations of QA numbers, questions, and answers.

[0126] The LLM service 4 outputs to the FAQ aggregation unit 27 a list of the combined FAQ and the QA numbers from which the aggregation was performed (excluding QA numbers unrelated to the aggregation), by aggregating similar FAQs other than the FAQ to be processed that overlap with the FAQ to be processed (i.e., are generally identical to the FAQ to be processed). For example, when data for the FAQ to be processed (3612895_20240808091227) and data for similar FAQs other than the target (20240509110113, 20240730_154144, 20240806_134821, 20240716_142702) are input, the aggregated FAQs and a list of the original QA numbers (3612895_20240808091227, 20240716_142702) are output, similar to Figure 15. In the second embodiment, aggregated FAQs were sometimes output based on similar FAQs other than the target FAQ (in this case, 20240716_142702), but this method ensures that FAQs are aggregated and output based on the target FAQ (in this case, 3612895_20240808091227).

[0127] As described above, embodiments have been explained as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these embodiments and can be applied to embodiments that have been modified, replaced, added, or omitted. Furthermore, it is possible to create new embodiments by combining the components described in the above embodiments. [Industrial applicability]

[0128] The FAQ acquisition method, FAQ acquisition system, and FAQ acquisition program related to this disclosure have the effect of reducing the workload of FAQ creators in creating FAQs while acquiring convenient FAQs that aggregate similar FAQs, and are useful as an FAQ acquisition method, FAQ acquisition system, and FAQ acquisition program that acquire FAQs that include combinations of questions and answers. [Explanation of symbols]

[0129] 1: FAQ Acquisition System 2: PBX 3: Voice recognition service 4: LLM Service 5: Embedded Model Service 6: Search Service 7: Main Control Unit 8: Aggregated FAQ Database 11: Call audio storage unit 12: Call information storage unit 15: Indexing and Search Engines 16: FAQ Search Database 21: Voice acquisition unit 22: Speech Recognition Unit 23: Text Extraction Unit 24:FAQ creation department 25: Index Creation Section 26: Query Search Section 27: FAQ Compilation Department 28: Display section 30:FAQ storage section 31: Selection section for FAQs to be processed 32: Existing FAQ Storage Section

Claims

1. A method for obtaining FAQs using an FAQ acquisition system that acquires FAQs including combinations of questions and their answers, By inputting information about the response to be processed into a large-scale language model, the FAQ related to the response is obtained as the FAQ to be processed based on the output of the large-scale language model. The FAQs to be processed are registered in the FAQ search database. From among the multiple FAQs related to existing responses registered in the FAQ search database, one or more FAQs containing questions similar to the question of the FAQ to be processed are extracted as similar FAQs. An FAQ acquisition method, which involves inputting the aforementioned similar FAQs into the large-scale language model, and then obtaining aggregated FAQs based on the output of the large-scale language model.

2. The multiple FAQs registered in the aforementioned FAQ search database are managed by an index. The FAQ acquisition method according to claim 1, wherein the similar FAQs are extracted from the plurality of FAQs based on the index.

3. The aforementioned index is associated with a vector obtained by transforming the corresponding FAQ question. The FAQ acquisition method according to claim 2, wherein a question similar to the question of the FAQ to be processed is determined based on the similarity between the vector obtained by transforming the question of the FAQ to be processed and the vector obtained by transforming each of the plurality of FAQs.

4. The FAQ acquisition method according to claim 1, wherein the question similar to the question of the FAQ to be processed is determined based on an evaluation of the semantic relationship between the question of the FAQ to be processed and the plurality of FAQs.

5. The FAQ acquisition method according to claim 1, wherein the aggregated FAQs are obtained by deleting some of the similar FAQs so that they do not overlap with each other.

6. The FAQ acquisition method according to claim 1, wherein the aggregated FAQ is obtained by combining at least some of the similar FAQs.

7. The aggregated information regarding the aforementioned aggregated FAQs is displayed on the screen. The FAQ acquisition method according to any one of claims 1 to 6, wherein the aggregated result information includes the number of similar FAQs that were deleted in the acquisition of the aggregated FAQs.

8. The FAQ acquisition method according to claim 1, wherein the information regarding the response to be processed that is input to the large-scale language model includes text data obtained by converting the voice data of the response or text data input in the response.

9. The aforementioned processing targets include telephone conversations, The FAQ acquisition method according to claim 1, wherein if the telephone conversation includes at least one of a conversation by a new operator, a wrong number, and a prank call, the FAQs to be processed related to that conversation are excluded from the target for acquiring the aggregated FAQs.

10. An FAQ acquisition system including a processor that performs the process of acquiring an FAQ that includes a combination of questions and their answers, The aforementioned processor, By inputting information about the response to be processed into a large-scale language model, the FAQ related to the response is obtained as the FAQ to be processed based on the output of the large-scale language model. The FAQs to be processed are registered in the FAQ search database. From among the multiple FAQs related to existing responses registered in the FAQ search database, one or more FAQs containing questions similar to the question of the FAQ to be processed are extracted as similar FAQs. An FAQ acquisition system that inputs the aforementioned similar FAQs into the large-scale language model, and acquires aggregated FAQs based on the output of the large-scale language model.

11. An FAQ retrieval program for performing the process of obtaining an FAQ that includes a combination of questions and their answers, On the computer, By inputting information about the response to be processed into a large-scale language model, the FAQ related to the response is obtained as the FAQ to be processed based on the output of the large-scale language model. The FAQ to be processed is registered in the FAQ search database. From among multiple FAQs related to existing responses registered in the FAQ search database, one or more FAQs containing questions similar to the question of the FAQ to be processed are extracted as similar FAQs. An FAQ acquisition program that inputs the aforementioned similar FAQs into the large-scale language model, and then obtains aggregated FAQs based on the output of the large-scale language model.

Images