System, method, and program for generating a response to a user's voice

Abstract

To obtain a voice response system capable of outputting fast and accurate answers. [Solution] The system for generating a response to a user's speech of the present invention comprises an identification means for identifying at least one element contained in the speech by analyzing the speech, a storage means for storing information about the user, and a generation means for generating a response to the speech based on the at least one element, wherein the generation means has a language model for generating text, and the language model is capable of outputting a confidence level for the text along with the text.

Description

[Technical Field]

[0001] The present invention relates to a system, method, and program for generating a response to a user's speech. [Background technology]

[0002] Currently, there are several personal AI assistants on the market (such as Apple Siri, Google Assistant, and Amazon Alexa) (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2024-519261 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a voice response system that can output fast and accurate responses. [Means for solving the problem]

[0005] The present invention provides a system that includes a language model for text generation as well as a storage means for storing user data, and generates answers by relying on information from the storage means when the accuracy of the output from the language model is low. The present invention provides, for example, the following items. (Item 1) 1. A system for generating a response to a user's speech, the system comprising: an identification means for identifying at least one element contained in the audio by analyzing the audio; a storage means for storing information about the user; A generating means for generating an answer to the speech based on the at least one element, the generating means including a language model for generating text, the language model being capable of outputting a confidence level for the text together with the text, and the generating means for generating an answer to the speech based on the at least one element, the generating text for said element; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from the storage means; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A generating means A system comprising: (Item 2) Determining a first similarity between the acquired information and the element includes: vectorizing the obtained information to generate a first vector; generating a reference vector by vectorizing the elements; determining a similarity between the first vector and the reference vector; The system according to the preceding item, comprising: (Item 3) The information processing system further includes a search unit that can connect to an external information source and searches for information from the external information source, and the generating unit if the first similarity is less than the first similarity threshold, searching the external information source for information related to the element; determining a second similarity between the retrieved information and the element; determining whether the second similarity is greater than a second similarity threshold; adopting the retrieved information if the second similarity is higher than the second similarity threshold; abandoning generating the answer if the second similarity is lower than the second similarity threshold; The system according to any one of the preceding items, further comprising: (Item 4) Determining a second similarity between the retrieved information and the element includes: vectorizing the retrieved information to generate a second vector; generating a reference vector by vectorizing the elements; determining a similarity between the second vector and the reference vector; 2. The system according to claim 1, further comprising: (Item 5) The system described in any one of the preceding items further comprises an update means for updating the storage means with the acquired information when it is determined that the first similarity is lower than the first similarity threshold. (Item 6) The system described in any one of the preceding items further includes an update means for updating the storage means with the acquired information when it is determined that the first similarity is lower than the first similarity threshold, wherein the update means updates the storage means with the searched information for which it is determined that the second similarity is higher than the second similarity threshold. (Item 7) The system according to any one of the preceding items, wherein the storage means stores a dialogue log with the user. (Item 8) 2. The system of claim 1, wherein the storage means stores the user's information in a structured manner, and the external information source stores the information in an unstructured manner. (Item 9) 1. A method for generating a response to a user's speech, the method comprising: analyzing the audio to identify at least one element contained in the audio; generating a response to the speech based on the at least one element, the generating including, for each element of the at least one element: generating text for the element using a language model for generating text, the language model being capable of outputting a confidence level for the text along with the text; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from a storage means for storing information about the user; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A method comprising: (Item 9A) Item 10. The method according to item 9, comprising the features according to any one of the preceding items. (Item 10) A program for generating a response to a user's voice, the program being executed on a computer having a processor, the program comprising: analyzing the audio to identify at least one element contained in the audio; generating a response to the speech based on the at least one element, the generating including, for each element of the at least one element: generating text for the element using a language model for generating text, the language model being capable of outputting a confidence level for the text along with the text; generating text for said element; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from a storage means for storing information about the user; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A program that causes the processor to execute a process including the steps of: (Item 10A) Item 11. A program according to item 10, comprising the features according to any one of the preceding items. (Item 10B) A computer-readable storage medium storing the program according to item 10A or 10B. [Effects of the Invention]

[0006] The present invention provides a voice response system capable of outputting fast and accurate responses, which has the advantage of ensuring the accuracy of responses based on information search while maintaining fast real-time performance, and eliminating inconsistencies and variations in quality in the output. [Brief explanation of the drawings]

[0007] [Figure 1] A diagram showing an example of a flow for obtaining a response from a voice response system. [Figure 2] FIG. 1 shows an example of the configuration of a system 100 for generating a response to a user's voice. [Figure 3A] FIG. 1 is a diagram showing an example of a specific configuration of a system 100 for generating a response to a user's voice. [Figure 3B]FIG. 1 illustrates an example of an alternative embodiment of a system 100 for generating a response to a user's speech. [Figure 4] FIG. 1 illustrates an example of a configuration for implementing the system 100 for generating a response to a user's speech. [Figure 5] 1 is a flowchart illustrating an example process (process 500) by system 100 for generating a response to a speech by a user. [Figure 6] Flowchart showing an example of the process executed in step S502 DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0009] 1. Voice response agent The inventor of the present invention has attempted to speed up responses from voice response systems such as personal AI assistants. However, when pursuing faster response speeds, the information search process becomes a bottleneck, limiting the speed of response. When attempting to address this issue by omitting the information search process, a problem arises in which inconsistencies arise in the response content, such as answers that are not factual. Thus, the inventor of the present invention has faced the challenge of difficulty in achieving a voice response system that can output fast and accurate answers.

[0010] As a result of extensive research, the inventors of the present invention have developed a system that overcomes this difficulty and dramatically improves response speed while ensuring the accuracy of answers. This system utilizes SLMs (small language models) that enable high-speed responses and a database that stores user data. The database is continuously updated with information obtained from the dialogue log between the user and the system, so that the latest user information can always be maintained. Alternatively, the database may be continuously updated with information (especially personal information) entered by the user into the user's terminal device.

[0011] Generating an initial answer from SLM ensures a fast response time, but if the confidence level of the answer from SLM is low, information is retrieved from the database to ensure the accuracy of the answer. Furthermore, if it is determined that even the information from the database is inaccurate, the accuracy of the answer is ensured by generating an answer using LLM (large-scale language model) and even RAG (search expansion generation). Alternatively, RAG can be used to supplement the answer or expand the query to enable re-search, making the answers from SLM and the database more detailed. If the confidence level of the answer is still low even after using LLM and RAG, the system decides not to generate an answer, avoiding the response itself with an inaccurate answer.

[0012] This system can generate answers to users with a reaction speed equal to or faster than that of a human (for example, between 2 seconds and 0.2 seconds, preferably 0.2 seconds or less), and can achieve 100% accuracy. This is because if there is no confidence in the accuracy of the answer, no answer is generated, and the answer accuracy is not reduced.

[0013] FIG. 1 shows an example of a flow for obtaining a response from a voice response system.

[0014] In this example, the user U uses a smart speaker 10 that can realize the above-described mechanism.

[0015] In step S1, the user U inputs voice into the smart speaker 10.

[0016] At this time, the user U can activate the smart speaker 10 by vocally inputting a wake word. However, a wake word is not necessarily required, and the smart speaker 10 may be activated by any other means.

[0017] For example, the user U can input general questions such as "What's the weather today?" by voice, and can also input personal questions such as "What's the zip code of my workplace?". For example, the user U can also input company-specific questions such as "What's the stock of product A?", "Tell me about recent sales," and "What are the PC setup procedures for new employees?". In this case, the user of the system can be considered to be the company, not the user U. When the smart speaker 10 receives these inputs, it will generate answers to these inputs.

[0018] The above-mentioned smart speaker 10 can be provided with a system 100 for generating a response to the voice of the user U, which will be described later, and the response can be generated by the system 100.

[0019] In step S2, the generated answer is output as voice from the smart speaker 10. The voice from the smart speaker 10 can be output with a reaction speed equal to or faster than that of a human (for example, between 2 seconds and 0.2 seconds, preferably 0.2 seconds or less) after the user inputs the question by voice in step S1. This makes it possible to realize a natural conversational experience close to that of a human conversation.

[0020] Furthermore, the system 100 proactively evaluates the reliability or accuracy of answers and does not output answers with low reliability or accuracy, thereby actively preventing "plausible lies" and ensuring the reliability of information.

[0021] Furthermore, if the reliability or accuracy of the generated answer is evaluated as low, the information in the database from which the answer was generated can be updated with more accurate information, thereby enabling autonomous performance improvement.

[0022] The system 100 for generating a response to a user's voice may have, for example, the following configuration.

[0023] 2. System configuration for generating responses to user voice FIG. 2 shows an example of the configuration of a system 100 for generating a response to a user's voice.

[0024] A system 100 for generating a response to a user's voice may be implemented in a smart speaker 10. The smart speaker 10 and the system 100 may be connected to a server device 200 and a user's terminal device 300 via a network N. The connection may be made in any manner, and may be wireless or wired, or may be via the Internet or a LAN.

[0025] The system 100 may also be connected to a database DB. In Fig. 2, the database DB is shown as a local database for the system 100, but the database DB may also be a cloud database connected via a network N. The database DB may also be connected to a terminal device 300.

[0026] The smart speaker 10 includes a microphone and a speaker. Voice from the user can be acquired via the microphone and input to the system 100. Responses generated by the system 100 can be output as voice via the speaker. Alternatively, the smart speaker 10 can transmit responses generated by the system 100 to the terminal device 300 or the server device 200.

[0027] The server device 200 may be a device capable of implementing an answer generation process using a large-scale language model (LLM). The server device 200 has a database and stores various pieces of knowledge or information. The server device 200 can generate answers using the LLM, and can also generate answers using search expansion generation (RAG).

[0028] The terminal device 300 may be a terminal device owned by a user, and is not limited to the illustrated smartphone, but may also be a tablet computer, a laptop computer, a desktop computer, a smartwatch, smart glasses, etc. The terminal device 300 may acquire data about the user and provide it to the system 100.

[0029] FIG. 3A shows an example of a specific configuration of the system 100 for generating a response to a user's voice.

[0030] The system 100 includes a specifying unit 101 , a generating unit 102 , a storing unit 103 , and a searching unit 104 .

[0031] When a voice is input to the system 100 , the input voice is passed to the identifying means 101 .

[0032] The identification means 101 is configured to identify at least one element contained in the speech by analyzing the speech. The identification means 101 can analyze the speech by, for example, natural language processing. The element identified by the identification means 101 is an element that may be involved in answer generation. For example, by analyzing a speech saying, "What's the weather like today?", the elements "today" and "weather" can be identified. For example, by analyzing a speech saying, "What's the postal code of my workplace?", the elements "me," "workplace," and "postal code" can be identified.

[0033] The identification means 101 can perform any natural language processing as long as it can identify elements contained in speech.

[0034] The generation means 102 is configured to generate an answer to the speech based on at least one identified element. The generation means 102 includes a language model that generates text. The language model may preferably be a small language model (SLM). In this specification, a large language model (LLM) is a language model provided as an API by so-called Big Tech companies such as Google or OpenAI, and may have a huge number of parameters (e.g., one trillion or more). While an LLM has high answer accuracy in general topics, it may generate an incorrect answer when generating an answer based on knowledge that is not publicly available on the Internet or when generating an answer based on knowledge in a specific specialized field if the knowledge is insufficient. Furthermore, due to the large size of the model, it tends to take a long time to generate an answer. In contrast, in this specification, a small language model (SLM) is a small model that may have one-tenth or less of the number of parameters of an LLM (e.g., one hundred billion or less). The LLM has undergone additional training based on information not publicly available on the internet and / or additional training specific to a specific subject (including further miniaturization through quantization processing), which may allow the SLM to generate fast and accurate answers for specific subjects, although it may fall short of the LLM for general subjects.

[0035] When a language model generates text, it can output a confidence level for that text. Confidence can be represented, for example, by a parameter called Temperature. Temperature is a parameter that adjusts the diversity of the output from the language model. When Temperature is high, the output from the language model becomes more diverse and less consistent, resulting in a more creative output. On the other hand, when Temperature is low, the output from the language model becomes less diverse and more consistent, resulting in a loss of creativity. The lower the Temperature, the more consistent the answer, which can be said to be more reliable. Alternatively, confidence can be represented by token probability. Token probability allows the confidence of the model to be measured from the prediction probability for each word. Alternatively, confidence can be represented by human evaluation. Alternatively, confidence can be a combination of these indicators.

[0036] The generating means 102 acquires information for one element or each of the multiple elements identified by the identifying means 101, and generates an answer from the acquired information. The generating means 102 can generate an answer for each element from information generated from a language model, from information acquired from the storage means 103, or from information searched by the searching means 104. At this time, if there is an element among the multiple elements for which information has already been obtained, information about that element can also be taken into consideration to obtain information for each element.

[0037] First, the generation means 102 uses a language model to generate text for an element. For example, if the element is "today," it can generate text describing the date of that day. For example, if information about "today" has already been obtained and the element is "weather," it can generate text describing the weather of that day.

[0038] At this time, the generation means 102 also outputs the reliability of the generated text. This makes it possible to evaluate the accuracy of the generated text. For example, if the reliability is higher than a threshold, it can be determined that the accuracy of the generated text is high, and the generated text can be adopted. On the other hand, for example, if the reliability is lower than the threshold, it can be determined that the accuracy of the generated text is low, and the text generated by the language model will not be adopted, and information will be obtained from another source. Here, the threshold may be a fixed value or a variable value. The threshold may be a preset value or a value set by a user or a service provider.

[0039] If the generated text is not adopted, the generating means 102 will obtain information from the storage means 103, for example.

[0040] The storage means 103 is configured to store information about users.

[0041] The information about the user may be, for example, the user's personal information, or the user's subjective information and / or the user's objective information. The information about the user may be, for example, a dialogue log obtained by dialogue with the user or information derived from the dialogue log. The information about the user may be, for example, information input by the user to his / her terminal device 300.

[0042] The storage means 103 can store information about users in a structured manner. This allows for fast access to the information stored in the storage means 103. Preferably, the information stored in the storage means 103 can be accessed within 0.1 seconds. This can be achieved, for example, by a known fast search algorithm.

[0043] The storage unit 103 may store information in, for example, a relational database format. This allows information to be acquired quickly (for example, within approximately 0.1 seconds) using SQL. The storage unit 103 may store information in, for example, a knowledge graph format.

[0044] For example, the storage means 103 stores the user's personal information, such as a postal code, address, and telephone number, in association with the user's home. For example, the storage means 103 stores the user's name, postal code, address, telephone number, and route from home in association with the user's workplace. For example, the storage means 103 stores the user's personal information, such as a name, age, relationship, postal code, address, and telephone number, in association with the user's family members.

[0045] With such structural storage, for example, for an element "my workplace," a "name" associated with the user's workplace can be obtained. For example, for an element "my workplace phone number," a "phone number" associated with the user's workplace can be obtained.

[0046] The above-mentioned information is merely an example, and other information can be similarly stored in a structured manner, thereby enabling high-speed searches.

[0047] For example, if the reliability of the information obtained from the storage means 103 is higher than a threshold, it can be determined that the accuracy of the obtained information is high, and the generation means 102 can adopt the obtained information. On the other hand, for example, if the reliability of the information obtained from the storage means 103 is lower than a threshold, it can be determined that the accuracy of the obtained information is low, and the information obtained from the storage means 103 will not be adopted as is. The generation means 102 may determine not to generate an answer and stop the answer generation process, or may obtain information from another source.

[0048] The generating means 102 can evaluate the reliability of the information acquired from the storage means 103 by using a vector. Specifically, the generating means 102 generates a first vector by vectorizing the information acquired from the storage means 103. This can be performed, for example, by embedding. Furthermore, the generating means 102 generates a reference vector by vectorizing an element. This can also be performed, for example, by embedding. By determining the similarity between the first vector and the reference vector, the reliability of the information acquired from the storage means 103 can be evaluated as the similarity between the reliability of the information acquired from the storage means 103 and the element (hereinafter also referred to as "first similarity"). The similarity between the first vector and the reference vector can be evaluated by any method for calculating similarity, for example, by cosine similarity.

[0049] The first similarity is compared with a threshold (hereinafter also referred to as "first similarity threshold"). If the first similarity is higher than the first similarity threshold, the reliability of the information acquired from the storage means 103 may be determined to be high. If the first similarity is lower than the first similarity threshold, the reliability of the information acquired from the storage means 103 may be determined to be low. Here, the first similarity threshold may be a fixed value or a variable value. The first similarity threshold may be a preset value or a value set by a user or a service provider. The first similarity threshold may be dynamically adjusted.

[0050] As described above, if the reliability of the information acquired from the storage means 103 is high, the generation means 102 can adopt the acquired information, but if the reliability of the information acquired from the storage means 103 is low, for example, the generation means 102 will not adopt the acquired information as is, but will instead search for information from the search means 104.

[0051] The retrieval means 104 is connectable to an external information source and is configured to retrieve information from the external information source.

[0052] An external information source is an information source that stores various types of information, and may store information in an unstructured manner, for example. An external information source may be, for example, an information source on the Internet, and may be searchable using a search query or the like. Alternatively, an external information source may be a specific database of a particular company, and the specific database may store internal regulations, manuals, past meeting minutes, recent emails, chat history, business manuals, an internal knowledge base, past inquiry history, etc.

[0053] The retrieval means 104 may provide information retrieved from an external information source as is, or may provide information generated by using the information retrieved from the external information source to generate information in a language model (small language model (SLM) or large language model (LLM)) (so-called RAG (search expansion generation)). In either case, this specification may refer to retrieving information from an external information source. Here, the language model may be one included in the generation means 102, or may be a language model separate from the one included in the generation means 102. Preferably, the language model included in the generation means 102 is a small language model (SLM), and the language model used by the retrieval means 104 may be a large language model (LLM). This speeds up information generation by the generation means 102, while enabling the retrieval means 104 to reference a large amount of information, thereby ensuring accuracy in information generation.

[0054] For example, the search means 104 may summarize the text generated by the language model of the generation means 102 and / or the information retrieved from the storage means 103, or extract keywords from the text generated by the language model of the generation means 102 and / or the information retrieved from the storage means 103, and search for information from external information sources using these summaries and / or keywords as queries. The information retrieved in this manner may be used to generate an answer, or may be used as supplementary information to the text generated by the language model of the generation means 102 and / or the information retrieved from the storage means 103.

[0055] For example, if the content of the voice question is ambiguous and the language model of the generation means 102 cannot generate text with sufficient accuracy, or information cannot be obtained from the storage means 103, the search means 104 can use the ambiguous question as a query to search for information from an external information source. Then, by inferring the intent of the question from the search results and converting or expanding the query based on the inferred intent, it may be possible to generate text using the language model of the generation means 102 and / or obtain information from the storage means 103 again.

[0056] For example, if the reliability of the information retrieved by the search means 104 is higher than a threshold, it can be determined that the accuracy of the retrieved information is high, and the generation means 102 can adopt the retrieved information. On the other hand, for example, if the reliability of the information retrieved by the search means 104 is lower than a threshold, it can be determined that the accuracy of the retrieved information is low, and the information retrieved by the search means 104 is not adopted. The generation means 102 may determine not to generate an answer and abort the answer generation process, or may obtain information from another source, but it is preferable to abandon the answer generation process. This is because generating an answer takes a considerable amount of time, making it impossible to ensure high speed.

[0057] The generating means 102 can evaluate the reliability of the information retrieved from the searching means 104 by using a vector. Specifically, the generating means 102 generates a second vector by vectorizing the information retrieved from the searching means 104. This can be performed, for example, by embedding. Furthermore, the generating means 102 generates a reference vector by vectorizing elements. This is the reference vector described above. By determining the similarity between the first vector and the reference vector, the reliability of the information retrieved from the searching means 104 can be evaluated as the similarity between the reliability of the information retrieved from the searching means 104 and the element (hereinafter also referred to as "second similarity").

[0058] The second similarity is compared with a threshold (hereinafter also referred to as "second similarity threshold"). If the second similarity is higher than the second similarity threshold, the reliability of the information retrieved from the search means 104 may be determined to be high. If the second similarity is lower than the second similarity threshold, the reliability of the information retrieved from the search means 104 may be determined to be low. Here, the second similarity threshold may be a fixed value or a variable value. The second similarity threshold may be a preset value or a value set by a user or a service provider. The second similarity threshold may be dynamically adjusted.

[0059] As described above, if the reliability of the information retrieved from the search means 104 is high, the generation means 102 can adopt the retrieved information, but if the reliability of the information retrieved from the search means 104 is low, for example, the generation means 102 may not adopt the retrieved information and give up on generating an answer.

[0060] The generating means 102 generates an answer to the voice based on the text or information determined to be adopted for the identified element or for each of the identified elements. For example, the answer "Today's weather is sunny" may be generated based on the information determined to be adopted for each of the elements "today" and "weather." For example, the answer "Your work phone number is xxx-xxx-xxx" may be generated based on the information determined to be adopted for each of the elements "me," "workplace," and "phone number."

[0061] For example, in response to the question "What is the inventory of product A?", the answer "50 units" is generated based on the information determined to be adopted from the information acquired from the language model and storage means 103 for the element "inventory of product A." At this time, if it is determined that the reliability of the information acquired from the language model and storage means 103 is low, the search means 104 searches an external information source that stores internal company notices about product A, and when it finds information that "shipment of product A will be temporarily suspended next week due to a change in specifications," supplemental information is added, and finally the answer "The current inventory is 50 units. However, please note that shipments will be temporarily suspended from next week due to a change in specifications." can be generated.

[0062] For example, in response to the question "Tell me about recent sales," the language model and storage means 103 attempted to generate an answer for the elements "recent" and "sales," but was unable to adopt accurate information because it was unclear what period "recent" meant. The search means 104 searches an external information source that stores minutes of a company's management meetings, and upon finding that the topic was "second-quarter sales report," creates a query in which the element "recent" is replaced with "second quarter." This allows the language model and storage means 103 to re-search and re-generate the elements "second quarter" and "sales," ultimately generating an answer such as, "You're talking about second-quarter sales, which were discussed at the most recent management meeting. Sales were 100 million yen, up 5% from the same period last year."

[0063] For example, in response to the question, "What are the PC setup procedures for new employees?", the language model and storage means 103 attempted to generate an answer for the elements "for new employees" and "PC setup procedures," but was unable to obtain the relevant information. The search means 104 then searches external information sources that store all company information (e.g., business manuals, internal knowledge bases, past inquiry histories, etc.), finds a file called "PC Setup Manual.pdf" created by the company's IT department, and generates an answer based on the contents of that file. For example, the search means 104 may ultimately generate an answer such as, "Yes, it's the PC setup procedures for new employees. According to the internal manual, first do XX, then install △△. For more information, please refer to this link: [link]."

[0064] In this way, the system 100 can ensure both a high speed of generating an answer and high accuracy of the answer. The system 100 can output an answer with a reaction speed equal to or faster than that of a human (for example, 2 seconds to 0.2 seconds, preferably 0.2 seconds or less) and with 100% accuracy. For example, if the accuracy of the answer is insufficient, the system will abandon outputting the answer. This actively prevents "plausible lies" and ensures the reliability of the information.

[0065] FIG. 3B illustrates an example of an alternative embodiment of the system 100 for generating a response to a user's speech.

[0066] The configuration shown in Fig. 3B differs from the configuration described above with reference to Fig. 3A in that it further includes update means 105. Components similar to those described above with reference to Fig. 3A are given the same reference numerals, and detailed description thereof will be omitted here.

[0067] The system 100 includes a specifying unit 101 , a generating unit 102 , a storing unit 103 , a searching unit 104 , and an updating unit 105 .

[0068] When a voice is input to the system 100 , the input voice is passed to the identifying means 101 .

[0069] The identification unit 101 is configured to identify at least one element contained in the speech by analyzing the speech. The identification unit 101 can perform any natural language processing as long as it can identify the element contained in the speech.

[0070] The generating means 102 is configured to generate an answer to the speech based on at least one identified element. The generating means 102 can generate an answer for one element or each of a plurality of elements identified by the identifying means 101 from information generated from a language model, from information acquired from the storage means 103, or from information retrieved by the retrieval means 104.

[0071] The update means 105 is configured to update the information stored in the storage means 103. When information is acquired from the storage means 103, the update means 105 compares the reliability of the information acquired from the storage means 103 and a first similarity between the element and the information with a first similarity threshold, and when it is determined that the first similarity is lower than the first similarity threshold, the update means 105 can update the storage means 103 with the information. This can prevent inaccurate information from being continuously stored in the storage means 103 and contribute to building the storage means 103 with accurate information.

[0072] For example, for information determined to have a first similarity lower than the first similarity threshold, the update means 105 may update the storage means 103 with information searched for by the subsequent search means 104, or may update the storage means 103 with information input by the user, or may update the storage means 103 in a different manner. Preferably, the update means 103 may be updated with information searched for by the subsequent search means 104 and determined to have high reliability. This is because information determined to have high reliability is considered to be more accurate.

[0073] In this way, by continuously updating the storage means 103, it is possible to autonomously improve the performance of the system 100. This can also lead to a reduction in the maintenance costs of the system 100.

[0074] The user information stored in the storage means 103 and kept up to date can also be used to build an SLM.

[0075] FIG. 4 illustrates an example of a configuration for implementing the system 100 for generating a response to a user's speech.

[0076] The system 100 comprises an interface section 110, a processor section 120, and a memory section .

[0077] The interface unit 110 exchanges information with the outside of the system 100. The processor unit 120 of the system 100 can receive information from the outside of the system 100 and can transmit information to the outside of the system 100 via the interface unit 110. The interface unit 110 can exchange information in any format. The interface unit 110 includes, for example, an input unit that allows information to be input to the system 100 and an output unit that allows information to be output from the system 100. At least a portion of the above-mentioned identification means 101 can be implemented by the interface unit 110 and can receive voice via the interface unit 110. A portion of the above-mentioned generation means 102 can be implemented by the interface unit 110 and can output a generated answer to the external environment, the server device 200, or the terminal device 300 via the interface unit 110.

[0078] The processor unit 120 executes the processing of the system 100 and controls the operation of the entire system 100. The processor unit 120 reads and executes a program stored in the memory unit 130. This allows the system 100 to function as a system that executes desired steps. The processor unit 120 may be implemented by a single processor or by multiple processors. At least a portion of the above-mentioned identifying means 101, at least a portion of the generating means 102, a portion of the storing means 103, at least a portion of the searching means 104, and at least a portion of the updating means 105 may be implemented by the processor unit 120.

[0079] The memory unit 130 stores programs required to execute the processes of the system 100, data required to execute the programs, and the like. The memory unit 130 may also store programs for causing the processor unit 120 to execute processes to assist a user (e.g., a program that realizes the processes shown in FIG. 5, which will be described later). Here, how the programs are stored in the memory unit 130 is not important. For example, the programs may be pre-installed in the memory unit 130. Alternatively, the programs may be stored in a non-transitory computer-readable storage medium and installed by reading the storage medium. Alternatively, the programs may be installed in the memory unit 130 by being downloaded via a network. In this case, the type of network does not matter. The memory unit 130 may be implemented by any storage means. At least a portion of the storage means 103 described above may be implemented by the memory unit 130.

[0080] The database DB stores, for example, information about users. The database DB can store information about users in a structured manner. At least a part of the above-mentioned storage means 103 can be implemented by the database DB.

[0081] 4, the database DB is provided outside the system 100, but the present invention is not limited to this. At least a part of the database DB can also be provided inside the system 100.

[0082] Although system 100 is preferably implemented within smart speaker 10, it does not necessarily have to be implemented within smart speaker 10 and may be provided outside smart speaker 10. Furthermore, each component of system 100 does not have to reside within a single device; a distributed configuration in which the components are distributed across multiple devices is also within the scope of the present invention.

[0083] Each component of the system 100 described above may be composed of a single hardware component or multiple hardware components. When composed of multiple hardware components, the manner in which the hardware components are connected does not matter. The hardware components may be connected wirelessly or by wire. The system 100 is not limited to a specific hardware configuration. It is also within the scope of the present invention for the processor unit 120 to be configured using analog circuits rather than digital circuits. The configuration of the system 100 is not limited to the one described above as long as it can realize its functions.

[0084] 3. System processing to generate a response to the user's voice 5 is a flow chart illustrating an example process (process 500) by system 100 for generating a response to a speech for a user. Process 500 is performed in processor unit 120 of system 100.

[0085] For example, when a voice is input to the smart speaker 10, the input voice is input to the processor unit 120 via the interface unit 110. The processor unit 120 receives the input voice.

[0086] In step S501, the processor unit 120 analyzes the speech to identify at least one element contained in the speech. The processor unit 120 can perform any natural language processing as long as it can identify the element contained in the speech.

[0087] In step S502, processor unit 120 generates a response to the voice based on at least one element identified in step S501. Processor unit 120 acquires information for one element identified in step S501 or for each of the multiple elements identified in step S501 by processing described below with reference to Fig. 6. Processor unit 120 can use the acquired information to generate a response.

[0088] The generated answer can be output from the smart speaker 10. By outputting the answer within a reaction speed of about 2 seconds to about 0.2 seconds, preferably about 0.2 seconds or less, after the user inputs a voice to the smart speaker 10, it is possible to realize a natural conversational experience close to that of a human conversation. In addition, the accuracy of the answer can be ensured.

[0089] Fig. 6 is a flowchart showing an example of the processing executed in step S502. Processor unit 120 may perform the processing shown in Fig. 6 for one element identified in step S501. If there is an element for which the processing shown in Fig. 6 has already been executed, information obtained for that element may also be taken into consideration.

[0090] In step S601, the processor unit 120 generates text for an element using a language model. The language model may preferably be a small language model (SLM). When generating text, the language model can output a confidence level for the text. The confidence level may be represented by, for example, a parameter called Temperature. The confidence level may be represented by token probability, by human evaluation, or by a combination of these indicators.

[0091] In step S602, the processor unit 120 compares the reliability output in step S601 with a threshold. If the reliability is higher than the threshold (Yes), the process proceeds to step S603. If the reliability is lower than the threshold (No), the process proceeds to step S604.

[0092] In step S603, the processor unit 120 uses the text generated in step S601 because it can be determined that the generated text is highly accurate. The text is then used as information for the element in question to generate an answer. If there are other elements, an answer can be generated together with the information used for the other elements.

[0093] If the reliability of the text generated in step S601 is low, the accuracy of the generated text is determined to be low, and the text generated in step S601 is not adopted. In this case, in step S604, the processor unit 120 obtains information about the element from the database DB. Information about the user can be stored in the database DB in a structured manner, thereby enabling high-speed access to the information stored in the storage means 103.

[0094] In step S605, the processor unit 120 compares the first similarity between the acquired information and the element with a first similarity threshold to evaluate the reliability of the information acquired in step S604. Specifically, the information acquired in step S604 is vectorized to generate a first vector. Then, the element is vectorized to generate a reference vector. Then, the first similarity between the first vector and the reference vector is determined. The first similarity may be expressed, for example, by cosine similarity. If the first similarity is higher than the first similarity threshold (Yes), the process proceeds to step S606. If the first similarity is lower than the first similarity threshold (No), the process proceeds to step S607.

[0095] In step S606, the processor unit 120 uses the information acquired in step S604 because it can be determined that the acquired information is highly accurate. Then, the processor unit 120 generates an answer using the acquired information as information for the element. If there are other elements, an answer can be generated together with the information adopted for the other elements.

[0096] If the reliability of the information acquired in step S604 is low, the accuracy of the acquired information is determined to be low, and the information acquired in step S604 is not adopted. In this case, in step S607, the processor unit 120 searches for information on the element from an external information source. The external information source is an information source that stores various information, and may store information in an unstructured manner, for example. Searching for information on the element includes providing the information retrieved from the external information source as is, or providing information generated by using the information retrieved from the external information source to generate information using a language model (so-called RAG).

[0097] In step S608, the processor unit 120 compares the second similarity between the retrieved information and the element with a second similarity threshold to evaluate the reliability of the information retrieved in step S607. Specifically, the processor unit 120 vectorizes the information retrieved in step S607 to generate a second vector. Next, the processor unit 120 determines the second similarity between the reference vector obtained by vectorizing the element and the second vector. The second similarity may be expressed, for example, by cosine similarity. If the second similarity is higher than the second similarity threshold (Yes), the process proceeds to step S609. If the second similarity is lower than the second similarity threshold (No), the process proceeds to step S610.

[0098] In step S609, the processor unit 120 adopts the information retrieved in step S607. This is because it can be determined that the accuracy of the retrieved information is high. Then, an answer is generated using the acquired information as information for the element. If there are other elements, an answer may be generated together with information adopted for the other elements. Alternatively, the acquired information is adopted as supplementary information to the information obtained in step S601 and / or step S604. Alternatively, the acquired information may be used to convert or expand the search query, and the processes from step S601 onward may be performed again. This is effective when the input voice is an ambiguous question.

[0099] If the reliability of the information retrieved in step S607 is low, the accuracy of the retrieved information is determined to be low, the information retrieved in step S607 is not adopted, and the answer generation process is abandoned. As a result, the processor unit 120 does not generate an answer to the voice, and can output, for example, that an answer is not possible.

[0100] In the above example, it has been explained that the process proceeds to step S607 if the first similarity is lower than the first similarity threshold (No) in step S605, but steps S607 to S609 may be omitted and the process may proceed directly to step S610. This allows the generation of an answer to be abandoned if the reliability of the information acquired from the database DB is low, thereby realizing faster processing. In this case, high answer accuracy can also be ensured.

[0101] For example, by adjusting the hardware specifications, the processing from step S601 to step S604 can be executed within approximately 0.1 seconds, the processing from step S605 to step S606 can be executed within approximately 0.05 seconds, and the processing from step S607 to step S609 can be executed within approximately 0.05 seconds, thereby realizing high-speed answer generation within approximately 0.2 seconds.

[0102] What was unexpected was that this high-speed answer generation also ensures the accuracy of the answers.

[0103] In the examples described above with reference to Figures 5 and 6, the processes are described as being performed in a specific order, but the order of each process is not limited to that described and may be performed in any order that is logically possible.

[0104] In the example described above with reference to Figures 5 and 6, the processing of each step shown in Figures 5 and 6 can be realized by the processor unit 120 and a program stored in the memory unit 130, but at least one of the processing of each step shown in Figures 5 and 6 may also be realized by a hardware configuration such as a control circuit.

[0105] The present invention is not limited to the above-described embodiments. It is understood that the scope of the present invention should be interpreted only by the claims. It is understood that a person skilled in the art can implement an equivalent scope based on the description of the present invention and common technical knowledge from the description of specific preferred embodiments of the present invention. [Industrial Applicability]

[0106] The present invention is useful for providing a voice response system that can output fast and accurate responses. [Explanation of symbols]

[0107] U User 10 Smart Speakers 100 systems 200 Server device 300 Terminal Equipment DB Database

Claims

1. 1. A system for generating a response to a user's speech, the system comprising: an identification means for analyzing the audio to identify at least one element contained in the audio; a storage means for storing information about the user; A generating means for generating an answer to the speech based on the at least one element, the generating means including a language model for generating text, the language model being capable of outputting a confidence level for the text together with the text, the generating means performing, for each element of the at least one element: generating text for said element; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from the storage means; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A generating means A system comprising:

2. Determining a first similarity between the obtained information and the element includes: vectorizing the obtained information to generate a first vector; generating a reference vector by vectorizing the elements; determining a similarity between the first vector and the reference vector; The system of claim 1 , comprising:

3. The information processing system further includes a search unit that can connect to an external information source and searches for information from the external information source, and the generating unit if the first similarity is less than the first similarity threshold, retrieving information related to the element from the external information source; determining a second similarity between the retrieved information and the element; determining whether the second similarity is greater than a second similarity threshold; adopting the retrieved information if the second similarity is higher than the second similarity threshold; abandoning generating the answer if the second similarity is lower than the second similarity threshold; The system of claim 1 , further comprising:

4. Determining a second similarity between the retrieved information and the element includes: vectorizing the retrieved information to generate a second vector; generating a reference vector by vectorizing the elements; determining a similarity between the second vector and the reference vector; The system of claim 3 , comprising:

5. The system of claim 1 , further comprising: an update means for updating the storage means with the obtained information when the first similarity is determined to be lower than the first similarity threshold.

6. 5. The system of claim 4, further comprising an update means for updating the storage means with the retrieved information when it is determined that the first similarity is lower than the first similarity threshold, wherein the update means updates the storage means with the retrieved information for which it is determined that the second similarity is higher than the second similarity threshold.

7. The system of claim 1 , wherein the storage means stores a log of interactions with the user.

8. 4. The system of claim 3, wherein said storage means stores said user's information in a structured manner and said external information source stores information in an unstructured manner.

9. 1. A method for generating a response to a user's speech, the method comprising: analyzing the audio to identify at least one element contained in the audio; generating a response to the speech based on the at least one element, the generating including, for each element of the at least one element: generating text for the element using a language model for generating text, the language model being capable of outputting a confidence level for the text along with the text; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from a storage means for storing information about the user; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A method comprising:

10. A program for generating a response to a user's voice, the program being executed on a computer having a processor, the program comprising: analyzing the audio to identify at least one element contained in the audio; generating a response to the speech based on the at least one element, the generating including, for each element of the at least one element: generating text for the element using a language model for generating text, the language model being capable of outputting a confidence level for the text along with the text; generating text for said element; determining whether the confidence of the generated text is higher than a threshold; adopting the generated text if the confidence is higher than the threshold; If the reliability is lower than the threshold, obtaining information related to the element from a storage means for storing information about the user; determining a first similarity between the obtained information and the element; determining whether the first similarity is greater than a first similarity threshold; adopting the acquired information if the first similarity is higher than the first similarity threshold; A program that causes the processor to execute a process including the steps of:

Images