system

Abstract

We provide the system. [Solution] A means of receiving inquiries from users in natural language, A means for analyzing the aforementioned query and extracting keywords, A means of obtaining related information using vector search technology with extracted keywords, A means of automatically generating a response in natural language based on the acquired information, A means of providing the generated answer to the user, A means for monitoring the operation of the aforementioned means and accumulating feedback for use in learning, An information processing system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, the labor shortage due to population decline and aging has become serious, and there is a demand for efficient and accurate service provision, especially in call centers and window services. Also, it is necessary to respond quickly and accurately to diverse and complex inquiries from users. However, since conventional automatic response systems can only provide limited answers to specific inquiries, the development of technologies for realizing effective automatic responses is demanded.

Means for Solving the Problems

[0005] This invention provides a system that first receives a natural language inquiry from a user, analyzes it, and extracts keywords. Next, it uses the extracted keywords to obtain related information using vector search technology. Then, based on the obtained information, it automatically generates an answer in natural language and finally provides this answer to the user. Furthermore, by accumulating feedback obtained from users and reflecting it in improving the accuracy of the system, it achieves effective responses to a wide range of inquiries.

[0006] A "user" refers to an individual or organization that makes an inquiry using an information system.

[0007] "Natural language" refers to languages ​​that humans use on a daily basis and that have a complex linguistic structure, including grammar and vocabulary.

[0008] An "inquiry" refers to a question or request that a user makes to a system in order to obtain information.

[0009] "Analysis" refers to the process of breaking down received information and processing it in order to understand its meaning.

[0010] A "keyword" refers to a word or phrase that is important when searching for information.

[0011] "Vector search technology" refers to a technique that represents keywords and documents as high-dimensional vectors and searches for information based on similarity.

[0012] "Information" refers to the content of the knowledge or data in question, including the information necessary to answer the inquiry.

[0013] "Automatic generation" refers to the creation of information using artificial intelligence or algorithms without human intervention.

[0014] "Feedback" refers to opinions and evaluations of a system provided by users, and is information used to improve the system.

[0015] "Response" refers to the answer or information provided by the system in response to an inquiry.

Brief Description of the Drawings

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which multiple emotions are mapped. [Figure 10] It shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiment for Implementing the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides an embodiment of an information provision system based on natural language queries. This system mainly consists of three components: a user, a terminal, and a server.

[0038] When users need information, they input questions in natural language using devices such as computers or smartphones. This input is done through an interface available to the user. The interface includes features such as text input fields and voice input functions, and is designed with user-friendliness in mind.

[0039] The terminal receives input from the user and transmits it to the server via the network. The terminal incorporates a communication module to facilitate this transmission. The terminal also receives responses from the server and presents them appropriately to the user. Screen display and audio output functions are utilized to allow the user to receive information visually or audibly.

[0040] After receiving a query from a terminal, the server analyzes the query using a natural language processing engine. Specifically, it extracts important information through processes such as tokenization, syntactic analysis, and keyword extraction. Subsequently, it retrieves related information from a database using vector search technology. This information is represented as a multidimensional vector, and the most suitable one is selected through similarity calculation.

[0041] Furthermore, the server uses a generative AI model to automatically generate natural language responses based on relevant information. These generated responses are optimized to be natural and easy to understand. This process takes care to ensure, for example, that the responses are consistent, non-contradictory, and helpful to the user's understanding.

[0042] As a concrete example, consider a case where a user asks, "What are the opening hours of my nearest city hall?" The terminal receiving this inquiry sends the content to the server. The server refers to local information and public facility data and generates a specific answer, such as, "City hall opening hours are from 9 am to 5 pm on weekdays." This answer is then provided to the user in real time via the terminal.

[0043] Through these embodiments, the present invention aims to improve the efficiency of information provision and significantly enhance user convenience by responding quickly and accurately to a wide range of user inquiries.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] The user enters questions requiring information into the device using natural language. Input can be in text or voice format.

[0047] Step 2:

[0048] The terminal receives input from the user and appropriately converts the input data into a digital format. It then prepares the converted data to be sent to the server via the network.

[0049] Step 3:

[0050] The server receives user inquiries sent from the terminal. The inquiry content is analyzed using a natural language processing engine, tokenized, tagged with parts of speech, and key keywords are extracted.

[0051] Step 4:

[0052] The server vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. The most suitable set of information is then selected through similarity calculation.

[0053] Step 5:

[0054] Based on the information acquired by the server, a generative AI model is used to automatically generate responses in natural language. The responses are structured to be easily understood by the user.

[0055] Step 6:

[0056] The server prepares to send the final generated response to the terminal. If necessary, it performs minor formatting to maintain the integrity of the output data.

[0057] Step 7:

[0058] The terminal displays the response received from the server to the user. The display is presented in a format that is convenient for the user, such as on-screen text or audio output.

[0059] Step 8:

[0060] Check if the user was satisfied with the displayed answer and provide feedback to request further details if necessary.

[0061] The feedback is sent to the server for future accuracy improvements.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] Existing information provision systems struggle to generate quick and accurate responses to diverse natural language inquiries from users. Furthermore, there is a need for efficient methods to effectively evaluate the relevance of information and generate optimal answers.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting information units, and a device for using computational techniques to acquire related information using the information units. This makes it possible to respond quickly to complex user inquiries and generate optimal and consistent answers.

[0067] A "user" refers to an individual or organization that seeks to obtain information through a system.

[0068] "Natural language" refers to the linguistic forms that humans use on a daily basis, and which require special processing for computers to understand.

[0069] An "inquiry" refers to a question expressed in natural language that a user wants to know.

[0070] "Device" refers to hardware or software components used to perform a function.

[0071] "Analysis" refers to the process of understanding the structure and content of input natural language and extracting its meaning.

[0072] An "information unit" refers to a semantically important element or keyword obtained through analysis.

[0073] "Computational techniques" refer to algorithms and methods for efficiently processing and retrieving data.

[0074] "Related information" refers to the information necessary to generate an appropriate response to an inquiry.

[0075] "Consistency" refers to a state where information or answers are not contradictory and are logically consistent.

[0076] This embodiment of the invention provides a configuration primarily consisting of three elements: a user, a terminal, and a server.

[0077] Users can make inquiries using natural language via devices such as computers and smartphones. This allows users to intuitively obtain information. Since the devices are equipped with interfaces that allow text input and voice input, users only need to input their questions.

[0078] The terminal receives natural language inquiries from the user and transmits this data to the server via the internet. To ensure smooth communication, the terminal is equipped with an appropriate communication module. Furthermore, after receiving a response from the server, it is equipped with a display device or audio output device to appropriately present that response to the user.

[0079] The server receives user queries sent from terminals and performs analysis using a natural language processing engine. Specifically, it tokenizes the queries, parses them syntactically, and extracts information units. After this, it uses computational techniques to retrieve related information from the database based on the extracted information units. For data retrieval, it uses vector search techniques to perform similarity calculations and select the most relevant information.

[0080] Furthermore, the server utilizes a generative AI model to automatically generate responses in natural language using the acquired relevant information. These responses are optimized to be natural, easy to understand, and consistent for the user.

[0081] A concrete example is when a user asks, "Please tell me the location of the nearest hospital." The device sends this inquiry to the server. The server refers to local information and facility data, generates a specific answer such as, "The nearest hospital is XX Hospital, and its address is YY," and provides it to the user through the device.

[0082] An example of a prompt might be: "The user has asked, 'What is the weather forecast for this weekend?' Generate an appropriate natural language response to this inquiry. Refer to weather forecast information and ensure consistency."

[0083] In this way, the present invention enables rapid and accurate responses to diverse user inquiries and provides clear and easy-to-understand information.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] Users input inquiries in natural language using a smartphone or computer interface. For example, they might send a message like "What's the weather forecast for tomorrow?" to their device via text or voice input. At this stage, natural language text data is input.

[0087] Step 2:

[0088] The terminal receives input data from the user and sends it to the server via the network. The communication module on the terminal packets this data and sends it over the internet. At this stage, the output is passed to the server as structured data.

[0089] Step 3:

[0090] The server receives data sent from the terminal. Using a natural language processing engine, it first tokenizes the input text, breaking down the sentence into individual words and phrases. Then, it performs syntactic analysis to extract important information units and keywords. The input is natural language text, and the output is the analyzed keywords.

[0091] Step 4:

[0092] Using the extracted keywords, the server employs vector search techniques to explore relevant databases. Similarity calculations, such as Euclidean distance, are performed to find the information most closely related to the input keywords. The output of this process is obtained as a collection of retrieved relevant information.

[0093] Step 5:

[0094] The server uses a generative AI model to generate natural language responses based on the relevant information it has acquired. Specifically, this is a process of assembling information in context and creating coherent sentences. The output here is natural language text to be provided to the user.

[0095] Step 6:

[0096] The server sends the generated response to the terminal. The terminal receives this information and presents it to the user visually or audibly. For example, it may display text on a screen or read it aloud using a text-to-speech system. The output of this step is the response displayed on the user's terminal.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In emergencies, providing timely and accurate information is difficult, and there is a need to quickly deliver appropriate route guidance information to users. Furthermore, the quality of responses to natural language inquiries and the system's own learning capabilities need to be improved.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting specific information, and a device for providing appropriate route guidance information in emergencies. This enables the rapid and accurate provision of information in emergencies and improves the quality of route guidance.

[0102] A "user" is someone who makes queries to the system using natural language.

[0103] "Natural language" refers to the form of language that humans use on a daily basis, and the form used as input to a system.

[0104] A "device for receiving inquiries" is a device for receiving questions from users that are written in natural language.

[0105] A "device for extracting specific information" is a device that analyzes received inquiries and extracts the necessary information.

[0106] "Multidimensional search technology" is a technique that represents information within a database as multidimensional vectors and performs searches based on similarity.

[0107] A "device for acquiring related information" is a device for retrieving related information from a database based on specific information that has been extracted.

[0108] A "device that automatically generates responses" is a device that generates answers for the user in a natural way based on acquired information.

[0109] "Route guidance information" refers to instructional information provided as a means of appropriate evacuation or movement during emergencies.

[0110] A "device that monitors operation and stores evaluation information for learning" is a device that monitors the performance of a system and stores data to help improve the system.

[0111] The system for implementing this invention is configured to allow users to make inquiries in natural language using a smartphone or smart device. The natural language inquiry entered by the user is transmitted to the server via the network through the terminal.

[0112] The server first parses the query and uses a natural language processing engine to extract specific information. Specifically, it uses spaCy, an open-source natural language processing library, for tokenization, parsing, and information extraction. Then, vector search technology is used to quickly and efficiently retrieve data related to the extracted information. A database management system (e.g., PostgreSQL) can be used for this process.

[0113] Based on the acquired information, the server uses a generative AI model (e.g., OpenAI®'s GPT) to generate natural language responses to provide to the user. These responses are optimized to be realistic and easy to understand.

[0114] Furthermore, this system is also practical in emergencies. For example, if a user asks "What is the evacuation route?" during a fire, the server will instantly retrieve the optimal evacuation route from the database and generate specific instructions such as "The nearest emergency exit from your current location is 50 meters northeast."

[0115] The following is an example of a prompt message: "Please provide the location of the nearest emergency exit. The user is located in front of a home computer."

[0116] In this way, the system aims to enhance user safety and satisfaction by enabling the rapid and accurate provision of information in a variety of situations, including emergencies.

[0117] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0118] Step 1:

[0119] The user enters a query in natural language using a smartphone or smart device. This input is made via the device's interface in text or voice format. The device then prepares to send this input data to the server over the network.

[0120] Step 2:

[0121] The server receives natural language queries sent from the terminal. The input text data is parsed by a natural language processing engine (e.g., spaCy), undergoing tokenization and syntactic analysis. This extracts specific information from the query. The output is data containing the extracted specific information.

[0122] Step 3:

[0123] The server uses the extracted specific information to perform vector search techniques. It uses a database system (e.g., PostgreSQL) to search for and retrieve relevant information. In this process, the information is represented in a multidimensional vector format, and similarity calculations are performed to select the most relevant information. The output is the most relevant information.

[0124] Step 4:

[0125] The server uses the acquired relevant information to automatically generate a natural language response using a generative AI model (e.g., OpenAI's GPT). This generation process considers the consistency and accuracy of the information. The output is a natural language response provided to the user.

[0126] Step 5:

[0127] The server sends the generated response to the terminal over the network. The terminal displays the received response to the user. This display is done via text or voice output. The user can receive the information provided through the terminal in real time.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] This invention provides an embodiment of an information processing system that recognizes a user's emotional state in response to a natural language query and provides appropriate information. This system consists of a user, a terminal, a server, and an emotion engine.

[0130] When users input questions or requests for information in natural language, their emotional state in that situation is also taken into consideration. Users make inquiries via text or voice using a computer or smartphone. The input data is then processed on the device.

[0131] The device receives input from the user and uses an emotion engine to analyze the user's emotions. This analysis includes various parameters such as voice tone, word choice, and input speed. The results of the analysis are generated as data indicating the user's emotional state. This emotional state data is also sent to the server.

[0132] The server analyzes the input from the terminal using a natural language processing engine based on the inquiry content and sentiment data received, and extracts keywords. Then, it vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. At this point, the server selects a response from the retrieved information that is adjusted based on sentiment.

[0133] The server uses a generative AI model to automatically generate responses that adjust tone and expression based on data from the emotion engine. These generated responses include emotional consideration for the user. For example, if the user is feeling anxious, calm and friendly language will be used.

[0134] As a concrete example, consider a scenario where a user enters a message into the device stating, "I'm feeling anxious about buying a new mobile phone." In this case, the emotion engine recognizes the user's anxiety based on their tone of voice and chosen words. Based on this information, the server generates a response that is both helpful and reassuring. It then presents the user with a response such as, "Don't worry, we can help you choose a mobile phone."

[0135] This system aims to improve the user experience by flexibly responding to the user's psychological state while accurately providing the necessary information.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] Users input questions and requests for information using natural language on their own devices. This input can be in text or voice format.

[0139] Step 2:

[0140] The device receives input from the user and records it as digital data. The received input is then passed directly to the emotion engine.

[0141] Step 3:

[0142] The device uses an emotion engine to analyze user input and recognize the user's emotional state from their voice tone and expressions. The recognized emotion data is sent to the server along with the inquiry content.

[0143] Step 4:

[0144] The server analyzes the inquiry content and sentiment data received from the terminal. A natural language processing engine is used to extract keywords from the inquiry.

[0145] Step 5:

[0146] The server uses the extracted keywords to retrieve relevant information from the database using vector search technology. At this time, it extracts the most suitable information based on similarity calculations.

[0147] Step 6:

[0148] The server uses an AI model based on sentiment data to adjust the tone and expression when generating responses, constructing responses that are sensitive to the user's emotions.

[0149] Step 7:

[0150] The server sends the generated response to the terminal. The response is formatted on the terminal in a way that the user can easily understand.

[0151] Step 8:

[0152] The device receives a response from the server and displays it to the user. The display format is either text or audio, and the expression is tailored to the user's emotions.

[0153] Step 9:

[0154] The user reviews the displayed answers and asks additional questions or provides feedback if necessary. This feedback is sent to the server to improve the system.

[0155] (Example 2)

[0156] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0157] In modern information processing systems, when users make inquiries using natural language, there is a need to provide appropriate information that takes their emotional state into consideration. However, conventional systems have suffered from insufficient response generation based on user emotions, resulting in a limited user experience. There is a need for technology that enables the provision of information in a tone that matches the user's psychological state.

[0158] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0159] In this invention, the server includes means for analyzing user inquiries and identifying emotional states, means for extracting keywords based on emotional states and inquiry content, and means for automatically generating responses in an emotionally appropriate tone from the acquired information. This makes it possible to provide information that takes the user's emotions into consideration.

[0160] A "user" refers to an individual or organization that uses an information processing system to make queries in natural language.

[0161] "Natural language" refers to forms of communication composed of language structures that humans use on a daily basis.

[0162] "Inquiry" refers to text or audio information entered by a user with the intention of requesting information from the system.

[0163] "Emotional state" refers to data that analyzes and classifies the psychological and emotional state that users exhibit when making inquiries.

[0164] "Keywords" are important words or phrases extracted from the inquiry text and used for information retrieval and response generation.

[0165] "Vector search technology" refers to a technical method for efficiently searching for related information by converting extracted keywords into numerical vectors.

[0166] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to automatically generate responses in natural language.

[0167] "Feedback" refers to the data obtained from evaluating the operation of an information processing system and using it for future improvements and learning.

[0168] In this information processing system, the user, terminal, server, and emotion engine are the main components. Users can ask questions and request information in natural language using a computer or smartphone. Input methods include text or voice, and the data is transmitted to the terminal immediately.

[0169] The device receives a user inquiry and first uses an emotion engine to analyze the user's emotional state. This emotion analysis uses a variety of data parameters, such as voice tone, word choice, and input speed. This analysis generates data indicating the user's emotional state.

[0170] After sentiment analysis, the data is sent from the terminal to the server. The server uses a natural language processing engine to analyze the received query and extract keywords. These extracted keywords are then vectorized, and the server uses vector search technology to retrieve relevant information from the database. Similarity evaluation is also performed during the information retrieval process to select the most relevant information.

[0171] The server utilizes a generative AI model to automatically generate responses with adjusted tone and expression based on the user's emotional state. These generated responses are then delivered to the user in an emotionally sensitive manner. For example, if a user inquires that they are feeling anxious about purchasing a new mobile phone, the emotion engine recognizes this anxiety, and the server generates a friendly response such as, "Don't worry, we can help you choose a mobile phone."

[0172] An example of a prompt sentence to input into a generative AI model is, "Provide the user with advice on purchasing a mobile phone that will alleviate their anxiety." In this way, it becomes possible to improve the user experience by understanding the user's emotions and providing appropriate and useful information.

[0173] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0174] Step 1:

[0175] Users make inquiries using natural language via computers or smartphones. Input is either text or voice, and this data is sent to the device. The inquiry data is then processed in the next step.

[0176] Step 2:

[0177] The device receives user inquiry data and uses an emotion engine to analyze the user's emotional state. This analysis generates data by examining multiple parameters such as voice tone, input speed, and word choice. The output is data indicating the emotional state determined by the analysis.

[0178] Step 3:

[0179] The terminal sends the analyzed sentiment data and query content to the server. Once the sentiment data and query content are transferred to the server, they are used for the next processing step.

[0180] Step 4:

[0181] The server uses a natural language processing engine to extract keywords based on the received inquiry content and sentiment data. It identifies the most relevant keywords from the input content and generates vectorized keywords as output data.

[0182] Step 5:

[0183] The server utilizes vectorized keywords and employs vector search technology to retrieve relevant information from the database. It evaluates the similarity of the information and selects the most appropriate information. The output is filtered relevant information.

[0184] Step 6:

[0185] The server uses a generative AI model to automatically generate natural language responses with appropriate tone and expression based on the user's emotional state. This process takes acquired information and emotional data as input, and the output is a refined response that takes the user's emotions into consideration.

[0186] Step 7:

[0187] The server sends the generated response back to the terminal, which then presents it to the user. Based on that response, the user can decide on their next course of action with a sense of security and confidence.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0190] In today's world, users routinely acquire vast amounts of information through digital platforms. Consequently, there is a growing demand for information tailored to users' emotional states. However, conventional systems struggle to adequately consider user emotions in this regard. This is particularly true in brick-and-mortar stores, where accurately perceiving and responding to customer emotions directly impacts customer satisfaction and loyalty. Yet, there is a lack of systems capable of real-time analysis and information provision for diverse emotions.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for receiving inquiries from users in natural language, means for analyzing the inquiries, extracting keywords, and recognizing the emotional state, and means for automatically generating natural language responses that correspond to the user's emotional state based on the acquired information. This makes it possible to provide appropriate information considering the user's emotional state and improve the quality of customer service in physical stores.

[0193] "Means of receiving inquiries from users in natural language" refers to technologies that allow a system to acquire inquiries made by users using natural language in voice or text format as input.

[0194] "Methods for analyzing inquiries, extracting keywords, and recognizing emotional states" refers to technologies that analyze natural language data entered by users to identify important keywords, and simultaneously determine the user's emotions based on their voice and context.

[0195] "Methods for obtaining related information using vector search technology" refers to a technology that quantifies extracted keywords and efficiently searches for related information from a database based on similarity.

[0196] "Methods for automatically generating responses in natural language" refers to technologies that automatically generate responses using natural phrasing based on the information acquired by the system and the user's emotional state.

[0197] "Means of presenting information that includes emotional considerations" refers to techniques that use expressions that reflect the user's emotional state to present information to the user in a more user-friendly way.

[0198] "Means of accumulating feedback" refers to technologies that record user reactions to the results output by a system and use them for future improvements.

[0199] The system for carrying out this invention consists of a user, a terminal, and a server. The user makes inquiries using natural language through an application on a smart device. The terminal acquires these inquiries as voice or text data and uses an emotion analysis engine to determine the user's emotional state. Specific hardware includes smartphones and tablets, and the software can utilize Google® Cloud Natural Language API or IBM Watson® Natural Language Understanding for emotion analysis.

[0200] The analyzed data is sent to a server, where a natural language processing engine analyzes the query and vectorizes the keywords. This process uses a common database management system (e.g., MySQL®, PostgreSQL). The server then uses a generative AI model to generate responses tailored to the user's emotional state. During the generation stage, the tone and content of the text are adjusted to provide customers with a sense of security and friendliness.

[0201] For example, if a user inputs into the app that they are "anxious about buying a new mobile phone," the sentiment analysis engine will sense the user's anxiety from their tone of voice and the content of their speech. Based on this, the server will generate a tailored response such as, "Don't worry, we can provide support regarding your mobile phone selection." An example of a prompt to input into the generating AI model would be, "The customer appears anxious about purchasing a new product. Please provide reassuring information."

[0202] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0203] Step 1:

[0204] Users input questions and information requests in natural language, either by voice or text, through applications on their smart devices. This input is data (voice signals or text data) acquired by the device. The data is then passed to the application through the device's input system.

[0205] Step 2:

[0206] The device sends the acquired voice or text data to an emotion analysis engine to analyze the user's emotional state. This analysis takes into account parameters such as voice tone, speaking speed, and selected words. The analysis engine uses the Google Cloud Natural Language API to identify the type of emotion (e.g., anxiety, joy). As a result, data indicating the user's emotional state is generated and sent to the server.

[0207] Step 3:

[0208] The server receives user inquiries and sentiment data sent from the terminal. A natural language processing engine is used to extract keywords from the inquiries. This process uses text analysis techniques to identify important words and converts the results into a vector format. The extracted keywords are then used to perform database searches within the server.

[0209] Step 4:

[0210] The server uses vectorized keywords and vector search technology to retrieve relevant information from the database. This retrieval process uses a similarity evaluation algorithm to select the most relevant information. The resulting information is then used to generate answers.

[0211] Step 5:

[0212] The server automatically generates natural language responses using a generative AI model based on the acquired information and user sentiment data. The generative AI model has the ability to construct information with a tone and expression appropriate to the emotional state. The responses generated at this stage are adjusted to include appropriate emotional considerations for the user.

[0213] Step 6:

[0214] Finally, the server sends the generated response to the terminal for the user to use. The terminal displays the response in natural language on the user's screen, or reads it aloud using speech synthesis technology. Based on this information, the user can then make appropriate decisions and take appropriate actions.

[0215] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0216] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0217] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0218] [Second Embodiment]

[0219] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0220] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0221] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0222] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0223] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0224] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0225] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0226] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0227] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0228] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0229] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0230] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0231] This invention provides an embodiment of an information provision system based on natural language queries. This system mainly consists of three components: a user, a terminal, and a server.

[0232] When users need information, they input questions in natural language using devices such as computers or smartphones. This input is done through an interface available to the user. The interface includes features such as text input fields and voice input functions, and is designed with user-friendliness in mind.

[0233] The terminal receives input from the user and transmits it to the server via the network. The terminal incorporates a communication module to facilitate this transmission. The terminal also receives responses from the server and presents them appropriately to the user. Screen display and audio output functions are utilized to allow the user to receive information visually or audibly.

[0234] After receiving a query from a terminal, the server analyzes the query using a natural language processing engine. Specifically, it extracts important information through processes such as tokenization, syntactic analysis, and keyword extraction. Subsequently, it retrieves related information from a database using vector search technology. This information is represented as a multidimensional vector, and the most suitable one is selected through similarity calculation.

[0235] Furthermore, the server uses a generative AI model to automatically generate natural language responses based on relevant information. These generated responses are optimized to be natural and easy to understand. This process takes care to ensure, for example, that the responses are consistent, non-contradictory, and helpful to the user's understanding.

[0236] As a concrete example, consider a case where a user asks, "What are the opening hours of my nearest city hall?" The terminal receiving this inquiry sends the content to the server. The server refers to local information and public facility data and generates a specific answer, such as, "City hall opening hours are from 9 am to 5 pm on weekdays." This answer is then provided to the user in real time via the terminal.

[0237] Through these embodiments, the present invention aims to improve the efficiency of information provision and significantly enhance user convenience by responding quickly and accurately to a wide range of user inquiries.

[0238] The following describes the processing flow.

[0239] Step 1:

[0240] The user enters questions requiring information into the device using natural language. Input can be in text or voice format.

[0241] Step 2:

[0242] The terminal receives input from the user and appropriately converts the input data into a digital format. It then prepares the converted data to be sent to the server via the network.

[0243] Step 3:

[0244] The server receives user inquiries sent from the terminal. The inquiry content is analyzed using a natural language processing engine, tokenized, tagged with parts of speech, and key keywords are extracted.

[0245] Step 4:

[0246] The server vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. The most suitable set of information is then selected through similarity calculation.

[0247] Step 5:

[0248] Based on the information acquired by the server, a generative AI model is used to automatically generate responses in natural language. The responses are structured to be easily understood by the user.

[0249] Step 6:

[0250] The server prepares to send the final generated response to the terminal. If necessary, it performs minor formatting to maintain the integrity of the output data.

[0251] Step 7:

[0252] The terminal displays the response received from the server to the user. The display is presented in a format that is convenient for the user, such as on-screen text or audio output.

[0253] Step 8:

[0254] Check if the user was satisfied with the displayed answer and provide feedback to request further details if necessary.

[0255] The feedback is sent to the server for future accuracy improvements.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] Existing information provision systems struggle to generate quick and accurate responses to diverse natural language inquiries from users. Furthermore, there is a need for efficient methods to effectively evaluate the relevance of information and generate optimal answers.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting information units, and a device for using computational techniques to acquire related information using the information units. This makes it possible to respond quickly to complex user inquiries and generate optimal and consistent answers.

[0261] A "user" refers to an individual or organization that seeks to obtain information through a system.

[0262] "Natural language" refers to the linguistic forms that humans use on a daily basis, and which require special processing for computers to understand.

[0263] An "inquiry" refers to a question expressed in natural language that a user wants to know.

[0264] "Device" refers to hardware or software components used to perform a function.

[0265] "Analysis" refers to the process of understanding the structure and content of input natural language and extracting its meaning.

[0266] An "information unit" refers to a semantically important element or keyword obtained through analysis.

[0267] "Computational techniques" refer to algorithms and methods for efficiently processing and retrieving data.

[0268] "Related information" refers to the information necessary to generate an appropriate response to an inquiry.

[0269] "Consistency" refers to a state where information or answers are not contradictory and are logically consistent.

[0270] This embodiment of the invention provides a configuration primarily consisting of three elements: a user, a terminal, and a server.

[0271] Users can make inquiries using natural language via devices such as computers and smartphones. This allows users to intuitively obtain information. Since the devices are equipped with interfaces that allow text input and voice input, users only need to input their questions.

[0272] The terminal receives natural language inquiries from the user and transmits this data to the server via the internet. To ensure smooth communication, the terminal is equipped with an appropriate communication module. Furthermore, after receiving a response from the server, it is equipped with a display device or audio output device to appropriately present that response to the user.

[0273] The server receives user queries sent from terminals and performs analysis using a natural language processing engine. Specifically, it tokenizes the queries, parses them syntactically, and extracts information units. After this, it uses computational techniques to retrieve related information from the database based on the extracted information units. For data retrieval, it uses vector search techniques to perform similarity calculations and select the most relevant information.

[0274] Furthermore, the server utilizes a generative AI model to automatically generate responses in natural language using the acquired relevant information. These responses are optimized to be natural, easy to understand, and consistent for the user.

[0275] A concrete example is when a user asks, "Please tell me the location of the nearest hospital." The device sends this inquiry to the server. The server refers to local information and facility data, generates a specific answer such as, "The nearest hospital is XX Hospital, and its address is YY," and provides it to the user through the device.

[0276] An example of a prompt might be: "The user has asked, 'What is the weather forecast for this weekend?' Generate an appropriate natural language response to this inquiry. Refer to weather forecast information and ensure consistency."

[0277] In this way, the present invention enables rapid and accurate responses to diverse user inquiries and provides clear and easy-to-understand information.

[0278] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0279] Step 1:

[0280] The user inputs an inquiry in natural language using the interface of a smartphone or computer. For example, the user sends text such as "What's the weather forecast for tomorrow?" or voice input to the terminal. At this stage, natural language text data is inputted.

[0281] Step 2:

[0282] The terminal receives the input data from the user and sends it to the server via the network. The communication module on the terminal packetizes these data and sends it to the Internet. The output at this stage is passed to the server as structured data.

[0283] Step 3:

[0284] The server receives the data sent from the terminal. Using a natural language processing engine, first the input text is tokenized, and the sentence is decomposed into individual words and phrases. Then, syntactic analysis is performed to extract important information units and keywords. The input is natural language text, and the output is the parsed keywords.

[0285] Step 4:

[0286] Using the extracted keywords, the server searches the relevant database using vector search technology. Here, similarity calculations such as Euclidean distance are performed to find the information most closely related to the input keywords. The output of this process is obtained as a set of retrieved relevant information.

[0287] Step 5:

[0288] The server generates a natural language answer using an AI model based on the acquired relevant information. Specifically, it is a process of assembling the information along the context into a coherent text. The output here is natural language text for providing to the user.

[0289] Step 6:

[0290] The server sends the generated response to the terminal. The terminal receives this information and presents it to the user visually or audibly. For example, it may display text on a screen or read it aloud using a text-to-speech system. The output of this step is the response displayed on the user's terminal.

[0291] (Application Example 1)

[0292] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0293] In emergencies, providing timely and accurate information is difficult, and there is a need to quickly deliver appropriate route guidance information to users. Furthermore, the quality of responses to natural language inquiries and the system's own learning capabilities need to be improved.

[0294] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0295] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting specific information, and a device for providing appropriate route guidance information in emergencies. This enables the rapid and accurate provision of information in emergencies and improves the quality of route guidance.

[0296] A "user" is someone who makes queries to the system using natural language.

[0297] "Natural language" refers to the form of language that humans use on a daily basis, and the form used as input to a system.

[0298] A "device for receiving inquiries" is a device for receiving questions from users that are written in natural language.

[0299] A "device for extracting specific information" is a device that analyzes received inquiries and extracts the necessary information.

[0300] "Multidimensional search technology" is a technique that represents information within a database as multidimensional vectors and performs searches based on similarity.

[0301] A "device for acquiring related information" is a device for retrieving related information from a database based on specific information that has been extracted.

[0302] A "device that automatically generates responses" is a device that generates answers for the user in a natural way based on acquired information.

[0303] "Route guidance information" refers to instructional information provided as a means of appropriate evacuation or movement during emergencies.

[0304] A "device that monitors operation and stores evaluation information for learning" is a device that monitors the performance of a system and stores data to help improve the system.

[0305] The system for implementing this invention is configured to allow users to make inquiries in natural language using a smartphone or smart device. The natural language inquiry entered by the user is transmitted to the server via the network through the terminal.

[0306] The server first parses the query and uses a natural language processing engine to extract specific information. Specifically, it uses spaCy, an open-source natural language processing library, for tokenization, parsing, and information extraction. Then, vector search technology is used to quickly and efficiently retrieve data related to the extracted information. A database management system (e.g., PostgreSQL) can be used for this process.

[0307] Based on the acquired information, the server uses a generative AI model (e.g., GPT of OpenAI) to generate a natural language answer for the user. This answer is optimized to be in a realistic and easy-to-understand form.

[0308] Furthermore, this system is practical even in emergencies. For example, when a user asks "Tell me the evacuation route" during a fire, the server immediately obtains the optimal evacuation route from the database and generates a specific guidance such as "The nearest emergency exit from your current location is 50 meters northeast."

[0309] The following is an example of a prompt sentence: "Please provide the location information of the nearest emergency exit. The user is located in front of a home computer."

[0310] In this way, this system aims to enable quick and accurate information provision in various situations including emergencies, enhancing the safety and satisfaction of users.

[0311] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0312] Step 1:

[0313] The user uses a smartphone or smart device to input a natural language inquiry. This input is made via the interface of the terminal in text or voice form. The terminal prepares to send this input data to the server through the network.

[0314] Step 2:

[0315] The server receives the natural language inquiry sent from the terminal. The input text data is analyzed by a natural language processing engine (e.g., spaCy), and tokenization and syntactic analysis are performed. As a result, the specific information of the inquiry is extracted. The output is data containing the extracted specific information.

[0316] Step 3:

[0317] The server uses the extracted specific information to perform vector search techniques. It uses a database system (e.g., PostgreSQL) to search for and retrieve relevant information. In this process, the information is represented in a multidimensional vector format, and similarity calculations are performed to select the most relevant information. The output is the most relevant information.

[0318] Step 4:

[0319] The server uses the acquired relevant information to automatically generate a natural language response using a generative AI model (e.g., OpenAI's GPT). This generation process considers the consistency and accuracy of the information. The output is a natural language response provided to the user.

[0320] Step 5:

[0321] The server sends the generated response to the terminal over the network. The terminal displays the received response to the user. This display is done via text or voice output. The user can receive the information provided through the terminal in real time.

[0322] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0323] This invention provides an embodiment of an information processing system that recognizes a user's emotional state in response to a natural language query and provides appropriate information. This system consists of a user, a terminal, a server, and an emotion engine.

[0324] When users input questions or requests for information in natural language, their emotional state in that situation is also taken into consideration. Users make inquiries via text or voice using a computer or smartphone. The input data is then processed on the device.

[0325] The device receives input from the user and uses an emotion engine to analyze the user's emotions. This analysis includes various parameters such as voice tone, word choice, and input speed. The results of the analysis are generated as data indicating the user's emotional state. This emotional state data is also sent to the server.

[0326] The server analyzes the input from the terminal using a natural language processing engine based on the inquiry content and sentiment data received, and extracts keywords. Then, it vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. At this point, the server selects a response from the retrieved information that is adjusted based on sentiment.

[0327] The server uses a generative AI model to automatically generate responses that adjust tone and expression based on data from the emotion engine. These generated responses include emotional consideration for the user. For example, if the user is feeling anxious, calm and friendly language will be used.

[0328] As a concrete example, consider a scenario where a user enters a message into the device stating, "I'm feeling anxious about buying a new mobile phone." In this case, the emotion engine recognizes the user's anxiety based on their tone of voice and chosen words. Based on this information, the server generates a response that is both helpful and reassuring. It then presents the user with a response such as, "Don't worry, we can help you choose a mobile phone."

[0329] This system aims to improve the user experience by flexibly responding to the user's psychological state while accurately providing the necessary information.

[0330] The following describes the processing flow.

[0331] Step 1:

[0332] Users input questions and requests for information using natural language on their own devices. This input can be in text or voice format.

[0333] Step 2:

[0334] The device receives input from the user and records it as digital data. The received input is then passed directly to the emotion engine.

[0335] Step 3:

[0336] The device uses an emotion engine to analyze user input and recognize the user's emotional state from their voice tone and expressions. The recognized emotion data is sent to the server along with the inquiry content.

[0337] Step 4:

[0338] The server analyzes the inquiry content and sentiment data received from the terminal. A natural language processing engine is used to extract keywords from the inquiry.

[0339] Step 5:

[0340] The server uses the extracted keywords to retrieve relevant information from the database using vector search technology. At this time, it extracts the most suitable information based on similarity calculations.

[0341] Step 6:

[0342] The server uses an AI model based on sentiment data to adjust the tone and expression when generating responses, constructing responses that are sensitive to the user's emotions.

[0343] Step 7:

[0344] The server sends the generated response to the terminal. The response is formatted on the terminal in a way that the user can easily understand.

[0345] Step 8:

[0346] The device receives a response from the server and displays it to the user. The display format is either text or audio, and the expression is tailored to the user's emotions.

[0347] Step 9:

[0348] The user reviews the displayed answers and asks additional questions or provides feedback if necessary. This feedback is sent to the server to improve the system.

[0349] (Example 2)

[0350] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] In modern information processing systems, when users make inquiries using natural language, there is a need to provide appropriate information that takes their emotional state into consideration. However, conventional systems have suffered from insufficient response generation based on user emotions, resulting in a limited user experience. There is a need for technology that enables the provision of information in a tone that matches the user's psychological state.

[0352] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0353] In this invention, the server includes means for analyzing user inquiries and identifying emotional states, means for extracting keywords based on emotional states and inquiry content, and means for automatically generating responses in an emotionally appropriate tone from the acquired information. This makes it possible to provide information that takes the user's emotions into consideration.

[0354] A "user" refers to an individual or organization that uses an information processing system to make queries in natural language.

[0355] "Natural language" refers to forms of communication composed of language structures that humans use on a daily basis.

[0356] "Inquiry" refers to text or audio information entered by a user with the intention of requesting information from the system.

[0357] "Emotional state" refers to data that analyzes and classifies the psychological and emotional state that users exhibit when making inquiries.

[0358] "Keywords" are important words or phrases extracted from the inquiry text and used for information retrieval and response generation.

[0359] "Vector search technology" refers to a technical method for efficiently searching for related information by converting extracted keywords into numerical vectors.

[0360] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to automatically generate responses in natural language.

[0361] "Feedback" refers to the data obtained from evaluating the operation of an information processing system and using it for future improvements and learning.

[0362] In this information processing system, the user, terminal, server, and emotion engine are the main components. Users can ask questions and request information in natural language using a computer or smartphone. Input methods include text or voice, and the data is transmitted to the terminal immediately.

[0363] The device receives a user inquiry and first uses an emotion engine to analyze the user's emotional state. This emotion analysis uses a variety of data parameters, such as voice tone, word choice, and input speed. This analysis generates data indicating the user's emotional state.

[0364] After sentiment analysis, the data is sent from the terminal to the server. The server uses a natural language processing engine to analyze the received query and extract keywords. These extracted keywords are then vectorized, and the server uses vector search technology to retrieve relevant information from the database. Similarity evaluation is also performed during the information retrieval process to select the most relevant information.

[0365] The server utilizes a generative AI model to automatically generate responses with adjusted tone and expression based on the user's emotional state. These generated responses are then delivered to the user in an emotionally sensitive manner. For example, if a user inquires that they are feeling anxious about purchasing a new mobile phone, the emotion engine recognizes this anxiety, and the server generates a friendly response such as, "Don't worry, we can help you choose a mobile phone."

[0366] An example of a prompt sentence to input into a generative AI model is, "Provide the user with advice on purchasing a mobile phone that will alleviate their anxiety." In this way, it becomes possible to improve the user experience by understanding the user's emotions and providing appropriate and useful information.

[0367] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0368] Step 1:

[0369] Users make inquiries using natural language via computers or smartphones. Input is either text or voice, and this data is sent to the device. The inquiry data is then processed in the next step.

[0370] Step 2:

[0371] The device receives user inquiry data and uses an emotion engine to analyze the user's emotional state. This analysis generates data by examining multiple parameters such as voice tone, input speed, and word choice. The output is data indicating the emotional state determined by the analysis.

[0372] Step 3:

[0373] The terminal sends the analyzed sentiment data and query content to the server. Once the sentiment data and query content are transferred to the server, they are used for the next processing step.

[0374] Step 4:

[0375] The server uses a natural language processing engine to extract keywords based on the received inquiry content and sentiment data. It identifies the most relevant keywords from the input content and generates vectorized keywords as output data.

[0376] Step 5:

[0377] The server utilizes vectorized keywords and employs vector search technology to retrieve relevant information from the database. It evaluates the similarity of the information and selects the most appropriate information. The output is filtered relevant information.

[0378] Step 6:

[0379] The server uses a generative AI model to automatically generate natural language responses with appropriate tone and expression based on the user's emotional state. This process takes acquired information and emotional data as input, and the output is a refined response that takes the user's emotions into consideration.

[0380] Step 7:

[0381] The server sends the generated response back to the terminal, which then presents it to the user. Based on that response, the user can decide on their next course of action with a sense of security and confidence.

[0382] (Application Example 2)

[0383] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0384] In today's world, users routinely acquire vast amounts of information through digital platforms. Consequently, there is a growing demand for information tailored to users' emotional states. However, conventional systems struggle to adequately consider user emotions in this regard. This is particularly true in brick-and-mortar stores, where accurately perceiving and responding to customer emotions directly impacts customer satisfaction and loyalty. Yet, there is a lack of systems capable of real-time analysis and information provision for diverse emotions.

[0385] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0386] In this invention, the server includes means for receiving inquiries from users in natural language, means for analyzing the inquiries, extracting keywords, and recognizing the emotional state, and means for automatically generating natural language responses that correspond to the user's emotional state based on the acquired information. This makes it possible to provide appropriate information considering the user's emotional state and improve the quality of customer service in physical stores.

[0387] "Means of receiving inquiries from users in natural language" refers to technologies that allow a system to acquire inquiries made by users using natural language in voice or text format as input.

[0388] "Methods for analyzing inquiries, extracting keywords, and recognizing emotional states" refers to technologies that analyze natural language data entered by users to identify important keywords, and simultaneously determine the user's emotions based on their voice and context.

[0389] "Methods for obtaining related information using vector search technology" refers to a technology that quantifies extracted keywords and efficiently searches for related information from a database based on similarity.

[0390] "Methods for automatically generating responses in natural language" refers to technologies that automatically generate responses using natural phrasing based on the information acquired by the system and the user's emotional state.

[0391] "Means of presenting information that includes emotional considerations" refers to techniques that use expressions that reflect the user's emotional state to present information to the user in a more user-friendly way.

[0392] "Means of accumulating feedback" refers to technologies that record user reactions to the results output by a system and use them for future improvements.

[0393] The system for carrying out this invention consists of a user, a terminal, and a server. The user makes inquiries using natural language through an application on a smart device. The terminal acquires these inquiries as voice or text data and uses an emotion analysis engine to determine the user's emotional state. Specific hardware includes smartphones and tablets, and the software can utilize Google Cloud Natural Language API or IBM Watson Natural Language Understanding for emotion analysis.

[0394] The analyzed data is sent to a server, where a natural language processing engine analyzes the query and vectorizes the keywords. This process utilizes a common database management system (e.g., MySQL, PostgreSQL). The server then uses a generative AI model to generate responses tailored to the user's emotional state. During the generation stage, the tone and content of the text are adjusted to provide customers with a sense of security and friendliness.

[0395] For example, if a user inputs into the app that they are "anxious about buying a new mobile phone," the sentiment analysis engine will sense the user's anxiety from their tone of voice and the content of their speech. Based on this, the server will generate a tailored response such as, "Don't worry, we can provide support regarding your mobile phone selection." An example of a prompt to input into the generating AI model would be, "The customer appears anxious about purchasing a new product. Please provide reassuring information."

[0396] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0397] Step 1:

[0398] Users input questions and information requests in natural language, either by voice or text, through applications on their smart devices. This input is data (voice signals or text data) acquired by the device. The data is then passed to the application through the device's input system.

[0399] Step 2:

[0400] The device sends the acquired voice or text data to an emotion analysis engine to analyze the user's emotional state. This analysis takes into account parameters such as voice tone, speaking speed, and selected words. The analysis engine uses the Google Cloud Natural Language API to identify the type of emotion (e.g., anxiety, joy). As a result, data indicating the user's emotional state is generated and sent to the server.

[0401] Step 3:

[0402] The server receives user inquiries and sentiment data sent from the terminal. A natural language processing engine is used to extract keywords from the inquiries. This process uses text analysis techniques to identify important words and converts the results into a vector format. The extracted keywords are then used to perform database searches within the server.

[0403] Step 4:

[0404] The server uses vectorized keywords and vector search technology to retrieve relevant information from the database. This retrieval process uses a similarity evaluation algorithm to select the most relevant information. The resulting information is then used to generate answers.

[0405] Step 5:

[0406] The server automatically generates natural language responses using a generative AI model based on the acquired information and user sentiment data. The generative AI model has the ability to construct information with a tone and expression appropriate to the emotional state. The responses generated at this stage are adjusted to include appropriate emotional considerations for the user.

[0407] Step 6:

[0408] Finally, the server sends the generated response to the terminal for the user to use. The terminal displays the response in natural language on the user's screen, or reads it aloud using speech synthesis technology. Based on this information, the user can then make appropriate decisions and take appropriate actions.

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0410] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0412] [Third Embodiment]

[0413] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0414] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0415] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0417] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0419] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0420] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0421] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0424] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0425] This invention provides an embodiment of an information provision system based on natural language queries. This system mainly consists of three components: a user, a terminal, and a server.

[0426] When users need information, they input questions in natural language using devices such as computers or smartphones. This input is done through an interface available to the user. The interface includes features such as text input fields and voice input functions, and is designed with user-friendliness in mind.

[0427] The terminal receives input from the user and transmits it to the server via the network. The terminal incorporates a communication module to facilitate this transmission. The terminal also receives responses from the server and presents them appropriately to the user. Screen display and audio output functions are utilized to allow the user to receive information visually or audibly.

[0428] After receiving a query from a terminal, the server analyzes the query using a natural language processing engine. Specifically, it extracts important information through processes such as tokenization, syntactic analysis, and keyword extraction. Subsequently, it retrieves related information from a database using vector search technology. This information is represented as a multidimensional vector, and the most suitable one is selected through similarity calculation.

[0429] Furthermore, the server uses a generative AI model to automatically generate natural language responses based on relevant information. These generated responses are optimized to be natural and easy to understand. This process takes care to ensure, for example, that the responses are consistent, non-contradictory, and helpful to the user's understanding.

[0430] As a concrete example, consider a case where a user asks, "What are the opening hours of my nearest city hall?" The terminal receiving this inquiry sends the content to the server. The server refers to local information and public facility data and generates a specific answer, such as, "City hall opening hours are from 9 am to 5 pm on weekdays." This answer is then provided to the user in real time via the terminal.

[0431] Through these embodiments, the present invention aims to improve the efficiency of information provision and significantly enhance user convenience by responding quickly and accurately to a wide range of user inquiries.

[0432] The following describes the processing flow.

[0433] Step 1:

[0434] The user enters questions requiring information into the device using natural language. Input can be in text or voice format.

[0435] Step 2:

[0436] The terminal receives input from the user and appropriately converts the input data into a digital format. It then prepares the converted data to be sent to the server via the network.

[0437] Step 3:

[0438] The server receives user inquiries sent from the terminal. The inquiry content is analyzed using a natural language processing engine, tokenized, tagged with parts of speech, and key keywords are extracted.

[0439] Step 4:

[0440] The server vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. The most suitable set of information is then selected through similarity calculation.

[0441] Step 5:

[0442] Based on the information acquired by the server, a generative AI model is used to automatically generate responses in natural language. The responses are structured to be easily understood by the user.

[0443] Step 6:

[0444] The server prepares to send the final generated response to the terminal. If necessary, it performs minor formatting to maintain the integrity of the output data.

[0445] Step 7:

[0446] The terminal displays the response received from the server to the user. The display is presented in a format that is convenient for the user, such as on-screen text or audio output.

[0447] Step 8:

[0448] Check if the user was satisfied with the displayed answer and provide feedback to request further details if necessary.

[0449] The feedback is sent to the server for future accuracy improvements.

[0450] (Example 1)

[0451] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0452] Existing information provision systems struggle to generate quick and accurate responses to diverse natural language inquiries from users. Furthermore, there is a need for efficient methods to effectively evaluate the relevance of information and generate optimal answers.

[0453] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0454] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting information units, and a device for using computational techniques to acquire related information using the information units. This makes it possible to respond quickly to complex user inquiries and generate optimal and consistent answers.

[0455] A "user" refers to an individual or organization that seeks to obtain information through a system.

[0456] "Natural language" refers to the linguistic forms that humans use on a daily basis, and which require special processing for computers to understand.

[0457] An "inquiry" refers to a question expressed in natural language that a user wants to know.

[0458] "Device" refers to hardware or software components used to perform a function.

[0459] "Analysis" refers to the process of understanding the structure and content of input natural language and extracting its meaning.

[0460] An "information unit" refers to a semantically important element or keyword obtained through analysis.

[0461] "Computational techniques" refer to algorithms and methods for efficiently processing and retrieving data.

[0462] "Related information" refers to the information necessary to generate an appropriate response to an inquiry.

[0463] "Consistency" refers to a state where information or answers are not contradictory and are logically consistent.

[0464] This embodiment of the invention provides a configuration primarily consisting of three elements: a user, a terminal, and a server.

[0465] Users can make inquiries using natural language via devices such as computers and smartphones. This allows users to intuitively obtain information. Since the devices are equipped with interfaces that allow text input and voice input, users only need to input their questions.

[0466] The terminal receives natural language inquiries from the user and transmits this data to the server via the internet. To ensure smooth communication, the terminal is equipped with an appropriate communication module. Furthermore, after receiving a response from the server, it is equipped with a display device or audio output device to appropriately present that response to the user.

[0467] The server receives user queries sent from terminals and performs analysis using a natural language processing engine. Specifically, it tokenizes the queries, parses them syntactically, and extracts information units. After this, it uses computational techniques to retrieve related information from the database based on the extracted information units. For data retrieval, it uses vector search techniques to perform similarity calculations and select the most relevant information.

[0468] Furthermore, the server utilizes a generative AI model to automatically generate responses in natural language using the acquired relevant information. These responses are optimized to be natural, easy to understand, and consistent for the user.

[0469] A concrete example is when a user asks, "Please tell me the location of the nearest hospital." The device sends this inquiry to the server. The server refers to local information and facility data, generates a specific answer such as, "The nearest hospital is XX Hospital, and its address is YY," and provides it to the user through the device.

[0470] An example of a prompt might be: "The user has asked, 'What is the weather forecast for this weekend?' Generate an appropriate natural language response to this inquiry. Refer to weather forecast information and ensure consistency."

[0471] In this way, the present invention enables rapid and accurate responses to diverse user inquiries and provides clear and easy-to-understand information.

[0472] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0473] Step 1:

[0474] Users input inquiries in natural language using a smartphone or computer interface. For example, they might send a message like "What's the weather forecast for tomorrow?" to their device via text or voice input. At this stage, natural language text data is input.

[0475] Step 2:

[0476] The terminal receives input data from the user and sends it to the server via the network. The communication module on the terminal packets this data and sends it over the internet. At this stage, the output is passed to the server as structured data.

[0477] Step 3:

[0478] The server receives data sent from the terminal. Using a natural language processing engine, it first tokenizes the input text, breaking down the sentence into individual words and phrases. Then, it performs syntactic analysis to extract important information units and keywords. The input is natural language text, and the output is the analyzed keywords.

[0479] Step 4:

[0480] Using the extracted keywords, the server employs vector search techniques to explore relevant databases. Similarity calculations, such as Euclidean distance, are performed to find the information most closely related to the input keywords. The output of this process is obtained as a collection of retrieved relevant information.

[0481] Step 5:

[0482] The server uses a generative AI model to generate natural language responses based on the relevant information it has acquired. Specifically, this is a process of assembling information in context and creating coherent sentences. The output here is natural language text to be provided to the user.

[0483] Step 6:

[0484] The server sends the generated response to the terminal. The terminal receives this information and presents it to the user visually or audibly. For example, it may display text on a screen or read it aloud using a text-to-speech system. The output of this step is the response displayed on the user's terminal.

[0485] (Application Example 1)

[0486] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0487] In emergencies, providing timely and accurate information is difficult, and there is a need to quickly deliver appropriate route guidance information to users. Furthermore, the quality of responses to natural language inquiries and the system's own learning capabilities need to be improved.

[0488] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0489] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting specific information, and a device for providing appropriate route guidance information in emergencies. This enables the rapid and accurate provision of information in emergencies and improves the quality of route guidance.

[0490] A "user" is someone who makes queries to the system using natural language.

[0491] "Natural language" refers to the form of language that humans use on a daily basis, and the form used as input to a system.

[0492] A "device for receiving inquiries" is a device for receiving questions from users that are written in natural language.

[0493] A "device for extracting specific information" is a device that analyzes received inquiries and extracts the necessary information.

[0494] "Multidimensional search technology" is a technique that represents information within a database as multidimensional vectors and performs searches based on similarity.

[0495] A "device for acquiring related information" is a device for retrieving related information from a database based on specific information that has been extracted.

[0496] A "device that automatically generates responses" is a device that generates answers for the user in a natural way based on acquired information.

[0497] "Route guidance information" refers to instructional information provided as a means of appropriate evacuation or movement during emergencies.

[0498] A "device that monitors operation and stores evaluation information for learning" is a device that monitors the performance of a system and stores data to help improve the system.

[0499] The system for implementing this invention is configured to allow users to make inquiries in natural language using a smartphone or smart device. The natural language inquiry entered by the user is transmitted to the server via the network through the terminal.

[0500] The server first parses the query and uses a natural language processing engine to extract specific information. Specifically, it uses spaCy, an open-source natural language processing library, for tokenization, parsing, and information extraction. Then, vector search technology is used to quickly and efficiently retrieve data related to the extracted information. A database management system (e.g., PostgreSQL) can be used for this process.

[0501] Based on the acquired information, the server uses a generative AI model (e.g., OpenAI's GPT) to generate natural language responses to provide to the user. These responses are optimized to be realistic and easy to understand.

[0502] Furthermore, this system is also practical in emergencies. For example, if a user asks "What is the evacuation route?" during a fire, the server will instantly retrieve the optimal evacuation route from the database and generate specific instructions such as "The nearest emergency exit from your current location is 50 meters northeast."

[0503] The following is an example of a prompt message: "Please provide the location of the nearest emergency exit. The user is located in front of a home computer."

[0504] In this way, the system aims to enhance user safety and satisfaction by enabling the rapid and accurate provision of information in a variety of situations, including emergencies.

[0505] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0506] Step 1:

[0507] The user enters a query in natural language using a smartphone or smart device. This input is made via the device's interface in text or voice format. The device then prepares to send this input data to the server over the network.

[0508] Step 2:

[0509] The server receives natural language queries sent from the terminal. The input text data is parsed by a natural language processing engine (e.g., spaCy), undergoing tokenization and syntactic analysis. This extracts specific information from the query. The output is data containing the extracted specific information.

[0510] Step 3:

[0511] The server uses the extracted specific information to perform vector search techniques. It uses a database system (e.g., PostgreSQL) to search for and retrieve relevant information. In this process, the information is represented in a multidimensional vector format, and similarity calculations are performed to select the most relevant information. The output is the most relevant information.

[0512] Step 4:

[0513] The server uses the acquired relevant information to automatically generate a natural language response using a generative AI model (e.g., OpenAI's GPT). This generation process considers the consistency and accuracy of the information. The output is a natural language response provided to the user.

[0514] Step 5:

[0515] The server sends the generated response to the terminal over the network. The terminal displays the received response to the user. This display is done via text or voice output. The user can receive the information provided through the terminal in real time.

[0516] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0517] This invention provides an embodiment of an information processing system that recognizes a user's emotional state in response to a natural language query and provides appropriate information. This system consists of a user, a terminal, a server, and an emotion engine.

[0518] When users input questions or requests for information in natural language, their emotional state in that situation is also taken into consideration. Users make inquiries via text or voice using a computer or smartphone. The input data is then processed on the device.

[0519] The device receives input from the user and uses an emotion engine to analyze the user's emotions. This analysis includes various parameters such as voice tone, word choice, and input speed. The results of the analysis are generated as data indicating the user's emotional state. This emotional state data is also sent to the server.

[0520] The server analyzes the input from the terminal using a natural language processing engine based on the inquiry content and sentiment data received, and extracts keywords. Then, it vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. At this point, the server selects a response from the retrieved information that is adjusted based on sentiment.

[0521] The server uses a generative AI model to automatically generate responses that adjust tone and expression based on data from the emotion engine. These generated responses include emotional consideration for the user. For example, if the user is feeling anxious, calm and friendly language will be used.

[0522] As a concrete example, consider a scenario where a user enters a message into the device stating, "I'm feeling anxious about buying a new mobile phone." In this case, the emotion engine recognizes the user's anxiety based on their tone of voice and chosen words. Based on this information, the server generates a response that is both helpful and reassuring. It then presents the user with a response such as, "Don't worry, we can help you choose a mobile phone."

[0523] This system aims to improve the user experience by flexibly responding to the user's psychological state while accurately providing the necessary information.

[0524] The following describes the processing flow.

[0525] Step 1:

[0526] Users input questions and requests for information using natural language on their own devices. This input can be in text or voice format.

[0527] Step 2:

[0528] The device receives input from the user and records it as digital data. The received input is then passed directly to the emotion engine.

[0529] Step 3:

[0530] The device uses an emotion engine to analyze user input and recognize the user's emotional state from their voice tone and expressions. The recognized emotion data is sent to the server along with the inquiry content.

[0531] Step 4:

[0532] The server analyzes the inquiry content and sentiment data received from the terminal. A natural language processing engine is used to extract keywords from the inquiry.

[0533] Step 5:

[0534] The server uses the extracted keywords to retrieve relevant information from the database using vector search technology. At this time, it extracts the most suitable information based on similarity calculations.

[0535] Step 6:

[0536] The server uses an AI model based on sentiment data to adjust the tone and expression when generating responses, constructing responses that are sensitive to the user's emotions.

[0537] Step 7:

[0538] The server sends the generated response to the terminal. The response is formatted on the terminal in a way that the user can easily understand.

[0539] Step 8:

[0540] The device receives a response from the server and displays it to the user. The display format is either text or audio, and the expression is tailored to the user's emotions.

[0541] Step 9:

[0542] The user reviews the displayed answers and asks additional questions or provides feedback if necessary. This feedback is sent to the server to improve the system.

[0543] (Example 2)

[0544] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0545] In modern information processing systems, when users make inquiries using natural language, there is a need to provide appropriate information that takes their emotional state into consideration. However, conventional systems have suffered from insufficient response generation based on user emotions, resulting in a limited user experience. There is a need for technology that enables the provision of information in a tone that matches the user's psychological state.

[0546] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0547] In this invention, the server includes means for analyzing user inquiries and identifying emotional states, means for extracting keywords based on emotional states and inquiry content, and means for automatically generating responses in an emotionally appropriate tone from the acquired information. This makes it possible to provide information that takes the user's emotions into consideration.

[0548] A "user" refers to an individual or organization that uses an information processing system to make queries in natural language.

[0549] "Natural language" refers to forms of communication composed of language structures that humans use on a daily basis.

[0550] "Inquiry" refers to text or audio information entered by a user with the intention of requesting information from the system.

[0551] "Emotional state" refers to data that analyzes and classifies the psychological and emotional state that users exhibit when making inquiries.

[0552] "Keywords" are important words or phrases extracted from the inquiry text and used for information retrieval and response generation.

[0553] "Vector search technology" refers to a technical method for efficiently searching for related information by converting extracted keywords into numerical vectors.

[0554] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to automatically generate responses in natural language.

[0555] "Feedback" refers to the data obtained from evaluating the operation of an information processing system and using it for future improvements and learning.

[0556] In this information processing system, the user, terminal, server, and emotion engine are the main components. Users can ask questions and request information in natural language using a computer or smartphone. Input methods include text or voice, and the data is transmitted to the terminal immediately.

[0557] The device receives a user inquiry and first uses an emotion engine to analyze the user's emotional state. This emotion analysis uses a variety of data parameters, such as voice tone, word choice, and input speed. This analysis generates data indicating the user's emotional state.

[0558] After sentiment analysis, the data is sent from the terminal to the server. The server uses a natural language processing engine to analyze the received query and extract keywords. These extracted keywords are then vectorized, and the server uses vector search technology to retrieve relevant information from the database. Similarity evaluation is also performed during the information retrieval process to select the most relevant information.

[0559] The server utilizes a generative AI model to automatically generate responses with adjusted tone and expression based on the user's emotional state. These generated responses are then delivered to the user in an emotionally sensitive manner. For example, if a user inquires that they are feeling anxious about purchasing a new mobile phone, the emotion engine recognizes this anxiety, and the server generates a friendly response such as, "Don't worry, we can help you choose a mobile phone."

[0560] An example of a prompt sentence to input into a generative AI model is, "Provide the user with advice on purchasing a mobile phone that will alleviate their anxiety." In this way, it becomes possible to improve the user experience by understanding the user's emotions and providing appropriate and useful information.

[0561] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0562] Step 1:

[0563] Users make inquiries using natural language via computers or smartphones. Input is either text or voice, and this data is sent to the device. The inquiry data is then processed in the next step.

[0564] Step 2:

[0565] The device receives user inquiry data and uses an emotion engine to analyze the user's emotional state. This analysis generates data by examining multiple parameters such as voice tone, input speed, and word choice. The output is data indicating the emotional state determined by the analysis.

[0566] Step 3:

[0567] The terminal sends the analyzed sentiment data and query content to the server. Once the sentiment data and query content are transferred to the server, they are used for the next processing step.

[0568] Step 4:

[0569] The server uses a natural language processing engine to extract keywords based on the received inquiry content and sentiment data. It identifies the most relevant keywords from the input content and generates vectorized keywords as output data.

[0570] Step 5:

[0571] The server utilizes vectorized keywords and employs vector search technology to retrieve relevant information from the database. It evaluates the similarity of the information and selects the most appropriate information. The output is filtered relevant information.

[0572] Step 6:

[0573] The server uses a generative AI model to automatically generate natural language responses with appropriate tone and expression based on the user's emotional state. This process takes acquired information and emotional data as input, and the output is a refined response that takes the user's emotions into consideration.

[0574] Step 7:

[0575] The server sends the generated response back to the terminal, which then presents it to the user. Based on that response, the user can decide on their next course of action with a sense of security and confidence.

[0576] (Application Example 2)

[0577] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0578] In today's world, users routinely acquire vast amounts of information through digital platforms. Consequently, there is a growing demand for information tailored to users' emotional states. However, conventional systems struggle to adequately consider user emotions in this regard. This is particularly true in brick-and-mortar stores, where accurately perceiving and responding to customer emotions directly impacts customer satisfaction and loyalty. Yet, there is a lack of systems capable of real-time analysis and information provision for diverse emotions.

[0579] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0580] In this invention, the server includes means for receiving inquiries from users in natural language, means for analyzing the inquiries, extracting keywords, and recognizing the emotional state, and means for automatically generating natural language responses that correspond to the user's emotional state based on the acquired information. This makes it possible to provide appropriate information considering the user's emotional state and improve the quality of customer service in physical stores.

[0581] "Means of receiving inquiries from users in natural language" refers to technologies that allow a system to acquire inquiries made by users using natural language in voice or text format as input.

[0582] "Methods for analyzing inquiries, extracting keywords, and recognizing emotional states" refers to technologies that analyze natural language data entered by users to identify important keywords, and simultaneously determine the user's emotions based on their voice and context.

[0583] "Methods for obtaining related information using vector search technology" refers to a technology that quantifies extracted keywords and efficiently searches for related information from a database based on similarity.

[0584] "Methods for automatically generating responses in natural language" refers to technologies that automatically generate responses using natural phrasing based on the information acquired by the system and the user's emotional state.

[0585] "Means of presenting information that includes emotional considerations" refers to techniques that use expressions that reflect the user's emotional state to present information to the user in a more user-friendly way.

[0586] "Means of accumulating feedback" refers to technologies that record user reactions to the results output by a system and use them for future improvements.

[0587] The system for carrying out this invention consists of a user, a terminal, and a server. The user makes inquiries using natural language through an application on a smart device. The terminal acquires these inquiries as voice or text data and uses an emotion analysis engine to determine the user's emotional state. Specific hardware includes smartphones and tablets, and the software can utilize Google Cloud Natural Language API or IBM Watson Natural Language Understanding for emotion analysis.

[0588] The analyzed data is sent to a server, where a natural language processing engine analyzes the query and vectorizes the keywords. This process utilizes a common database management system (e.g., MySQL, PostgreSQL). The server then uses a generative AI model to generate responses tailored to the user's emotional state. During the generation stage, the tone and content of the text are adjusted to provide customers with a sense of security and friendliness.

[0589] For example, if a user inputs into the app that they are "anxious about buying a new mobile phone," the sentiment analysis engine will sense the user's anxiety from their tone of voice and the content of their speech. Based on this, the server will generate a tailored response such as, "Don't worry, we can provide support regarding your mobile phone selection." An example of a prompt to input into the generating AI model would be, "The customer appears anxious about purchasing a new product. Please provide reassuring information."

[0590] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0591] Step 1:

[0592] Users input questions and information requests in natural language, either by voice or text, through applications on their smart devices. This input is data (voice signals or text data) acquired by the device. The data is then passed to the application through the device's input system.

[0593] Step 2:

[0594] The device sends the acquired voice or text data to an emotion analysis engine to analyze the user's emotional state. This analysis takes into account parameters such as voice tone, speaking speed, and selected words. The analysis engine uses the Google Cloud Natural Language API to identify the type of emotion (e.g., anxiety, joy). As a result, data indicating the user's emotional state is generated and sent to the server.

[0595] Step 3:

[0596] The server receives user inquiries and sentiment data sent from the terminal. A natural language processing engine is used to extract keywords from the inquiries. This process uses text analysis techniques to identify important words and converts the results into a vector format. The extracted keywords are then used to perform database searches within the server.

[0597] Step 4:

[0598] The server uses vectorized keywords and vector search technology to retrieve relevant information from the database. This retrieval process uses a similarity evaluation algorithm to select the most relevant information. The resulting information is then used to generate answers.

[0599] Step 5:

[0600] The server automatically generates natural language responses using a generative AI model based on the acquired information and user sentiment data. The generative AI model has the ability to construct information with a tone and expression appropriate to the emotional state. The responses generated at this stage are adjusted to include appropriate emotional considerations for the user.

[0601] Step 6:

[0602] Finally, the server sends the generated response to the terminal for the user to use. The terminal displays the response in natural language on the user's screen, or reads it aloud using speech synthesis technology. Based on this information, the user can then make appropriate decisions and take appropriate actions.

[0603] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0604] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0605] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0606] [Fourth Embodiment]

[0607] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0608] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0609] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0610] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0611] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0612] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0613] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0614] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0615] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0616] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0617] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0618] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0619] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0620] This invention provides an embodiment of an information provision system based on natural language queries. This system mainly consists of three components: a user, a terminal, and a server.

[0621] When users need information, they input questions in natural language using devices such as computers or smartphones. This input is done through an interface available to the user. The interface includes features such as text input fields and voice input functions, and is designed with user-friendliness in mind.

[0622] The terminal receives input from the user and transmits it to the server via the network. The terminal incorporates a communication module to facilitate this transmission. The terminal also receives responses from the server and presents them appropriately to the user. Screen display and audio output functions are utilized to allow the user to receive information visually or audibly.

[0623] After receiving a query from a terminal, the server analyzes the query using a natural language processing engine. Specifically, it extracts important information through processes such as tokenization, syntactic analysis, and keyword extraction. Subsequently, it retrieves related information from a database using vector search technology. This information is represented as a multidimensional vector, and the most suitable one is selected through similarity calculation.

[0624] Furthermore, the server uses a generative AI model to automatically generate natural language responses based on relevant information. These generated responses are optimized to be natural and easy to understand. This process takes care to ensure, for example, that the responses are consistent, non-contradictory, and helpful to the user's understanding.

[0625] As a concrete example, consider a case where a user asks, "What are the opening hours of my nearest city hall?" The terminal receiving this inquiry sends the content to the server. The server refers to local information and public facility data and generates a specific answer, such as, "City hall opening hours are from 9 am to 5 pm on weekdays." This answer is then provided to the user in real time via the terminal.

[0626] Through these embodiments, the present invention aims to improve the efficiency of information provision and significantly enhance user convenience by responding quickly and accurately to a wide range of user inquiries.

[0627] The following describes the processing flow.

[0628] Step 1:

[0629] The user enters questions requiring information into the device using natural language. Input can be in text or voice format.

[0630] Step 2:

[0631] The terminal receives input from the user and appropriately converts the input data into a digital format. It then prepares the converted data to be sent to the server via the network.

[0632] Step 3:

[0633] The server receives user inquiries sent from the terminal. The inquiry content is analyzed using a natural language processing engine, tokenized, tagged with parts of speech, and key keywords are extracted.

[0634] Step 4:

[0635] The server vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. The most suitable set of information is then selected through similarity calculation.

[0636] Step 5:

[0637] Based on the information acquired by the server, a generative AI model is used to automatically generate responses in natural language. The responses are structured to be easily understood by the user.

[0638] Step 6:

[0639] The server prepares to send the final generated response to the terminal. If necessary, it performs minor formatting to maintain the integrity of the output data.

[0640] Step 7:

[0641] The terminal displays the response received from the server to the user. The display is presented in a format that is convenient for the user, such as on-screen text or audio output.

[0642] Step 8:

[0643] Check if the user was satisfied with the displayed answer and provide feedback to request further details if necessary.

[0644] The feedback is sent to the server for future accuracy improvements.

[0645] (Example 1)

[0646] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0647] Existing information provision systems struggle to generate quick and accurate responses to diverse natural language inquiries from users. Furthermore, there is a need for efficient methods to effectively evaluate the relevance of information and generate optimal answers.

[0648] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0649] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting information units, and a device for using computational techniques to acquire related information using the information units. This makes it possible to respond quickly to complex user inquiries and generate optimal and consistent answers.

[0650] A "user" refers to an individual or organization that seeks to obtain information through a system.

[0651] "Natural language" refers to the linguistic forms that humans use on a daily basis, and which require special processing for computers to understand.

[0652] An "inquiry" refers to a question expressed in natural language that a user wants to know.

[0653] "Device" refers to hardware or software components used to perform a function.

[0654] "Analysis" refers to the process of understanding the structure and content of input natural language and extracting its meaning.

[0655] An "information unit" refers to a semantically important element or keyword obtained through analysis.

[0656] "Computational techniques" refer to algorithms and methods for efficiently processing and retrieving data.

[0657] "Related information" refers to the information necessary to generate an appropriate response to an inquiry.

[0658] "Consistency" refers to a state where information or answers are not contradictory and are logically consistent.

[0659] This embodiment of the invention provides a configuration primarily consisting of three elements: a user, a terminal, and a server.

[0660] Users can make inquiries using natural language via devices such as computers and smartphones. This allows users to intuitively obtain information. Since the devices are equipped with interfaces that allow text input and voice input, users only need to input their questions.

[0661] The terminal receives natural language inquiries from the user and transmits this data to the server via the internet. To ensure smooth communication, the terminal is equipped with an appropriate communication module. Furthermore, after receiving a response from the server, it is equipped with a display device or audio output device to appropriately present that response to the user.

[0662] The server receives user queries sent from terminals and performs analysis using a natural language processing engine. Specifically, it tokenizes the queries, parses them syntactically, and extracts information units. After this, it uses computational techniques to retrieve related information from the database based on the extracted information units. For data retrieval, it uses vector search techniques to perform similarity calculations and select the most relevant information.

[0663] Furthermore, the server utilizes a generative AI model to automatically generate responses in natural language using the acquired relevant information. These responses are optimized to be natural, easy to understand, and consistent for the user.

[0664] A concrete example is when a user asks, "Please tell me the location of the nearest hospital." The device sends this inquiry to the server. The server refers to local information and facility data, generates a specific answer such as, "The nearest hospital is XX Hospital, and its address is YY," and provides it to the user through the device.

[0665] An example of a prompt might be: "The user has asked, 'What is the weather forecast for this weekend?' Generate an appropriate natural language response to this inquiry. Refer to weather forecast information and ensure consistency."

[0666] In this way, the present invention enables rapid and accurate responses to diverse user inquiries and provides clear and easy-to-understand information.

[0667] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0668] Step 1:

[0669] Users input inquiries in natural language using a smartphone or computer interface. For example, they might send a message like "What's the weather forecast for tomorrow?" to their device via text or voice input. At this stage, natural language text data is input.

[0670] Step 2:

[0671] The terminal receives input data from the user and sends it to the server via the network. The communication module on the terminal packets this data and sends it over the internet. At this stage, the output is passed to the server as structured data.

[0672] Step 3:

[0673] The server receives data sent from the terminal. Using a natural language processing engine, it first tokenizes the input text, breaking down the sentence into individual words and phrases. Then, it performs syntactic analysis to extract important information units and keywords. The input is natural language text, and the output is the analyzed keywords.

[0674] Step 4:

[0675] Using the extracted keywords, the server employs vector search techniques to explore relevant databases. Similarity calculations, such as Euclidean distance, are performed to find the information most closely related to the input keywords. The output of this process is obtained as a collection of retrieved relevant information.

[0676] Step 5:

[0677] The server uses a generative AI model to generate natural language responses based on the relevant information it has acquired. Specifically, this is a process of assembling information in context and creating coherent sentences. The output here is natural language text to be provided to the user.

[0678] Step 6:

[0679] The server sends the generated response to the terminal. The terminal receives this information and presents it to the user visually or audibly. For example, it may display text on a screen or read it aloud using a text-to-speech system. The output of this step is the response displayed on the user's terminal.

[0680] (Application Example 1)

[0681] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0682] In emergencies, providing timely and accurate information is difficult, and there is a need to quickly deliver appropriate route guidance information to users. Furthermore, the quality of responses to natural language inquiries and the system's own learning capabilities need to be improved.

[0683] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0684] In this invention, the server includes a device for receiving inquiries from users in natural language, a device for analyzing the inquiries and extracting specific information, and a device for providing appropriate route guidance information in emergencies. This enables the rapid and accurate provision of information in emergencies and improves the quality of route guidance.

[0685] A "user" is someone who makes queries to the system using natural language.

[0686] "Natural language" refers to the form of language that humans use on a daily basis, and the form used as input to a system.

[0687] A "device for receiving inquiries" is a device for receiving questions from users that are written in natural language.

[0688] A "device for extracting specific information" is a device that analyzes received inquiries and extracts the necessary information.

[0689] "Multidimensional search technology" is a technique that represents information within a database as multidimensional vectors and performs searches based on similarity.

[0690] A "device for acquiring related information" is a device for retrieving related information from a database based on specific information that has been extracted.

[0691] A "device that automatically generates responses" is a device that generates answers for the user in a natural way based on acquired information.

[0692] "Route guidance information" refers to instructional information provided as a means of appropriate evacuation or movement during emergencies.

[0693] A "device that monitors operation and stores evaluation information for learning" is a device that monitors the performance of a system and stores data to help improve the system.

[0694] The system for implementing this invention is configured to allow users to make inquiries in natural language using a smartphone or smart device. The natural language inquiry entered by the user is transmitted to the server via the network through the terminal.

[0695] The server first parses the query and uses a natural language processing engine to extract specific information. Specifically, it uses spaCy, an open-source natural language processing library, for tokenization, parsing, and information extraction. Then, vector search technology is used to quickly and efficiently retrieve data related to the extracted information. A database management system (e.g., PostgreSQL) can be used for this process.

[0696] Based on the acquired information, the server uses a generative AI model (e.g., OpenAI's GPT) to generate natural language responses to provide to the user. These responses are optimized to be realistic and easy to understand.

[0697] Furthermore, this system is also practical in emergencies. For example, if a user asks "What is the evacuation route?" during a fire, the server will instantly retrieve the optimal evacuation route from the database and generate specific instructions such as "The nearest emergency exit from your current location is 50 meters northeast."

[0698] The following is an example of a prompt message: "Please provide the location of the nearest emergency exit. The user is located in front of a home computer."

[0699] In this way, the system aims to enhance user safety and satisfaction by enabling the rapid and accurate provision of information in a variety of situations, including emergencies.

[0700] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0701] Step 1:

[0702] The user enters a query in natural language using a smartphone or smart device. This input is made via the device's interface in text or voice format. The device then prepares to send this input data to the server over the network.

[0703] Step 2:

[0704] The server receives natural language queries sent from the terminal. The input text data is parsed by a natural language processing engine (e.g., spaCy), undergoing tokenization and syntactic analysis. This extracts specific information from the query. The output is data containing the extracted specific information.

[0705] Step 3:

[0706] The server uses the extracted specific information to perform vector search techniques. It uses a database system (e.g., PostgreSQL) to search for and retrieve relevant information. In this process, the information is represented in a multidimensional vector format, and similarity calculations are performed to select the most relevant information. The output is the most relevant information.

[0707] Step 4:

[0708] The server uses the acquired relevant information to automatically generate a natural language response using a generative AI model (e.g., OpenAI's GPT). This generation process considers the consistency and accuracy of the information. The output is a natural language response provided to the user.

[0709] Step 5:

[0710] The server sends the generated response to the terminal over the network. The terminal displays the received response to the user. This display is done via text or voice output. The user can receive the information provided through the terminal in real time.

[0711] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0712] This invention provides an embodiment of an information processing system that recognizes a user's emotional state in response to a natural language query and provides appropriate information. This system consists of a user, a terminal, a server, and an emotion engine.

[0713] When users input questions or requests for information in natural language, their emotional state in that situation is also taken into consideration. Users make inquiries via text or voice using a computer or smartphone. The input data is then processed on the device.

[0714] The device receives input from the user and uses an emotion engine to analyze the user's emotions. This analysis includes various parameters such as voice tone, word choice, and input speed. The results of the analysis are generated as data indicating the user's emotional state. This emotional state data is also sent to the server.

[0715] The server analyzes the input from the terminal using a natural language processing engine based on the inquiry content and sentiment data received, and extracts keywords. Then, it vectorizes the extracted keywords and uses vector search technology to retrieve relevant information from the database. At this point, the server selects a response from the retrieved information that is adjusted based on sentiment.

[0716] The server uses a generative AI model to automatically generate responses that adjust tone and expression based on data from the emotion engine. These generated responses include emotional consideration for the user. For example, if the user is feeling anxious, calm and friendly language will be used.

[0717] As a concrete example, consider a scenario where a user enters a message into the device stating, "I'm feeling anxious about buying a new mobile phone." In this case, the emotion engine recognizes the user's anxiety based on their tone of voice and chosen words. Based on this information, the server generates a response that is both helpful and reassuring. It then presents the user with a response such as, "Don't worry, we can help you choose a mobile phone."

[0718] This system aims to improve the user experience by flexibly responding to the user's psychological state while accurately providing the necessary information.

[0719] The following describes the processing flow.

[0720] Step 1:

[0721] Users input questions and requests for information using natural language on their own devices. This input can be in text or voice format.

[0722] Step 2:

[0723] The device receives input from the user and records it as digital data. The received input is then passed directly to the emotion engine.

[0724] Step 3:

[0725] The device uses an emotion engine to analyze user input and recognize the user's emotional state from their voice tone and expressions. The recognized emotion data is sent to the server along with the inquiry content.

[0726] Step 4:

[0727] The server analyzes the inquiry content and sentiment data received from the terminal. A natural language processing engine is used to extract keywords from the inquiry.

[0728] Step 5:

[0729] The server uses the extracted keywords to retrieve relevant information from the database using vector search technology. At this time, it extracts the most suitable information based on similarity calculations.

[0730] Step 6:

[0731] The server uses an AI model based on sentiment data to adjust the tone and expression when generating responses, constructing responses that are sensitive to the user's emotions.

[0732] Step 7:

[0733] The server sends the generated response to the terminal. The response is formatted on the terminal in a way that the user can easily understand.

[0734] Step 8:

[0735] The device receives a response from the server and displays it to the user. The display format is either text or audio, and the expression is tailored to the user's emotions.

[0736] Step 9:

[0737] The user reviews the displayed answers and asks additional questions or provides feedback if necessary. This feedback is sent to the server to improve the system.

[0738] (Example 2)

[0739] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0740] In modern information processing systems, when users make inquiries using natural language, there is a need to provide appropriate information that takes their emotional state into consideration. However, conventional systems have suffered from insufficient response generation based on user emotions, resulting in a limited user experience. There is a need for technology that enables the provision of information in a tone that matches the user's psychological state.

[0741] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0742] In this invention, the server includes means for analyzing user inquiries and identifying emotional states, means for extracting keywords based on emotional states and inquiry content, and means for automatically generating responses in an emotionally appropriate tone from the acquired information. This makes it possible to provide information that takes the user's emotions into consideration.

[0743] A "user" refers to an individual or organization that uses an information processing system to make queries in natural language.

[0744] "Natural language" refers to forms of communication composed of language structures that humans use on a daily basis.

[0745] "Inquiry" refers to text or audio information entered by a user with the intention of requesting information from the system.

[0746] "Emotional state" refers to data that analyzes and classifies the psychological and emotional state that users exhibit when making inquiries.

[0747] "Keywords" are important words or phrases extracted from the inquiry text and used for information retrieval and response generation.

[0748] "Vector search technology" refers to a technical method for efficiently searching for related information by converting extracted keywords into numerical vectors.

[0749] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to automatically generate responses in natural language.

[0750] "Feedback" refers to the data obtained from evaluating the operation of an information processing system and using it for future improvements and learning.

[0751] In this information processing system, the user, terminal, server, and emotion engine are the main components. Users can ask questions and request information in natural language using a computer or smartphone. Input methods include text or voice, and the data is transmitted to the terminal immediately.

[0752] The device receives a user inquiry and first uses an emotion engine to analyze the user's emotional state. This emotion analysis uses a variety of data parameters, such as voice tone, word choice, and input speed. This analysis generates data indicating the user's emotional state.

[0753] After sentiment analysis, the data is sent from the terminal to the server. The server uses a natural language processing engine to analyze the received query and extract keywords. These extracted keywords are then vectorized, and the server uses vector search technology to retrieve relevant information from the database. Similarity evaluation is also performed during the information retrieval process to select the most relevant information.

[0754] The server utilizes a generative AI model to automatically generate responses with adjusted tone and expression based on the user's emotional state. These generated responses are then delivered to the user in an emotionally sensitive manner. For example, if a user inquires that they are feeling anxious about purchasing a new mobile phone, the emotion engine recognizes this anxiety, and the server generates a friendly response such as, "Don't worry, we can help you choose a mobile phone."

[0755] An example of a prompt sentence to input into a generative AI model is, "Provide the user with advice on purchasing a mobile phone that will alleviate their anxiety." In this way, it becomes possible to improve the user experience by understanding the user's emotions and providing appropriate and useful information.

[0756] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0757] Step 1:

[0758] Users make inquiries using natural language via computers or smartphones. Input is either text or voice, and this data is sent to the device. The inquiry data is then processed in the next step.

[0759] Step 2:

[0760] The device receives user inquiry data and uses an emotion engine to analyze the user's emotional state. This analysis generates data by examining multiple parameters such as voice tone, input speed, and word choice. The output is data indicating the emotional state determined by the analysis.

[0761] Step 3:

[0762] The terminal sends the analyzed sentiment data and query content to the server. Once the sentiment data and query content are transferred to the server, they are used for the next processing step.

[0763] Step 4:

[0764] The server uses a natural language processing engine to extract keywords based on the received inquiry content and sentiment data. It identifies the most relevant keywords from the input content and generates vectorized keywords as output data.

[0765] Step 5:

[0766] The server utilizes vectorized keywords and employs vector search technology to retrieve relevant information from the database. It evaluates the similarity of the information and selects the most appropriate information. The output is filtered relevant information.

[0767] Step 6:

[0768] The server uses a generative AI model to automatically generate natural language responses with appropriate tone and expression based on the user's emotional state. This process takes acquired information and emotional data as input, and the output is a refined response that takes the user's emotions into consideration.

[0769] Step 7:

[0770] The server sends the generated response back to the terminal, which then presents it to the user. Based on that response, the user can decide on their next course of action with a sense of security and confidence.

[0771] (Application Example 2)

[0772] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0773] In today's world, users routinely acquire vast amounts of information through digital platforms. Consequently, there is a growing demand for information tailored to users' emotional states. However, conventional systems struggle to adequately consider user emotions in this regard. This is particularly true in brick-and-mortar stores, where accurately perceiving and responding to customer emotions directly impacts customer satisfaction and loyalty. Yet, there is a lack of systems capable of real-time analysis and information provision for diverse emotions.

[0774] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0775] In this invention, the server includes means for receiving inquiries from users in natural language, means for analyzing the inquiries, extracting keywords, and recognizing the emotional state, and means for automatically generating natural language responses that correspond to the user's emotional state based on the acquired information. This makes it possible to provide appropriate information considering the user's emotional state and improve the quality of customer service in physical stores.

[0776] "Means of receiving inquiries from users in natural language" refers to technologies that allow a system to acquire inquiries made by users using natural language in voice or text format as input.

[0777] "Methods for analyzing inquiries, extracting keywords, and recognizing emotional states" refers to technologies that analyze natural language data entered by users to identify important keywords, and simultaneously determine the user's emotions based on their voice and context.

[0778] "Methods for obtaining related information using vector search technology" refers to a technology that quantifies extracted keywords and efficiently searches for related information from a database based on similarity.

[0779] "Methods for automatically generating responses in natural language" refers to technologies that automatically generate responses using natural phrasing based on the information acquired by the system and the user's emotional state.

[0780] "Means of presenting information that includes emotional considerations" refers to techniques that use expressions that reflect the user's emotional state to present information to the user in a more user-friendly way.

[0781] "Means of accumulating feedback" refers to technologies that record user reactions to the results output by a system and use them for future improvements.

[0782] The system for carrying out this invention consists of a user, a terminal, and a server. The user makes inquiries using natural language through an application on a smart device. The terminal acquires these inquiries as voice or text data and uses an emotion analysis engine to determine the user's emotional state. Specific hardware includes smartphones and tablets, and the software can utilize Google Cloud Natural Language API or IBM Watson Natural Language Understanding for emotion analysis.

[0783] The analyzed data is sent to a server, where a natural language processing engine analyzes the query and vectorizes the keywords. This process utilizes a common database management system (e.g., MySQL, PostgreSQL). The server then uses a generative AI model to generate responses tailored to the user's emotional state. During the generation stage, the tone and content of the text are adjusted to provide customers with a sense of security and friendliness.

[0784] For example, if a user inputs into the app that they are "anxious about buying a new mobile phone," the sentiment analysis engine will sense the user's anxiety from their tone of voice and the content of their speech. Based on this, the server will generate a tailored response such as, "Don't worry, we can provide support regarding your mobile phone selection." An example of a prompt to input into the generating AI model would be, "The customer appears anxious about purchasing a new product. Please provide reassuring information."

[0785] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0786] Step 1:

[0787] Users input questions and information requests in natural language, either by voice or text, through applications on their smart devices. This input is data (voice signals or text data) acquired by the device. The data is then passed to the application through the device's input system.

[0788] Step 2:

[0789] The device sends the acquired voice or text data to an emotion analysis engine to analyze the user's emotional state. This analysis takes into account parameters such as voice tone, speaking speed, and selected words. The analysis engine uses the Google Cloud Natural Language API to identify the type of emotion (e.g., anxiety, joy). As a result, data indicating the user's emotional state is generated and sent to the server.

[0790] Step 3:

[0791] The server receives user inquiries and sentiment data sent from the terminal. A natural language processing engine is used to extract keywords from the inquiries. This process uses text analysis techniques to identify important words and converts the results into a vector format. The extracted keywords are then used to perform database searches within the server.

[0792] Step 4:

[0793] The server uses vectorized keywords and vector search technology to retrieve relevant information from the database. This retrieval process uses a similarity evaluation algorithm to select the most relevant information. The resulting information is then used to generate answers.

[0794] Step 5:

[0795] The server automatically generates natural language responses using a generative AI model based on the acquired information and user sentiment data. The generative AI model has the ability to construct information with a tone and expression appropriate to the emotional state. The responses generated at this stage are adjusted to include appropriate emotional considerations for the user.

[0796] Step 6:

[0797] Finally, the server sends the generated response to the terminal for the user to use. The terminal displays the response in natural language on the user's screen, or reads it aloud using speech synthesis technology. Based on this information, the user can then make appropriate decisions and take appropriate actions.

[0798] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0799] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0801] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0802] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0803] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0804] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0805] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0806] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0807] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0808] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0809] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0810] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0811] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0812] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0813] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0814] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0815] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0816] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0817] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0818] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0819] The following is further disclosed regarding the embodiments described above.

[0820] (Claim 1)

[0821] A means of receiving inquiries from users in natural language,

[0822] A means for analyzing the aforementioned query and extracting keywords,

[0823] A means of obtaining related information using vector search technology with extracted keywords,

[0824] A means of automatically generating a response in natural language based on the acquired information,

[0825] A means of providing the generated answer to the user,

[0826] A means for monitoring the operation of the aforementioned means and accumulating feedback for use in learning,

[0827] An information processing system that includes this.

[0828] (Claim 2)

[0829] The information processing system according to claim 1, wherein the information acquisition means evaluates the similarity of the acquired information and selects the optimal information.

[0830] (Claim 3)

[0831] The information processing system according to claim 1, wherein the response generation means generates a response by integrating information obtained from multiple information sources.

[0832] "Example 1"

[0833] (Claim 1)

[0834] A device that accepts inquiries from users in natural language,

[0835] A device that analyzes the aforementioned query and extracts information units,

[0836] A device that uses extracted information units to acquire related information using computational techniques,

[0837] A device that automatically generates responses in natural language based on acquired information,

[0838] A device that provides the generated answer to the user,

[0839] A device for observing the operation of the aforementioned device and accumulating information for use in learning,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, wherein the information acquisition device evaluates the similarity of the acquired information and selects the optimal information.

[0843] (Claim 3)

[0844] The system according to claim 1, wherein the response generation device generates a response by integrating information obtained from multiple information sources.

[0845] "Application Example 1"

[0846] (Claim 1)

[0847] A device that accepts inquiries from users in natural language,

[0848] A device that analyzes the aforementioned query and extracts specific information,

[0849] A device that uses extracted specific information to acquire related information using multidimensional search technology,

[0850] A device that automatically generates responses in natural language based on acquired information,

[0851] A device that provides the generated response to the user,

[0852] A device that provides appropriate route guidance information in emergencies,

[0853] A device that monitors the operation of the aforementioned device and stores evaluation information for learning,

[0854] An information management system that includes this.

[0855] (Claim 2)

[0856] The information management system according to claim 1, wherein the related information acquisition device evaluates the similarity of the acquired information and selects the optimal information.

[0857] (Claim 3)

[0858] The information management system according to claim 1, wherein the response generation device integrates information obtained from multiple information sources to generate a response.

[0859] "Example 2 of combining an emotion engine"

[0860] (Claim 1)

[0861] A means of receiving inquiries from users in natural language,

[0862] A means for analyzing the aforementioned inquiry and identifying the user's emotional state,

[0863] A means for extracting keywords based on the aforementioned emotional state and inquiry content,

[0864] A means of obtaining relevant information using vector search technology with extracted keywords,

[0865] A means to automatically generate natural language responses in a tone appropriate to the user's emotional state from the acquired information,

[0866] A means of providing the generated answer to the user,

[0867] A means for monitoring the emotion analysis and response generation processes and accumulating feedback for use in learning,

[0868] A system that includes this.

[0869] (Claim 2)

[0870] The system according to claim 1, wherein the information acquisition means evaluates the similarity of the acquired information, selects the optimal information, and adjusts the information based on the user's emotional state.

[0871] (Claim 3)

[0872] The system according to claim 1, wherein the response generation means integrates information obtained from multiple sources and generates a response that takes into account the user's feelings.

[0873] "Application example 2 of combining emotional engines"

[0874] (Claim 1)

[0875] A means of receiving inquiries from users in natural language,

[0876] A means for analyzing the aforementioned inquiry, extracting keywords, and recognizing the emotional state,

[0877] A means of obtaining relevant information using vector search technology with extracted keywords and emotional states,

[0878] A means to automatically generate natural language responses that correspond to the user's emotional state based on the acquired information,

[0879] A means of providing the user with the generated response and presenting information, including emotional considerations, through an interface,

[0880] A means for monitoring the operation of the aforementioned means and accumulating feedback for use in learning,

[0881] A system that includes this.

[0882] (Claim 2)

[0883] The system according to claim 1, wherein the information acquisition means evaluates the similarity of the acquired information and selects the optimal information based on the user's emotional state.

[0884] (Claim 3)

[0885] The system according to claim 1, wherein the response generation means integrates information obtained from multiple sources and generates a response in a tone appropriate to the user's emotional state. [Explanation of Symbols]

[0886] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving inquiries from users in natural language, A means for analyzing the aforementioned query and extracting keywords, A means of obtaining related information using vector search technology with extracted keywords, A means of automatically generating a response in natural language based on the acquired information, A means of providing the generated answer to the user, A means for monitoring the operation of the aforementioned means and accumulating feedback for use in learning, An information processing system that includes this.

2. The information processing system according to claim 1, wherein the information acquisition means evaluates the similarity of the acquired information and selects the optimal information.

3. The information processing system according to claim 1, wherein the response generation means generates a response by integrating information obtained from multiple information sources.

Images