system

Abstract

An object of the system according to the embodiment is to generate an appropriate response to an utterance of a user and to cooperate with an external service.SOLUTION: A system according to an embodiment includes a reception unit, an analysis unit, a provision unit, and a cooperation unit. The reception unit receives an utterance of a user. The analyzer analyzes the utterance received by the receiver and generates a response. The providing unit provides the response generated by the analyzing unit. The cooperation unit cooperates with an external service through an API.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies do not adequately generate appropriate responses to user comments or link with external services, so there is room for improvement.

[0005] The system according to the embodiment aims to generate an appropriate response to a user's comment and to cooperate with external services. [Means for solving the problem]

[0006] The system according to the embodiment includes a reception unit, an analysis unit, a provision unit, and a linking unit. The reception unit receives user utterances. The analysis unit analyzes the utterances received by the reception unit and generates a response. The provision unit provides the response generated by the analysis unit. The linking unit links with external services via APIs. [Effects of the Invention]

[0007] The system according to the embodiment can generate an appropriate response to a user's comment and can cooperate with external services. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A support system according to an embodiment of the present invention accepts user utterances, analyzes them with a generation AI, provides appropriate responses, and connects with external services. The support system allows users to receive support through conversations with characters selected by the user. The generation AI utilizes natural language processing and speech recognition technology to understand the user's utterances and generate appropriate responses. The system also connects with various external services via an API to provide support tailored to the user's needs. For example, the support system can manage schedules, set reminders, search for information, and play music. For example, a user launches an application and selects their favorite character. The support system then initiates a conversation with the character and discusses the support they require. For example, the system can issue an instruction such as, "Set a reminder for tomorrow's meeting." The generation AI analyzes the user's utterances and generates appropriate responses. Speech recognition technology converts the user's utterances into text, which the generation AI then analyzes. Furthermore, the support system connects with external services via an API to provide support tailored to the user's needs. For example, the system can connect with a calendar app to manage schedules or with a music streaming service to play favorite music. The information search function can also be used to quickly provide users with the information they need. This allows the support system to enable the user to efficiently perform daily tasks and reduce stress.This allows the support system to efficiently perform daily tasks and reduce stress.

[0029] The support system according to the embodiment includes a reception unit, an analysis unit, a provision unit, and a linking unit. The reception unit receives a user's utterance. The user's utterance may be, for example, voice, text, or gesture, but is not limited to these examples. The reception unit converts the user's utterance into text using, for example, speech recognition technology. The reception unit can also accept text input or gesture input. For example, the reception unit accurately converts the user's utterance into text using speech recognition technology. The analysis unit uses a generation AI to analyze the utterance received by the reception unit and generate an appropriate response. The analysis is performed based on, for example, natural language processing or a machine learning algorithm, but is not limited to these examples. For example, the analysis unit uses a generation AI (for example, a natural language generation model such as GPT-4 (registered trademark) or Gemini) to analyze the user's utterance and generate an appropriate response. The analysis unit can also use the generation AI to understand the context of the user's utterance and generate an appropriate response. The provision unit provides the response generated by the analysis unit to the user. The response may be provided in the form of, for example, a voice response, a text response, an image response, or the like, but is not limited to these examples. For example, the providing unit uses Text-to-Speech (TTS) technology to provide the generated response as voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. The linking unit links with external services through an API. Examples of external services include, but are not limited to, calendar apps, music streaming services, and information search services. For example, the linking unit links with a calendar app to manage a schedule. The linking unit can also link with a music streaming service to play music. The linking unit can also provide information that the user wants to know by using an information search function. As a result, the support system according to the embodiment can efficiently analyze user utterances, provide appropriate responses, and link with external services to provide support that meets the user's needs.

[0030] The reception unit can convert the user's utterance into text using speech recognition technology. Examples of speech recognition technology include, but are not limited to, deep learning-based speech recognition and HMM-based speech recognition. The reception unit can convert the user's utterance into text using, for example, deep learning-based speech recognition technology. The reception unit can also convert the user's utterance into text using HMM-based speech recognition technology. The reception unit can also support multiple languages ​​using speech recognition technology. For example, the reception unit recognizes speech input in multiple languages, such as English, Japanese, and French, and converts it into text data. Thus, the use of speech recognition technology allows the user's utterance to be accurately converted into text. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input speech data acquired using speech recognition technology into a generation AI and have the generation AI convert the speech data into text data.

[0031] The analysis unit can analyze the user's utterance using a generation AI and generate an appropriate response. Examples of the generation AI include, but are not limited to, natural language generation models such as GPT-4 and Gemini. The analysis unit can analyze the user's utterance using, for example, GPT-4 and generate an appropriate response. The analysis unit can also analyze the user's utterance using Gemini and generate an appropriate response. The analysis unit can also understand the context of the user's utterance and generate an appropriate response using a generation AI. For example, the analysis unit can understand the context of the user's utterance using a generation AI and generate an appropriate response. This allows the generation AI to accurately analyze the user's utterance and generate an appropriate response. Some or all of the above-described processing in the analysis unit can be performed using, for example, an AI, or can be performed without using an AI. For example, the analysis unit can input the user's utterance into the generation AI and cause the generation AI to generate an appropriate response.

[0032] The providing unit can provide the generated response to the user by voice or text. For example, the providing unit uses Text-to-Speech (TTS) technology to provide the generated response by voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. For example, the providing unit uses TTS technology to provide the generated response by voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. In this way, by providing the generated response by voice or text, information can be conveyed to the user in an appropriate form. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the generated response to a generation AI and cause the generation AI to provide the response by voice or text.

[0033] The linking unit can manage the schedule in cooperation with a calendar app. Examples of calendar apps include, but are not limited to, Google (registered trademark) Calendar and Outlook (registered trademark) Calendar. The linking unit can manage the schedule in cooperation with Google Calendar, for example. The linking unit can also manage the schedule in cooperation with Outlook (registered trademark) Calendar. The linking unit can also automatically update the user's schedule in cooperation with the calendar app. For example, the linking unit can automatically update the user's schedule in cooperation with the calendar app. This allows for more efficient schedule management for the user by linking with the calendar app. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input schedule data acquired in cooperation with the calendar app into the generation AI and cause the generation AI to perform schedule management.

[0034] The linking unit can play music in cooperation with a music streaming service. Examples of music streaming services include, but are not limited to, LINE MUSIC (registered trademark) and Apple Music (registered trademark). For example, the linking unit can play music in cooperation with LINE MUSIC. The linking unit can also play music in cooperation with Apple Music. The linking unit can also automatically update a user's playlist in cooperation with a music streaming service. For example, the linking unit can link with a music streaming service to automatically update a user's playlist. This allows the user to easily play their favorite music by linking with a music streaming service. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input music data obtained in cooperation with a music streaming service to a generation AI and cause the generation AI to play music.

[0035] The collaboration unit can provide the information the user wants to know by utilizing an information search function. Examples of information search functions include, but are not limited to, a search engine and a database search. For example, the collaboration unit can provide the information the user wants to know by utilizing a search engine. The collaboration unit can also provide the information the user wants to know by utilizing a database search. The collaboration unit can also quickly provide the information the user wants to know by utilizing the information search function. For example, the collaboration unit can quickly provide the information the user wants to know by utilizing the information search function. In this way, the user can quickly obtain the information they need by utilizing the information search function. Some or all of the above-described processing in the collaboration unit may be performed, for example, using AI or may be performed without using AI. For example, the collaboration unit can input information data acquired by utilizing the information search function into the generation AI and cause the generation AI to provide the information.

[0036] The reception unit can analyze the user's past speech history and select a reception method. For example, the reception unit prioritizes reception of phrases that the user has frequently used in the past. The reception unit can also select a reception method suitable for a specific time period from the user's past speech history. The reception unit can also select an optimal reception means (voice, text, etc.) based on the user's past speech history. In this way, the optimal reception method can be selected by analyzing the user's past speech history. Some or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's past speech history data into a generation AI and have the generation AI select the optimal reception method.

[0037] The reception unit can perform filtering based on the user's current situation and areas of interest when receiving utterances. For example, the reception unit receives only topics related to the user's current situation. The reception unit can also preferentially receive utterances containing specific keywords based on the user's areas of interest. The reception unit can also perform appropriate filtering based on the user's current situation (e.g., at work, on a break). In this way, by filtering based on the user's current situation and areas of interest, highly relevant utterances can be preferentially received. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's current situation data and area of ​​interest data to the generation AI and have the generation AI perform filtering.

[0038] When receiving a utterance, the reception unit can select a reception means according to the user's input method. For example, if the user is using voice input, the reception unit can receive the utterance using voice recognition technology. Furthermore, if the user is using text input, the reception unit can also receive the utterance using text analysis technology. Furthermore, if the user is using gesture input, the reception unit can also receive the utterance using gesture recognition technology. This improves the accuracy of utterance reception by selecting the optimal reception means according to the user's input method. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's input method data into the generation AI and cause the generation AI to select the optimal reception means.

[0039] When receiving a utterance, the reception unit can prioritize receiving highly relevant utterances by taking into account the user's geographical location information. For example, when the user is in a specific location, the reception unit prioritizes receiving utterances related to that location. The reception unit can also prioritize receiving utterances related to nearby information based on the user's current location. The reception unit can also prioritize receiving utterances related to the user's travel destination when the user is traveling. In this way, highly relevant utterances can be prioritized by taking the user's geographical location information into account. Some or all of the above-described processing in the reception unit may be performed using AI, for example, or may be performed without using AI. For example, the reception unit can input the user's geographical location information data into the generation AI and cause the generation AI to prioritize receiving highly relevant utterances.

[0040] The reception unit can analyze the user's social media activity when receiving a comment and receive related comments. For example, the reception unit can prioritize receiving comments related to content posted by the user on social media. The reception unit can also prioritize receiving topics that the user is likely to be interested in based on the user's social media activity. The reception unit can also prioritize receiving comments related to the activities of the user's friends on social media. In this way, by analyzing the user's social media activity, related comments can be preferentially received. Some or all of the above-described processing by the reception unit can be performed using, for example, AI, or can be performed without using AI. For example, the reception unit can input the user's social media activity data into the generation AI and cause the generation AI to receive related comments.

[0041] The reception unit can customize the reception method by reflecting the user's past feedback when receiving a utterance. For example, the reception unit preferentially uses the reception method that the user has previously preferred. The reception unit can also select the optimal reception method based on the user's past feedback. The reception unit can also continuously improve the reception method by reflecting the user's past feedback. In this way, the optimal reception method can be provided by reflecting the user's past feedback. Some or all of the above-described processing in the reception unit may be performed using AI, for example, or may be performed without using AI. For example, the reception unit can input the user's past feedback data into the generation AI and have the generation AI customize the reception method.

[0042] During analysis, the analysis unit can adjust the level of detail of the analysis based on the importance of the statement. For example, the analysis unit performs a detailed analysis on important statements to generate a highly accurate response. The analysis unit can also perform a standard analysis on general statements to generate an appropriate response. The analysis unit can also perform a simplified analysis on less urgent statements to generate a quick response. In this way, an appropriate response can be generated by adjusting the level of detail of the analysis based on the importance of the statement. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input statement importance data to a generation AI and have the generation AI adjust the level of detail of the analysis.

[0043] During analysis, the analysis unit can apply different analysis algorithms depending on the category of the utterance. For example, the analysis unit can apply a schedule analysis algorithm to utterances related to schedule management. The analysis unit can also apply an information search algorithm to utterances related to information search. The analysis unit can also apply a music analysis algorithm to utterances related to music playback. In this way, by applying different analysis algorithms depending on the category of the utterance, highly accurate responses can be generated. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input utterance category data into the generation AI and cause the generation AI to apply different analysis algorithms.

[0044] During analysis, the analysis unit can improve the accuracy of the analysis by referring to the user's past analysis results. For example, the analysis unit generates a highly accurate response to a similar utterance based on the user's past analysis results. The analysis unit can also adjust the analysis algorithm by referring to the user's past analysis results. The analysis unit can also accumulate the user's past analysis results and continuously improve the accuracy of the analysis. In this way, the accuracy of the analysis can be improved by referring to the user's past analysis results. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the user's past analysis result data into the generation AI and cause the generation AI to improve the accuracy of the analysis.

[0045] During analysis, the analysis unit can determine the priority of analysis based on the time when the comment was submitted. For example, the analysis unit prioritizes analysis of highly urgent comments and quickly generates a response. The analysis unit can also analyze general comments with standard priority. The analysis unit can also analyze less urgent comments later. In this way, by determining the priority of analysis based on the time when the comment was submitted, it is possible to respond quickly to highly urgent comments. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input data on the time when the comment was submitted to the generation AI and have the generation AI determine the analysis priority.

[0046] During analysis, the analysis unit can adjust the order of analysis based on the relevance of the utterances. For example, the analysis unit prioritizes analysis of highly relevant utterances and quickly generates a response. The analysis unit can also analyze utterances of general relevance in a standard order. The analysis unit can also analyze utterances of low relevance later. In this way, by adjusting the order of analysis based on the relevance of the utterances, it is possible to quickly respond to highly relevant utterances. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input utterance relevance data to the generation AI and have the generation AI adjust the order of analysis.

[0047] During analysis, the analysis unit can adjust the use of technical terminology in the analysis according to the user's level of expertise. For example, if the user has technical expertise, the analysis unit generates a response using technical terminology. Furthermore, if the user has general knowledge, the analysis unit can also generate a response in easy-to-understand language. Furthermore, if the user is a beginner, the analysis unit can also generate a response in simple language. By adjusting the use of technical terminology in the analysis according to the user's level of expertise, a response that is easy for the user to understand can be provided. Some or all of the above-described processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the user's level of expertise data into the generation AI and have the generation AI execute the use of technical terminology.

[0048] The providing unit can adjust the content of the information provided based on the importance of the response when providing the information. For example, the providing unit provides detailed information for an important response. The providing unit can also provide standard information for a general response. The providing unit can also provide simplified information for a low-urgency response. In this way, by adjusting the level of detail provided based on the importance of the response, appropriate information can be provided. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input response importance data to the generating AI and cause the generating AI to adjust the content of the information provided.

[0049] The providing unit can apply different providing means depending on the category of the response when providing the information. For example, the providing unit can provide a response related to schedule management through a calendar app. The providing unit can also provide search results as a link for a response related to an information search. The providing unit can also provide a response related to music playback through a music streaming service. In this way, by applying different providing means depending on the category of the response, it is possible to provide information in an optimal form for the user. Some or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input response category data to the generation AI and cause the generation AI to apply different providing means.

[0050] The providing unit can improve the accuracy of the provision by referring to the user's past provision results when providing the response. For example, the providing unit provides a highly accurate response to a similar utterance based on the user's past provision results. The providing unit can also adjust the provision means by referring to the user's past provision results. The providing unit can also accumulate the user's past provision results and continuously improve the accuracy of the provision. In this way, the accuracy of the provision can be improved by referring to the user's past provision results. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's past provision result data into the generation AI and cause the generation AI to improve the accuracy of the provision.

[0051] The providing unit can select the optimal providing method by taking into account the user's device information when providing information. For example, if the user is using a smartphone, the providing unit can provide a display method that matches the screen size. Furthermore, if the user is using a tablet, the providing unit can also provide a display method that is optimized for a large screen. Furthermore, if the user is using a smartwatch, the providing unit can also provide a display method that is simple and highly visible. This allows information to be provided in the optimal form by taking into account the user's device information. Some or all of the above-described processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's device information data into the generation AI and cause the generation AI to select the optimal providing method.

[0052] The providing unit can provide multilingual content according to the user's language setting when providing the content. The providing unit can automatically set the response language based on, for example, the language setting of the user's device. The providing unit can also provide a language switching function when the user uses multiple languages. The providing unit can also provide responses in a specific language when the user selects that language. This allows information to be provided in a format that is easy for the user to understand by providing multilingual support according to the user's language setting. Some or all of the above-described processing by the providing unit can be performed, for example, using AI or without AI. For example, the providing unit can input the user's language setting data into a generation AI and cause the generation AI to provide multilingual support.

[0053] The providing unit can customize the delivery method by reflecting user feedback when providing information. For example, the providing unit can preferentially use a delivery method that the user has previously preferred. The providing unit can also select the optimal delivery means based on the user's past feedback. The providing unit can also continuously improve the delivery method by reflecting the user's past feedback. In this way, information can be provided in an optimal form by reflecting the user's feedback. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input user feedback data into the generation AI and cause the generation AI to customize the delivery method.

[0054] At the time of linking, the linking unit can analyze the user's past linking history and select the optimal linking method. For example, the linking unit prioritizes linking with services that the user has frequently used in the past. The linking unit can also select a linking method suitable for a specific time period from the user's past linking history. The linking unit can also select the optimal linking means based on the user's past linking history. In this way, the optimal linking method can be selected by analyzing the user's past linking history. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input the user's past linking history data into the generation AI and cause the generation AI to select the optimal linking method.

[0055] The linking unit can customize the linking means based on the user's current situation during linking. For example, when the user is at work, the linking unit can prioritize linking with work-related services. Furthermore, when the user is on a break, the linking unit can prioritize linking with services that allow the user to relax. Furthermore, when the user is traveling, the linking unit can prioritize linking with services related to travel. This allows for customizing the linking means based on the user's current situation, thereby providing more appropriate services. Some or all of the above-described processing in the linking unit may be performed using, or without, AI, for example. For example, the linking unit can input the user's current situation data into the generation AI and cause the generation AI to customize the linking means.

[0056] The collaboration unit can improve the collaboration method by reflecting user feedback during collaboration. The collaboration unit, for example, selects the optimal collaboration means based on the user's past feedback. The collaboration unit can also continuously improve the collaboration method by reflecting user feedback. The collaboration unit can also propose new collaboration services based on user feedback. This allows the collaboration method to be continuously improved by reflecting user feedback. Some or all of the above-mentioned processing in the collaboration unit may be performed using AI, for example, or may be performed without using AI. For example, the collaboration unit can input user feedback data into the generation AI and cause the generation AI to improve the collaboration method.

[0057] When linking, the linking unit can select a linking method taking into account the user's geographical location information. For example, when the user is in a specific location, the linking unit prioritizes linking with services related to that location. The linking unit can also prioritize linking with services related to nearby information based on the user's current location. The linking unit can also prioritize linking with services related to the user's travel destination when the user is traveling. This makes it possible to select an optimal linking method by taking the user's geographical location information into consideration. Some or all of the above-described processing in the linking unit may be performed using AI, for example, or may be performed without using AI. For example, the linking unit can input the user's geographical location information data into the generation AI and cause the generation AI to select an optimal linking method.

[0058] At the time of linking, the linking unit can analyze the user's social media activity and suggest services to link. For example, the linking unit prioritizes linking with services related to content posted by the user on social media. The linking unit can also prioritize linking with services that are likely to interest the user based on the user's social media activity. The linking unit can also prioritize linking with services related to the activities of the user's friends on social media. In this way, by analyzing the user's social media activity, related services can be linked with priority. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input the user's social media activity data into the generation AI and cause the generation AI to suggest services to link with.

[0059] The collaboration unit can customize the collaboration method by reflecting the user's past feedback during collaboration. For example, the collaboration unit can prioritize the use of collaboration methods that the user has previously preferred. The collaboration unit can also select the optimal collaboration method based on the user's past feedback. The collaboration unit can also continuously improve the collaboration method by reflecting the user's past feedback. In this way, the optimal collaboration method can be provided by reflecting the user's past feedback. Some or all of the above-described processing in the collaboration unit may be performed using AI, for example, or may be performed without using AI. For example, the collaboration unit can input the user's past feedback data into the generation AI and cause the generation AI to customize the collaboration method.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] When analyzing a user's comments, the analysis unit refers to the user's past behavioral history to more accurately understand the intention of the comment. For example, it prioritizes analysis of phrases and keywords that the user has frequently used in the past. It can also generate responses appropriate for specific time periods from the user's past behavioral history. Furthermore, it can select the optimal response method (voice, text, etc.) based on the user's past behavioral history. In this way, by referring to the user's past behavioral history, it is possible to more accurately understand the intention of the comment and generate an appropriate response.

[0062] When providing the generated response to the user, the providing unit may select a providing method taking into consideration the remaining battery level of the user's device. For example, if the remaining battery level of the user's device is low, a text response may be provided preferentially. Also, if the remaining battery level is sufficient, a voice response may be provided. Furthermore, if the remaining battery level is very low, a brief response may be provided to reduce battery consumption of the device. In this way, information can be provided in an optimal form by taking into consideration the remaining battery level of the user's device.

[0063] When linking with an external service, the linking unit can select a linking method taking into account the user's current network connection status. For example, if the user is connected to Wi-Fi, high-speed data transfer can be performed. Also, if the user is using mobile data communication, it can transfer only the minimum amount of data necessary to reduce the amount of data transferred. Furthermore, if the user is offline, it can temporarily suspend linking and attempt linking again when the network connection is restored. This makes it possible to select the optimal linking method by taking into account the user's network connection status.

[0064] When accepting a user's speech, the accepting unit can analyze background sounds surrounding the user's speech and complement the content of the speech. For example, if the user is speaking in a noisy environment, the accepting unit can analyze the background sounds to remove noise and clarify the content of the speech. Also, if the user is speaking in a quiet environment, the accepting unit can analyze the background sounds to complement the content of the speech. Furthermore, if the user is making a specific sound (e.g., an alarm sound or music), the accepting unit can provide information related to that sound. In this way, by analyzing the background sounds surrounding the user's speech, the content of the speech can be complemented and an appropriate response can be taken.

[0065] When providing the generated response to the user, the providing unit can select a providing method by taking into consideration the user's current activity status. For example, if the user is driving, a voice response can be provided preferentially. Also, if the user is in a meeting, a text response can be provided. Furthermore, if the user is exercising, a concise response can be provided so as not to interfere with the user's activity. In this way, information can be provided in an optimal form by taking into consideration the user's current activity status.

[0066] When accepting a user's utterance, the reception unit can analyze the speed of the user's utterance and complement the content of the utterance. For example, if the user is speaking quickly, the reception unit can analyze the utterance speed to extract important keywords and clarify the content of the utterance. Also, if the user is speaking slowly, the reception unit can analyze the utterance speed to understand the intention of the utterance. Furthermore, if the user is speaking intermittently, the reception unit can analyze the utterance speed to complement the content of the utterance. In this way, by analyzing the utterance speed of the user, the content of the utterance can be complemented and an appropriate response can be taken.

[0067] The processing flow of the first embodiment will be briefly explained below.

[0068] Step 1: The reception unit receives user utterances. User utterances include voice, text, and gestures. The reception unit converts the user utterances into text using speech recognition technology. It can also receive text input and gesture input. Step 2: The analysis unit uses the generation AI to analyze the utterance received by the reception unit and generate an appropriate response. The analysis is based on natural language processing and machine learning algorithms. For example, the generation AI (a natural language generation model such as GPT-4 or Gemini) is used to analyze the user's utterance, understand the context, and generate an appropriate response. Step 3: The providing unit provides the response generated by the analysis unit to the user. The response may be provided in the form of a voice response, a text response, an image response, or the like. For example, the providing unit may provide the generated response by voice using Text-to-Speech (TTS) technology. Alternatively, the response may be provided by text using a chatbot or a messaging app. Step 4: The integration unit integrates with external services via API. External services include calendar apps, music streaming services, and information search services. For example, the integration unit integrates with a calendar app to manage schedules, with a music streaming service to play music, and with the information search function to provide the information the user wants.

[0069] (Example 2) A support system according to an embodiment of the present invention accepts user utterances, analyzes them with a generation AI, provides appropriate responses, and connects with external services. The support system allows users to receive support through conversations with characters selected by the user. The generation AI utilizes natural language processing and speech recognition technology to understand the user's utterances and generate appropriate responses. The system also connects with various external services via an API to provide support tailored to the user's needs. For example, the support system can manage schedules, set reminders, search for information, and play music. For example, a user launches an application and selects their favorite character. The support system then initiates a conversation with the character and discusses the support they require. For example, the system can issue an instruction such as, "Set a reminder for tomorrow's meeting." The generation AI analyzes the user's utterances and generates appropriate responses. Speech recognition technology converts the user's utterances into text, which the generation AI then analyzes. Furthermore, the support system connects with external services via an API to provide support tailored to the user's needs. For example, the system can connect with a calendar app to manage schedules or with a music streaming service to play favorite music. The information search function can also be used to quickly provide users with the information they need. This allows the support system to enable the user to efficiently perform daily tasks and reduce stress.This allows the support system to efficiently perform daily tasks and reduce stress.

[0070] The support system according to the embodiment includes a reception unit, an analysis unit, a provision unit, and a linking unit. The reception unit receives a user's utterance. The user's utterance may be, for example, voice, text, or gesture, but is not limited to these examples. The reception unit converts the user's utterance into text using, for example, voice recognition technology. The reception unit can also accept text input or gesture input. For example, the reception unit accurately converts the user's utterance into text using voice recognition technology. The analysis unit uses a generation AI to analyze the utterance received by the reception unit and generate an appropriate response. The analysis is performed based on, for example, natural language processing or a machine learning algorithm, but is not limited to these examples. For example, the analysis unit uses a generation AI (for example, a natural language generation model such as GPT-4 or Gemini) to analyze the user's utterance and generate an appropriate response. The analysis unit can also use the generation AI to understand the context of the user's utterance and generate an appropriate response. The provision unit provides the response generated by the analysis unit to the user. The response may be provided in the form of, for example, a voice response, a text response, an image response, or the like, but is not limited to these examples. For example, the providing unit uses Text-to-Speech (TTS) technology to provide the generated response as voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. The linking unit links with external services through an API. Examples of external services include, but are not limited to, calendar apps, music streaming services, and information search services. For example, the linking unit links with a calendar app to manage a schedule. The linking unit can also link with a music streaming service to play music. The linking unit can also provide information that the user wants to know by using an information search function. As a result, the support system according to the embodiment can efficiently analyze user utterances, provide appropriate responses, and link with external services to provide support that meets the user's needs.

[0071] The reception unit can convert the user's utterance into text using speech recognition technology. Examples of speech recognition technology include, but are not limited to, deep learning-based speech recognition and HMM-based speech recognition. The reception unit can convert the user's utterance into text using, for example, deep learning-based speech recognition technology. The reception unit can also convert the user's utterance into text using HMM-based speech recognition technology. The reception unit can also support multiple languages ​​using speech recognition technology. For example, the reception unit recognizes speech input in multiple languages, such as English, Japanese, and French, and converts it into text data. Thus, the use of speech recognition technology allows the user's utterance to be accurately converted into text. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input speech data acquired using speech recognition technology into a generation AI and have the generation AI convert the speech data into text data.

[0072] The analysis unit can analyze the user's utterance using a generation AI and generate an appropriate response. Examples of the generation AI include, but are not limited to, natural language generation models such as GPT-4 and Gemini. The analysis unit can analyze the user's utterance using, for example, GPT-4 and generate an appropriate response. The analysis unit can also analyze the user's utterance using Gemini and generate an appropriate response. The analysis unit can also understand the context of the user's utterance and generate an appropriate response using a generation AI. For example, the analysis unit can understand the context of the user's utterance using a generation AI and generate an appropriate response. This allows the generation AI to accurately analyze the user's utterance and generate an appropriate response. Some or all of the above-described processing in the analysis unit can be performed using, for example, an AI, or can be performed without using an AI. For example, the analysis unit can input the user's utterance into the generation AI and cause the generation AI to generate an appropriate response.

[0073] The providing unit can provide the generated response to the user by voice or text. For example, the providing unit uses Text-to-Speech (TTS) technology to provide the generated response by voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. For example, the providing unit uses TTS technology to provide the generated response by voice. The providing unit can also use a chatbot or a messaging app to provide the generated response as text. In this way, by providing the generated response by voice or text, information can be conveyed to the user in an appropriate form. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the generated response to a generation AI and cause the generation AI to provide the response by voice or text.

[0074] The linking unit can manage the schedule in cooperation with a calendar app. Examples of calendar apps include, but are not limited to, Google Calendar and Outlook Calendar. For example, the linking unit can manage the schedule in cooperation with Google Calendar. The linking unit can also manage the schedule in cooperation with Outlook Calendar. The linking unit can also automatically update the user's schedule in cooperation with the calendar app. For example, the linking unit can automatically update the user's schedule in cooperation with the calendar app. This allows for more efficient schedule management for the user by linking with the calendar app. Some or all of the above-described processing in the linking unit may be performed using, or without, AI. For example, the linking unit can input schedule data acquired in cooperation with the calendar app into the generation AI and have the generation AI perform schedule management.

[0075] The linking unit can play music in cooperation with a music streaming service. Examples of music streaming services include, but are not limited to, LINE MUSIC and Apple Music. For example, the linking unit can play music in cooperation with LINE MUSIC. The linking unit can also play music in cooperation with Apple Music. The linking unit can also automatically update a user's playlist in cooperation with a music streaming service. For example, the linking unit can automatically update a user's playlist in cooperation with a music streaming service. This allows a user to easily play their favorite music by linking with a music streaming service. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input music data obtained in cooperation with a music streaming service to a generation AI and cause the generation AI to play music.

[0076] The collaboration unit can provide the information the user wants to know by utilizing an information search function. Examples of information search functions include, but are not limited to, a search engine and a database search. For example, the collaboration unit can provide the information the user wants to know by utilizing a search engine. The collaboration unit can also provide the information the user wants to know by utilizing a database search. The collaboration unit can also quickly provide the information the user wants to know by utilizing the information search function. For example, the collaboration unit can quickly provide the information the user wants to know by utilizing the information search function. In this way, the user can quickly obtain the information they need by utilizing the information search function. Some or all of the above-described processing in the collaboration unit may be performed, for example, using AI or may be performed without using AI. For example, the collaboration unit can input information data acquired by utilizing the information search function into the generation AI and cause the generation AI to provide the information.

[0077] The reception unit can estimate the user's emotions and adjust the timing of utterance acceptance based on the estimated user emotions. For example, if the user is feeling stressed, the reception unit can delay the timing of utterance acceptance to allow the user to relax. Furthermore, if the user is excited, the reception unit can also accelerate the timing of utterance acceptance to respond quickly. Furthermore, if the user is tired, the reception unit can adjust the timing of utterance acceptance and start with a simple question. By adjusting the timing of utterance acceptance according to the user's emotions, utterances can be accepted at a more appropriate time. Emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data into the generation AI and cause the generation AI to adjust the timing of utterance acceptance.

[0078] The reception unit can analyze the user's past speech history and select a reception method. For example, the reception unit prioritizes reception of phrases that the user has frequently used in the past. The reception unit can also select a reception method suitable for a specific time period from the user's past speech history. The reception unit can also select an optimal reception means (voice, text, etc.) based on the user's past speech history. In this way, the optimal reception method can be selected by analyzing the user's past speech history. Some or all of the above-mentioned processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's past speech history data into a generation AI and have the generation AI select the optimal reception method.

[0079] The reception unit can perform filtering based on the user's current situation and areas of interest when receiving utterances. For example, the reception unit receives only topics related to the user's current situation. The reception unit can also preferentially receive utterances containing specific keywords based on the user's areas of interest. The reception unit can also perform appropriate filtering based on the user's current situation (e.g., at work, on a break). In this way, by filtering based on the user's current situation and areas of interest, highly relevant utterances can be preferentially received. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's current situation data and area of ​​interest data to the generation AI and have the generation AI perform filtering.

[0080] When receiving a utterance, the reception unit can select a reception means according to the user's input method. For example, if the user is using voice input, the reception unit can receive the utterance using voice recognition technology. Furthermore, if the user is using text input, the reception unit can also receive the utterance using text analysis technology. Furthermore, if the user is using gesture input, the reception unit can also receive the utterance using gesture recognition technology. This improves the accuracy of utterance reception by selecting the optimal reception means according to the user's input method. Some or all of the above-described processing in the reception unit may be performed using, for example, AI, or may be performed without using AI. For example, the reception unit can input the user's input method data into the generation AI and cause the generation AI to select the optimal reception means.

[0081] The reception unit can estimate the user's emotions and determine the priority of utterances to be received based on the estimated user emotions. For example, when the user is feeling stressed, the reception unit can prioritize important utterances. Furthermore, when the user is relaxed, the reception unit can equally receive all utterances. Furthermore, when the user is in a hurry, the reception unit can prioritize urgent utterances. Thus, by determining the priority of utterances according to the user's emotions, important utterances can be preferentially received. The emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the reception unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the reception unit can input the user's emotion data into the generation AI and have the generation AI determine the priority of utterances.

[0082] When receiving a utterance, the reception unit can prioritize receiving highly relevant utterances by taking into account the user's geographical location information. For example, when the user is in a specific location, the reception unit prioritizes receiving utterances related to that location. The reception unit can also prioritize receiving utterances related to nearby information based on the user's current location. The reception unit can also prioritize receiving utterances related to the user's travel destination when the user is traveling. In this way, highly relevant utterances can be prioritized by taking the user's geographical location information into account. Some or all of the above-described processing in the reception unit may be performed using AI, for example, or may be performed without using AI. For example, the reception unit can input the user's geographical location information data into the generation AI and cause the generation AI to prioritize receiving highly relevant utterances.

[0083] The reception unit can analyze the user's social media activity when receiving a comment and receive related comments. For example, the reception unit can prioritize receiving comments related to content posted by the user on social media. The reception unit can also prioritize receiving topics that the user is likely to be interested in based on the user's social media activity. The reception unit can also prioritize receiving comments related to the activities of the user's friends on social media. In this way, by analyzing the user's social media activity, related comments can be preferentially received. Some or all of the above-described processing by the reception unit can be performed using, for example, AI, or can be performed without using AI. For example, the reception unit can input the user's social media activity data into the generation AI and cause the generation AI to receive related comments.

[0084] The reception unit can customize the reception method by reflecting the user's past feedback when receiving a utterance. For example, the reception unit preferentially uses the reception method that the user has previously preferred. The reception unit can also select the optimal reception method based on the user's past feedback. The reception unit can also continuously improve the reception method by reflecting the user's past feedback. In this way, the optimal reception method can be provided by reflecting the user's past feedback. Some or all of the above-described processing in the reception unit may be performed using AI, for example, or may be performed without using AI. For example, the reception unit can input the user's past feedback data into the generation AI and have the generation AI customize the reception method.

[0085] The analysis unit can estimate the user's emotions and adjust the way a response is expressed based on the estimated user's emotions. For example, if the user is stressed, the analysis unit can generate a response using gentle language. If the user is relaxed, the analysis unit can also generate a friendly response. If the user is in a hurry, the analysis unit can also generate a concise and quick response. This allows for adjusting the way a response is expressed based on the user's emotions, thereby providing a more appropriate response. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the analysis unit can be performed using, for example, an AI, or without an AI. For example, the analysis unit can input the user's emotion data into the generation AI and have the generation AI adjust the way a response is expressed.

[0086] During analysis, the analysis unit can adjust the level of detail of the analysis based on the importance of the statement. For example, the analysis unit performs a detailed analysis on important statements to generate a highly accurate response. The analysis unit can also perform a standard analysis on general statements to generate an appropriate response. The analysis unit can also perform a simplified analysis on less urgent statements to generate a quick response. In this way, an appropriate response can be generated by adjusting the level of detail of the analysis based on the importance of the statement. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input statement importance data to a generation AI and have the generation AI adjust the level of detail of the analysis.

[0087] During analysis, the analysis unit can apply different analysis algorithms depending on the category of the utterance. For example, the analysis unit can apply a schedule analysis algorithm to utterances related to schedule management. The analysis unit can also apply an information search algorithm to utterances related to information search. The analysis unit can also apply a music analysis algorithm to utterances related to music playback. In this way, by applying different analysis algorithms depending on the category of the utterance, highly accurate responses can be generated. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input utterance category data into the generation AI and cause the generation AI to apply different analysis algorithms.

[0088] During analysis, the analysis unit can improve the accuracy of the analysis by referring to the user's past analysis results. For example, the analysis unit generates a highly accurate response to a similar utterance based on the user's past analysis results. The analysis unit can also adjust the analysis algorithm by referring to the user's past analysis results. The analysis unit can also accumulate the user's past analysis results and continuously improve the accuracy of the analysis. In this way, the accuracy of the analysis can be improved by referring to the user's past analysis results. Some or all of the above-described processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the user's past analysis result data into the generation AI and cause the generation AI to improve the accuracy of the analysis.

[0089] The analysis unit can estimate the user's emotions and adjust the length of the response based on the estimated user emotions. For example, if the user is stressed, the analysis unit can generate a short, to-the-point response. If the user is relaxed, the analysis unit can generate a longer response with detailed explanations. If the user is in a hurry, the analysis unit can generate a concise, quick response. This allows for adjusting the length of the response according to the user's emotions, thereby providing a more appropriate response. The emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the analysis unit can be performed using, for example, an AI, or without an AI. For example, the analysis unit can input the user's emotion data into the generation AI and have the generation AI adjust the length of the response.

[0090] During analysis, the analysis unit can determine the priority of analysis based on the time when the comment was submitted. For example, the analysis unit prioritizes analysis of highly urgent comments and quickly generates a response. The analysis unit can also analyze general comments with standard priority. The analysis unit can also analyze less urgent comments later. In this way, by determining the priority of analysis based on the time when the comment was submitted, it is possible to respond quickly to highly urgent comments. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input data on the time when the comment was submitted to the generation AI and have the generation AI determine the analysis priority.

[0091] During analysis, the analysis unit can adjust the order of analysis based on the relevance of the utterances. For example, the analysis unit prioritizes analysis of highly relevant utterances and quickly generates a response. The analysis unit can also analyze utterances of general relevance in a standard order. The analysis unit can also analyze utterances of low relevance later. In this way, by adjusting the order of analysis based on the relevance of the utterances, it is possible to quickly respond to highly relevant utterances. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input utterance relevance data to the generation AI and have the generation AI adjust the order of analysis.

[0092] During analysis, the analysis unit can adjust the use of technical terminology in the analysis according to the user's level of expertise. For example, if the user has technical expertise, the analysis unit generates a response using technical terminology. Furthermore, if the user has general knowledge, the analysis unit can also generate a response in easy-to-understand language. Furthermore, if the user is a beginner, the analysis unit can also generate a response in simple language. By adjusting the use of technical terminology in the analysis according to the user's level of expertise, a response that is easy for the user to understand can be provided. Some or all of the above-described processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the user's level of expertise data into the generation AI and have the generation AI execute the use of technical terminology.

[0093] The providing unit can estimate the user's emotions and adjust the response provision method based on the estimated user's emotions. For example, if the user is feeling stressed, the providing unit can provide a response in a gentle voice. If the user is relaxed, the providing unit can also provide a friendly voice. If the user is in a hurry, the providing unit can also provide a quick and concise response. This allows the response provision method to be adjusted according to the user's emotions, thereby providing a more appropriate response. The emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the user's emotion data into the generation AI and cause the generation AI to adjust the response provision method.

[0094] The providing unit can adjust the content of the information provided based on the importance of the response when providing the information. For example, the providing unit provides detailed information for an important response. The providing unit can also provide standard information for a general response. The providing unit can also provide simplified information for a low-urgency response. In this way, by adjusting the level of detail provided based on the importance of the response, appropriate information can be provided. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input response importance data to the generating AI and cause the generating AI to adjust the content of the information provided.

[0095] The providing unit can apply different providing means depending on the category of the response when providing the information. For example, the providing unit can provide a response related to schedule management through a calendar app. The providing unit can also provide search results as a link for a response related to an information search. The providing unit can also provide a response related to music playback through a music streaming service. In this way, by applying different providing means depending on the category of the response, it is possible to provide information in an optimal form for the user. Some or all of the above-mentioned processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input response category data to the generation AI and cause the generation AI to apply different providing means.

[0096] The providing unit can improve the accuracy of the provision by referring to the user's past provision results when providing the response. For example, the providing unit provides a highly accurate response to a similar utterance based on the user's past provision results. The providing unit can also adjust the provision means by referring to the user's past provision results. The providing unit can also accumulate the user's past provision results and continuously improve the accuracy of the provision. In this way, the accuracy of the provision can be improved by referring to the user's past provision results. Some or all of the above-described processing in the providing unit may be performed using, for example, AI, or may be performed without using AI. For example, the providing unit can input the user's past provision result data into the generation AI and cause the generation AI to improve the accuracy of the provision.

[0097] The providing unit can estimate the user's emotions and adjust the timing of providing a response based on the estimated user's emotions. For example, if the user is feeling stressed, the providing unit can delay the timing of providing a response to allow the user to relax. Furthermore, if the user is excited, the providing unit can also accelerate the timing of providing a response to provide a quicker response. Furthermore, if the user is tired, the providing unit can adjust the timing of providing a response and start with a simple question. This allows the response to be provided at a more appropriate time by adjusting the timing of providing a response according to the user's emotions. The emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the providing unit can input the user's emotion data into the generation AI and cause the generation AI to adjust the timing of providing a response.

[0098] The providing unit can select the optimal providing method by taking into account the user's device information when providing information. For example, if the user is using a smartphone, the providing unit can provide a display method that matches the screen size. Furthermore, if the user is using a tablet, the providing unit can also provide a display method that is optimized for a large screen. Furthermore, if the user is using a smartwatch, the providing unit can also provide a display method that is simple and highly visible. This allows information to be provided in the optimal form by taking into account the user's device information. Some or all of the above-described processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input the user's device information data into the generation AI and cause the generation AI to select the optimal providing method.

[0099] The providing unit can provide multilingual content according to the user's language setting when providing the content. The providing unit can automatically set the response language based on, for example, the language setting of the user's device. The providing unit can also provide a language switching function when the user uses multiple languages. The providing unit can also provide responses in a specific language when the user selects that language. This allows information to be provided in a format that is easy for the user to understand by providing multilingual support according to the user's language setting. Some or all of the above-described processing by the providing unit can be performed, for example, using AI or without AI. For example, the providing unit can input the user's language setting data into a generation AI and cause the generation AI to provide multilingual support.

[0100] The providing unit can customize the delivery method by reflecting user feedback when providing information. For example, the providing unit can preferentially use a delivery method that the user has previously preferred. The providing unit can also select the optimal delivery means based on the user's past feedback. The providing unit can also continuously improve the delivery method by reflecting the user's past feedback. In this way, information can be provided in an optimal form by reflecting the user's feedback. Some or all of the above-mentioned processing in the providing unit may be performed using AI, for example, or may be performed without using AI. For example, the providing unit can input user feedback data into the generation AI and cause the generation AI to customize the delivery method.

[0101] The linking unit can estimate the user's emotions and determine the priority of linked services based on the estimated user emotions. For example, if the user is feeling stressed, the linking unit can prioritize linking services that help the user relax. Furthermore, if the user is excited, the linking unit can prioritize linking entertainment services. Furthermore, if the user is tired, the linking unit can prioritize linking reminder or schedule management services. By determining the priority of linked services according to the user's emotions, more appropriate services can be provided. The emotion estimation is realized using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the linking unit can be performed using, for example, an AI, or can be performed without using an AI. For example, the linking unit can input the user's emotion data into the generation AI and have the generation AI determine the priority of linked services.

[0102] At the time of linking, the linking unit can analyze the user's past linking history and select the optimal linking method. For example, the linking unit prioritizes linking with services that the user has frequently used in the past. The linking unit can also select a linking method suitable for a specific time period from the user's past linking history. The linking unit can also select the optimal linking means based on the user's past linking history. In this way, the optimal linking method can be selected by analyzing the user's past linking history. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input the user's past linking history data into the generation AI and cause the generation AI to select the optimal linking method.

[0103] The linking unit can customize the linking means based on the user's current situation during linking. For example, when the user is at work, the linking unit can prioritize linking with work-related services. Furthermore, when the user is on a break, the linking unit can prioritize linking with services that allow the user to relax. Furthermore, when the user is traveling, the linking unit can prioritize linking with services related to travel. This allows for customizing the linking means based on the user's current situation, thereby providing more appropriate services. Some or all of the above-described processing in the linking unit may be performed using, or without, AI, for example. For example, the linking unit can input the user's current situation data into the generation AI and cause the generation AI to customize the linking means.

[0104] The collaboration unit can improve the collaboration method by reflecting user feedback during collaboration. The collaboration unit, for example, selects the optimal collaboration means based on the user's past feedback. The collaboration unit can also continuously improve the collaboration method by reflecting user feedback. The collaboration unit can also propose new collaboration services based on user feedback. This allows the collaboration method to be continuously improved by reflecting user feedback. Some or all of the above-mentioned processing in the collaboration unit may be performed using AI, for example, or may be performed without using AI. For example, the collaboration unit can input user feedback data into the generation AI and cause the generation AI to improve the collaboration method.

[0105] The linking unit can estimate the user's emotions and adjust the display method of the linked service based on the estimated user's emotions. For example, if the user is feeling stressed, the linking unit can provide a simple, highly visible display method. Furthermore, if the user is relaxed, the linking unit can provide a display method that includes detailed information. Furthermore, if the user is in a hurry, the linking unit can provide a display method that focuses on the main points. This allows the display method of the linked service to be adjusted according to the user's emotions, thereby providing information in a more appropriate form. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the linking unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the linking unit can input the user's emotion data into the generation AI and cause the generation AI to adjust the display method of the linked service.

[0106] When linking, the linking unit can select a linking method taking into account the user's geographical location information. For example, when the user is in a specific location, the linking unit prioritizes linking with services related to that location. The linking unit can also prioritize linking with services related to nearby information based on the user's current location. The linking unit can also prioritize linking with services related to the user's travel destination when the user is traveling. This makes it possible to select an optimal linking method by taking the user's geographical location information into consideration. Some or all of the above-described processing in the linking unit may be performed using AI, for example, or may be performed without using AI. For example, the linking unit can input the user's geographical location information data into the generation AI and cause the generation AI to select an optimal linking method.

[0107] At the time of linking, the linking unit can analyze the user's social media activity and suggest services to link. For example, the linking unit prioritizes linking with services related to content posted by the user on social media. The linking unit can also prioritize linking with services that are likely to interest the user based on the user's social media activity. The linking unit can also prioritize linking with services related to the activities of the user's friends on social media. In this way, by analyzing the user's social media activity, related services can be linked with priority. Some or all of the above-described processing in the linking unit may be performed using, for example, AI, or may be performed without using AI. For example, the linking unit can input the user's social media activity data into the generation AI and cause the generation AI to suggest services to link with.

[0108] The collaboration unit can customize the collaboration method by reflecting the user's past feedback during collaboration. For example, the collaboration unit can prioritize the use of collaboration methods that the user has previously preferred. The collaboration unit can also select the optimal collaboration method based on the user's past feedback. The collaboration unit can also continuously improve the collaboration method by reflecting the user's past feedback. In this way, the optimal collaboration method can be provided by reflecting the user's past feedback. Some or all of the above-described processing in the collaboration unit may be performed using AI, for example, or may be performed without using AI. For example, the collaboration unit can input the user's past feedback data into the generation AI and cause the generation AI to customize the collaboration method. === Hard Collateral 1-1 === Each of the above-described elements, including the reception unit, analysis unit, provision unit, and linkage unit, is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the reception unit receives a user's utterance using the microphone 38B or touch panel 38A of the smart device 14, and converts the utterance into text using voice recognition technology by the control unit 46A. The analysis unit is implemented, for example, by the specific processing unit 290 of the data processing device 12, analyzes the user's utterance using a generation AI, and generates an appropriate response. The provision unit provides the generated response to the user using, for example, the speaker 40B or display 40A of the smart device 14. The linkage unit links with an external service via, for example, the communication I / F 26 of the data processing device 12, and provides support according to the user's needs. === Hard Collateral 1-2 === Each of the multiple elements including the above-described reception unit, analysis unit, provision unit, and linkage unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the reception unit receives a user's utterance using the microphone 238 of the smart glasses 214 and converts it into text using voice recognition technology by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the user's utterance using a generation AI, and generates an appropriate response. The provision unit provides the generated response to the user, for example, using the speaker 240 of the smart glasses 214. The linkage unit links with an external service via, for example, the communication I / F 26 of the data processing device 12, and provides support according to the user's needs. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and linkage unit is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the reception unit receives a user's utterance using the microphone 238 of the headset-type terminal 314, and converts the utterance into text using voice recognition technology by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the user's utterance using a generation AI to generate an appropriate response. The provision unit provides the generated response to the user, for example, by using the speaker 240 of the headset-type terminal 314. The linkage unit links with an external service via, for example, the communication I / F 26 of the data processing device 12, and provides support according to the user's needs. === Hard Collateral 1-4 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and linkage unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the reception unit receives a user's utterance using the microphone 238 of the robot 414 and converts it into text using voice recognition technology by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, analyzes the user's utterance using a generation AI, and generates an appropriate response. The provision unit provides the generated response to the user, for example, using the speaker 240 of the robot 414. The linkage unit links with an external service via, for example, the communication I / F 26 of the data processing device 12, and provides support according to the user's needs.

[0109] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0110] When receiving a user's speech, the reception unit can analyze the volume and tone of the user's speech and estimate the importance of the speech. For example, if the user is speaking loudly, it can be determined that the urgency is high and the analysis can be performed with priority. On the other hand, if the user is speaking quietly, it can be determined that the urgency is low and the analysis can be performed with normal priority. Furthermore, if the user's tone indicates anger or sadness, an emotion estimation function can be used to take an appropriate response. This makes it possible to estimate the importance of the speech based on the volume and tone of the user's speech and take an appropriate response.

[0111] When analyzing a user's comments, the analysis unit refers to the user's past behavioral history to more accurately understand the intention of the comment. For example, it prioritizes analysis of phrases and keywords that the user has frequently used in the past. It can also generate responses appropriate for specific time periods from the user's past behavioral history. Furthermore, it can select the optimal response method (voice, text, etc.) based on the user's past behavioral history. In this way, by referring to the user's past behavioral history, it is possible to more accurately understand the intention of the comment and generate an appropriate response.

[0112] When providing the generated response to the user, the providing unit may select a providing method taking into consideration the remaining battery level of the user's device. For example, if the remaining battery level of the user's device is low, a text response may be provided preferentially. Also, if the remaining battery level is sufficient, a voice response may be provided. Furthermore, if the remaining battery level is very low, a brief response may be provided to reduce battery consumption of the device. In this way, information can be provided in an optimal form by taking into consideration the remaining battery level of the user's device.

[0113] When linking with an external service, the linking unit can select a linking method taking into account the user's current network connection status. For example, if the user is connected to Wi-Fi, high-speed data transfer can be performed. Also, if the user is using mobile data communication, it can transfer only the minimum amount of data necessary to reduce the amount of data transferred. Furthermore, if the user is offline, it can temporarily suspend linking and attempt linking again when the network connection is restored. This makes it possible to select the optimal linking method by taking into account the user's network connection status.

[0114] When accepting a user's speech, the accepting unit can analyze background sounds surrounding the user's speech and complement the content of the speech. For example, if the user is speaking in a noisy environment, the accepting unit can analyze the background sounds to remove noise and clarify the content of the speech. Also, if the user is speaking in a quiet environment, the accepting unit can analyze the background sounds to complement the content of the speech. Furthermore, if the user is making a specific sound (e.g., an alarm sound or music), the accepting unit can provide information related to that sound. In this way, by analyzing the background sounds surrounding the user's speech, the content of the speech can be complemented and an appropriate response can be taken.

[0115] When analyzing a user's utterances, the analysis unit can estimate the user's emotions and adjust the tone of the response based on the estimated emotions. For example, if the user is angry, the analysis unit can generate a response in a calm and collected tone. If the user is sad, the analysis unit can generate a response in a gentle tone. Furthermore, if the user is happy, the analysis unit can generate a response in a bright tone. This allows the system to provide a more appropriate response by adjusting the tone of the response according to the user's emotions.

[0116] When providing the generated response to the user, the providing unit can select a providing method by taking into consideration the user's current activity status. For example, if the user is driving, a voice response can be provided preferentially. Also, if the user is in a meeting, a text response can be provided. Furthermore, if the user is exercising, a concise response can be provided so as not to interfere with the user's activity. In this way, information can be provided in an optimal form by taking into consideration the user's current activity status.

[0117] When linking with an external service, the linking unit can estimate the user's emotions and select the type of service to link with based on the estimated emotions. For example, if the user is feeling stressed, it can prioritize linking with relaxation services (e.g., meditation apps or relaxation music). Also, if the user is excited, it can prioritize linking with entertainment services (e.g., games or video streaming). Furthermore, if the user is tired, it can prioritize linking with reminder or schedule management services. In this way, by selecting the type of service to link with based on the user's emotions, it is possible to provide more appropriate services.

[0118] When accepting a user's utterance, the reception unit can analyze the speed of the user's utterance and complement the content of the utterance. For example, if the user is speaking quickly, the reception unit can analyze the utterance speed to extract important keywords and clarify the content of the utterance. Also, if the user is speaking slowly, the reception unit can analyze the utterance speed to understand the intention of the utterance. Furthermore, if the user is speaking intermittently, the reception unit can analyze the utterance speed to complement the content of the utterance. In this way, by analyzing the utterance speed of the user, the content of the utterance can be complemented and an appropriate response can be taken.

[0119] When analyzing a user's utterances, the analysis unit can estimate the user's emotions and adjust the content of the response based on the estimated emotions. For example, if the user is feeling stressed, a concise and to-the-point response can be generated. Alternatively, if the user is relaxed, a response including detailed explanations can be generated. Furthermore, if the user is in a hurry, a quick and concise response can be generated. This allows the content of the response to be adjusted according to the user's emotions, making it possible to provide a more appropriate response.

[0120] The processing flow of the second embodiment will be briefly explained below.

[0121] Step 1: The reception unit receives user utterances. User utterances include voice, text, and gestures. The reception unit converts the user utterances into text using speech recognition technology. It can also receive text input and gesture input. Step 2: The analysis unit uses the generation AI to analyze the utterance received by the reception unit and generate an appropriate response. The analysis is based on natural language processing and machine learning algorithms. For example, the generation AI (a natural language generation model such as GPT-4 or Gemini) is used to analyze the user's utterance, understand the context, and generate an appropriate response. Step 3: The providing unit provides the response generated by the analysis unit to the user. The response may be provided in the form of a voice response, a text response, an image response, or the like. For example, the providing unit may provide the generated response by voice using Text-to-Speech (TTS) technology. Alternatively, the response may be provided by text using a chatbot or a messaging app. Step 4: The integration unit integrates with external services via API. External services include calendar apps, music streaming services, and information search services. For example, the integration unit integrates with a calendar app to manage schedules, with a music streaming service to play music, and with the information search function to provide the information the user wants.

[0122] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0123] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0124] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0125] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0126] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0127] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0128] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0129] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0130] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0131] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0132] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0133] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0134] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0135] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0136] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0137] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0138] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0139] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0140] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0141] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0142] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0143] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0144] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0145] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0146] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0147] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0148] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0149] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0150] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0151] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0152] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0153] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0154] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0155] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0156] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0157] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0158] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0159] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0160] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0161] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0162] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0163] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0164] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0165] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0166] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0167] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0168] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0169] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0170] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0171] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0172] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0173] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0174] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0175] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0176] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0177] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0178] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0179] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0180] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0181] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0182] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0183] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0184] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0185] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0186] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0187] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0188] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0189] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0190] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0191] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0192] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0193] [Explanation of symbols]

[0194] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a reception unit that receives user comments; an analysis unit that analyzes the message received by the reception unit and generates a response; a providing unit that provides a response generated by the analysis unit; A linking unit that links with external services through APIs. A system characterized by:

2. The reception unit Converts user speech into text using voice recognition technology 2. The system of claim 1.

3. The analysis unit Generative AI analyzes user comments and generates appropriate responses 2. The system of claim 1.

4. The providing unit Providing the generated response to the user via speech or text 2. The system of claim 1.

5. The linking unit is Manage your schedule by linking with a calendar app 2. The system of claim 1.

6. The linking unit is Play music in conjunction with music streaming services 2. The system of claim 1.

7. The linking unit is Use information search functions to provide users with the information they want to know 2. The system of claim 1.

8. The reception unit Estimates the user's emotions and adjusts the timing of speech acceptance based on the estimated user emotions.

2. The system of claim 1.

Images