System

Abstract

To provide a system according to an embodiment that records and analyzes conversation contents, and generates proper replies.SOLUTION: A system according to an embodiment comprises: a recording unit; an analysis unit; and a conversation unit. The recording unit is configured to record the conversation contents. The analysis unit is configured to analyze the conversation content recorded by the recording unit. The conversation unit is configured to have a conversation on the basis of a result analyzed by the analysis unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has the problem that it is not sufficient to effectively record and analyze conversation content and generate appropriate responses.

[0005] The system according to the embodiment aims to record and analyze the contents of the conversation and generate an appropriate response. [Means for solving the problem]

[0006] The system according to the embodiment includes a recording unit, an analysis unit, and a conversation unit. The recording unit records the conversation content. The analysis unit analyzes the conversation content recorded by the recording unit. The conversation unit conducts a conversation based on the result of the analysis by the analysis unit. Effect of the Invention

[0007] The system according to the embodiment can record and analyze the conversation content and generate an appropriate response. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The conversation learning system according to the embodiment of the present invention is a system in which an AI learns conversations with a generation AI (for example, GPT or Gemini) to learn the contents of the conversation for each scene, and creates an alter ego of the user. This conversation learning system is a system in which a user has a conversation with a generation AI, records the conversation contents, analyzes the recorded conversation contents, and learns the user's conversation patterns and phrases, so that the AI ​​can converse on behalf of the user in a specific scene. For example, when a child talks to the AI ​​on a messenger app, the AI ​​replies on behalf of the user. The AI ​​can behave as if the user himself is replying by using phrases and expressions that the user normally uses. This allows the child to have an experience as if he or she were talking to a parent. Furthermore, this mechanism can be used in various scenes, such as nursing care and customer service at a hairdresser. For example, in the field of nursing care, the AI ​​can converse with the elderly instead of the caregiver, thereby reducing the burden on the caregiver. In addition, in customer service at a hairdresser, the AI ​​can converse with the customer instead of the hairdresser, thereby making the hairdresser's work more efficient. In this way, by having the AI ​​learn conversations with the generating AI, the AI ​​can learn the content of conversations for each scene and create an avatar of itself. This allows the conversation learning system to learn the user's conversation patterns and expressions and to converse on behalf of the user in specific scenes.

[0029] The conversation learning system according to the embodiment includes a recording unit, an analysis unit, and a conversation unit. The recording unit records the conversation content between the user and the generation AI. The conversation content between the user and the generation AI includes, for example, a text-based conversation, an audio conversation, a video conversation, and the like, but is not limited to such examples. The recording unit, for example, saves the text-based conversation in a log file. The recording unit can also record the audio conversation and save it as an audio file. Furthermore, the recording unit can record the video conversation and save it as a video file. For example, the recording unit records the text-based conversation in real time and saves it in a log file. The audio conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed, for example, using natural language processing technology, but is not limited to such examples. For example, the analysis unit uses natural language processing technology to extract keywords from the conversation content. The analysis unit can also estimate emotions from the conversation content using emotion analysis technology. Furthermore, the analysis unit can also extract important parts of the conversation content using a keyword extraction technique. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content and identify the topic of the conversation. The emotion analysis technique estimates the emotion of the conversation content and calculates an emotion score. The keyword extraction technique extracts important parts of the conversation content and generates a summary. The conversation unit conducts a conversation based on the result analyzed by the analysis unit. The conversation is conducted, for example, using an interactive AI system, but is not limited to such an example. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system. Furthermore, the conversation unit can also conduct a conversation on behalf of the user in a specific scene based on the analysis result. Furthermore, the conversation unit can reproduce the user's conversation pattern and phrases based on the analysis result. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system and reproduces the user's conversation pattern and phrases. As a result, the conversation learning system according to the embodiment can learn the user's conversation pattern and phrases and conduct a conversation on behalf of the user in a specific scene.A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can have a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs a conversation.

[0030] The recording unit records the contents of the conversation between the user and the generation AI. The contents of the conversation between the user and the generation AI include, for example, text-based conversation, voice conversation, video conversation, etc., but are not limited to such examples. The recording unit, for example, saves the text-based conversation in a log file. The recording unit can also record the voice conversation and save it as an audio file. Furthermore, the recording unit can record the video conversation and save it as a video file. For example, the recording unit records the text-based conversation in real time and saves it in a log file. The voice conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. The recording unit can use a database system to efficiently manage these data. The database system organizes the conversation contents in chronological order, making it easy to search and filter. For example, the conversation contents can be searched based on a specific keyword or date and time. In addition, the recording unit can use encryption technology to ensure the security of the data. This makes it possible to maintain the integrity of the data while protecting the privacy of the user. Furthermore, the recording unit has a data backup function, and measures can be taken against data loss or damage. For example, by regularly backing up data and storing it in different locations, the risk of disasters and system failures can be reduced. This allows the recording unit to reliably record the content of conversations between the user and the generating AI, providing a foundation for using it for later analysis and learning.

[0031] The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed, for example, using a natural language processing technique, but is not limited to such an example. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content. The analysis unit can also use a sentiment analysis technique to estimate the sentiment of the conversation content. Furthermore, the analysis unit can also extract important parts of the conversation content using a keyword extraction technique. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content and identify the topic of the conversation. The sentiment analysis technique estimates the sentiment of the conversation content and calculates a sentiment score. The keyword extraction technique extracts important parts of the conversation content and generates a summary. The analysis unit can combine these techniques to grasp the overall picture of the conversation content and understand the user's intentions and emotions. For example, the natural language processing technique is used to analyze the context of the conversation and identify the user's questions and requests. The sentiment analysis technique is used to evaluate the user's emotional state and provide information for generating an appropriate response. Furthermore, the analysis unit can analyze trends in the conversation content and grasp the user's interests and concerns. For example, past conversation data can be used to identify frequently discussed topics and keywords that users use in future conversations, providing a foundation for the analytics department to better understand users' needs and emotions and provide more personalized responses.

[0032] The conversation unit conducts a conversation based on the results of the analysis by the analysis unit. The conversation is conducted, for example, using an interactive AI system, but is not limited to such an example. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system. The conversation unit can also conduct a conversation on behalf of the user in a specific scene based on the analysis result. Furthermore, the conversation unit can reproduce the user's conversation pattern and phrases based on the analysis result. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system and reproduces the user's conversation pattern and phrases. Thereby, the conversation learning system according to the embodiment can learn the user's conversation pattern and phrases and conduct a conversation on behalf of the user in a specific scene. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can conduct a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs a conversation. The conversation unit can learn the user's preferences and style based on the user's past conversation data and provide a more natural and consistent conversation. For example, the conversation unit learns frequently used phrases and expressions and reflects them in the conversation to generate responses that are familiar to the user. The conversation unit can also monitor the user's reactions in real time and provide responses at appropriate timing. This allows the conversation unit to smoothly progress interactions with the user and improve user satisfaction. Furthermore, the conversation unit can simulate multiple dialogue scenarios and select the optimal response. For example, multiple response candidates are generated in response to a user's question, and the most appropriate one is selected and provided. This allows the conversation unit to flexibly respond to the user's needs and provide a more advanced conversation experience.

[0033] The recording unit can record the contents of the conversation between the user and the interactive AI system. The interactive AI system includes, for example, a generation AI, Siri, Alexa, etc., but is not limited to such examples. For example, the recording unit records the contents of the conversation between the user and the generation AI as a text log. In addition, the recording unit can also record the contents of the conversation between the user and Siri as an audio file. Furthermore, the recording unit can also record the contents of the conversation between the user and Alexa as a video file. For example, the recording unit records the text-based conversation between the user and the generation AI in real time and saves it in a log file. The voice conversation between the user and Siri is recorded using a microphone and saved as an audio file. The video conversation between the user and Alexa is recorded using a camera and saved as a video file. In this way, the conversation between the user and the interactive AI system can be recorded and used for later analysis. Some or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can record the contents of the conversation between the user and the interactive AI system in real time and save it in a log file using an AI model.

[0034] The analysis unit can analyze the recorded conversation content and learn the user's conversation patterns and expressions. The conversation patterns include, for example, frequent phrases, grammatical structures, and topic flow, but are not limited to such examples. The analysis unit, for example, extracts frequent phrases from the recorded conversation content and learns the user's conversation patterns. The analysis unit can also analyze the grammatical structure of the recorded conversation content and learn the user's conversation patterns. Furthermore, the analysis unit can analyze the topic flow of the recorded conversation content and learn the user's conversation patterns. For example, the analysis unit extracts frequent phrases from the recorded conversation content and identifies the user's conversation patterns. The analysis of the grammatical structure extracts grammatical features of the conversation content and learns the user's conversation patterns. The analysis of the topic flow analyzes the transition of the topic of the conversation content and learns the user's conversation patterns. The phrases include, for example, honorific language, casual expressions, technical terms, etc., but are not limited to such examples. The analysis unit, for example, extracts a pattern of using honorific language from the recorded conversation content and learns the user's expressions. The analysis unit can also extract a pattern of using casual expressions from the recorded conversation content and learn the user's expressions. The analysis unit can also extract a pattern of using technical terms from the recorded conversation content and learn the user's expressions. For example, the analysis unit extracts a pattern of using honorific language from the recorded conversation content and identifies the user's expressions. The analysis of casual expressions extracts a pattern of using casual expressions from the conversation content and learns the user's expressions. The analysis of technical terms extracts a pattern of using technical terms from the conversation content and learns the user's expressions. This makes it possible to have a more natural conversation by learning the user's conversation patterns and expressions. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the recorded conversation content into an AI model and have the AI ​​model learn the user's conversation patterns and expressions.

[0035] The conversation unit can have a conversation on behalf of the user in a specific scene based on the analysis result. The specific scene includes, for example, a business meeting, a casual conversation, and a response to an emergency, but is not limited to such examples. For example, the conversation unit has a conversation on behalf of the user in a business meeting. The conversation unit can also have a conversation on behalf of the user in a casual conversation. Furthermore, the conversation unit can have a conversation on behalf of the user in a response to an emergency. For example, the conversation unit has a conversation on behalf of the user in a business meeting and uses business phrases and expressions. In a casual conversation, the conversation is conducted using the user's usual phrases and expressions. In a response to an emergency, the conversation is conducted using phrases and expressions for a quick and appropriate response. This reduces the burden on the user by having a conversation on behalf of the user in a specific scene. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the analysis result into an AI model and cause the AI ​​model to execute a conversation in a specific scene.

[0036] The conversation unit can reply on behalf of the user when the child speaks to the conversation unit through a messaging application. Examples of messaging applications include, but are not limited to, LINE (registered trademark), WhatsApp (registered trademark), and Facebook (registered trademark) Messenger. For example, the conversation unit replies on behalf of the user when the child speaks to the conversation unit through LINE. Furthermore, the conversation unit can also reply on behalf of the user when the child speaks to the conversation unit through WhatsApp. Furthermore, the conversation unit can also reply on behalf of the user when the child speaks to the conversation unit through Facebook Messenger. For example, when the child speaks to the conversation unit through LINE, the conversation unit replies using the user's usual phrases and expressions. In a conversation through WhatsApp, the conversation unit replies using the user's usual phrases and expressions. In a conversation through Facebook Messenger, the conversation unit replies using the user's usual phrases and expressions. This makes it possible to provide an experience as if the child were talking to a parent. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the contents of the conversation in the messaging application to an AI model and cause the AI ​​model to execute a process of replying on behalf of the user.

[0037] The conversation unit can converse with the elderly instead of the caregiver at the nursing site. Nursing sites include, for example, nursing homes, home nursing care, day services, and the like, but are not limited to such examples. For example, the conversation unit converses with the elderly instead of the caregiver at the nursing home. The conversation unit can also converse with the elderly instead of the caregiver at home nursing care. Furthermore, the conversation unit can also converse with the elderly instead of the caregiver at day services. For example, the conversation unit converses with the elderly instead of the caregiver at the nursing home, and uses the caregiver's usual phrases and expressions. In home nursing care, the conversation with the elderly is performed using the caregiver's usual phrases and expressions. In day services, the conversation with the elderly is performed using the caregiver's usual phrases and expressions. This can reduce the burden on the caregiver. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the content of the conversation at the nursing site into an AI model, and cause the AI ​​model to execute a process of conversing with the elderly instead of the caregiver.

[0038] The conversation unit can converse with the customer on behalf of the hairdresser in the hairdresser's customer service work. The hairdresser's customer service work includes, for example, counseling, conversation during treatment, explanation of aftercare, etc., but is not limited to such examples. The conversation unit, for example, converses with the customer on behalf of the hairdresser in counseling. The conversation unit can also converse with the customer on behalf of the hairdresser in conversation during treatment. Furthermore, the conversation unit can converse with the customer on behalf of the hairdresser in explanation of aftercare. For example, the conversation unit converses with the customer on behalf of the hairdresser in counseling, and uses the hairdresser's usual phrases and expressions. In the conversation during treatment, the conversation with the customer is carried out using the hairdresser's usual phrases and expressions. In the explanation of aftercare, the conversation with the customer is carried out using the hairdresser's usual phrases and expressions. This can make the work of the hairdresser more efficient. A part or all of the above-mentioned processing in the conversation unit may be carried out using, for example, AI, or may be carried out without using AI. For example, the conversation unit can input the content of a conversation that a hairdresser has with a customer into the AI ​​model and have the AI ​​model execute a process of conversing with the customer on behalf of the hairdresser.

[0039] The recording unit can analyze the user's past conversation history and select the optimal recording method. The past conversation history includes, for example, text logs, voice recordings, video recordings, and the like, but is not limited to such examples. For example, the recording unit preferentially records phrases that the user has frequently used in the past. In addition, the recording unit can also analyze the user's past conversation patterns and automatically extract and record important parts. Furthermore, the recording unit can preferentially record conversation contents in specific scenes from the user's past conversation history. For example, the recording unit extracts phrases that the user has frequently used in the past and saves them in a log file. The recording unit analyzes the user's past conversation patterns and automatically extracts important parts and saves them in a log file. The recording unit preferentially records conversation contents in specific scenes from the user's past conversation history and saves them in a log file. In this way, the optimal recording method can be selected by analyzing the past conversation history. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the user's past conversation history into an AI model and cause the AI ​​model to select the optimal recording method.

[0040] The recording unit can perform filtering based on the user's current situation and field of interest when recording the conversation content. The current situation includes, for example, a location, time, surrounding environment, etc., but is not limited to such examples. The field of interest includes, for example, hobbies, work, learning contents, etc., but is not limited to such examples. For example, when the user is at work, the recording unit preferentially records the conversation content related to work. In addition, when the user is talking about a hobby, the recording unit can also focus on recording the conversation content related to the hobby. Furthermore, when the user is working on a specific project, the recording unit can also preferentially record the conversation content related to the project. For example, when the user is at work, the recording unit extracts the conversation content related to work and saves it in a log file. When the user is talking about a hobby, the recording unit extracts the conversation content related to the hobby and saves it in a log file. When the user is working on a specific project, the recording unit extracts the conversation content related to the project and saves it in a log file. In this way, important conversation content can be preferentially recorded by filtering based on the user's situation and field of interest. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input data on the user's current situation and areas of interest into the AI ​​model and have the AI ​​model perform filtering.

[0041] When recording the conversation contents, the recording unit can preferentially record the conversation contents with high relevance in consideration of the geographical location information of the user. The geographical location information includes, for example, GPS data, address information, location sensor data, etc., but is not limited to such examples. For example, when the user is in a specific location, the recording unit preferentially records the conversation contents related to the location. In addition, when the user is traveling, the recording unit can also focus on recording the conversation contents related to the travel destination. Furthermore, when the user is at home, the recording unit can preferentially record the conversation contents related to the home. For example, when the user is in a specific location, the recording unit extracts the conversation contents related to the location and saves them in a log file. When the user is traveling, the recording unit extracts the conversation contents related to the travel destination and saves them in a log file. When the user is at home, the recording unit extracts the conversation contents related to the home and saves them in a log file. In this way, by considering the geographical location information of the user, the conversation contents with high relevance can be preferentially recorded. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the user's geographical location information into the AI ​​model and cause the AI ​​model to extract highly relevant conversation content.

[0042] The recording unit can analyze the user's social media activity and record related conversations when recording the conversation content. Social media activities include, for example, posting content, comments, the number of likes, and the like, but are not limited to such examples. For example, the recording unit preferentially records phrases that the user frequently uses on social media. The recording unit can also focus on recording conversation content related to the user's social media interest field. Furthermore, the recording unit can automatically extract and record important conversation content based on the user's social media activity history. For example, the recording unit extracts phrases that the user frequently uses on social media and saves them in a log file. The recording unit extracts conversation content related to the user's social media interest field and saves them in a log file. The recording unit automatically extracts important conversation content based on the user's social media activity history and saves them in a log file. In this way, by analyzing the user's social media activity, related conversation content can be preferentially recorded. Part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can input the user's social media activity data into an AI model and cause the AI ​​model to extract related conversation content.

[0043] The analysis unit can adjust the level of detail of the analysis based on the importance of the conversation during the analysis. The importance includes, for example, the content of the conversation, the importance of the speaker, the context of the conversation, and the like, but is not limited to such examples. For example, the analysis unit performs a detailed analysis of important conversation contents. The analysis unit can also perform a concise analysis of general conversation contents. Furthermore, the analysis unit can perform a detailed analysis of conversation contents in a specific scene according to the scene. For example, the analysis unit performs a detailed analysis of important conversation contents and displays the results. For general conversation contents, the analysis unit performs a concise analysis and displays the results. For conversation contents in a specific scene, the analysis unit performs a detailed analysis according to the scene and displays the results. As a result, important conversation contents can be analyzed in detail by adjusting the level of detail of the analysis based on the importance of the conversation. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input importance data of the conversation contents to an AI model and cause the AI ​​model to adjust the level of detail of the analysis.

[0044] The analysis unit can apply different analysis algorithms depending on the category of the conversation during analysis. The categories include, for example, business, private, academic, etc., but are not limited to such examples. For example, in the case of a business conversation, the analysis unit applies a business analysis algorithm. In addition, in the case of a private conversation, the analysis unit can also apply a private analysis algorithm. Furthermore, the analysis unit can apply an analysis algorithm according to the scene to the content of the conversation in a specific scene. For example, in the case of a business conversation, the analysis unit applies an analysis algorithm for business and displays the result. In the case of a private conversation, the analysis unit applies an analysis algorithm for private and displays the result. In the case of a specific scene, the analysis unit applies an analysis algorithm according to the scene and displays the result. In this way, by applying different analysis algorithms depending on the category of the conversation, more appropriate analysis results can be provided. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input category data of the content of the conversation to an AI model and cause the AI ​​model to apply the analysis algorithm.

[0045] The analysis unit can determine the priority of the analysis based on the time of occurrence of the conversation during the analysis. The time of occurrence includes, for example, the date and time, the season, before and after an event, and the like, but is not limited to such examples. For example, the analysis unit prioritizes the analysis of the most recent conversation content. The analysis unit can also focus on analyzing the conversation content that occurred in a specific period. Furthermore, the analysis unit can prioritize the analysis of the conversation content in a period specified by the user. For example, the analysis unit prioritizes the analysis of the most recent conversation content and displays the results. The analysis unit prioritizes the analysis of the conversation content that occurred in a specific period and displays the results. The analysis unit prioritizes the analysis of the conversation content in a period specified by the user and displays the results. As a result, the analysis priority is determined based on the time of occurrence of the conversation, so that the latest conversation content can be analyzed with priority. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the data on the time of occurrence of the conversation content into an AI model and cause the AI ​​model to execute the determination of the analysis priority.

[0046] The analysis unit can adjust the order of analysis based on the relevance of the conversation during analysis. The relevance includes, for example, the degree of matching of topics, co-occurrence of keywords, similarity of context, and the like, but is not limited to such examples. For example, the analysis unit prioritizes analysis of highly relevant conversation contents. The analysis unit can also postpone conversation contents with low relevance. Furthermore, the analysis unit can analyze conversation contents in a specific scene in an order according to the scene. For example, the analysis unit prioritizes analysis of highly relevant conversation contents and displays the results. Conversation contents with low relevance are postponed and the results are displayed. Conversation contents in a specific scene are analyzed in an order according to the scene and the results are displayed. In this way, by adjusting the order of analysis based on the relevance of the conversation, highly relevant conversation contents can be analyzed with priority. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input relevance data of the conversation contents to an AI model and cause the AI ​​model to adjust the order of analysis.

[0047] The conversation unit can select an appropriate conversation method by referring to the user's past conversation patterns during conversation. The past conversation patterns include, for example, frequent phrases, grammatical structures, and topic flow, but are not limited to such examples. The conversation unit uses, for example, phrases that the user has frequently used in the past. The conversation unit can also analyze the user's past conversation patterns and select an optimal conversation method. Furthermore, the conversation unit can select an optimal conversation method for a specific scene from the user's past conversation history. For example, the conversation unit extracts phrases that the user has frequently used in the past and uses them in the conversation. The conversation unit analyzes the user's past conversation patterns, selects an optimal conversation method, and uses it in the conversation. The conversation unit selects an optimal conversation method for a specific scene from the user's past conversation history and uses it in the conversation. In this way, the optimal conversation method can be selected by referring to the user's past conversation patterns. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's past conversation pattern data into an AI model and cause the AI ​​model to select an optimal conversation method.

[0048] The conversation unit can customize the content of the conversation based on the current situation of the user during the conversation. The current situation includes, for example, a location, a time, a surrounding environment, and the like, but is not limited to such examples. For example, when the user is at work, the conversation unit provides a conversation content related to the work. In addition, when the user is talking about a hobby, the conversation unit can also provide a conversation content related to the hobby. Furthermore, when the user is working on a specific project, the conversation unit can also provide a conversation content related to the project. For example, when the user is at work, the conversation unit extracts a conversation content related to the work and uses it for the conversation. When the user is talking about a hobby, the conversation unit extracts a conversation content related to the hobby and uses it for the conversation. When the user is working on a specific project, the conversation unit extracts a conversation content related to the project and uses it for the conversation. In this way, by customizing the content of the conversation based on the current situation of the user, a more appropriate conversation can be provided. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's current situation data into an AI model and cause the AI ​​model to execute customization of the conversation content.

[0049] The conversation unit can select an appropriate conversation method in consideration of the geographical location information of the user during the conversation. The geographical location information includes, for example, GPS data, address information, location sensor data, etc., but is not limited to such examples. For example, when the user is in a specific location, the conversation unit provides conversation content related to the location. In addition, when the user is traveling, the conversation unit can also provide conversation content related to the travel destination. Furthermore, when the user is at home, the conversation unit can also provide conversation content related to the home. For example, when the user is in a specific location, the conversation unit extracts conversation content related to the location and uses it for the conversation. When the user is traveling, the conversation unit extracts conversation content related to the travel destination and uses it for the conversation. When the user is at home, the conversation unit extracts conversation content related to the home and uses it for the conversation. In this way, by considering the geographical location information of the user, it is possible to provide highly relevant conversation content. Part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the geographical location information data of the user to an AI model and cause the AI ​​model to select an appropriate conversation method.

[0050] The conversation unit can analyze the user's social media activity during the conversation to suggest the content of the conversation. The social media activity includes, for example, the content of posts, comments, the number of likes, and the like, but is not limited to such examples. The conversation unit, for example, uses phrases that the user frequently uses on social media. The conversation unit can also provide conversation content related to the user's field of interest on social media. Furthermore, the conversation unit can automatically suggest important conversation content based on the user's social media activity history. For example, the conversation unit extracts phrases that the user frequently uses on social media and uses them in the conversation. The conversation unit extracts conversation content related to the user's field of interest on social media and uses them in the conversation. Based on the user's social media activity history, important conversation content is automatically suggested and used in the conversation. In this way, by analyzing the user's social media activity, it is possible to provide highly relevant conversation content. Part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's social media activity data into an AI model and cause the AI ​​model to execute the suggestion of the conversation content.

[0051] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0052] The recording unit can automatically create a memo based on the content of the user's conversation, triggered by a specific keyword or phrase. For example, if the user says "important," that part is emphasized and recorded. Also, if the user says "check later," that part can be added to another list. Furthermore, if the user says "meeting," the content of the conversation can be saved as a meeting memo. This makes it possible to efficiently record important information based on the user's instructions. Some or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the content of the user's conversation into an AI model and cause the AI ​​model to execute a process of creating a memo triggered by a specific keyword or phrase.

[0053] The analysis unit can analyze the contents of the user's conversation and estimate the user's interests and concerns. For example, if the user frequently talks about a specific topic, the analysis unit prioritizes the analysis of information related to that topic. In addition, if the user frequently uses a specific keyword, the analysis unit can also focus on analyzing information related to that keyword. Furthermore, if the user talks about a specific person or place, the analysis unit can also analyze information related to that person or place. This makes it possible to provide more useful information by performing analysis based on the user's interests and concerns. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the contents of the user's conversation into an AI model and cause the AI ​​model to execute an estimation of the interests and concerns.

[0054] The recording unit can translate the contents of the user's conversation in real time and record it in multiple languages. For example, when the user speaks in English, the recording unit translates the contents into Japanese, Spanish, etc. and records them. Also, when the user speaks in Japanese, the recording unit can translate the contents into English, French, etc. and record them. Furthermore, when the user speaks a mixture of multiple languages, translation corresponding to each language can be performed and recorded. This can facilitate communication between users who use different languages. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can input the contents of the user's conversation into an AI model and have the AI ​​model perform real-time translation and recording.

[0055] The recording unit can analyze the content of the user's conversation and automatically collect and record information related to a specific topic. For example, if the user is talking about travel, the recording unit collects and records information related to the travel. Also, if the user is talking about health, the recording unit can collect and record information related to health. Furthermore, if the user is talking about work, the recording unit can collect and record information related to work. This makes it possible to efficiently collect and record related information based on the content of the user's conversation. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the content of the user's conversation into an AI model and cause the AI ​​model to collect and record related information.

[0056] The analysis unit can analyze the content of the user's conversation and estimate the user's learning style. For example, if the user prefers visual information, the analysis unit can provide visual materials. If the user prefers auditory information, the analysis unit can also provide audio materials. Furthermore, if the user prefers practical learning, the analysis unit can also provide practical tasks. This makes it possible to provide appropriate learning materials according to the user's learning style. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the content of the user's conversation into an AI model and cause the AI ​​model to execute the estimation of the learning style.

[0057] The process flow of the first embodiment will be briefly described below.

[0058] Step 1: The recording unit records the content of the conversation between the user and the generated AI. The recording unit saves the text-based conversation in a log file, records the voice conversation and saves it as an audio file, and records the video conversation and saves it as a video file. For example, the text-based conversation is recorded in real time and saved in a log file. The voice conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. Step 2: The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed using natural language processing technology, and includes extracting keywords from the conversation content, sentiment analysis, and extracting important parts. For example, the analysis unit uses natural language processing technology to extract keywords from the conversation content and identify the topic of the conversation. It uses sentiment analysis technology to estimate the sentiment of the conversation content and calculate a sentiment score. It uses keyword extraction technology to extract important parts of the conversation content and generate a summary. Step 3: The conversation unit conducts a conversation based on the results of the analysis by the analysis unit. The conversation is conducted using a dialogue-based AI system, and can conduct the conversation on behalf of the user. For example, the conversation unit conducts a conversation on behalf of the user in a specific scene based on the analysis results, reproducing the user's conversation patterns and phrases. The processing in the conversation unit may be performed using AI or may be performed without using AI. For example, the conversation unit conducts a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs the conversation.

[0059] (Example 2) The conversation learning system according to the embodiment of the present invention is a system in which an AI learns conversations with a generation AI (for example, GPT- or Gemini) to learn the contents of the conversation for each scene, and creates an alter ego of the user. This conversation learning system is a system in which a user has a conversation with a generation AI, records the conversation contents, analyzes the recorded conversation contents, and learns the user's conversation patterns and phrases, so that the AI ​​can converse on behalf of the user in a specific scene. For example, when a child talks to the AI ​​on a messenger app, the AI ​​replies on behalf of the user. The AI ​​can behave as if the user himself is replying by using phrases and expressions that the user normally uses. This allows the child to have an experience as if he or she were talking to a parent. Furthermore, this mechanism can be used in various scenes, such as nursing care and customer service at a hairdresser. For example, in the field of nursing care, the AI ​​can converse with the elderly instead of the caregiver, thereby reducing the burden on the caregiver. In addition, in customer service at a hairdresser, the AI ​​can converse with the customer instead of the hairdresser, thereby making the hairdresser's work more efficient. In this way, by having the AI ​​learn conversations with the generating AI, the AI ​​can learn the content of conversations for each scene and create an avatar of itself. This allows the conversation learning system to learn the user's conversation patterns and expressions and to converse on behalf of the user in specific scenes.

[0060] The conversation learning system according to the embodiment includes a recording unit, an analysis unit, and a conversation unit. The recording unit records the conversation content between the user and the generation AI. The conversation content between the user and the generation AI includes, for example, a text-based conversation, an audio conversation, a video conversation, and the like, but is not limited to such examples. The recording unit, for example, saves the text-based conversation in a log file. The recording unit can also record the audio conversation and save it as an audio file. Furthermore, the recording unit can record the video conversation and save it as a video file. For example, the recording unit records the text-based conversation in real time and saves it in a log file. The audio conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed, for example, using natural language processing technology, but is not limited to such examples. For example, the analysis unit uses natural language processing technology to extract keywords from the conversation content. The analysis unit can also estimate emotions from the conversation content using emotion analysis technology. Furthermore, the analysis unit can also extract important parts of the conversation content using a keyword extraction technique. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content and identify the topic of the conversation. The emotion analysis technique estimates the emotion of the conversation content and calculates an emotion score. The keyword extraction technique extracts important parts of the conversation content and generates a summary. The conversation unit conducts a conversation based on the result analyzed by the analysis unit. The conversation is conducted, for example, using an interactive AI system, but is not limited to such an example. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system. Furthermore, the conversation unit can also conduct a conversation on behalf of the user in a specific scene based on the analysis result. Furthermore, the conversation unit can reproduce the user's conversation pattern and phrases based on the analysis result. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system and reproduces the user's conversation pattern and phrases. As a result, the conversation learning system according to the embodiment can learn the user's conversation pattern and phrases and conduct a conversation on behalf of the user in a specific scene.A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can have a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs a conversation.

[0061] The recording unit records the contents of the conversation between the user and the generation AI. The contents of the conversation between the user and the generation AI include, for example, text-based conversation, voice conversation, video conversation, etc., but are not limited to such examples. The recording unit, for example, saves the text-based conversation in a log file. The recording unit can also record the voice conversation and save it as an audio file. Furthermore, the recording unit can record the video conversation and save it as a video file. For example, the recording unit records the text-based conversation in real time and saves it in a log file. The voice conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. The recording unit can use a database system to efficiently manage these data. The database system organizes the conversation contents in chronological order, making it easy to search and filter. For example, the conversation contents can be searched based on a specific keyword or date and time. In addition, the recording unit can use encryption technology to ensure the security of the data. This makes it possible to maintain the integrity of the data while protecting the privacy of the user. Furthermore, the recording unit has a data backup function, and measures can be taken against data loss or damage. For example, by regularly backing up data and storing it in different locations, the risk of disasters and system failures can be reduced. This allows the recording unit to reliably record the content of conversations between the user and the generating AI, providing a foundation for using it for later analysis and learning.

[0062] The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed, for example, using a natural language processing technique, but is not limited to such an example. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content. The analysis unit can also use a sentiment analysis technique to estimate the sentiment of the conversation content. Furthermore, the analysis unit can also extract important parts of the conversation content using a keyword extraction technique. For example, the analysis unit uses a natural language processing technique to extract keywords from the conversation content and identify the topic of the conversation. The sentiment analysis technique estimates the sentiment of the conversation content and calculates a sentiment score. The keyword extraction technique extracts important parts of the conversation content and generates a summary. The analysis unit can combine these techniques to grasp the overall picture of the conversation content and understand the user's intentions and emotions. For example, the natural language processing technique is used to analyze the context of the conversation and identify the user's questions and requests. The sentiment analysis technique is used to evaluate the user's emotional state and provide information for generating an appropriate response. Furthermore, the analysis unit can analyze trends in the conversation content and grasp the user's interests and concerns. For example, past conversation data can be used to identify frequently discussed topics and keywords that users use in future conversations, providing a foundation for the analytics department to better understand users' needs and emotions and provide more personalized responses.

[0063] The conversation unit conducts a conversation based on the results of the analysis by the analysis unit. The conversation is conducted, for example, using an interactive AI system, but is not limited to such an example. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system. The conversation unit can also conduct a conversation on behalf of the user in a specific scene based on the analysis result. Furthermore, the conversation unit can reproduce the user's conversation pattern and phrases based on the analysis result. For example, the conversation unit conducts a conversation on behalf of the user using an interactive AI system and reproduces the user's conversation pattern and phrases. Thereby, the conversation learning system according to the embodiment can learn the user's conversation pattern and phrases and conduct a conversation on behalf of the user in a specific scene. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can conduct a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs a conversation. The conversation unit can learn the user's preferences and style based on the user's past conversation data and provide a more natural and consistent conversation. For example, the conversation unit learns frequently used phrases and expressions and reflects them in the conversation to generate responses that are familiar to the user. The conversation unit can also monitor the user's reactions in real time and provide responses at appropriate timing. This allows the conversation unit to smoothly progress interactions with the user and improve user satisfaction. Furthermore, the conversation unit can simulate multiple dialogue scenarios and select the optimal response. For example, multiple response candidates are generated in response to a user's question, and the most appropriate one is selected and provided. This allows the conversation unit to flexibly respond to the user's needs and provide a more advanced conversation experience.

[0064] The recording unit can record the contents of the conversation between the user and the interactive AI system. The interactive AI system includes, for example, a generation AI, Siri, Alexa, etc., but is not limited to such examples. For example, the recording unit records the contents of the conversation between the user and the generation AI as a text log. In addition, the recording unit can also record the contents of the conversation between the user and Siri as an audio file. Furthermore, the recording unit can also record the contents of the conversation between the user and Alexa as a video file. For example, the recording unit records the text-based conversation between the user and the generation AI in real time and saves it in a log file. The voice conversation between the user and Siri is recorded using a microphone and saved as an audio file. The video conversation between the user and Alexa is recorded using a camera and saved as a video file. In this way, the conversation between the user and the interactive AI system can be recorded and used for later analysis. Some or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can record the contents of the conversation between the user and the interactive AI system in real time and save it in a log file using an AI model.

[0065] The analysis unit can analyze the recorded conversation content and learn the user's conversation patterns and expressions. The conversation patterns include, for example, frequent phrases, grammatical structures, and topic flow, but are not limited to such examples. The analysis unit, for example, extracts frequent phrases from the recorded conversation content and learns the user's conversation patterns. The analysis unit can also analyze the grammatical structure of the recorded conversation content and learn the user's conversation patterns. Furthermore, the analysis unit can analyze the topic flow of the recorded conversation content and learn the user's conversation patterns. For example, the analysis unit extracts frequent phrases from the recorded conversation content and identifies the user's conversation patterns. The analysis of the grammatical structure extracts grammatical features of the conversation content and learns the user's conversation patterns. The analysis of the topic flow analyzes the transition of the topic of the conversation content and learns the user's conversation patterns. The phrases include, for example, honorific language, casual expressions, technical terms, etc., but are not limited to such examples. The analysis unit, for example, extracts a pattern of using honorific language from the recorded conversation content and learns the user's expressions. The analysis unit can also extract a pattern of using casual expressions from the recorded conversation content and learn the user's expressions. The analysis unit can also extract a pattern of using technical terms from the recorded conversation content and learn the user's expressions. For example, the analysis unit extracts a pattern of using honorific language from the recorded conversation content and identifies the user's expressions. The analysis of casual expressions extracts a pattern of using casual expressions from the conversation content and learns the user's expressions. The analysis of technical terms extracts a pattern of using technical terms from the conversation content and learns the user's expressions. This makes it possible to have a more natural conversation by learning the user's conversation patterns and expressions. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the recorded conversation content into an AI model and have the AI ​​model learn the user's conversation patterns and expressions.

[0066] The conversation unit can have a conversation on behalf of the user in a specific scene based on the analysis result. The specific scene includes, for example, a business meeting, a casual conversation, and a response to an emergency, but is not limited to such examples. For example, the conversation unit has a conversation on behalf of the user in a business meeting. The conversation unit can also have a conversation on behalf of the user in a casual conversation. Furthermore, the conversation unit can have a conversation on behalf of the user in a response to an emergency. For example, the conversation unit has a conversation on behalf of the user in a business meeting and uses business phrases and expressions. In a casual conversation, the conversation is conducted using the user's usual phrases and expressions. In a response to an emergency, the conversation is conducted using phrases and expressions for a quick and appropriate response. This reduces the burden on the user by having a conversation on behalf of the user in a specific scene. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the analysis result into an AI model and cause the AI ​​model to execute a conversation in a specific scene.

[0067] The conversation unit can reply on behalf of the user when the child speaks to the conversation unit through a messaging application. Examples of messaging applications include, but are not limited to, LINE, WhatsApp, Facebook Messenger, and the like. For example, when the child speaks to the conversation unit through LINE, the conversation unit replies on behalf of the user. In addition, the conversation unit can also reply on behalf of the user when the child speaks to the conversation unit through WhatsApp. In addition, the conversation unit can also reply on behalf of the user when the child speaks to the conversation unit through Facebook Messenger. For example, when the child speaks to the conversation unit through LINE, the conversation unit replies using the user's usual phrases and expressions. In a conversation through WhatsApp, the conversation unit replies using the user's usual phrases and expressions. In a conversation through Facebook Messenger, the conversation unit replies using the user's usual phrases and expressions. This makes it possible to provide an experience as if the child were talking to a parent. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the contents of the conversation in the messaging application to an AI model and cause the AI ​​model to execute a process of replying on behalf of the user.

[0068] The conversation unit can converse with the elderly instead of the caregiver at the nursing site. Nursing sites include, for example, nursing homes, home nursing care, day services, and the like, but are not limited to such examples. For example, the conversation unit converses with the elderly instead of the caregiver at the nursing home. The conversation unit can also converse with the elderly instead of the caregiver at home nursing care. Furthermore, the conversation unit can also converse with the elderly instead of the caregiver at day services. For example, the conversation unit converses with the elderly instead of the caregiver at the nursing home, and uses the caregiver's usual phrases and expressions. In home nursing care, the conversation with the elderly is performed using the caregiver's usual phrases and expressions. In day services, the conversation with the elderly is performed using the caregiver's usual phrases and expressions. This can reduce the burden on the caregiver. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit can input the content of the conversation at the nursing site into an AI model, and cause the AI ​​model to execute a process of conversing with the elderly instead of the caregiver.

[0069] The conversation unit can converse with the customer on behalf of the hairdresser in the hairdresser's customer service work. The hairdresser's customer service work includes, for example, counseling, conversation during treatment, explanation of aftercare, etc., but is not limited to such examples. The conversation unit, for example, converses with the customer on behalf of the hairdresser in counseling. The conversation unit can also converse with the customer on behalf of the hairdresser in conversation during treatment. Furthermore, the conversation unit can converse with the customer on behalf of the hairdresser in explanation of aftercare. For example, the conversation unit converses with the customer on behalf of the hairdresser in counseling, and uses the hairdresser's usual phrases and expressions. In the conversation during treatment, the conversation with the customer is carried out using the hairdresser's usual phrases and expressions. In the explanation of aftercare, the conversation with the customer is carried out using the hairdresser's usual phrases and expressions. This can make the work of the hairdresser more efficient. A part or all of the above-mentioned processing in the conversation unit may be carried out using, for example, AI, or may be carried out without using AI. For example, the conversation unit can input the content of a conversation that a hairdresser has with a customer into the AI ​​model and have the AI ​​model execute a process of conversing with the customer on behalf of the hairdresser.

[0070] The recording unit can estimate the user's emotion and adjust the timing of recording the conversation content based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is excited, the recording unit records important parts of the conversation with priority. Also, when the user is relaxed, the recording unit can record the entire conversation evenly. Furthermore, when the user is stressed, the recording unit can focus on recording parts that cause stress. For example, when the user is excited, the recording unit records important parts of the conversation with priority and saves them in a log file. When the user is relaxed, the recording unit records the entire conversation evenly and saves them in a log file. When the user is stressed, the recording unit records parts that cause stress with priority and saves them in a log file. In this way, important conversation content can be recorded with priority by adjusting the recording timing according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the recording unit may input the user's emotion data to the generation AI and cause the generation AI to estimate the emotion.

[0071] The recording unit can analyze the user's past conversation history and select the optimal recording method. The past conversation history includes, for example, text logs, voice recordings, video recordings, and the like, but is not limited to such examples. For example, the recording unit preferentially records phrases that the user has frequently used in the past. In addition, the recording unit can also analyze the user's past conversation patterns and automatically extract and record important parts. Furthermore, the recording unit can preferentially record conversation contents in specific scenes from the user's past conversation history. For example, the recording unit extracts phrases that the user has frequently used in the past and saves them in a log file. The recording unit analyzes the user's past conversation patterns and automatically extracts important parts and saves them in a log file. The recording unit preferentially records conversation contents in specific scenes from the user's past conversation history and saves them in a log file. In this way, the optimal recording method can be selected by analyzing the past conversation history. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the user's past conversation history into an AI model and cause the AI ​​model to select the optimal recording method.

[0072] The recording unit can perform filtering based on the user's current situation and field of interest when recording the conversation content. The current situation includes, for example, a location, time, surrounding environment, etc., but is not limited to such examples. The field of interest includes, for example, hobbies, work, learning contents, etc., but is not limited to such examples. For example, when the user is at work, the recording unit preferentially records the conversation content related to work. In addition, when the user is talking about a hobby, the recording unit can also focus on recording the conversation content related to the hobby. Furthermore, when the user is working on a specific project, the recording unit can also preferentially record the conversation content related to the project. For example, when the user is at work, the recording unit extracts the conversation content related to work and saves it in a log file. When the user is talking about a hobby, the recording unit extracts the conversation content related to the hobby and saves it in a log file. When the user is working on a specific project, the recording unit extracts the conversation content related to the project and saves it in a log file. In this way, important conversation content can be preferentially recorded by filtering based on the user's situation and field of interest. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input data on the user's current situation and areas of interest into the AI ​​model and have the AI ​​model perform filtering.

[0073] The recording unit can estimate the user's emotion and determine the priority of the conversation contents to be recorded based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is excited, the recording unit preferentially records the conversation contents related to the heightened emotion. Also, when the user is relaxed, the recording unit can record the entire conversation evenly. Furthermore, when the user is stressed, the recording unit can focus on recording the part that causes stress. For example, when the user is excited, the recording unit extracts the conversation contents related to the heightened emotion and saves it in a log file. When the user is relaxed, the recording unit evenly records the entire conversation and saves it in a log file. When the user is stressed, the recording unit extracts the part that causes stress and saves it in a log file. In this way, by determining the priority of the conversation contents to be recorded according to the user's emotion, important conversation contents can be preferentially recorded. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the recording unit may input the user's emotion data to the generation AI and cause the generation AI to estimate the emotion.

[0074] When recording the conversation contents, the recording unit can preferentially record the conversation contents with high relevance in consideration of the geographical location information of the user. The geographical location information includes, for example, GPS data, address information, location sensor data, etc., but is not limited to such examples. For example, when the user is in a specific location, the recording unit preferentially records the conversation contents related to the location. In addition, when the user is traveling, the recording unit can also focus on recording the conversation contents related to the travel destination. Furthermore, when the user is at home, the recording unit can preferentially record the conversation contents related to the home. For example, when the user is in a specific location, the recording unit extracts the conversation contents related to the location and saves them in a log file. When the user is traveling, the recording unit extracts the conversation contents related to the travel destination and saves them in a log file. When the user is at home, the recording unit extracts the conversation contents related to the home and saves them in a log file. In this way, by considering the geographical location information of the user, the conversation contents with high relevance can be preferentially recorded. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the user's geographical location information into the AI ​​model and cause the AI ​​model to extract highly relevant conversation content.

[0075] The recording unit can analyze the user's social media activity and record related conversations when recording the conversation content. Social media activities include, for example, posting content, comments, the number of likes, and the like, but are not limited to such examples. For example, the recording unit preferentially records phrases that the user frequently uses on social media. The recording unit can also focus on recording conversation content related to the user's social media interest field. Furthermore, the recording unit can automatically extract and record important conversation content based on the user's social media activity history. For example, the recording unit extracts phrases that the user frequently uses on social media and saves them in a log file. The recording unit extracts conversation content related to the user's social media interest field and saves them in a log file. The recording unit automatically extracts important conversation content based on the user's social media activity history and saves them in a log file. In this way, by analyzing the user's social media activity, related conversation content can be preferentially recorded. Part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can input the user's social media activity data into an AI model and cause the AI ​​model to extract related conversation content.

[0076] The analysis unit can estimate the user's emotion and adjust the expression method of the analysis based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is relaxed, the analysis unit provides a detailed analysis result. In addition, when the user is in a hurry, the analysis unit can also provide a concise analysis result with the main points. Furthermore, when the user is excited, the analysis unit can also provide an analysis result with a visually stimulating effect. For example, when the user is relaxed, the analysis unit generates and displays a detailed analysis result. When the user is in a hurry, the analysis unit generates and displays a concise analysis result with the main points. When the user is excited, the analysis unit generates and displays an analysis result with a visually stimulating effect. In this way, by adjusting the expression method of the analysis according to the user's emotion, a more appropriate analysis result can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit may input the user's emotion data to the generation AI and cause the generation AI to estimate the emotion.

[0077] The analysis unit can adjust the level of detail of the analysis based on the importance of the conversation during the analysis. The importance includes, for example, the content of the conversation, the importance of the speaker, the context of the conversation, and the like, but is not limited to such examples. For example, the analysis unit performs a detailed analysis of important conversation contents. The analysis unit can also perform a concise analysis of general conversation contents. Furthermore, the analysis unit can perform a detailed analysis of conversation contents in a specific scene according to the scene. For example, the analysis unit performs a detailed analysis of important conversation contents and displays the results. For general conversation contents, the analysis unit performs a concise analysis and displays the results. For conversation contents in a specific scene, the analysis unit performs a detailed analysis according to the scene and displays the results. As a result, important conversation contents can be analyzed in detail by adjusting the level of detail of the analysis based on the importance of the conversation. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input importance data of the conversation contents to an AI model and cause the AI ​​model to adjust the level of detail of the analysis.

[0078] The analysis unit can apply different analysis algorithms depending on the category of the conversation during analysis. The categories include, for example, business, private, academic, etc., but are not limited to such examples. For example, in the case of a business conversation, the analysis unit applies a business analysis algorithm. In addition, in the case of a private conversation, the analysis unit can also apply a private analysis algorithm. Furthermore, the analysis unit can apply an analysis algorithm according to the scene to the content of the conversation in a specific scene. For example, in the case of a business conversation, the analysis unit applies an analysis algorithm for business and displays the result. In the case of a private conversation, the analysis unit applies an analysis algorithm for private and displays the result. In the case of a specific scene, the analysis unit applies an analysis algorithm according to the scene and displays the result. In this way, by applying different analysis algorithms depending on the category of the conversation, more appropriate analysis results can be provided. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input category data of the content of the conversation to an AI model and cause the AI ​​model to apply the analysis algorithm.

[0079] The analysis unit can estimate the user's emotion and adjust the length of the analysis based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is in a hurry, the analysis unit performs a short and to the point analysis. Also, when the user is relaxed, the analysis unit can perform a detailed analysis. Furthermore, when the user is excited, the analysis unit can perform an analysis with visually stimulating effects added. For example, when the user is in a hurry, the analysis unit performs a short and to the point analysis and displays the result. When the user is relaxed, the analysis unit performs a detailed analysis and displays the result. When the user is excited, the analysis unit performs an analysis with visually stimulating effects added and displays the result. In this way, by adjusting the length of the analysis according to the user's emotion, a more appropriate analysis result can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit may input the user's emotion data to the generation AI and cause the generation AI to estimate the emotion.

[0080] The analysis unit can determine the priority of the analysis based on the time of occurrence of the conversation during the analysis. The time of occurrence includes, for example, the date and time, the season, before and after an event, and the like, but is not limited to such examples. For example, the analysis unit prioritizes the analysis of the most recent conversation content. The analysis unit can also focus on analyzing the conversation content that occurred in a specific period. Furthermore, the analysis unit can prioritize the analysis of the conversation content in a period specified by the user. For example, the analysis unit prioritizes the analysis of the most recent conversation content and displays the results. The analysis unit prioritizes the analysis of the conversation content that occurred in a specific period and displays the results. The analysis unit prioritizes the analysis of the conversation content in a period specified by the user and displays the results. As a result, the analysis priority is determined based on the time of occurrence of the conversation, so that the latest conversation content can be analyzed with priority. Part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the data on the time of occurrence of the conversation content into an AI model and cause the AI ​​model to execute the determination of the analysis priority.

[0081] The analysis unit can adjust the order of analysis based on the relevance of the conversation during analysis. The relevance includes, for example, the degree of matching of topics, co-occurrence of keywords, similarity of context, and the like, but is not limited to such examples. For example, the analysis unit prioritizes analysis of highly relevant conversation contents. The analysis unit can also postpone conversation contents with low relevance. Furthermore, the analysis unit can analyze conversation contents in a specific scene in an order according to the scene. For example, the analysis unit prioritizes analysis of highly relevant conversation contents and displays the results. Conversation contents with low relevance are postponed and the results are displayed. Conversation contents in a specific scene are analyzed in an order according to the scene and the results are displayed. In this way, by adjusting the order of analysis based on the relevance of the conversation, highly relevant conversation contents can be analyzed with priority. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input relevance data of the conversation contents to an AI model and cause the AI ​​model to adjust the order of analysis.

[0082] The conversation unit can estimate the user's emotion and adjust the expression method of the conversation based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is relaxed, the conversation unit uses a gentle expression method. Also, when the user is in a hurry, the conversation unit can use a concise and quick expression method. Furthermore, when the user is excited, the conversation unit can use an expression method with a visually stimulating effect added. For example, when the user is relaxed, the conversation unit uses a gentle expression method to conduct the conversation. When the user is in a hurry, the conversation unit uses a concise and quick expression method to conduct the conversation. When the user is excited, the conversation unit uses an expression method with a visually stimulating effect added. In this way, by adjusting the expression method of the conversation according to the user's emotion, a more appropriate conversation can be provided. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the conversation unit can input a user's emotion data to the generation AI and have the generation AI perform emotion estimation.

[0083] The conversation unit can select an appropriate conversation method by referring to the user's past conversation patterns during conversation. The past conversation patterns include, for example, frequent phrases, grammatical structures, and topic flow, but are not limited to such examples. The conversation unit uses, for example, phrases that the user has frequently used in the past. The conversation unit can also analyze the user's past conversation patterns and select an optimal conversation method. Furthermore, the conversation unit can select an optimal conversation method for a specific scene from the user's past conversation history. For example, the conversation unit extracts phrases that the user has frequently used in the past and uses them in the conversation. The conversation unit analyzes the user's past conversation patterns, selects an optimal conversation method, and uses it in the conversation. The conversation unit selects an optimal conversation method for a specific scene from the user's past conversation history and uses it in the conversation. In this way, the optimal conversation method can be selected by referring to the user's past conversation patterns. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's past conversation pattern data into an AI model and cause the AI ​​model to select an optimal conversation method.

[0084] The conversation unit can customize the content of the conversation based on the current situation of the user during the conversation. The current situation includes, for example, a location, a time, a surrounding environment, and the like, but is not limited to such examples. For example, when the user is at work, the conversation unit provides a conversation content related to the work. In addition, when the user is talking about a hobby, the conversation unit can also provide a conversation content related to the hobby. Furthermore, when the user is working on a specific project, the conversation unit can also provide a conversation content related to the project. For example, when the user is at work, the conversation unit extracts a conversation content related to the work and uses it for the conversation. When the user is talking about a hobby, the conversation unit extracts a conversation content related to the hobby and uses it for the conversation. When the user is working on a specific project, the conversation unit extracts a conversation content related to the project and uses it for the conversation. In this way, by customizing the content of the conversation based on the current situation of the user, a more appropriate conversation can be provided. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's current situation data into an AI model and cause the AI ​​model to execute customization of the conversation content.

[0085] The conversation unit can estimate the user's emotion and determine the priority of the conversation based on the estimated user's emotion. The emotion includes, for example, joy, sadness, anger, surprise, etc., but is not limited to such examples. For example, when the user is excited, the conversation unit provides the conversation content related to the heightened emotion with priority. Also, when the user is relaxed, the conversation unit can provide the entire conversation evenly. Furthermore, when the user is stressed, the conversation unit can provide the part that causes the stress with priority. For example, when the user is excited, the conversation unit extracts the conversation content related to the heightened emotion and uses it for the conversation. When the user is relaxed, the entire conversation is provided evenly and used for the conversation. When the user is stressed, the part that causes the stress is extracted and used for the conversation. In this way, important conversation content can be provided with priority by determining the priority of the conversation according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, AI, or may be performed without using AI. For example, the conversation unit may input the user's emotion data to the generation AI and cause the generation AI to estimate the emotion.

[0086] The conversation unit can select an appropriate conversation method in consideration of the geographical location information of the user during the conversation. The geographical location information includes, for example, GPS data, address information, location sensor data, etc., but is not limited to such examples. For example, when the user is in a specific location, the conversation unit provides conversation content related to the location. In addition, when the user is traveling, the conversation unit can also provide conversation content related to the travel destination. Furthermore, when the user is at home, the conversation unit can also provide conversation content related to the home. For example, when the user is in a specific location, the conversation unit extracts conversation content related to the location and uses it for the conversation. When the user is traveling, the conversation unit extracts conversation content related to the travel destination and uses it for the conversation. When the user is at home, the conversation unit extracts conversation content related to the home and uses it for the conversation. In this way, by considering the geographical location information of the user, it is possible to provide highly relevant conversation content. Part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the geographical location information data of the user to an AI model and cause the AI ​​model to select an appropriate conversation method.

[0087] The conversation unit can analyze the user's social media activity during the conversation to suggest the content of the conversation. The social media activity includes, for example, the content of posts, comments, the number of likes, and the like, but is not limited to such examples. The conversation unit, for example, uses phrases that the user frequently uses on social media. The conversation unit can also provide conversation content related to the user's field of interest on social media. Furthermore, the conversation unit can automatically suggest important conversation content based on the user's social media activity history. For example, the conversation unit extracts phrases that the user frequently uses on social media and uses them in the conversation. The conversation unit extracts conversation content related to the user's field of interest on social media and uses them in the conversation. Based on the user's social media activity history, important conversation content is automatically suggested and used in the conversation. In this way, by analyzing the user's social media activity, it is possible to provide highly relevant conversation content. Part or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the user's social media activity data into an AI model and cause the AI ​​model to execute the suggestion of the conversation content.

[0088] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0089] The conversation unit can analyze the tone and speed of the user's voice and adjust the tempo and tone of the conversation based on this. For example, if the user speaks quickly, the conversation unit can respond by speeding up the tempo. Also, if the user speaks slowly, the conversation unit can respond by slowing down the tempo. Furthermore, if the user's voice tone is high, the conversation unit can respond with a bright tone, and if the user's voice tone is low, the conversation unit can respond with a calm tone. This enables a natural conversation that matches the user's speaking style. Some or all of the above-mentioned processing in the conversation unit may be performed, for example, using AI, or may be performed without using AI. For example, the conversation unit can input the tone and speed data of the user's voice into an AI model and cause the AI ​​model to adjust the tempo and tone of the conversation.

[0090] The recording unit can automatically create a memo based on the content of the user's conversation, triggered by a specific keyword or phrase. For example, if the user says "important," that part is emphasized and recorded. Also, if the user says "check later," that part can be added to another list. Furthermore, if the user says "meeting," the content of the conversation can be saved as a meeting memo. This makes it possible to efficiently record important information based on the user's instructions. Some or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the content of the user's conversation into an AI model and cause the AI ​​model to execute a process of creating a memo triggered by a specific keyword or phrase.

[0091] The analysis unit can analyze the contents of the user's conversation and estimate the user's interests and concerns. For example, if the user frequently talks about a specific topic, the analysis unit prioritizes the analysis of information related to that topic. In addition, if the user frequently uses a specific keyword, the analysis unit can also focus on analyzing information related to that keyword. Furthermore, if the user talks about a specific person or place, the analysis unit can also analyze information related to that person or place. This makes it possible to provide more useful information by performing analysis based on the user's interests and concerns. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the contents of the user's conversation into an AI model and cause the AI ​​model to execute an estimation of the interests and concerns.

[0092] The conversation unit can estimate the user's emotions and adjust the content of the conversation based on the estimated emotions. For example, if the user is sad, the conversation unit can provide comforting content. Also, if the user is happy, the conversation unit can provide empathetic content. Furthermore, if the user is angry, the conversation unit can provide content that responds calmly. This makes it possible to provide appropriate conversation content according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the conversation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0093] The recording unit can translate the contents of the user's conversation in real time and record it in multiple languages. For example, when the user speaks in English, the recording unit translates the contents into Japanese, Spanish, etc. and records them. Also, when the user speaks in Japanese, the recording unit can translate the contents into English, French, etc. and record them. Furthermore, when the user speaks a mixture of multiple languages, translation corresponding to each language can be performed and recorded. This can facilitate communication between users who use different languages. A part or all of the above-mentioned processing in the recording unit may be performed using, for example, AI, or may be performed without using AI. For example, the recording unit can input the contents of the user's conversation into an AI model and have the AI ​​model perform real-time translation and recording.

[0094] The analysis unit can analyze the contents of the user's conversation and estimate the user's health condition. For example, if the user frequently says that he or she is tired, the analysis unit estimates that the user is tired. In addition, if the user complains of a headache, the analysis unit can also estimate that the user is feeling a headache. Furthermore, if the user is feeling stressed, the analysis unit can analyze the cause and suggest a method of reducing stress. This makes it possible to provide appropriate advice based on the user's health condition. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the contents of the user's conversation into an AI model and cause the AI ​​model to execute an estimation of the health condition.

[0095] The conversation unit can estimate the user's emotions and adjust the timing of the conversation based on the estimated emotions. For example, if the user is busy, the conversation unit can respond short and to the point. Also, if the user is relaxed, the conversation unit can provide a detailed explanation. Furthermore, if the user is excited, the conversation unit can respond quickly and provide content that will ease the user's excitement. This enables a conversation at an appropriate time according to the user's emotions. The emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversation unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the conversation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0096] The recording unit can analyze the content of the user's conversation and automatically collect and record information related to a specific topic. For example, if the user is talking about travel, the recording unit collects and records information related to the travel. Also, if the user is talking about health, the recording unit can collect and record information related to health. Furthermore, if the user is talking about work, the recording unit can collect and record information related to work. This makes it possible to efficiently collect and record related information based on the content of the user's conversation. A part or all of the above-mentioned processing in the recording unit may be performed, for example, using AI, or may be performed without using AI. For example, the recording unit can input the content of the user's conversation into an AI model and cause the AI ​​model to collect and record related information.

[0097] The analysis unit can analyze the content of the user's conversation and estimate the user's learning style. For example, if the user prefers visual information, the analysis unit can provide visual materials. If the user prefers auditory information, the analysis unit can also provide audio materials. Furthermore, if the user prefers practical learning, the analysis unit can also provide practical tasks. This makes it possible to provide appropriate learning materials according to the user's learning style. Some or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the content of the user's conversation into an AI model and cause the AI ​​model to execute the estimation of the learning style.

[0098] The conversation unit can estimate the user's emotions and personalize the content of the conversation based on the estimated emotions. For example, if the user is stressed, the conversation unit can provide relaxing content. Also, if the user is excited, the conversation unit can provide content that shares the excitement. Furthermore, if the user is sad, the conversation unit can provide comforting content. This makes it possible to provide personalized conversation content according to the user's emotions. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the conversation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the conversation unit can input the user's emotion data to the generation AI and cause the generation AI to execute emotion estimation.

[0099] The process flow of the second embodiment will be briefly described below.

[0100] Step 1: The recording unit records the content of the conversation between the user and the generated AI. The recording unit saves the text-based conversation in a log file, records the voice conversation and saves it as an audio file, and records the video conversation and saves it as a video file. For example, the text-based conversation is recorded in real time and saved in a log file. The voice conversation is recorded using a microphone and saved as an audio file. The video conversation is recorded using a camera and saved as a video file. Step 2: The analysis unit analyzes the conversation content recorded by the recording unit. The analysis is performed using natural language processing technology, and includes extracting keywords from the conversation content, sentiment analysis, and extracting important parts. For example, the analysis unit uses natural language processing technology to extract keywords from the conversation content and identify the topic of the conversation. It uses sentiment analysis technology to estimate the sentiment of the conversation content and calculate a sentiment score. It uses keyword extraction technology to extract important parts of the conversation content and generate a summary. Step 3: The conversation unit conducts a conversation based on the results of the analysis by the analysis unit. The conversation is conducted using a dialogue-based AI system, and can conduct the conversation on behalf of the user. For example, the conversation unit conducts a conversation on behalf of the user in a specific scene based on the analysis results, reproducing the user's conversation patterns and phrases. The processing in the conversation unit may be performed using AI or may be performed without using AI. For example, the conversation unit conducts a conversation using an AI model that inputs the results of the analysis by the analysis unit and outputs the conversation.

[0101] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0102] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0103] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0104] Each of the multiple elements including the recording unit, the analysis unit, and the conversation unit described above is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the recording unit is realized by saving the conversation content in the storage 50 of the smart device 14. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the recorded conversation content. The conversation unit is realized, for example, by the control unit 46A of the smart device 14, and conducts a conversation based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above example, and various changes are possible.

[0105] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0106] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0107] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0108] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0109] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0110] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0111] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0112] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0113] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0114] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0115] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0116] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0117] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0118] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0119] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0120] Each of the multiple elements including the above-mentioned recording unit, analysis unit, and conversation unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the recording unit is realized by saving the conversation content in the storage 50 of the smart glasses 214. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the recorded conversation content. The conversation unit is realized, for example, by the control unit 46A of the smart glasses 214, and conducts a conversation based on the analysis result. The correspondence relationship between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0121] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0122] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0124] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0125] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0126] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0128] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0129] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0130] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0131] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0132] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0134] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0135] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0136] Each of the multiple elements including the above-mentioned recording unit, analysis unit, and conversation unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the recording unit is realized by saving the conversation content in the storage 50 of the headset type terminal 314. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the recorded conversation content. The conversation unit is realized, for example, by the control unit 46A of the headset type terminal 314, and conducts a conversation based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0137] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0138] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0140] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0141] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0142] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0144] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0145] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0146] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0148] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0149] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0150] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0151] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0152] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0153] Each of the multiple elements including the above-mentioned recording unit, analysis unit, and conversation unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the recording unit is realized by saving the conversation content in the storage 50 of the robot 414. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the recorded conversation content. The conversation unit is realized, for example, by the control unit 46A of the robot 414, and conducts a conversation based on the analysis result. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0154] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0155] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0156] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0157] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0158] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0159] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0160] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0161] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0162] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0163] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0164] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0165] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0166] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0167] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0168] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0169] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0170] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0171] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0172] (Appendix 1) A recording unit for recording the contents of the conversation; an analysis unit that analyzes the conversation content recorded by the recording unit; a conversation unit that conducts a conversation based on the result of the analysis by the analysis unit; Equipped A system characterized by: (Appendix 2) The recording unit is Recording conversations between users and conversational AI systems 2. The system of claim 1, (Appendix 3) The analysis unit is Analyzes recorded conversations and learns user conversation patterns and expressions 2. The system of claim 1, (Appendix 4) The conversation unit includes: Based on the analysis results, the system will converse on behalf of the user in specific situations. 2. The system of claim 1, (Appendix 5) The conversation unit includes: Reply on your behalf when your child talks to you in messaging apps 2. The system of claim 1, (Appendix 6) The conversation unit includes: Conversing with the elderly instead of caregivers at care sites 2. The system of claim 1, (Appendix 7) The conversation unit includes: Conversing with customers on behalf of hairdressers in customer service roles 2. The system of claim 1, (Appendix 8) The recording unit is The user's emotions are estimated, and the timing of recording the conversation content is adjusted based on the estimated user's emotions. 2. The system of claim 1, (Appendix 9) The recording unit is Analyze the user's past conversation history and select the appropriate recording method 2. The system of claim 1, (Appendix 10) The recording unit is Filter conversations based on the user's current situation and interests when recording them 2. The system of claim 1, (Appendix 11) The recording unit is Estimate the user's emotions and prioritize the conversation contents to be recorded based on the estimated user's emotions. 2. The system of claim 1, (Appendix 12) The recording unit is When recording conversation content, the geographic location of the user is taken into consideration to prioritize recording of relevant conversation content. 2. The system of claim 1, (Appendix 13) The recording unit is When recording conversations, analyze users' social media activity and record related conversations 2. The system of claim 1, (Appendix 14) The analysis unit is Infer user emotions and adjust the presentation of analytics based on the inferred user emotions 2. The system of claim 1, (Appendix 15) The analysis unit is During analysis, adjust the level of analysis detail based on the importance of the conversation 2. The system of claim 1, (Appendix 16) The analysis unit is During analysis, different analysis algorithms are applied depending on the category of the conversation. 2. The system of claim 1, (Appendix 17) The analysis unit is Infer user sentiment and adjust the length of the analysis based on the inferred user sentiment 2. The system of claim 1, (Appendix 18) The analysis unit is During analysis, prioritize analysis based on when the conversation occurred 2. The system of claim 1, (Appendix 19) The analysis unit is During analysis, adjust the order of analysis based on conversation relevance 2. The system of claim 1, (Appendix 20) The conversation unit includes: Estimate the user's emotions and adjust the way the conversation is expressed based on the estimated user's emotions 2. The system of claim 1, (Appendix 21) The conversation unit includes: During a conversation, the system selects an appropriate conversation method by referring to the user's past conversation patterns. 2. The system of claim 1, (Appendix 22) The conversation unit includes: During a conversation, customize the content of the conversation based on the user's current situation 2. The system of claim 1, (Appendix 23) The conversation unit includes: Estimate user emotions and prioritize conversations based on the estimated user emotions 2. The system of claim 1, (Appendix 24) The conversation unit includes: During a conversation, the appropriate conversation method is selected based on the user's geographic location. 2. The system of claim 1, (Appendix 25) The conversation unit includes: During conversations, the app analyzes users' social media activity to suggest conversation topics 2. The system of claim 1, [Explanation of symbols]

[0173] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A recording unit for recording the contents of the conversation; an analysis unit that analyzes the conversation content recorded by the recording unit; a conversation unit that conducts a conversation based on the result of the analysis by the analysis unit; Equipped A system characterized by:

2. The recording unit is Recording the content of conversations between users and conversational AI systems 2. The system of claim 1 .

3. The analysis unit is Analyzes recorded conversations and learns user conversation patterns and expressions 2. The system of claim 1 .

4. The conversation unit includes: Based on the analysis results, the system will converse on behalf of the user in specific situations.

2. The system of claim 1 .

5. The conversation unit includes: Reply on your behalf when your child talks to you in messaging apps 2. The system of claim 1 .

6. The conversation unit includes: Conversing with the elderly instead of caregivers at care sites 2. The system of claim 1 .

7. The conversation unit includes: Conversing with customers on behalf of hairdressers in customer service roles 2. The system of claim 1 .

8. The recording unit is The user's emotions are estimated, and the timing of recording the conversation content is adjusted based on the estimated user's emotions.

2. The system of claim 1 .

Images