system

Abstract

The system according to the embodiment aims to dig deep into the worries and complaints of users and identify the essential issues. [Solution] A system according to an embodiment includes a receiving unit, an analysis unit, a providing unit, and an identifying unit. The receiving unit receives user input. The analysis unit analyzes the input received by the receiving unit and generates probing questions. The providing unit provides the probing questions generated by the analysis unit. The identifying unit accumulates the user's worries and complaints based on the questions provided by the providing unit and identifies issues based on specific technical content.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies do not efficiently dig deep into users' concerns and complaints and do not adequately identify the underlying issues, so there is room for improvement.

[0005] The system according to the embodiment aims to dig deep into the worries and complaints of users and identify the essential issues. [Means for solving the problem]

[0006] The system according to the embodiment includes a receiving unit, an analyzing unit, a providing unit, and an identifying unit. The receiving unit receives user input. The analyzing unit analyzes the input received by the receiving unit and generates probing questions. The providing unit provides the probing questions generated by the analyzing unit. The identifying unit accumulates the user's worries and complaints based on the questions provided by the providing unit and identifies the issues based on specific technical content. [Effects of the Invention]

[0007] The system according to the embodiment can dig deep into the worries and complaints of users and identify the essential issues. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A system for listening to worries and complaints according to an embodiment of the present invention listens to a user's worries and complaints via voice calls or chats and extracts the essence of those worries and complaints. This system begins with the user inputting their worries or complaints via voice calls or chats. The system then replies with probing questions based on the input. By repeating this process, the essence of the user's worries and complaints is extracted and accumulated. For example, if a user inputs something like, "I'm busy at work and I'm stressed," this input is sent to the system. The system then replies with probing questions based on the user's input. For example, a question like, "What specific tasks cause you stress?" This question helps the user explain their worries in more detail. By repeating this process, the essence of the user's worries and complaints is extracted and accumulated. For example, if a user mentions a specific problem, such as, "I'm having trouble communicating with my boss," the system can grasp the details of that problem. While the service itself simply listens, the ultimate goal is to identify valuable issues by solving them from the accumulated issues and develop new services that solve those issues. Examples of such services include training programs for resolving communication issues with bosses and stress management applications. In this way, by digging deep into users' worries and complaints and extracting their essence, it becomes possible to find high-value issues and develop new services. This makes it possible for the worries and complaints listening system to efficiently dig deep into users' worries and complaints and identify high-value issues.

[0029] A system for listening to worries and complaints according to an embodiment includes a reception unit, an analysis unit, a provision unit, and an identification unit. The reception unit receives user input. The user input includes, but is not limited to, worries and complaints shared via voice calls or chats, for example. The reception unit receives the user's worries via voice calls, for example. The reception unit can also receive the user's complaints via chats. Furthermore, the reception unit can receive the user's input in real time. For example, the reception unit converts the content of the voice call into text in real time and transmits it to the analysis unit. The analysis unit analyzes the user's input and generates in-depth questions. The analysis unit analyzes the user's input using, for example, AI. The AI ​​analyzes the user's input using techniques such as machine learning and deep learning and generates in-depth questions. For example, the analysis unit analyzes the user's input using natural language processing techniques and generates appropriate in-depth questions. The analysis unit can also automatically collect related information based on the user's input and generate in-depth questions. The providing unit provides the probing questions generated by the analysis unit to the user. The providing unit provides the probing questions to the user in text format, for example. The providing unit can also provide the probing questions to the user in audio format. Furthermore, the providing unit can provide the probing questions at an appropriate timing in response to the user's input. For example, the providing unit provides the probing questions immediately after the user completes input. The identifying unit accumulates the user's worries and complaints based on the questions provided by the providing unit and identifies issues based on specific technical content. The identifying unit, for example, uses AI to find high-value issues from the accumulated data. The AI ​​analyzes the accumulated data using techniques such as machine learning and deep learning to identify high-value issues. For example, the identifying unit analyzes the accumulated data using clustering technology to identify highly relevant issues. The identifying unit can also identify issues with high business or technical value based on the accumulated data. As a result, the worries and complaints listening system according to the embodiment can efficiently dig deep into the user's worries and complaints and identify high-value issues.

[0030] The analysis unit can analyze the user's input using AI and generate in-depth questions. The analysis unit analyzes the user's input using, for example, machine learning technology. For example, the analysis unit collects the user's input as text data and inputs it into a machine learning model. The machine learning model analyzes the user's input and generates appropriate in-depth questions. The analysis unit can also analyze the user's input using deep learning technology. For example, the analysis unit analyzes the user's input using natural language processing technology and generates in-depth questions. Natural language processing technology is a technology for grammatically analyzing the user's input and understanding its meaning. Furthermore, the analysis unit can automatically collect related information based on the user's input and generate in-depth questions. For example, the analysis unit collects news articles and academic papers related to the user's input and generates in-depth questions based on them. In this way, the use of AI improves the accuracy of generating in-depth questions. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input user input to the generation AI, which can then generate probing questions.

[0031] The identification unit can use AI to find high-value issues from the accumulated data. The identification unit, for example, analyzes the accumulated data using machine learning technology. For example, the identification unit analyzes the accumulated data using clustering technology to identify highly related issues. Clustering technology is a technology for grouping data and consolidating data with common characteristics. The identification unit can also analyze the accumulated data using deep learning technology. For example, the identification unit inputs the accumulated data into a deep learning model to identify high-value issues. A deep learning model is a technology for learning large amounts of data and recognizing complex patterns. Furthermore, the identification unit can identify issues with high business or technical value based on the accumulated data. For example, the identification unit analyzes the accumulated data to identify issues with high business value. Issues with high business value are issues that contribute to corporate profits and growth. The identification unit can also identify issues with high technical value. Issues with high technical value are issues that contribute to technological advancement and innovation. As a result, the use of AI improves the accuracy of identifying high-value issues. Some or all of the above-described processing in the identification unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the identification unit may input accumulated data into the generation AI, which may then identify high-value issues.

[0032] The reception unit can accept user input through voice calls or chat. The reception unit, for example, accepts user concerns through voice calls. For example, the reception unit realizes voice calls using VoIP technology. VoIP technology is technology for making voice calls over the Internet. The reception unit can also accept user complaints through chat. For example, the reception unit accepts user input using text chat technology. Text chat technology is technology for sending and receiving text messages in real time. Furthermore, the reception unit can accept user input in real time. For example, the reception unit converts the content of the voice call into text in real time and transmits it to the analysis unit. This improves user convenience by accepting user input through voice calls or chat. Some or all of the above-described processing in the reception unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the reception unit can input the content of the voice call into a generation AI, which then converts it into text.

[0033] The providing unit can provide the probing questions to the user. The providing unit can provide the probing questions to the user in text format, for example. For example, the providing unit can display the probing questions through a chat window. The providing unit can also provide the probing questions to the user in audio format. For example, the providing unit can provide the probing questions by audio using speech synthesis technology. Speech synthesis technology is a technology for converting text data into audio. Furthermore, the providing unit can provide the probing questions at an appropriate timing in response to the user's input. For example, the providing unit can provide the probing questions immediately after the user completes input. In this way, by providing the probing questions to the user, the essence of the user's worries and complaints can be elicited. Some or all of the above-described processing in the providing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the providing unit can generate probing questions using a generation AI and provide them to the user.

[0034] The reception unit can analyze the user's past input history and select the optimal reception method. The reception unit, for example, stores the user's past input history in a database and analyzes it. For example, the reception unit preferentially suggests input methods (voice, text, etc.) that the user has frequently used in the past. The reception unit can also select the optimal reception method for a specific time period from the user's past input history. For example, the reception unit automatically displays an interface that the user has preferred in the past. In this way, the optimal reception method can be provided by analyzing the user's past input history. Some or all of the above-mentioned processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's past input history into a generation AI, which then selects the optimal reception method.

[0035] The reception unit can perform filtering based on the user's current situation and areas of interest when receiving input. The reception unit, for example, receives the user's current situation as input and performs filtering based on the input. For example, when the user inputs their current situation, the reception unit prioritizes receiving questions related to the situation. The reception unit can also prioritize receiving input of related topics based on the user's areas of interest. For example, the reception unit performs appropriate filtering and organizes the input content according to the user's current situation. This makes it possible to receive more appropriate input by filtering based on the user's current situation and areas of interest. Some or all of the above-described processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's current situation and areas of interest to the generation AI, and the generation AI can perform filtering.

[0036] When receiving input, the reception unit can prioritize receiving highly relevant input by taking into account the user's geographical location information. The reception unit, for example, acquires the user's geographical location information using GPS data or an IP address. For example, if the user is in a specific area, the reception unit can prioritize receiving input related to that area. The reception unit can also prioritize receiving input on related topics based on the user's current location. For example, the reception unit performs appropriate filtering and organizes the input content by taking into account the user's geographical location information. This allows highly relevant input to be prioritized by taking into account the user's geographical location information. Some or all of the above-described processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's geographical location information to the generation AI, which can then prioritize receiving highly relevant input.

[0037] The reception unit can analyze the user's social media activity and receive related input when receiving input. The reception unit, for example, stores the user's social media activity in a database and analyzes it. For example, the reception unit can analyze the user's current interests from the user's social media activity and preferentially receive related input. The reception unit can also preferentially receive input on related topics based on the user's social media posts. For example, the reception unit can appropriately filter and organize the input content, taking the user's social media activity into consideration. This allows the user's social media activity to be analyzed and related input to be preferentially received. Some or all of the above-described processing in the reception unit can be performed, for example, using a generation AI, or can be performed without using a generation AI. For example, the reception unit can input the user's social media activity to a generation AI, which can then preferentially receive related input.

[0038] During analysis, the analysis unit can adjust the level of detail of the probing questions based on the importance of the input. The analysis unit evaluates the importance of the input, for example, using an algorithm for evaluating the importance of the input. For example, the analysis unit analyzes the content of the user's input and evaluates its importance. For input with high importance, detailed probing questions are asked. Also, for input with low importance, brief probing questions can be asked. For example, the analysis unit can ask probing questions with appropriate level of detail for input with medium importance. This enables efficient analysis by adjusting the level of detail of the probing questions based on the importance of the input. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the importance of the input and adjusts the level of detail of the probing questions.

[0039] During analysis, the analysis unit can apply different analysis algorithms depending on the category of the input. The analysis unit, for example, classifies the input by category using an algorithm for classifying input categories. For example, the analysis unit analyzes the content of the user's input and identifies its category. A stress analysis algorithm can be applied to input related to stress. A communication analysis algorithm can also be applied to input related to communication. For example, the analysis unit applies a work analysis algorithm to input related to work. This improves the accuracy of analysis by applying different analysis algorithms depending on the category of the input. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which identifies the category of the input and applies an appropriate analysis algorithm.

[0040] During analysis, the analysis unit can determine the priority of probing questions based on the time of input submission. The analysis unit evaluates the time of input submission using, for example, an algorithm for evaluating the time of input submission. For example, the analysis unit analyzes the content of a user's input and evaluates the time of its submission. Probing questions can be prioritized for recently submitted input. Probing questions can also be postponed for older input. For example, the analysis unit can ask probing questions with a moderate priority for input submitted a medium time ago. This enables efficient analysis by determining the priority of probing questions based on the time of input submission. Some or all of the above-described processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the time of input submission and determines the priority of probing questions.

[0041] During analysis, the analysis unit can adjust the order of probing questions based on the relevance of the input. The analysis unit evaluates the relevance of the input, for example, using an algorithm for evaluating the relevance of the input. For example, the analysis unit analyzes the content of the user's input and evaluates its relevance. For input with high relevance, probing questions are asked first. Also, probing questions can be asked later for input with low relevance. For example, the analysis unit asks probing questions in an appropriate order for input with medium relevance. This enables efficient analysis by adjusting the order of probing questions based on the relevance of the input. Some or all of the above-described processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the relevance of the input and adjusts the order of probing questions.

[0042] When providing a question, the providing unit can select the most appropriate question by referring to the user's past answer history. The providing unit, for example, stores the user's past answer history in a database and refers to it. For example, the providing unit provides a related question based on the content of the user's past answers. The providing unit can also provide a question related to an unresolved problem from the user's past answer history. For example, the providing unit analyzes the user's past answer history and provides the most appropriate question. In this way, the most appropriate question can be provided by referring to the user's past answer history. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's past answer history into the generation AI, which can select the most appropriate question.

[0043] When providing a question, the providing unit can customize the content of the question based on the user's current situation. The providing unit, for example, accepts the user's current situation as input and customizes the content of the question based on that. For example, when the user inputs their current situation, the providing unit provides a question related to that situation. The providing unit can also customize and provide an appropriate question based on the user's current situation. For example, the providing unit adjusts the content of the question taking into account the user's current situation. In this way, by customizing the content of the question based on the user's current situation, a more appropriate question can be provided. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's current situation to the generation AI, which can then customize the content of the question.

[0044] When providing a question, the providing unit can provide an optimal question by taking into account the user's geographical location information. The providing unit, for example, acquires the user's geographical location information using GPS data or an IP address. For example, if the user is in a specific area, the providing unit provides questions related to that area. The providing unit can also provide questions on related topics based on the user's current location. For example, the providing unit provides an appropriate question by taking into account the user's geographical location information. This makes it possible to provide highly relevant questions by taking into account the user's geographical location information. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's geographical location information into the generation AI, which can then provide highly relevant questions.

[0045] When providing a question, the providing unit can adjust the content of the question by analyzing the user's social media activity. The providing unit, for example, stores the user's social media activity in a database and analyzes it. For example, the providing unit analyzes the user's current interests from the user's social media activity and provides related questions. The providing unit can also provide questions on related topics based on the user's social media comments. For example, the providing unit provides appropriate questions by taking the user's social media activity into consideration. In this way, highly relevant questions can be provided by analyzing the user's social media activity. Some or all of the above-mentioned processing in the providing unit may be performed using, or without, a generation AI. For example, the providing unit can input the user's social media activity into the generation AI, which then provides related questions.

[0046] When identifying a problem, the identification unit can improve the accuracy of the identification by taking into account the interrelationships of the accumulated data. The identification unit, for example, evaluates the interrelationships of the accumulated data using an algorithm that evaluates the interrelationships of data. For example, the identification unit analyzes the accumulated data and evaluates the interrelationships. The identification unit analyzes the interrelationships of the accumulated data and identifies highly related problems. The identification accuracy can also be improved by taking into account the interrelationships of the data. For example, the identification unit identifies the problem with the highest value based on the interrelationships of the accumulated data. In this way, the accuracy of the identification is improved by taking into account the interrelationships of the accumulated data. Some or all of the above-mentioned processing in the identification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the identification unit can input the accumulated data into a generation AI, which evaluates the interrelationships of the data and improves the accuracy of the identification.

[0047] The identification unit can take into account the user's attribute information when identifying a task. For example, the identification unit stores the user's attribute information in a database and performs identification taking that information into consideration. For example, the identification unit improves the accuracy of identification by taking into account attribute information such as the user's age and gender. The identification unit can also identify highly relevant tasks based on attribute information such as the user's occupation and hobbies. For example, the identification unit analyzes the user's attribute information and identifies the most valuable task. This improves the accuracy of identification by taking into account the user's attribute information. Some or all of the above-described processing in the identification unit may be performed using, or without, a generation AI. For example, the identification unit can input the user's attribute information into the generation AI, and the generation AI can perform identification taking into account the attribute information.

[0048] The identification unit can take the geographic distribution of data into consideration when identifying issues. The identification unit, for example, evaluates the geographic distribution of data using an algorithm for evaluating the geographic distribution of data. For example, the identification unit analyzes accumulated data and evaluates its geographic distribution. The identification unit analyzes the geographic distribution of data and identifies issues for each region. The accuracy of identification can also be improved by taking the geographic distribution into consideration. For example, the identification unit identifies the most valuable issue based on the geographic distribution. In this way, the accuracy of identification is improved by taking the geographic distribution of data into consideration. Some or all of the above-mentioned processing in the identification unit may be performed using, or without, the generation AI. For example, the identification unit can input accumulated data into the generation AI, which evaluates the geographic distribution and improves the accuracy of identification.

[0049] When identifying a problem, the identification unit can improve the accuracy of the identification by referring to related literature. For example, the identification unit stores related literature in a database and performs identification by referring to it. For example, the identification unit refers to related literature to improve the accuracy of the identification. It is also possible to identify highly relevant problems based on related literature. For example, the identification unit analyzes related literature to identify the most valuable problem. As a result, the accuracy of the identification is improved by referring to the related literature. Some or all of the above-mentioned processing in the identification unit may be performed using, or without, a generation AI. For example, the identification unit can input related literature into the generation AI, and the generation AI can improve the accuracy of the identification by referring to the related literature.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] When analyzing a user's input content, the analysis unit can improve the accuracy of the analysis by referring to the user's past input history. For example, the analysis unit may store the user's past input content in a database and refer to it during analysis. This allows the analysis unit to compare the user's past input content with the user's current input content to detect consistency or changes. The analysis unit can also learn the user's tendencies and patterns based on the past input history and generate more appropriate probing questions. Furthermore, the analysis unit can use the past input history to suggest solutions to the user's specific problems. In this way, by utilizing the user's past input history, the analysis accuracy can be improved and more effective support can be provided.

[0052] The providing unit can automatically search for related external resources based on the user's input and provide them to the user. For example, the providing unit can analyze the user's input and collect related news articles, academic papers, expert opinions, etc. from the Internet. The providing unit can also provide links to related online communities and forums based on the user's input. Furthermore, the providing unit can suggest resources such as books and videos related to the user's input. This allows the user to obtain multifaceted information on their worries and complaints and use it as reference material for problem solving.

[0053] When analyzing the user's input content, the identification unit can improve the accuracy of the analysis by referring to the user's social media activity. For example, the identification unit collects publicly posted content from the user's social media account and refers to it during analysis. This allows the identification unit to understand the user's online activities and interests and generate more appropriate in-depth questions. The identification unit can also estimate the user's emotions and stress level based on the user's social media activity and provide support accordingly. Furthermore, the identification unit can use the user's social media activity to suggest solutions to problems the user is facing. In this way, by utilizing the user's social media activity, the analysis accuracy can be improved and more effective support can be provided.

[0054] When accepting input content from a user, the acceptance unit can adjust the timing of accepting the input taking into account the user's current health condition. For example, the acceptance unit collects the user's health data (heart rate, blood pressure, sleep patterns, etc.) and adjusts the timing of accepting the input based on the collected data. This allows the acceptance of input to be refrained from when the user is feeling stressed or unwell. The acceptance unit can also adjust the frequency and method of accepting input according to the user's health condition. For example, when the user is relaxed, the acceptance unit can request more detailed input. Furthermore, the acceptance unit can provide appropriate feedback on the input content taking into account the user's health condition. This allows the acceptance of input at more appropriate times by taking the user's health condition into account, thereby reducing the burden on the user.

[0055] The providing unit can match the user with relevant experts or counselors based on the user's input. For example, the providing unit can analyze the user's input and recommend an appropriate expert or counselor based on the user's input. The providing unit can also automatically schedule an online consultation with an expert or counselor based on the user's input. Furthermore, the providing unit can collect feedback from experts or counselors based on the user's input and provide it to the user. This allows the user to obtain professional advice on their worries and complaints and find specific means to solve their problems.

[0056] The processing flow of the first embodiment will be briefly explained below.

[0057] Step 1: The reception unit receives user input. User input includes worries and complaints expressed through voice calls and chats. The reception unit converts the contents of the voice call into text in real time and sends it to the analysis unit. Step 2: The analysis unit analyzes the user's input and generates probing questions. The analysis unit uses AI to analyze the user's input and natural language processing technology to generate appropriate probing questions. It can also automatically collect related information and generate probing questions. Step 3: The providing unit provides the probing questions generated by the analysis unit to the user. The providing unit provides the probing questions to the user in text format or audio format, and provides the probing questions at an appropriate time according to the user's input. Step 4: The identification unit accumulates user concerns and complaints based on the questions provided by the provision unit, and identifies issues based on specific technical content. The identification unit uses AI to find high-value issues from the accumulated data, and identifies highly relevant issues using clustering technology.

[0058] (Example 2) A system for listening to worries and complaints according to an embodiment of the present invention listens to a user's worries and complaints via voice calls or chats and extracts the essence of those worries and complaints. This system begins with the user inputting their worries or complaints via voice calls or chats. The system then replies with probing questions based on the input. By repeating this process, the essence of the user's worries and complaints is extracted and accumulated. For example, if a user inputs something like, "I'm so busy at work, I'm stressed," this input is sent to the system. The system then replies with probing questions based on the user's input. For example, a question like, "What specific tasks cause you stress?" This question helps the user explain their worries in more detail. By repeating this process, the essence of the user's worries and complaints is extracted and accumulated. For example, if a user cites a specific problem, such as, "I'm having trouble communicating with my boss," the system can grasp the details of that problem. While the service itself simply listens, the ultimate goal is to identify valuable issues by solving them from the accumulated issues and develop new services that solve those issues. Examples of such services include training programs for resolving communication issues with bosses and stress management applications. In this way, by digging deep into users' worries and complaints and extracting their essence, it becomes possible to find high-value issues and develop new services. This makes it possible for the worries and complaints listening system to efficiently dig deep into users' worries and complaints and identify high-value issues.

[0059] A system for listening to worries and complaints according to an embodiment includes a reception unit, an analysis unit, a provision unit, and an identification unit. The reception unit receives user input. The user input includes, but is not limited to, worries and complaints shared via voice calls or chats, for example. The reception unit receives the user's worries via voice calls, for example. The reception unit can also receive the user's complaints via chats. Furthermore, the reception unit can receive the user's input in real time. For example, the reception unit converts the content of the voice call into text in real time and transmits it to the analysis unit. The analysis unit analyzes the user's input and generates in-depth questions. The analysis unit analyzes the user's input using, for example, AI. The AI ​​analyzes the user's input using techniques such as machine learning and deep learning and generates in-depth questions. For example, the analysis unit analyzes the user's input using natural language processing techniques and generates appropriate in-depth questions. The analysis unit can also automatically collect related information based on the user's input and generate in-depth questions. The providing unit provides the probing questions generated by the analysis unit to the user. The providing unit provides the probing questions to the user in text format, for example. The providing unit can also provide the probing questions to the user in audio format. Furthermore, the providing unit can provide the probing questions at an appropriate timing in response to the user's input. For example, the providing unit provides the probing questions immediately after the user completes input. The identifying unit accumulates the user's worries and complaints based on the questions provided by the providing unit and identifies issues based on specific technical content. The identifying unit, for example, uses AI to find high-value issues from the accumulated data. The AI ​​analyzes the accumulated data using techniques such as machine learning and deep learning to identify high-value issues. For example, the identifying unit analyzes the accumulated data using clustering technology to identify highly relevant issues. The identifying unit can also identify issues with high business or technical value based on the accumulated data. As a result, the worries and complaints listening system according to the embodiment can efficiently dig deep into the user's worries and complaints and identify high-value issues.

[0060] The analysis unit can analyze the user's input using AI and generate in-depth questions. The analysis unit analyzes the user's input using, for example, machine learning technology. For example, the analysis unit collects the user's input as text data and inputs it into a machine learning model. The machine learning model analyzes the user's input and generates appropriate in-depth questions. The analysis unit can also analyze the user's input using deep learning technology. For example, the analysis unit analyzes the user's input using natural language processing technology and generates in-depth questions. Natural language processing technology is a technology for grammatically analyzing the user's input and understanding its meaning. Furthermore, the analysis unit can automatically collect related information based on the user's input and generate in-depth questions. For example, the analysis unit collects news articles and academic papers related to the user's input and generates in-depth questions based on them. In this way, the use of AI improves the accuracy of generating in-depth questions. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input user input to the generation AI, which can then generate probing questions.

[0061] The identification unit can use AI to find high-value issues from the accumulated data. The identification unit, for example, analyzes the accumulated data using machine learning technology. For example, the identification unit analyzes the accumulated data using clustering technology to identify highly related issues. Clustering technology is a technology for grouping data and consolidating data with common characteristics. The identification unit can also analyze the accumulated data using deep learning technology. For example, the identification unit inputs the accumulated data into a deep learning model to identify high-value issues. A deep learning model is a technology for learning large amounts of data and recognizing complex patterns. Furthermore, the identification unit can identify issues with high business or technical value based on the accumulated data. For example, the identification unit analyzes the accumulated data to identify issues with high business value. Issues with high business value are issues that contribute to corporate profits and growth. The identification unit can also identify issues with high technical value. Issues with high technical value are issues that contribute to technological advancement and innovation. As a result, the use of AI improves the accuracy of identifying high-value issues. Some or all of the above-described processing in the identification unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the identification unit may input accumulated data into the generation AI, which may then identify high-value issues.

[0062] The reception unit can accept user input through voice calls or chat. The reception unit, for example, accepts user concerns through voice calls. For example, the reception unit realizes voice calls using VoIP technology. VoIP technology is technology for making voice calls over the Internet. The reception unit can also accept user complaints through chat. For example, the reception unit accepts user input using text chat technology. Text chat technology is technology for sending and receiving text messages in real time. Furthermore, the reception unit can accept user input in real time. For example, the reception unit converts the content of the voice call into text in real time and transmits it to the analysis unit. This improves user convenience by accepting user input through voice calls or chat. Some or all of the above-described processing in the reception unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the reception unit can input the content of the voice call into a generation AI, which then converts it into text.

[0063] The providing unit can provide the probing questions to the user. The providing unit can provide the probing questions to the user in text format, for example. For example, the providing unit can display the probing questions through a chat window. The providing unit can also provide the probing questions to the user in audio format. For example, the providing unit can provide the probing questions by audio using speech synthesis technology. Speech synthesis technology is a technology for converting text data into audio. Furthermore, the providing unit can provide the probing questions at an appropriate timing in response to the user's input. For example, the providing unit can provide the probing questions immediately after the user completes input. In this way, by providing the probing questions to the user, the essence of the user's worries and complaints can be elicited. Some or all of the above-described processing in the providing unit can be performed using, for example, a generation AI, or can be performed without using a generation AI. For example, the providing unit can generate probing questions using a generation AI and provide them to the user.

[0064] The reception unit can estimate the user's emotion and adjust the timing of input reception based on the estimated user emotion. The reception unit estimates the user's emotion using, for example, an emotion estimation algorithm. For example, the reception unit captures the user's facial expression with a camera and estimates the emotion using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expression and estimating emotions. The reception unit can also record the user's voice and estimate the emotion using voice analysis technology. For example, the reception unit analyzes the tone and speed of the voice to estimate the emotion. Furthermore, the reception unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate the emotion using an emotion estimation algorithm. For example, the reception unit estimates the emotion based on fluctuations in heart rate. This allows the user's stress to be reduced by adjusting the timing of input reception according to the user's emotion. Emotion estimation is realized using, for example, an emotion estimation function using an emotion engine or generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the reception unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the reception unit inputs image data of the user captured by a camera into the generation AI, and the generation AI can estimate emotions.

[0065] The reception unit can analyze the user's past input history and select the optimal reception method. The reception unit, for example, stores the user's past input history in a database and analyzes it. For example, the reception unit preferentially suggests input methods (voice, text, etc.) that the user has frequently used in the past. The reception unit can also select the optimal reception method for a specific time period from the user's past input history. For example, the reception unit automatically displays an interface that the user has preferred in the past. In this way, the optimal reception method can be provided by analyzing the user's past input history. Some or all of the above-mentioned processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's past input history into a generation AI, which then selects the optimal reception method.

[0066] The reception unit can perform filtering based on the user's current situation and areas of interest when receiving input. The reception unit, for example, receives the user's current situation as input and performs filtering based on the input. For example, when the user inputs their current situation, the reception unit prioritizes receiving questions related to the situation. The reception unit can also prioritize receiving input of related topics based on the user's areas of interest. For example, the reception unit performs appropriate filtering and organizes the input content according to the user's current situation. This makes it possible to receive more appropriate input by filtering based on the user's current situation and areas of interest. Some or all of the above-described processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's current situation and areas of interest to the generation AI, and the generation AI can perform filtering.

[0067] The reception unit can estimate the user's emotions and determine the priority of inputs to be received based on the estimated user emotions. The reception unit, for example, estimates the user's emotions using an emotion estimation algorithm. For example, the reception unit captures the user's facial expression with a camera and estimates the emotion using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The reception unit can also record the user's voice and estimate the emotion using voice analysis technology. For example, the reception unit analyzes the tone and speed of the voice to estimate the emotion. Furthermore, the reception unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate the emotion using an emotion estimation algorithm. For example, the reception unit estimates the emotion based on heart rate fluctuations. This enables the system to prioritize inputs to be received based on the user's emotions, thereby meeting the user's needs. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the reception unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the reception unit inputs image data of the user captured by a camera into the generation AI, and the generation AI can estimate emotions.

[0068] When receiving input, the reception unit can prioritize receiving highly relevant input by taking into account the user's geographical location information. The reception unit, for example, acquires the user's geographical location information using GPS data or an IP address. For example, if the user is in a specific area, the reception unit can prioritize receiving input related to that area. The reception unit can also prioritize receiving input on related topics based on the user's current location. For example, the reception unit performs appropriate filtering and organizes the input content by taking into account the user's geographical location information. This allows highly relevant input to be prioritized by taking into account the user's geographical location information. Some or all of the above-described processing in the reception unit may be performed using, or without, a generation AI. For example, the reception unit can input the user's geographical location information to the generation AI, which can then prioritize receiving highly relevant input.

[0069] The reception unit can analyze the user's social media activity and receive related input when receiving input. The reception unit, for example, stores the user's social media activity in a database and analyzes it. For example, the reception unit can analyze the user's current interests from the user's social media activity and preferentially receive related input. The reception unit can also preferentially receive input on related topics based on the user's social media posts. For example, the reception unit can appropriately filter and organize the input content, taking the user's social media activity into consideration. This allows the user's social media activity to be analyzed and related input to be preferentially received. Some or all of the above-described processing in the reception unit can be performed, for example, using a generation AI, or can be performed without using a generation AI. For example, the reception unit can input the user's social media activity to a generation AI, which can then preferentially receive related input.

[0070] The analysis unit can estimate the user's emotions and adjust the way in which probing questions are phrased based on the estimated user emotions. The analysis unit estimates the user's emotions using, for example, an emotion estimation algorithm. For example, the analysis unit captures the user's facial expressions with a camera and estimates the emotions using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The analysis unit can also record the user's voice and estimate emotions using voice analysis technology. For example, the analysis unit analyzes the tone and speed of the voice to estimate emotions. Furthermore, the analysis unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate emotions using an emotion estimation algorithm. For example, the analysis unit estimates emotions based on fluctuations in heart rate. This allows the system to adjust the way in which probing questions are phrased based on the user's emotions, thereby providing questions that are appropriate for the user. Emotion estimation is achieved using, for example, an emotion estimation function using an emotion engine or generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the analysis unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the analysis unit inputs image data of a user taken with a camera into the generation AI, and the generation AI can estimate emotions.

[0071] During analysis, the analysis unit can adjust the level of detail of the probing questions based on the importance of the input. The analysis unit evaluates the importance of the input, for example, using an algorithm for evaluating the importance of the input. For example, the analysis unit analyzes the content of the user's input and evaluates its importance. For input with high importance, detailed probing questions are asked. Also, for input with low importance, brief probing questions can be asked. For example, the analysis unit can ask probing questions with appropriate level of detail for input with medium importance. This enables efficient analysis by adjusting the level of detail of the probing questions based on the importance of the input. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the importance of the input and adjusts the level of detail of the probing questions.

[0072] During analysis, the analysis unit can apply different analysis algorithms depending on the category of the input. The analysis unit, for example, classifies the input by category using an algorithm for classifying input categories. For example, the analysis unit analyzes the content of the user's input and identifies its category. A stress analysis algorithm can be applied to input related to stress. A communication analysis algorithm can also be applied to input related to communication. For example, the analysis unit applies a work analysis algorithm to input related to work. This improves the accuracy of analysis by applying different analysis algorithms depending on the category of the input. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which identifies the category of the input and applies an appropriate analysis algorithm.

[0073] The analysis unit can estimate the user's emotions and adjust the length of probing questions based on the estimated user emotions. The analysis unit estimates the user's emotions using, for example, an emotion estimation algorithm. For example, the analysis unit captures the user's facial expressions with a camera and estimates the emotions using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The analysis unit can also record the user's voice and estimate emotions using voice analysis technology. For example, the analysis unit analyzes the tone and speed of the voice to estimate emotions. Furthermore, the analysis unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate emotions using an emotion estimation algorithm. For example, the analysis unit estimates emotions based on fluctuations in heart rate. This allows the length of probing questions to be adjusted according to the user's emotions, thereby providing questions appropriate to the user. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the analysis unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the analysis unit inputs image data of a user taken with a camera into the generation AI, and the generation AI can estimate emotions.

[0074] During analysis, the analysis unit can determine the priority of probing questions based on the time of input submission. The analysis unit evaluates the time of input submission using, for example, an algorithm for evaluating the time of input submission. For example, the analysis unit analyzes the content of a user's input and evaluates the time of its submission. Probing questions can be prioritized for recently submitted input. Probing questions can also be postponed for older input. For example, the analysis unit can ask probing questions with a moderate priority for input submitted a medium time ago. This enables efficient analysis by determining the priority of probing questions based on the time of input submission. Some or all of the above-described processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the time of input submission and determines the priority of probing questions.

[0075] During analysis, the analysis unit can adjust the order of probing questions based on the relevance of the input. The analysis unit evaluates the relevance of the input, for example, using an algorithm for evaluating the relevance of the input. For example, the analysis unit analyzes the content of the user's input and evaluates its relevance. For input with high relevance, probing questions are asked first. Also, probing questions can be asked later for input with low relevance. For example, the analysis unit asks probing questions in an appropriate order for input with medium relevance. This enables efficient analysis by adjusting the order of probing questions based on the relevance of the input. Some or all of the above-described processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the user's input to a generation AI, which evaluates the relevance of the input and adjusts the order of probing questions.

[0076] The providing unit can estimate the user's emotions and adjust the way questions are presented based on the estimated user emotions. The providing unit estimates the user's emotions using, for example, an emotion estimation algorithm. For example, the providing unit captures the user's facial expression with a camera and estimates the emotion using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The providing unit can also record the user's voice and estimate the emotion using voice analysis technology. For example, the providing unit analyzes the tone and speed of the voice to estimate the emotion. Furthermore, the providing unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate the emotion using an emotion estimation algorithm. For example, the providing unit estimates the emotion based on heart rate fluctuations. This allows the system to adjust the way questions are presented based on the user's emotions, thereby providing appropriate questions for the user. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using the generation AI. For example, the providing unit may input image data of a user taken with a camera to the generation AI, and the generation AI may estimate emotions.

[0077] When providing a question, the providing unit can select the most appropriate question by referring to the user's past answer history. The providing unit, for example, stores the user's past answer history in a database and refers to it. For example, the providing unit provides a related question based on the content of the user's past answers. The providing unit can also provide a question related to an unresolved problem from the user's past answer history. For example, the providing unit analyzes the user's past answer history and provides the most appropriate question. In this way, the most appropriate question can be provided by referring to the user's past answer history. Some or all of the above-mentioned processing in the providing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's past answer history into the generation AI, which can select the most appropriate question.

[0078] When providing a question, the providing unit can customize the content of the question based on the user's current situation. The providing unit, for example, accepts the user's current situation as input and customizes the content of the question based on that. For example, when the user inputs their current situation, the providing unit provides a question related to that situation. The providing unit can also customize and provide an appropriate question based on the user's current situation. For example, the providing unit adjusts the content of the question taking into account the user's current situation. In this way, by customizing the content of the question based on the user's current situation, a more appropriate question can be provided. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's current situation to the generation AI, which can then customize the content of the question.

[0079] The providing unit can estimate the user's emotions and prioritize questions based on the estimated user emotions. The providing unit estimates the user's emotions using, for example, an emotion estimation algorithm. For example, the providing unit captures the user's facial expression with a camera and estimates the emotions using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The providing unit can also record the user's voice and estimate emotions using voice analysis technology. For example, the providing unit analyzes the tone and speed of the voice to estimate emotions. Furthermore, the providing unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate emotions using an emotion estimation algorithm. For example, the providing unit estimates emotions based on heart rate fluctuations. This enables the system to prioritize questions based on the user's emotions and respond to the user's needs. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI can be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using the generation AI. For example, the providing unit may input image data of a user taken with a camera to the generation AI, and the generation AI may estimate emotions.

[0080] When providing a question, the providing unit can provide an optimal question by taking into account the user's geographical location information. The providing unit, for example, acquires the user's geographical location information using GPS data or an IP address. For example, if the user is in a specific area, the providing unit provides questions related to that area. The providing unit can also provide questions on related topics based on the user's current location. For example, the providing unit provides an appropriate question by taking into account the user's geographical location information. This makes it possible to provide highly relevant questions by taking into account the user's geographical location information. Some or all of the above-described processing in the providing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the providing unit can input the user's geographical location information into the generation AI, which can then provide highly relevant questions.

[0081] When providing a question, the providing unit can adjust the content of the question by analyzing the user's social media activity. The providing unit, for example, stores the user's social media activity in a database and analyzes it. For example, the providing unit analyzes the user's current interests from the user's social media activity and provides related questions. The providing unit can also provide questions on related topics based on the user's social media comments. For example, the providing unit provides appropriate questions by taking the user's social media activity into consideration. In this way, highly relevant questions can be provided by analyzing the user's social media activity. Some or all of the above-mentioned processing in the providing unit may be performed using, or without, a generation AI. For example, the providing unit can input the user's social media activity into the generation AI, which then provides related questions.

[0082] The identification unit can estimate the user's emotions and adjust the method for identifying high-value tasks based on the estimated user emotions. The identification unit, for example, estimates the user's emotions using an emotion estimation algorithm. For example, the identification unit captures the user's facial expressions with a camera and estimates the emotions using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The identification unit can also record the user's voice and estimate the emotions using voice analysis technology. For example, the identification unit analyzes the tone and speed of the voice to estimate the emotions. Furthermore, the identification unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate the emotions using an emotion estimation algorithm. For example, the identification unit estimates emotions based on heart rate fluctuations. This allows the method for identifying high-value tasks to be adjusted according to the user's emotions, thereby identifying more appropriate tasks. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the identification unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the identification unit inputs image data of a user captured by a camera into the generation AI, and the generation AI can estimate emotions.

[0083] When identifying a problem, the identification unit can improve the accuracy of the identification by taking into account the interrelationships of the accumulated data. The identification unit, for example, evaluates the interrelationships of the accumulated data using an algorithm that evaluates the interrelationships of data. For example, the identification unit analyzes the accumulated data and evaluates the interrelationships. The identification unit analyzes the interrelationships of the accumulated data and identifies highly related problems. The identification accuracy can also be improved by taking into account the interrelationships of the data. For example, the identification unit identifies the problem with the highest value based on the interrelationships of the accumulated data. In this way, the accuracy of the identification is improved by taking into account the interrelationships of the accumulated data. Some or all of the above-mentioned processing in the identification unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the identification unit can input the accumulated data into a generation AI, which evaluates the interrelationships of the data and improves the accuracy of the identification.

[0084] The identification unit can take into account the user's attribute information when identifying a task. For example, the identification unit stores the user's attribute information in a database and performs identification taking that information into consideration. For example, the identification unit improves the accuracy of identification by taking into account attribute information such as the user's age and gender. The identification unit can also identify highly relevant tasks based on attribute information such as the user's occupation and hobbies. For example, the identification unit analyzes the user's attribute information and identifies the most valuable task. This improves the accuracy of identification by taking into account the user's attribute information. Some or all of the above-described processing in the identification unit may be performed using, or without, a generation AI. For example, the identification unit can input the user's attribute information into the generation AI, and the generation AI can perform identification taking into account the attribute information.

[0085] The identification unit can estimate the user's emotions and adjust the display method of the identified task based on the estimated user's emotions. The identification unit, for example, estimates the user's emotions using an emotion estimation algorithm. For example, the identification unit captures the user's facial expression with a camera and estimates the emotion using an emotion estimation algorithm. The emotion estimation algorithm is a technology for analyzing changes in facial expressions and estimating emotions. The identification unit can also record the user's voice and estimate the emotion using voice analysis technology. For example, the identification unit analyzes the tone and speed of the voice to estimate the emotion. Furthermore, the identification unit can collect the user's biometric data (heart rate and electrodermal activity) using a sensor and estimate the emotion using an emotion estimation algorithm. For example, the identification unit estimates the emotion based on fluctuations in heart rate. This allows the display method of the identified task to be adjusted according to the user's emotions, thereby enabling an appropriate display for the user. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-described processing in the identification unit may be performed using the generation AI, for example, or may be performed without using the generation AI. For example, the identification unit inputs image data of a user captured by a camera into the generation AI, and the generation AI can estimate emotions.

[0086] The identification unit can take the geographic distribution of data into consideration when identifying issues. The identification unit, for example, evaluates the geographic distribution of data using an algorithm for evaluating the geographic distribution of data. For example, the identification unit analyzes accumulated data and evaluates its geographic distribution. The identification unit analyzes the geographic distribution of data and identifies issues for each region. The accuracy of identification can also be improved by taking the geographic distribution into consideration. For example, the identification unit identifies the most valuable issue based on the geographic distribution. In this way, the accuracy of identification is improved by taking the geographic distribution of data into consideration. Some or all of the above-mentioned processing in the identification unit may be performed using, or without, the generation AI. For example, the identification unit can input accumulated data into the generation AI, which evaluates the geographic distribution and improves the accuracy of identification.

[0087] When identifying a problem, the identification unit can improve the accuracy of the identification by referring to related literature. For example, the identification unit stores related literature in a database and performs identification by referring to it. For example, the identification unit refers to related literature to improve the accuracy of the identification. It is also possible to identify highly relevant problems based on related literature. For example, the identification unit analyzes related literature to identify the most valuable problem. As a result, the accuracy of the identification is improved by referring to the related literature. Some or all of the above-mentioned processing in the identification unit may be performed using, or without, a generation AI. For example, the identification unit can input related literature into the generation AI, and the generation AI can improve the accuracy of the identification by referring to the related literature. === Hard Collateral 1-1 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and identification unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the reception unit is realized by the reception device 38 of the smart device 14 and receives voice calls and chats from the user. The analysis unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and analyzes the user's input and generates in-depth questions. The provision unit is realized, for example, by the output device 40 of the smart device 14 and provides the in-depth questions generated by the analysis unit to the user. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and identifies high-value issues from accumulated data. === Hard Collateral 1-2 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and identification unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the reception unit is realized by the microphone 238 of the smart glasses 214 and receives voice calls and chats from the user. The analysis unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and analyzes the user's input and generates in-depth questions. The provision unit is realized, for example, by the speaker 240 of the smart glasses 214 and provides the in-depth questions generated by the analysis unit to the user. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and identifies high-value issues from accumulated data. === Hard Collateral 1-3 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and identification unit is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the reception unit is realized by the microphone 238 of the headset-type terminal 314 and receives voice calls and chats from the user. The analysis unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and analyzes the user's input and generates in-depth questions. The provision unit is realized, for example, by the speaker 240 of the headset-type terminal 314 and provides the in-depth questions generated by the analysis unit to the user. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and identifies high-value issues from accumulated data. === Hard Collateral 1-4 === Each of the multiple elements including the above-mentioned reception unit, analysis unit, provision unit, and identification unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the reception unit is realized by the microphone 238 of the robot 414 and receives voice calls and chats from the user. The analysis unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and analyzes the user's input and generates in-depth questions. The provision unit is realized, for example, by the speaker 240 of the robot 414 and provides the in-depth questions generated by the analysis unit to the user. The identification unit is realized, for example, by the identification processing unit 290 of the data processing device 12 and identifies high-value issues from accumulated data.

[0088] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0089] When analyzing a user's input content, the analysis unit can improve the accuracy of the analysis by referring to the user's past input history. For example, the analysis unit may store the user's past input content in a database and refer to it during analysis. This allows the analysis unit to compare the user's past input content with the user's current input content to detect consistency or changes. The analysis unit can also learn the user's tendencies and patterns based on the past input history and generate more appropriate probing questions. Furthermore, the analysis unit can use the past input history to suggest solutions to the user's specific problems. In this way, by utilizing the user's past input history, the analysis accuracy can be improved and more effective support can be provided.

[0090] The providing unit can automatically search for related external resources based on the user's input and provide them to the user. For example, the providing unit can analyze the user's input and collect related news articles, academic papers, expert opinions, etc. from the Internet. The providing unit can also provide links to related online communities and forums based on the user's input. Furthermore, the providing unit can suggest resources such as books and videos related to the user's input. This allows the user to obtain multifaceted information on their worries and complaints and use it as reference material for problem solving.

[0091] When analyzing the user's input content, the identification unit can improve the accuracy of the analysis by referring to the user's social media activity. For example, the identification unit collects publicly posted content from the user's social media account and refers to it during analysis. This allows the identification unit to understand the user's online activities and interests and generate more appropriate in-depth questions. The identification unit can also estimate the user's emotions and stress level based on the user's social media activity and provide support accordingly. Furthermore, the identification unit can use the user's social media activity to suggest solutions to problems the user is facing. In this way, by utilizing the user's social media activity, the analysis accuracy can be improved and more effective support can be provided.

[0092] When accepting input content from a user, the acceptance unit can adjust the timing of accepting the input taking into account the user's current health condition. For example, the acceptance unit collects the user's health data (heart rate, blood pressure, sleep patterns, etc.) and adjusts the timing of accepting the input based on the collected data. This allows the acceptance of input to be refrained from when the user is feeling stressed or unwell. The acceptance unit can also adjust the frequency and method of accepting input according to the user's health condition. For example, when the user is relaxed, the acceptance unit can request more detailed input. Furthermore, the acceptance unit can provide appropriate feedback on the input content taking into account the user's health condition. This allows the acceptance of input at more appropriate times by taking the user's health condition into account, thereby reducing the burden on the user.

[0093] The providing unit can match the user with relevant experts or counselors based on the user's input. For example, the providing unit can analyze the user's input and recommend an appropriate expert or counselor based on the user's input. The providing unit can also automatically schedule an online consultation with an expert or counselor based on the user's input. Furthermore, the providing unit can collect feedback from experts or counselors based on the user's input and provide it to the user. This allows the user to obtain professional advice on their worries and complaints and find specific means to solve their problems.

[0094] The analysis unit can estimate the user's emotions and adjust the difficulty of probing questions based on the estimated user emotions. For example, the analysis unit can analyze the user's facial expressions and voice to estimate emotions. If the emotion is negative, the analysis unit can start with simple questions and gradually increase the difficulty. If the emotion is positive, the analysis unit can ask more detailed questions. Furthermore, the analysis unit can adjust the tone and expression of the questions according to the user's emotions. This makes it possible to ask questions that take the user's emotions into consideration, providing an environment in which the user can answer in a relaxed manner.

[0095] The providing unit can estimate the user's emotions and adjust the frequency of providing questions based on the estimated user emotions. For example, the providing unit can analyze the user's facial expressions and voice to estimate emotions. If the emotions are negative, the frequency of providing questions can be reduced to encourage the user to take a rest. If the emotions are positive, the frequency of providing questions can be increased to motivate the user. Furthermore, the providing unit can adjust the timing of providing questions according to the user's emotions. In this way, by providing questions that take the user's emotions into consideration, it is possible to reduce the user's stress and provide more effective support.

[0096] The identification unit can estimate the user's emotions and prioritize high-value issues based on the estimated user emotions. For example, the identification unit can analyze the user's facial expressions and voice to estimate emotions. If the emotions are negative, it can prioritize and identify issues that are important to the user and propose solutions. If the emotions are positive, it can identify issues based on the user's interests and concerns, encouraging further growth and development. Furthermore, the identification unit can adjust the method for identifying issues according to the user's emotions. This allows for more effective support to be provided by identifying issues that take the user's emotions into consideration.

[0097] The providing unit can estimate the user's emotions and adjust the format of the questions based on the estimated user emotions. For example, the providing unit can analyze the user's facial expressions and voice to estimate the emotions. If the emotions are negative, the providing unit can provide text-format questions, allowing the user to answer at their own pace. If the emotions are positive, the providing unit can provide audio-format questions, promoting dialogue with the user. Furthermore, the providing unit can adjust the length and complexity of the questions according to the user's emotions. This makes it possible to provide questions that take the user's emotions into consideration, thereby providing an environment in which the user can answer in a relaxed manner.

[0098] The identification unit can estimate the user's emotions and propose solutions to the identified problem based on the estimated user's emotions. For example, the identification unit can analyze the user's facial expressions and voice to estimate emotions. If the emotions are negative, the identification unit can preferentially propose solutions to reduce the user's stress. If the emotions are positive, the identification unit can propose solutions to increase the user's motivation. Furthermore, the identification unit can adjust the method of proposing solutions according to the user's emotions. This makes it possible to provide more effective support by proposing solutions that take the user's emotions into consideration.

[0099] The processing flow of the second embodiment will be briefly explained below.

[0100] Step 1: The reception unit receives user input. User input includes worries and complaints expressed through voice calls and chats. The reception unit converts the contents of the voice call into text in real time and sends it to the analysis unit. Step 2: The analysis unit analyzes the user's input and generates probing questions. The analysis unit uses AI to analyze the user's input and natural language processing technology to generate appropriate probing questions. It can also automatically collect related information and generate probing questions. Step 3: The providing unit provides the probing questions generated by the analysis unit to the user. The providing unit provides the probing questions to the user in text format or audio format, and provides the probing questions at an appropriate time according to the user's input. Step 4: The identification unit accumulates user concerns and complaints based on the questions provided by the provision unit, and identifies issues based on specific technical content. The identification unit uses AI to find high-value issues from the accumulated data, and identifies highly relevant issues using clustering technology.

[0101] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0102] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0103] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0104] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0105] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0106] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0107] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0108] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0109] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0110] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0111] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0112] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0113] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0114] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0115] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0116] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0117] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0118] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0119] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0120] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0121] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0122] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0124] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0125] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0126] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0128] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0129] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0130] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0131] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0132] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0134] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0135] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0136] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0137] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0138] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0140] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0141] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0144] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0145] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0146] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0148] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0149] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0150] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0151] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0152] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0153] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0154] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0155] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0156] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0157] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0158] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0159] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0160] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0161] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0162] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0163] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0164] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0165] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0166] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0167] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0168] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0169] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0170] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0171] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0172] [Explanation of symbols]

[0173] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a reception unit that receives input from a user; an analysis unit that analyzes the input received by the reception unit and generates a probing question; a providing unit that provides the probing questions generated by the analysis unit; an identification unit that accumulates the user's worries and complaints based on the questions provided by the provision unit and identifies the problem based on specific technical content. A system characterized by:

2. The analysis unit Use AI to analyze user input and generate probing questions 2. The system of claim 1.

3. The identification unit Using AI to identify high-value issues from accumulated data 2. The system of claim 1.

4. The reception unit Accept user input through voice calls or chat 2. The system of claim 1.

5. The providing unit Probe questions for users 2. The system of claim 1.

6. The reception unit Estimate the user's emotions and adjust the timing of input acceptance based on the estimated user emotions.

2. The system of claim 1.

7. The reception unit Analyze the user's past input history and select the appropriate reception method 2. The system of claim 1.

8. The reception unit As input is received, it filters based on the user's current situation and interests.

2. The system of claim 1.

9. The reception unit Estimate the user's emotions and prioritize inputs based on the estimated user emotions.

2. The system of claim 1.

Images