system

Abstract

We provide the system. [Solution] Means for collecting past communication content, A means of using a generative model to analyze the content of the communication and understand the context, A means to anticipate misunderstandings among users and to present appropriate intent, A means of processing user feedback and improving the generative model, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern complex and multi-layered communications, misunderstandings and lack of understanding frequently occur, which is a problem that affects cooperation and business efficiency. Also, exchanges without the context of the past further increase these problems. Means to prevent such occurrences of misunderstandings and lack of understanding are required.

Means for Solving the Problems

[0005] To address this challenge, the present invention provides a system that predicts misunderstandings between users and presents appropriate intent by using a generative model that collects and analyzes past communication content to understand the context. Furthermore, it enables improved accuracy by processing user feedback and continuously refining the generative model. In collecting communication content, the system includes means for anonymizing data to protect user privacy. In addition, the analyzed contextual information is presented through an intuitive user interface to support smoother communication.

[0006] "Past communication content" refers to the content of messages and conversations previously exchanged between users, and is a set of data used as the basis for analysis.

[0007] A "generative model" is an advanced artificial intelligence algorithm used to analyze large amounts of data and understand and predict context and intent.

[0008] "Understanding context" means using information gleaned from past conversations and messages to accurately grasp the intentions and meanings behind statements and actions.

[0009] "Predicting misunderstandings" is the ability to identify potential misunderstandings or misinterpretations that may arise during a conversation, and to pinpoint their causes.

[0010] "Presenting the correct intent" means showing users the underlying intent and correct interpretation of the original message in order to resolve anticipated misunderstandings.

[0011] "Processing feedback" is the process of evaluating information about user experiences and reactions and using that information to improve the system in the future.

[0012] "Protecting privacy" means taking measures to anonymize or encrypt data in order to prevent personal information from being leaked to third parties.

[0013] An "intuitive user interface" is a system interface or catalyst designed to be easily understood and operated by the user. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]A sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention aims to realize a system for understanding the context of communication and preventing misunderstandings. The specific configuration and operation of this system are described below.

[0036] This system consists of three main elements: the user's terminal, a central server, and an AI-generated model. To begin using the system, the user must first log in by operating the terminal. The terminal extracts past communication history from its local storage and collects data to the extent agreed upon by the user. The collected data is anonymized for privacy protection and then sent to the server.

[0037] The server analyzes the received data and uses an AI generative model to analyze the context of the conversation. This process identifies points where misunderstandings are likely to occur from past conversation history. Based on the analysis results, the server predicts misunderstandings between users and provides interpretations to understand the correct intentions. These analysis results are sent back to the terminal and presented to the user in an easy-to-understand manner.

[0038] As a concrete example, suppose that in a project meeting held within a company, users A and B have different interpretations. The application on the terminal sends data to the server from past discussion records. Based on this information, the server predicts misunderstandings and uses an AI generative model to analyze and present the accurate intent. As a result, users A and B can proceed with the project based on a shared understanding.

[0039] This system allows users to prevent misunderstandings that may arise during communication and enables smooth transmission of intentions. The system also features a feedback processing function, which continuously improves the AI ​​generation model based on user feedback, providing more accurate and efficient analysis.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user launches the application and logs into the system. With consent, the device collects past communication history data from local storage.

[0043] Step 2:

[0044] The communication data collected by the device is anonymized for privacy protection and sent to the server using a secure protocol.

[0045] Step 3:

[0046] The server receives data, saves it to the database, and prepares the dataset. The data is organized and formatted into an easily accessible format.

[0047] Step 4:

[0048] The server uses an AI generative model to analyze data and understand the context of conversations between users. The analysis identifies the conversation's topic, sentiment, and intent.

[0049] Step 5:

[0050] The server analyzes the results to predict potential misunderstandings between users. It identifies the source of these misunderstandings and detects potential discrepancies in intent.

[0051] Step 6:

[0052] To prevent misunderstandings, the server reinterprets the intended meaning and sends it to the terminal. A selection of appropriate intentions and interpretations is then generated.

[0053] Step 7:

[0054] The analysis results received by the terminal are displayed to the user through the user interface in a format that is intuitively understandable to the user.

[0055] Step 8:

[0056] Based on the information provided by the user, the system adjusts communication to facilitate more effective dialogue.

[0057] Step 9:

[0058] Users provide feedback on their usage experience and the AI's intent analysis. This feedback is collected by the device and sent to the server.

[0059] Step 10:

[0060] The server analyzes the feedback it receives and uses it to improve the AI-generated model. Based on the feedback, the model is trained and its parameters are adjusted.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In today's information and communication environment, communication between users takes place through a variety of means, but frequent misunderstandings hinder smooth communication. Furthermore, advanced data analysis and privacy protection are required to prevent these misunderstandings and accurately convey users' intentions. This invention aims to address these challenges and improve communication between users.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for collecting past communication information, means for anonymizing the communication information and protecting privacy, and means for analyzing the anonymized data and understanding contextual information using a generative AI model. This prevents misunderstandings between users and enables accurate communication of intentions.

[0066] "Past communication information" refers to digital records of previous communications such as messages and phone calls made by the user.

[0067] "Anonymization" is the process of removing or transforming personally identifiable information so that the data does not infringe on privacy.

[0068] A "secure protocol" is a set of communication rules or standards for ensuring the secure transmission of data over the internet or a network.

[0069] A "generative AI model" refers to an artificial intelligence algorithm that takes data as input and performs analysis or generation based on a specific purpose.

[0070] "Contextual information" refers to all relevant information that indicates the background, situation, and intentions behind statements and actions in communication.

[0071] A "user interface" is a collection of screens and display elements that a user uses when interacting with a system.

[0072] This system is implemented as a network-based platform using user terminals, a central server, and a generative AI model. Users first log in to the system using their own terminals. After logging in, the terminals collect past communication information from local data storage to the extent agreed upon by the user. Specifically, they filter and retrieve communication history for a specified period from email clients and messaging applications on the terminals.

[0073] The device anonymizes the collected data. The anonymization algorithm is designed to protect user privacy by removing or transforming personally identifiable information. During this process, the data is transmitted to the server using a strong, secure protocol.

[0074] The server stores the received data in a pre-configured database and applies a generative AI model to analyze the contextual information. The generative AI model utilizes natural language processing techniques to understand the intentions and emotions behind the conversation and identify areas where misunderstandings may occur between users. This analysis makes it possible to predict mismatches in intent between users.

[0075] The analysis results are sent back from the server to the user's terminal and presented to the user through the terminal's user interface. At this time, the analyzed information is provided in an easy-to-understand manner through graphical displays and notifications, allowing the user to receive information based on the analysis.

[0076] As a concrete example, consider a scenario where past communication data from a project team within a company is analyzed. The terminal application sends a prompt message to the AI ​​model stating, "Analyze the intent behind the meeting agenda," and the server visualizes the analysis results for the user. This supports the efficient progress of the project.

[0077] An example of a prompt message might be, "Use an AI model to analyze the likelihood of misunderstandings between User A and User B based on their past discussion history."

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user logs into the system using their own device. The device receives user authentication information (e.g., user ID and password) as input. Upon successful login, a user authentication token is generated as output, and session information is established.

[0081] Step 2:

[0082] The device collects past communication information related to logged-in users. The input is communication history obtained from the device's email client and messaging apps. For data processing, messages and emails from a specified period are filtered and collected. The output is a list of the relevant historical data.

[0083] Step 3:

[0084] The terminal anonymizes the collected communication information. The communication history data obtained in step 2 is used as input. An algorithm is applied to mask the data by removing personally identifiable information. Anonymized history data is generated as output.

[0085] Step 4:

[0086] The terminal sends anonymized data to the server using a secure protocol. The input consists of anonymized data and communication protocol information. The output is a status indicating successful transmission.

[0087] Step 5:

[0088] The server analyzes the received data and uses a generative AI model to understand contextual information. Anonymized data is supplied to the server as input. The data processing involves the AI ​​model evaluating the data and analyzing the intent and emotions of the conversation. The output is the result of the contextual analysis.

[0089] Step 6:

[0090] Based on the analysis results, the server predicts potential misunderstandings between users and presents appropriate intent. The analysis results generated in step 5 are used as input. Data processing identifies points where misunderstandings may occur. The output is the interpretation of intent.

[0091] Step 7:

[0092] The server returns the interpretation result of the generated intent to the user's terminal via the user interface. The inputs used are the interpretation result of the intent and the user ID. The output is the analysis result displayed on the user's terminal.

[0093] Step 8:

[0094] The user reviews the presented analysis results and provides feedback. The user's evaluation and comments are entered on the device as input. The feedback data is sent to the server as output.

[0095] Step 9:

[0096] The server processes user feedback and improves the generated AI model. Feedback data is used as input. The AI ​​model's learning process is updated as data computation, and adjustments are made to improve the model's accuracy. The improved AI model is obtained as output.

[0097] (Application Example 1)

[0098] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0099] In modern commercial facilities and service industries, communication among employees and with customers is becoming increasingly complex, leading to challenges such as misunderstandings and miscommunication. Such misunderstandings can result in decreased operational efficiency and customer satisfaction. Furthermore, protecting privacy is crucial when using communication history.

[0100] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0101] In this invention, the server includes means for collecting past communication content and dialogue history, means for analyzing the communication content to understand the context and using a generative model to predict responses in real time, and means for predicting misunderstandings between users or between users and customers, presenting appropriate intentions, and supporting the optimal response method. This prevents misunderstandings in communication and enables smoother and more effective information transmission.

[0102] "Past communication content and dialogue history" refers to the record of all previous messages and conversations that took place between users or between users and customers.

[0103] A "generative model" refers to an algorithm that uses artificial intelligence technology to analyze communication content, understand the context, and predict responses.

[0104] "Methods for predicting misunderstandings between users and customers" refers to technologies that detect potential misunderstandings and miscommunications that may occur during communication, based on past data and contextual analysis.

[0105] "A means of presenting appropriate intent and supporting the optimal response method" refers to a system that clarifies intent based on analysis results and guides users in selecting the most effective response.

[0106] "Information equipment" refers to all electronic devices used by users, such as smartphones, tablets, and computers.

[0107] "Means of anonymizing data to protect privacy and applying stronger encryption techniques" refers to the process of processing data so that individuals cannot be identified and then using encryption technology to further enhance data security.

[0108] The system requires user communication devices, a central information processing unit, and software including an artificial intelligence generative model. The user's communication devices, such as smartphones or tablets, are responsible for collecting past communication content and conversation history. This data is collected with the user's consent, anonymized for privacy protection, and further protected by strong encryption methods such as 256-bit AES.

[0109] When data is transmitted to a central information processing unit, a generative AI model activates, analyzing the communication content to understand the context and predicting responses in real time. This model uses AI libraries such as TENSORFLOW® and employs natural language processing techniques to predict misunderstandings. The analysis results are presented to the user's communication device via a user interface, assisting in facilitating communication between users or between users and customers.

[0110] As a concrete example, when a new staff member in a store answers a customer's question, this system provides the optimal response in real time based on past conversations. The analysis results displayed on the staff member's communication device help to present the appropriate intent, improving the accuracy and efficiency of responses.

[0111] An example of a prompt message to input into a generative AI model would be, "Predict the best response based on past conversation history. Example: Based on past responses to 'Do you have it in stock?'" By using this prompt message, the model can focus on the issue it needs to address and derive a highly accurate response.

[0112] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0113] Step 1:

[0114] The user's device begins collecting communication history. Within the scope agreed upon by the user, past conversation content and interaction history are selected, anonymized, and encrypted. The input is the user's past communication content, and the output is anonymized and encrypted data.

[0115] Step 2:

[0116] The collected data is sent to the server. The server decrypts the received encrypted data and extracts the information necessary for analysis. The input is anonymized encrypted data, and the output is communication data in an analyzable format.

[0117] Step 3:

[0118] The server processes analyzable data using a generative AI model. In this process, it analyzes the context of the communication, past patterns, and common points of misunderstanding. The input is communication data in an analyzable format, and the output is predictions of misunderstandings and suggestions for appropriate responses.

[0119] Step 4:

[0120] The analysis results are sent back to the terminal via the user interface. The terminal displays the results in a format that is easy for the user to understand. The input is a prediction of misunderstandings and a suggestion of appropriate responses, while the output is user-oriented information displayed on the terminal.

[0121] Step 5:

[0122] Users can choose actions based on the information presented and provide feedback as needed. This feedback is used to improve the generative AI model, increasing the accuracy of subsequent analyses. The input is user feedback, and the output is the improved result of analyzing the feedback and reflecting it in the generative AI model.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention is a system that supports communication between users, and through a new configuration that includes an emotion engine, it can also reflect the user's emotions in the analysis results. The system mainly consists of a user terminal, a server, and a generative model equipped with an emotion engine.

[0125] When a user starts the system, the terminal performs a login process, collects past communication history data, anonymizes it with privacy protection measures, and sends it to the server. At this time, the terminal collects the user's emotions in real time, using an emotion engine to obtain emotional data from facial expressions and linguistic indicators.

[0126] The server integrates and analyzes the received communication data and sentiment data. In this analysis, a generative model understands the context, and the sentiment engine interprets the emotions. As a result of combining the data, the server understands the user's emotional state and reinterprets appropriate intentions based on that state.

[0127] Based on the analysis results, the server predicts communication misunderstandings and has the means to dynamically present intentions according to emotions. The terminal receives the analysis data obtained from the server and presents it in a format that is easy for the user to understand. This system provides more flexible and adaptive communication support by taking the user's emotional state into consideration.

[0128] As a concrete example, consider a scenario where members of a project team in remote locations are holding an online meeting. If the presenter is nervous, the emotion engine detects this, and the server clarifies the presenter's intentions to avoid misunderstandings, helping the audience accurately understand the presenter's intent. In this way, considering emotions improves the quality of communication.

[0129] This system allows users to objectively support their emotionally charged communication, thereby reducing misunderstandings. Furthermore, the accuracy improves over time through continuous refinement of the generative model using feedback.

[0130] The following describes the processing flow.

[0131] Step 1:

[0132] The user logs into the system using their device. The device collects past communication history, anonymizes the data to protect privacy, and then sends it to the server.

[0133] Step 2:

[0134] The device acquires real-time emotional data from the user. This utilizes an emotion engine that identifies emotions through facial expression analysis and voice tone analysis.

[0135] Step 3:

[0136] The server integrates received communication history data with real-time sentiment data. This data is centrally managed and prepared for analysis.

[0137] Step 4:

[0138] The server uses generative models to analyze the context of the communication content. Simultaneously, the emotion engine analyzes emotion data to identify the user's emotional state.

[0139] Step 5:

[0140] Based on contextual information and emotional states analyzed by the server, it predicts areas where misunderstandings are likely to occur. Considering emotional data enables more accurate intent analysis.

[0141] Step 6:

[0142] The server sends the results of the intent analysis to the terminal. The intent is dynamically adjusted based on the emotional situation and presented appropriately to the user.

[0143] Step 7:

[0144] The device displays the analysis results through an intuitive user interface. This allows users to easily understand their own emotions and intentions based on the content of the conversation.

[0145] Step 8:

[0146] Based on the information provided by the user, we will implement measures to improve communication. Users will provide feedback as needed, contributing to system improvements.

[0147] Step 9:

[0148] The server collects feedback and improves the generative model and sentiment engine. Based on the feedback, the model is retrained to improve the accuracy of the analysis.

[0149] (Example 2)

[0150] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0151] In user-to-user communication, misunderstandings of emotions and unclear intentions can hinder the smooth transmission of information. Furthermore, protecting privacy is difficult, and there is a lack of mechanisms to effectively utilize feedback and continuously improve generative models. These challenges need to be addressed to achieve emotionally sensitive communication.

[0152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0153] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for detecting the emotional state of the information sender and presenting appropriate intentions based on that emotion, means for acquiring emotional data from the information sender's facial expressions and voice and analyzing that information with an emotion engine, means for presenting the analysis results to the information receiver in a visualized form, and means for processing user feedback and continuously improving the generative model. This enables the provision of flexible and intuitive communication that responds to emotions.

[0154] "Past communication information" refers to all messages and call history previously exchanged between users, and includes communication history data.

[0155] "Generative modeling" refers to the process of using machine learning models to understand the context and intent behind information, utilizing artificial intelligence technology.

[0156] "The emotional state of the information provider" refers to the mental and emotional state of the user who is disseminating the information, and this includes emotions such as joy, sadness, and anger.

[0157] An "emotion engine" refers to software or hardware technology that analyzes user data such as facial expressions and voice to determine emotions.

[0158] "Feedback" refers to opinions, reactions, or data collected from users, which are used to improve systems and generative models.

[0159] "Anonymization" refers to the process of removing or transforming personally identifiable elements from data in order to protect privacy.

[0160] "Visualizing analysis results" means presenting the results of data analysis in a visually easy-to-understand format, providing information in a way that is easy for users to comprehend.

[0161] This invention is a system that supports user communication and aims to reduce misunderstandings while taking user emotions into consideration. The system mainly consists of a server, a terminal, and a generative model equipped with an emotion engine.

[0162] The device first collects past communication information when the user logs in. This includes messages and call history. The collected information is anonymized to protect privacy and then sent to the server. Anonymization is a process that removes or transforms personally identifiable information.

[0163] The device also collects emotional data in real time. The device works in conjunction with the emotion engine to analyze the user's facial expressions and voice, and obtains emotional data from them. The emotion engine uses a specific algorithm to determine the user's emotional state from the acquired data.

[0164] The server receives communication information and sentiment data transmitted from the terminal. A generative AI model is used to analyze this data and present appropriate intentions based on the user's emotional state. This generative AI model simultaneously performs contextual understanding and sentiment interpretation, dynamically reinterpreting intentions to reduce misunderstandings.

[0165] The server then returns the analysis results to the terminal, which presents them to the user through an intuitive interface. This system can continuously improve the generated AI model using user feedback. Based on the feedback, the system evolves to improve the user experience.

[0166] As a concrete example, consider a scenario where a project team is holding an online meeting. If the presenter is nervous, the device detects the presenter's voice and facial expressions and analyzes their emotions via the server. Based on the analyzed information, the server provides the audience with information to clarify the presenter's intentions. The presenter can also use this information to give their presentation with greater confidence.

[0167] An example of a prompt might be, "Please tell me how the server detected the tension that user A showed during the online meeting and how it conveyed that to the audience." In this way, the system supports tackling realistic challenges.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] When a user starts the system, the terminal displays a login screen where the user enters their authentication information. After the authentication information is entered, the terminal sends it to the server, and the login process is executed. This initiates the user's session. The inputs are a user ID and password, and the output is a message indicating login success or failure.

[0171] Step 2:

[0172] Once login is complete, the device collects past communication information. This information includes text messages and call history. The device encrypts and transforms this data to anonymize it. The input is raw communication data, and the output is anonymized communication data. The anonymized data is sent to the server.

[0173] Step 3:

[0174] The device also activates its camera and microphone to collect user emotion data. The collected facial and audio data is analyzed using an emotion engine and output as the user's real-time emotional state. The input for this step is facial expressions and audio, and the output is the analyzed emotion data.

[0175] Step 4:

[0176] The server receives anonymized communication data and sentiment datasets sent from the terminal. The server analyzes this data using a generating AI model and sentiment engine. This analysis enables an understanding of the user's context and emotional state. Raw data is the input, and the analysis results are the output.

[0177] Step 5:

[0178] The server uses a generative AI model to reinterpret the intent behind communication based on the analyzed emotional state and generates revised versions to avoid misunderstandings. In this step, the analysis results are used as input, and the revised intent is generated as output.

[0179] Step 6:

[0180] The server sends suggested revisions to the generated intent to the terminal, which then displays them. The user can then proceed with the conversation with confidence based on this information. The terminal visualizes the received data through an appropriate user interface and presents it to the user. The input is the suggested revision data from the server, and the output is the display on the user interface.

[0181] Step 7:

[0182] User feedback is collected through the device and sent to the server. The server uses this feedback to continuously improve the generated AI model. The input to this process is user feedback, and the output is an updated, improved model.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] In modern society, especially in family and daily communication, users' emotions can be a source of misunderstanding. In situations where direct dialogue is difficult, emotional misunderstandings can reduce the efficiency and quality of communication. Furthermore, emotional misunderstandings can negatively impact relationships with family and close friends. It is necessary to address this challenge and provide means to achieve smoother and more harmonious communication.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for predicting misunderstandings between users and presenting appropriate intentions, means for processing user feedback and improving the generative model, means for using an emotion engine that analyzes the user's emotional state, and means for presenting feedback and support to the user in a parental voice based on the analysis results. This enables more accurate and harmonious communication while taking into account the user's emotional state.

[0188] "Past communication information" refers to data related to conversations and messages previously exchanged between users.

[0189] "Analysis of communication information" refers to the process of analyzing collected communication data in order to understand its content and context.

[0190] A "generative model" refers to an algorithm or program that uses artificial intelligence technology to analyze data and generate appropriate output.

[0191] "Predicting misunderstandings between users" refers to identifying potential misunderstandings and misperceptions that may arise in communication between users.

[0192] "Presenting the correct intent" means explicitly providing information so that users can understand the intended message without misunderstanding.

[0193] "Feedback processing" refers to the process of collecting responses and opinions provided by users and using them to improve systems and models.

[0194] An "emotion engine" refers to software or a device that detects and evaluates emotions from a user's facial expressions, tone of voice, and other factors.

[0195] "Providing feedback and support" means providing users with necessary information in a timely manner based on the analysis results and supporting smooth communication.

[0196] "Parental voice" refers to outputting a warm voice that gives the user a sense of security and familiarity.

[0197] The system implementing this invention is particularly envisioned as a consumer robot for use in the home. The server utilizes a generative model that understands context by collecting and analyzing the user's past communication information. Based on the results of this analysis, it predicts potential misunderstandings between users and presents appropriate intentions. The server also processes user feedback and continuously improves the generative model.

[0198] The terminal, or consumer robot, is equipped with the ability to collect the user's facial expressions and tone of voice in real time and acquire emotional data through an emotion engine. This emotional data is comprehensively evaluated in conjunction with the analysis results on the server, and feedback and support are presented to the user in a parental voice. This kind of feedback facilitates smooth communication with the user and minimizes misunderstandings.

[0199] As a concrete example of implementing this system, consider a scenario where a robot with home helper functionality detects that a child is struggling with their homework. The robot would then reduce the child's stress by suggesting to the user, i.e., the child, via voice, "Shall we take a break? Or do you need any help?"

[0200] This invention enables more accurate and harmonious communication that takes into account the user's emotional state through user-friendly feedback. The system design utilizes software such as OpenCV (image processing), TensorFlow (generative AI modeling), and NLTK (natural language processing), and is implemented on a consumer robot platform.

[0201] An example of a prompt message would be, "The user's stress level is high. Please provide some conversational tips to help them relax."

[0202] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0203] Step 1:

[0204] The device uses a camera and microphone to collect data on the user's facial expressions and voice. The input is real-time facial images and audio from the user, and the output is emotion data based on these. An emotion engine is used to process the data and analyze emotions from changes in facial expressions and voice tone. Specifically, OpenCV is used to extract facial feature points and audio data is analyzed to identify the emotional state.

[0205] Step 2:

[0206] The device transmits collected emotional data to the server while protecting privacy. The input is anonymized emotional data, and the output is a secure data transmission to the server. The data is transformed into a form that cannot be linked to a specific user and transmitted securely using protocols such as HTTPS. Specifically, the data is encrypted before being transmitted to the server using secure network protocols for medical or educational purposes.

[0207] Step 3:

[0208] The server integrates received sentiment data with past communication information and performs analysis using a generative AI model. The input is sentiment data and past communication information, and the output is the prediction of misunderstandings between users and the proposed intent. The generative AI model understands the context and performs data calculations to formulate the optimal intent for the user. Specifically, it uses TensorFlow for natural language processing and NLTK to reinforce the sentiment context.

[0209] Step 4:

[0210] The server sends the generated intent and feedback information to the terminal. The input is the communication strategy generated within the server, and the output is the information transmitted to the terminal. The processing includes practical feedback provided by a generative AI model based on a specific scenario. Specifically, the server formalizes the feedback statement, forms an appropriate prompt statement, and then sends it to the terminal.

[0211] Step 5:

[0212] The terminal presents feedback received from the server to the user both audibly and visually. The input is the generated intent sent from the server, and the output is the presentation of feedback to the user. Using speech synthesis technology, the feedback is provided in a friendly voice. Specifically, a voice synthesizer is used to convey information to the user in a gentle, questioning manner.

[0213] Through these steps, users receive feedback and support that reflects their emotions, enabling smoother communication.

[0214] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0215] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0216] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0217] [Second Embodiment]

[0218] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0219] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0220] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0221] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0222] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0223] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0224] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0225] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0226] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0227] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0228] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0229] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0230] This invention aims to realize a system for understanding the context of communication and preventing misunderstandings. The specific configuration and operation of this system are described below.

[0231] This system consists of three main elements: the user's terminal, a central server, and an AI-generated model. To begin using the system, the user must first log in by operating the terminal. The terminal extracts past communication history from its local storage and collects data to the extent agreed upon by the user. The collected data is anonymized for privacy protection and then sent to the server.

[0232] The server analyzes the received data and uses an AI generative model to analyze the context of the conversation. This process identifies points where misunderstandings are likely to occur from past conversation history. Based on the analysis results, the server predicts misunderstandings between users and provides interpretations to understand the correct intentions. These analysis results are sent back to the terminal and presented to the user in an easy-to-understand manner.

[0233] As a concrete example, suppose that in a project meeting held within a company, users A and B have different interpretations. The application on the terminal sends data to the server from past discussion records. Based on this information, the server predicts misunderstandings and uses an AI generative model to analyze and present the accurate intent. As a result, users A and B can proceed with the project based on a shared understanding.

[0234] This system allows users to prevent misunderstandings that may arise during communication and enables smooth transmission of intentions. The system also features a feedback processing function, which continuously improves the AI ​​generation model based on user feedback, providing more accurate and efficient analysis.

[0235] The following describes the processing flow.

[0236] Step 1:

[0237] The user launches the application and logs into the system. With consent, the device collects past communication history data from local storage.

[0238] Step 2:

[0239] The communication data collected by the device is anonymized for privacy protection and sent to the server using a secure protocol.

[0240] Step 3:

[0241] The server receives data, saves it to the database, and prepares the dataset. The data is organized and formatted into an easily accessible format.

[0242] Step 4:

[0243] The server uses an AI generative model to analyze data and understand the context of conversations between users. The analysis identifies the conversation's topic, emotions, and intentions.

[0244] Step 5:

[0245] The server analyzes the results to predict potential misunderstandings between users. It identifies the source of these misunderstandings and detects potential discrepancies in intent.

[0246] Step 6:

[0247] To prevent misunderstandings, the server reinterprets the intended meaning and sends it to the terminal. A selection of appropriate intentions and interpretations is then generated.

[0248] Step 7:

[0249] The analysis results received by the terminal are displayed to the user through the user interface in a format that is intuitively understandable to the user.

[0250] Step 8:

[0251] Based on the information provided by the user, the system adjusts communication to facilitate more effective dialogue.

[0252] Step 9:

[0253] Users provide feedback on their usage experience and the AI's intent analysis. This feedback is collected by the device and sent to the server.

[0254] Step 10:

[0255] The server analyzes the feedback it receives and uses it to improve the AI-generated model. Based on the feedback, the model is trained and its parameters are adjusted.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0258] In today's information and communication environment, communication between users takes place through a variety of means, but frequent misunderstandings hinder smooth communication. Furthermore, advanced data analysis and privacy protection are required to prevent these misunderstandings and accurately convey users' intentions. This invention aims to address these challenges and improve communication between users.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes means for collecting past communication information, means for anonymizing the communication information and protecting privacy, and means for analyzing the anonymized data and understanding contextual information using a generative AI model. This prevents misunderstandings between users and enables accurate communication of intentions.

[0261] "Past communication information" refers to digital records of previous communications such as messages and phone calls made by the user.

[0262] "Anonymization" is the process of removing or transforming personally identifiable information so that the data does not infringe on privacy.

[0263] A "secure protocol" is a set of communication rules or standards for ensuring the secure transmission of data over the internet or a network.

[0264] A "generative AI model" refers to an artificial intelligence algorithm that takes data as input and performs analysis or generation based on a specific purpose.

[0265] "Contextual information" refers to all relevant information that indicates the background, situation, and intentions behind statements and actions in communication.

[0266] A "user interface" is a collection of screens and display elements that a user uses when interacting with a system.

[0267] This system is implemented as a network-based platform using user terminals, a central server, and a generative AI model. Users first log in to the system using their own terminals. After logging in, the terminals collect past communication information from local data storage to the extent agreed upon by the user. Specifically, they filter and retrieve communication history for a specified period from email clients and messaging applications on the terminals.

[0268] The device anonymizes the collected data. The anonymization algorithm is designed to protect user privacy by removing or transforming personally identifiable information. During this process, the data is transmitted to the server using a strong, secure protocol.

[0269] The server stores the received data in a pre-configured database and applies a generative AI model to analyze the contextual information. The generative AI model utilizes natural language processing techniques to understand the intentions and emotions behind the conversation and identify areas where misunderstandings may occur between users. This analysis makes it possible to predict mismatches in intent between users.

[0270] The analysis results are sent back from the server to the user's terminal and presented to the user through the terminal's user interface. At this time, the analyzed information is provided in an easy-to-understand manner through graphical displays and notifications, allowing the user to receive information based on the analysis.

[0271] As a concrete example, consider a scenario where past communication data from a project team within a company is analyzed. The terminal application sends a prompt message to the AI ​​model stating, "Analyze the intent behind the meeting agenda," and the server visualizes the analysis results for the user. This supports the efficient progress of the project.

[0272] An example of a prompt message might be, "Use an AI model to analyze the likelihood of misunderstandings between User A and User B based on their past discussion history."

[0273] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0274] Step 1:

[0275] The user logs into the system using their own device. The device receives user authentication information (e.g., user ID and password) as input. Upon successful login, a user authentication token is generated as output, and session information is established.

[0276] Step 2:

[0277] The device collects past communication information related to logged-in users. The input is communication history obtained from the device's email client and messaging apps. For data processing, messages and emails from a specified period are filtered and collected. The output is a list of the relevant historical data.

[0278] Step 3:

[0279] The terminal anonymizes the collected communication information. The communication history data obtained in step 2 is used as input. An algorithm is applied to mask the data by removing personally identifiable information. Anonymized history data is generated as output.

[0280] Step 4:

[0281] The terminal sends the anonymized data to the server using a secure protocol. As input, anonymized data and communication protocol information are used. As output, a transmission completion status is returned.

[0282] Step 5:

[0283] The server analyzes the received data and uses the generated AI model to understand the context information. As input, the anonymized data is supplied to the server. As data operations, the AI model evaluates the data and a process of analyzing the intention and sentiment of the conversation is performed. As output, context analysis results are generated.

[0284] Step 6:

[0285] Based on the analysis results, the server predicts the possibility of misunderstanding between users and presents appropriate intentions. As input, the analysis results generated in Step 5 are utilized. As data processing, points where misunderstandings may occur are identified. As output, intention interpretation results are generated.

[0286] Step 7:

[0287] The server returns the generated intention interpretation results to the user's terminal through the user interface. As input, the intention interpretation results and the user ID are used. As output, the analysis results are displayed on the user's terminal.

[0288] Step 8:

[0289] The user checks the presented analysis results and provides feedback. As input, the user's evaluation and comments are input on the terminal. As output, feedback data is sent to the server.

[0290] Step 9:

[0291] The server processes user feedback and improves the generated AI model. Feedback data is used as input. The AI ​​model's learning process is updated as data computation, and adjustments are made to improve the model's accuracy. The improved AI model is obtained as output.

[0292] (Application Example 1)

[0293] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0294] In modern commercial facilities and service industries, communication among employees and with customers is becoming increasingly complex, leading to challenges such as misunderstandings and miscommunication. Such misunderstandings can result in decreased operational efficiency and customer satisfaction. Furthermore, protecting privacy is crucial when using communication history.

[0295] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0296] In this invention, the server includes means for collecting past communication content and dialogue history, means for analyzing the communication content to understand the context and using a generative model to predict responses in real time, and means for predicting misunderstandings between users or between users and customers, presenting appropriate intentions, and supporting the optimal response method. This prevents misunderstandings in communication and enables smoother and more effective information transmission.

[0297] "Past communication content and dialogue history" refers to the record of all previous messages and conversations that took place between users or between users and customers.

[0298] A "generative model" refers to an algorithm that uses artificial intelligence technology to analyze communication content, understand the context, and predict responses.

[0299] "Methods for predicting misunderstandings between users and customers" refers to technologies that detect potential misunderstandings and miscommunications that may occur during communication, based on past data and contextual analysis.

[0300] "A means of presenting appropriate intent and supporting the optimal response method" refers to a system that clarifies intent based on analysis results and guides users in selecting the most effective response.

[0301] "Information equipment" refers to all electronic devices used by users, such as smartphones, tablets, and computers.

[0302] "Means of anonymizing data to protect privacy and applying stronger encryption techniques" refers to the process of processing data so that individuals cannot be identified and then using encryption technology to further enhance data security.

[0303] The system requires user communication devices, a central information processing unit, and software including an artificial intelligence generative model. The user's communication devices, such as smartphones or tablets, are responsible for collecting past communication content and conversation history. This data is collected with the user's consent, anonymized for privacy protection, and further protected by strong encryption methods such as 256-bit AES.

[0304] When data is sent to a central information processing unit, a generative AI model activates, analyzing the communication content to understand the context and predicting responses in real time. This model uses AI libraries such as TensorFlow and leverages natural language processing techniques to predict misunderstandings. The analysis results are presented to the user's communication device via a user interface, assisting in facilitating communication between users or between users and customers.

[0305] As a specific example, when a new staff member in a store responds to a customer's question, this system presents the optimal response in real time from past conversations. Based on the analysis results displayed on the staff member's communication device, the appropriate intent can be presented, improving the accuracy and efficiency of the response.

[0306] As an example of the prompt text input to the generative AI model, messages such as "Please predict the optimal response from the past conversation history. For example, based on the past response history of 'Is there any stock?' " can be considered. By using this prompt text, the model can focus on the issues to be addressed and derive highly accurate responses.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The user's terminal starts collecting communication history. Within the scope agreed by the user, past conversation contents and interaction histories are selected and anonymized and encrypted. The input is the user's past communication content, and the output is anonymized encrypted data.

[0310] Step 2:

[0311] The collected data is sent to the server. The server decrypts the received encrypted data and extracts the information necessary for analysis. The input is anonymized encrypted data, and the output is communication data in an analyzable format.

[0312] Step 3:

[0313] The server processes the analyzable data using the generative AI model. In this process, the context of the communication, past patterns, and common misunderstanding points are analyzed. The input is communication data in an analyzable format, and the output is a prediction of misunderstandings and a proposal of appropriate responses.

[0314] Step 4:

[0315] The analysis results are sent back to the terminal via the user interface. The terminal displays the results in a format that is easy for the user to understand. The input is a prediction of misunderstandings and a suggestion of appropriate responses, while the output is user-oriented information displayed on the terminal.

[0316] Step 5:

[0317] Users can choose actions based on the information presented and provide feedback as needed. This feedback is used to improve the generative AI model, increasing the accuracy of subsequent analyses. The input is user feedback, and the output is the improved result of analyzing the feedback and reflecting it in the generative AI model.

[0318] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0319] This invention is a system that supports communication between users, and through a new configuration that includes an emotion engine, it can also reflect the user's emotions in the analysis results. The system mainly consists of a user terminal, a server, and a generative model equipped with an emotion engine.

[0320] When a user starts the system, the terminal performs a login process, collects past communication history data, anonymizes it with privacy protection measures, and sends it to the server. At this time, the terminal collects the user's emotions in real time, using an emotion engine to obtain emotional data from facial expressions and linguistic indicators.

[0321] The server integrates and analyzes the received communication data and sentiment data. In this analysis, a generative model understands the context, and the sentiment engine interprets the emotions. As a result of combining the data, the server understands the user's emotional state and reinterprets appropriate intentions based on that state.

[0322] Based on the analysis results, the server predicts communication misunderstandings and has the means to dynamically present intentions according to emotions. The terminal receives the analysis data obtained from the server and presents it in a format that is easy for the user to understand. This system provides more flexible and adaptive communication support by taking the user's emotional state into consideration.

[0323] As a concrete example, consider a scenario where members of a project team in remote locations are holding an online meeting. If the presenter is nervous, the emotion engine detects this, and the server clarifies the presenter's intentions to avoid misunderstandings, helping the audience accurately understand the presenter's intent. In this way, considering emotions improves the quality of communication.

[0324] This system allows users to objectively support their emotionally charged communication, thereby reducing misunderstandings. Furthermore, the accuracy improves over time through continuous refinement of the generative model using feedback.

[0325] The following describes the processing flow.

[0326] Step 1:

[0327] The user logs into the system using their device. The device collects past communication history, anonymizes the data to protect privacy, and then sends it to the server.

[0328] Step 2:

[0329] The device acquires real-time emotional data from the user. This utilizes an emotion engine that identifies emotions through facial expression analysis and voice tone analysis.

[0330] Step 3:

[0331] The server integrates received communication history data with real-time sentiment data. This data is centrally managed and prepared for analysis.

[0332] Step 4:

[0333] The server uses generative models to analyze the context of the communication content. Simultaneously, the emotion engine analyzes emotion data to identify the user's emotional state.

[0334] Step 5:

[0335] Based on contextual information and emotional states analyzed by the server, it predicts areas where misunderstandings are likely to occur. Considering emotional data enables more accurate intent analysis.

[0336] Step 6:

[0337] The server sends the results of the intent analysis to the terminal. The intent is dynamically adjusted based on the emotional situation and presented appropriately to the user.

[0338] Step 7:

[0339] The device displays the analysis results through an intuitive user interface. This allows users to easily understand their own emotions and intentions based on the content of the conversation.

[0340] Step 8:

[0341] Based on the information provided by the user, we will implement measures to improve communication. Users will provide feedback as needed, contributing to system improvements.

[0342] Step 9:

[0343] The server collects feedback and improves the generative model and sentiment engine. Based on the feedback, the model is retrained to improve the accuracy of the analysis.

[0344] (Example 2)

[0345] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0346] In user-to-user communication, misunderstandings of emotions and unclear intentions can hinder the smooth transmission of information. Furthermore, protecting privacy is difficult, and there is a lack of mechanisms to effectively utilize feedback and continuously improve generative models. These challenges need to be addressed to achieve emotionally sensitive communication.

[0347] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0348] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for detecting the emotional state of the information sender and presenting appropriate intentions based on that emotion, means for acquiring emotional data from the information sender's facial expressions and voice and analyzing that information with an emotion engine, means for presenting the analysis results to the information receiver in a visualized form, and means for processing user feedback and continuously improving the generative model. This enables the provision of flexible and intuitive communication that responds to emotions.

[0349] "Past communication information" refers to all messages and call history previously exchanged between users, and includes communication history data.

[0350] "Generative modeling" refers to the process of using machine learning models to understand the context and intent behind information, utilizing artificial intelligence technology.

[0351] "The emotional state of the information provider" refers to the mental and emotional state of the user who is disseminating the information, and this includes emotions such as joy, sadness, and anger.

[0352] An "emotion engine" refers to software or hardware technology that analyzes user data such as facial expressions and voice to determine emotions.

[0353] "Feedback" refers to opinions, reactions, or data collected from users, which are used to improve systems and generative models.

[0354] "Anonymization" refers to the process of removing or transforming personally identifiable elements from data in order to protect privacy.

[0355] "Visualizing analysis results" means presenting the results of data analysis in a visually easy-to-understand format, providing information in a way that is easy for users to comprehend.

[0356] This invention is a system that supports user communication and aims to reduce misunderstandings while taking user emotions into consideration. The system mainly consists of a server, a terminal, and a generative model equipped with an emotion engine.

[0357] The device first collects past communication information when the user logs in. This includes messages and call history. The collected information is anonymized to protect privacy and then sent to the server. Anonymization is a process that removes or transforms personally identifiable information.

[0358] The device also collects emotional data in real time. The device works in conjunction with the emotion engine to analyze the user's facial expressions and voice, and obtains emotional data from them. The emotion engine uses a specific algorithm to determine the user's emotional state from the acquired data.

[0359] The server receives communication information and sentiment data transmitted from the terminal. A generative AI model is used to analyze this data and present appropriate intentions based on the user's emotional state. This generative AI model simultaneously performs contextual understanding and sentiment interpretation, dynamically reinterpreting intentions to reduce misunderstandings.

[0360] The server then returns the analysis results to the terminal, which presents them to the user through an intuitive interface. This system can continuously improve the generated AI model using user feedback. Based on the feedback, the system evolves to improve the user experience.

[0361] As a concrete example, consider a scenario where a project team is holding an online meeting. If the presenter is nervous, the device detects the presenter's voice and facial expressions and analyzes their emotions via the server. Based on the analyzed information, the server provides the audience with information to clarify the presenter's intentions. The presenter can also use this information to give their presentation with greater confidence.

[0362] An example of a prompt might be, "Please tell me how the server detected the tension that user A showed during the online meeting and how it conveyed that to the audience." In this way, the system supports tackling realistic challenges.

[0363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0364] Step 1:

[0365] When a user starts the system, the terminal displays a login screen where the user enters their authentication information. After the authentication information is entered, the terminal sends it to the server, and the login process is executed. This initiates the user's session. The inputs are a user ID and password, and the output is a message indicating login success or failure.

[0366] Step 2:

[0367] Once login is complete, the device collects past communication information. This information includes text messages and call history. The device encrypts and transforms this data to anonymize it. The input is raw communication data, and the output is anonymized communication data. The anonymized data is sent to the server.

[0368] Step 3:

[0369] The device also activates its camera and microphone to collect user emotion data. The collected facial and audio data is analyzed using an emotion engine and output as the user's real-time emotional state. The input for this step is facial expressions and audio, and the output is the analyzed emotion data.

[0370] Step 4:

[0371] The server receives anonymized communication data and sentiment datasets sent from the terminal. The server analyzes this data using a generating AI model and sentiment engine. This analysis enables an understanding of the user's context and emotional state. Raw data is the input, and the analysis results are the output.

[0372] Step 5:

[0373] The server uses a generative AI model to reinterpret the intent behind communication based on the analyzed emotional state and generates revised versions to avoid misunderstandings. In this step, the analysis results are used as input, and the revised intent is generated as output.

[0374] Step 6:

[0375] The server sends suggested revisions to the generated intent to the terminal, which then displays them. The user can then proceed with the conversation with confidence based on this information. The terminal visualizes the received data through an appropriate user interface and presents it to the user. The input is the suggested revision data from the server, and the output is the display on the user interface.

[0376] Step 7:

[0377] User feedback is collected through the device and sent to the server. The server uses this feedback to continuously improve the generated AI model. The input to this process is user feedback, and the output is an updated, improved model.

[0378] (Application Example 2)

[0379] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0380] In modern society, especially in family and daily communication, users' emotions can be a source of misunderstanding. In situations where direct dialogue is difficult, emotional misunderstandings can reduce the efficiency and quality of communication. Furthermore, emotional misunderstandings can negatively impact relationships with family and close friends. It is necessary to address this challenge and provide means to achieve smoother and more harmonious communication.

[0381] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0382] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for predicting misunderstandings between users and presenting appropriate intentions, means for processing user feedback and improving the generative model, means for using an emotion engine that analyzes the user's emotional state, and means for presenting feedback and support to the user in a parental voice based on the analysis results. This enables more accurate and harmonious communication while taking into account the user's emotional state.

[0383] "Past communication information" refers to data related to conversations and messages previously exchanged between users.

[0384] "Analysis of communication information" refers to the process of analyzing collected communication data in order to understand its content and context.

[0385] A "generative model" refers to an algorithm or program that uses artificial intelligence technology to analyze data and generate appropriate output.

[0386] "Predicting misunderstandings between users" refers to identifying potential misunderstandings and misperceptions that may arise in communication between users.

[0387] "Presenting the correct intent" means explicitly providing information so that users can understand the intended message without misunderstanding.

[0388] "Feedback processing" refers to the process of collecting responses and opinions provided by users and using them to improve systems and models.

[0389] An "emotion engine" refers to software or a device that detects and evaluates emotions from a user's facial expressions, tone of voice, and other factors.

[0390] "Providing feedback and support" means providing users with necessary information in a timely manner based on the analysis results and supporting smooth communication.

[0391] "Parental voice" refers to outputting a warm voice that gives the user a sense of security and familiarity.

[0392] The system implementing this invention is particularly envisioned as a consumer robot for use in the home. The server utilizes a generative model that understands context by collecting and analyzing the user's past communication information. Based on the results of this analysis, it predicts potential misunderstandings between users and presents appropriate intentions. The server also processes user feedback and continuously improves the generative model.

[0393] The terminal, or consumer robot, is equipped with the ability to collect the user's facial expressions and tone of voice in real time and acquire emotional data through an emotion engine. This emotional data is comprehensively evaluated in conjunction with the analysis results on the server, and feedback and support are presented to the user in a parental voice. This kind of feedback facilitates smooth communication with the user and minimizes misunderstandings.

[0394] As a concrete example of implementing this system, consider a scenario where a robot with home helper functionality detects that a child is struggling with their homework. The robot would then reduce the child's stress by suggesting to the user, i.e., the child, via voice, "Shall we take a break? Or do you need any help?"

[0395] This invention enables more accurate and harmonious communication that takes into account the user's emotional state through user-friendly feedback. The system design utilizes software such as OpenCV (image processing), TensorFlow (generative AI modeling), and NLTK (natural language processing), and is implemented on a consumer robot platform.

[0396] An example of a prompt message would be, "The user's stress level is high. Please provide some conversational tips to help them relax."

[0397] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0398] Step 1:

[0399] The device uses a camera and microphone to collect data on the user's facial expressions and voice. The input is real-time facial images and audio from the user, and the output is emotion data based on these. An emotion engine is used to process the data and analyze emotions from changes in facial expressions and voice tone. Specifically, OpenCV is used to extract facial feature points and audio data is analyzed to identify the emotional state.

[0400] Step 2:

[0401] The device transmits collected emotional data to the server while protecting privacy. The input is anonymized emotional data, and the output is a secure data transmission to the server. The data is transformed into a form that cannot be linked to a specific user and transmitted securely using protocols such as HTTPS. Specifically, the data is encrypted before being transmitted to the server using secure network protocols for medical or educational purposes.

[0402] Step 3:

[0403] The server integrates received sentiment data with past communication information and performs analysis using a generative AI model. The input is sentiment data and past communication information, and the output is the prediction of misunderstandings between users and the proposed intent. The generative AI model understands the context and performs data calculations to formulate the optimal intent for the user. Specifically, it uses TensorFlow for natural language processing and NLTK to reinforce the sentiment context.

[0404] Step 4:

[0405] The server sends the generated intent and feedback information to the terminal. The input is the communication strategy generated within the server, and the output is the information transmitted to the terminal. The processing includes practical feedback provided by a generative AI model based on a specific scenario. Specifically, the server formalizes the feedback statement, forms an appropriate prompt statement, and then sends it to the terminal.

[0406] Step 5:

[0407] The terminal presents feedback received from the server to the user both audibly and visually. The input is the generated intent sent from the server, and the output is the presentation of feedback to the user. Using speech synthesis technology, the feedback is provided in a friendly voice. Specifically, a voice synthesizer is used to convey information to the user in a gentle, questioning manner.

[0408] Through these steps, users receive feedback and support that reflects their emotions, enabling smoother communication.

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0410] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0412] [Third Embodiment]

[0413] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0414] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0415] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0417] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0419] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0420] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0421] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0424] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0425] This invention aims to realize a system for understanding the context of communication and preventing misunderstandings. The specific configuration and operation of this system are described below.

[0426] This system consists of three main elements: the user's terminal, a central server, and an AI-generated model. To begin using the system, the user must first log in by operating the terminal. The terminal extracts past communication history from its local storage and collects data to the extent agreed upon by the user. The collected data is anonymized for privacy protection and then sent to the server.

[0427] The server analyzes the received data and uses an AI generative model to analyze the context of the conversation. This process identifies points where misunderstandings are likely to occur from past conversation history. Based on the analysis results, the server predicts misunderstandings between users and provides interpretations to understand the correct intentions. These analysis results are sent back to the terminal and presented to the user in an easy-to-understand manner.

[0428] As a concrete example, suppose that in a project meeting held within a company, users A and B have different interpretations. The application on the terminal sends data to the server from past discussion records. Based on this information, the server predicts misunderstandings and uses an AI generative model to analyze and present the accurate intent. As a result, users A and B can proceed with the project based on a shared understanding.

[0429] This system allows users to prevent misunderstandings that may arise during communication and enables smooth transmission of intentions. The system also features a feedback processing function, which continuously improves the AI ​​generation model based on user feedback, providing more accurate and efficient analysis.

[0430] The following describes the processing flow.

[0431] Step 1:

[0432] The user launches the application and logs into the system. With consent, the device collects past communication history data from local storage.

[0433] Step 2:

[0434] The communication data collected by the device is anonymized for privacy protection and sent to the server using a secure protocol.

[0435] Step 3:

[0436] The server receives data, saves it to the database, and prepares the dataset. The data is organized and formatted into an easily accessible format.

[0437] Step 4:

[0438] The server uses an AI generative model to analyze data and understand the context of conversations between users. The analysis identifies the conversation's topic, emotions, and intentions.

[0439] Step 5:

[0440] The server analyzes the results to predict potential misunderstandings between users. It identifies the source of these misunderstandings and detects potential discrepancies in intent.

[0441] Step 6:

[0442] To prevent misunderstandings, the server reinterprets the intended meaning and sends it to the terminal. A selection of appropriate intentions and interpretations is then generated.

[0443] Step 7:

[0444] The analysis results received by the terminal are displayed to the user through the user interface in a format that is intuitively understandable to the user.

[0445] Step 8:

[0446] Based on the information provided by the user, the system adjusts communication to facilitate more effective dialogue.

[0447] Step 9:

[0448] Users provide feedback on their usage experience and the AI's intent analysis. This feedback is collected by the device and sent to the server.

[0449] Step 10:

[0450] The server analyzes the feedback it receives and uses it to improve the AI-generated model. Based on the feedback, the model is trained and its parameters are adjusted.

[0451] (Example 1)

[0452] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0453] In today's information and communication environment, communication between users takes place through a variety of means, but frequent misunderstandings hinder smooth communication. Furthermore, advanced data analysis and privacy protection are required to prevent these misunderstandings and accurately convey users' intentions. This invention aims to address these challenges and improve communication between users.

[0454] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0455] In this invention, the server includes means for collecting past communication information, means for anonymizing the communication information and protecting privacy, and means for analyzing the anonymized data and understanding contextual information using a generative AI model. This prevents misunderstandings between users and enables accurate communication of intentions.

[0456] "Past communication information" refers to digital records of previous communications such as messages and phone calls made by the user.

[0457] "Anonymization" is the process of removing or transforming personally identifiable information so that the data does not infringe on privacy.

[0458] A "secure protocol" is a set of communication rules or standards for ensuring the secure transmission of data over the internet or a network.

[0459] A "generative AI model" refers to an artificial intelligence algorithm that takes data as input and performs analysis or generation based on a specific purpose.

[0460] "Contextual information" refers to all relevant information that indicates the background, situation, and intentions behind statements and actions in communication.

[0461] A "user interface" is a collection of screens and display elements that a user uses when interacting with a system.

[0462] This system is implemented as a network-based platform using user terminals, a central server, and a generative AI model. Users first log in to the system using their own terminals. After logging in, the terminals collect past communication information from local data storage to the extent agreed upon by the user. Specifically, they filter and retrieve communication history for a specified period from email clients and messaging applications on the terminals.

[0463] The device anonymizes the collected data. The anonymization algorithm is designed to protect user privacy by removing or transforming personally identifiable information. During this process, the data is transmitted to the server using a strong, secure protocol.

[0464] The server stores the received data in a pre-configured database and applies a generative AI model to analyze the contextual information. The generative AI model utilizes natural language processing techniques to understand the intentions and emotions behind the conversation and identify areas where misunderstandings may occur between users. This analysis makes it possible to predict mismatches in intent between users.

[0465] The analysis results are sent back from the server to the user's terminal and presented to the user through the terminal's user interface. At this time, the analyzed information is provided in an easy-to-understand manner through graphical displays and notifications, allowing the user to receive information based on the analysis.

[0466] As a concrete example, consider a scenario where past communication data from a project team within a company is analyzed. The terminal application sends a prompt message to the AI ​​model stating, "Analyze the intent behind the meeting agenda," and the server visualizes the analysis results for the user. This supports the efficient progress of the project.

[0467] An example of a prompt message might be, "Use an AI model to analyze the likelihood of misunderstandings between User A and User B based on their past discussion history."

[0468] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0469] Step 1:

[0470] The user logs into the system using their own device. The device receives user authentication information (e.g., user ID and password) as input. Upon successful login, a user authentication token is generated as output, and session information is established.

[0471] Step 2:

[0472] The device collects past communication information related to logged-in users. The input is communication history obtained from the device's email client and messaging apps. For data processing, messages and emails from a specified period are filtered and collected. The output is a list of the relevant historical data.

[0473] Step 3:

[0474] The terminal anonymizes the collected communication information. The communication history data obtained in step 2 is used as input. An algorithm is applied to mask the data by removing personally identifiable information. Anonymized history data is generated as output.

[0475] Step 4:

[0476] The terminal sends anonymized data to the server using a secure protocol. Anonymized data and communication protocol information are used as input. The output is a status indicating successful transmission.

[0477] Step 5:

[0478] The server analyzes the received data and uses a generative AI model to understand contextual information. Anonymized data is supplied to the server as input. The data processing involves the AI ​​model evaluating the data and analyzing the intent and emotions of the conversation. The output is the result of the contextual analysis.

[0479] Step 6:

[0480] Based on the analysis results, the server predicts the possibility of misunderstandings between users and presents the appropriate intent. The analysis results generated in step 5 are used as input. As data processing, points where misunderstandings may occur are identified. As output, the interpretation of intent is generated.

[0481] Step 7:

[0482] The server returns the interpretation result of the generated intent to the user's terminal via the user interface. The inputs used are the interpretation result of the intent and the user ID. The output is the analysis result displayed on the user's terminal.

[0483] Step 8:

[0484] The user reviews the presented analysis results and provides feedback. As input, the user's evaluation and comments are entered on the device. As output, the feedback data is sent to the server.

[0485] Step 9:

[0486] The server processes user feedback and improves the generated AI model. Feedback data is used as input. The AI ​​model's learning process is updated as data computation, and adjustments are made to improve the model's accuracy. The improved AI model is obtained as output.

[0487] (Application Example 1)

[0488] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] In modern commercial facilities and service industries, communication among employees and with customers is becoming increasingly complex, leading to challenges such as misunderstandings and miscommunication. Such misunderstandings can result in decreased operational efficiency and customer satisfaction. Furthermore, protecting privacy is crucial when using communication history.

[0490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0491] In this invention, the server includes means for collecting past communication content and dialogue history, means for analyzing the communication content to understand the context and using a generative model to predict responses in real time, and means for predicting misunderstandings between users or between users and customers, presenting appropriate intentions, and supporting the optimal response method. This prevents misunderstandings in communication and enables smoother and more effective information transmission.

[0492] "Past communication content and dialogue history" refers to the record of all previous messages and conversations that took place between users or between users and customers.

[0493] A "generative model" refers to an algorithm that uses artificial intelligence technology to analyze communication content, understand the context, and predict responses.

[0494] "Methods for predicting misunderstandings between users and customers" refers to technologies that detect potential misunderstandings and miscommunications that may occur during communication, based on past data and contextual analysis.

[0495] "A means of presenting appropriate intent and supporting the optimal response method" refers to a system that clarifies intent based on analysis results and guides users in selecting the most effective response.

[0496] "Information equipment" refers to all electronic devices used by users, such as smartphones, tablets, and computers.

[0497] "Means of anonymizing data to protect privacy and applying stronger encryption techniques" refers to the process of processing data so that individuals cannot be identified and then using encryption technology to further enhance data security.

[0498] The system requires user communication devices, a central information processing unit, and software including an artificial intelligence generative model. The user's communication devices, such as smartphones or tablets, are responsible for collecting past communication content and conversation history. This data is collected with the user's consent, anonymized for privacy protection, and further protected by strong encryption methods such as 256-bit AES.

[0499] When data is sent to a central information processing unit, a generative AI model activates, analyzing the communication content to understand the context and predicting responses in real time. This model uses AI libraries such as TensorFlow and leverages natural language processing techniques to predict misunderstandings. The analysis results are presented to the user's communication device via a user interface, assisting in facilitating communication between users or between users and customers.

[0500] As a concrete example, when a new staff member in a store answers a customer's question, this system provides the optimal response in real time based on past conversations. The analysis results displayed on the staff member's communication device help to present the appropriate intent, improving the accuracy and efficiency of responses.

[0501] An example of a prompt message to input into a generative AI model would be, "Predict the best response based on past conversation history. Example: Based on past responses to 'Do you have it in stock?'" By using this prompt message, the model can focus on the issue it needs to address and derive a highly accurate response.

[0502] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0503] Step 1:

[0504] The user's device begins collecting communication history. Within the scope agreed upon by the user, past conversation content and interaction history are selected, anonymized, and encrypted. The input is the user's past communication content, and the output is anonymized and encrypted data.

[0505] Step 2:

[0506] The collected data is sent to the server. The server decrypts the received encrypted data and extracts the information necessary for analysis. The input is anonymized encrypted data, and the output is communication data in an analyzable format.

[0507] Step 3:

[0508] The server processes analyzable data using a generative AI model. In this process, it analyzes the context of the communication, past patterns, and common points of misunderstanding. The input is communication data in an analyzable format, and the output is predictions of misunderstandings and suggestions for appropriate responses.

[0509] Step 4:

[0510] The analysis results are sent back to the terminal via the user interface. The terminal displays the results in a format that is easy for the user to understand. The input is a prediction of misunderstandings and a suggestion of appropriate responses, while the output is user-oriented information displayed on the terminal.

[0511] Step 5:

[0512] Users can choose actions based on the information presented and provide feedback as needed. This feedback is used to improve the generative AI model, increasing the accuracy of subsequent analyses. The input is user feedback, and the output is the improved result of analyzing the feedback and reflecting it in the generative AI model.

[0513] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0514] This invention is a system that supports communication between users, and through a new configuration that includes an emotion engine, it can also reflect the user's emotions in the analysis results. The system mainly consists of a user terminal, a server, and a generative model equipped with an emotion engine.

[0515] When a user starts the system, the terminal performs a login process, collects past communication history data, anonymizes it with privacy protection measures, and sends it to the server. At this time, the terminal collects the user's emotions in real time, using an emotion engine to obtain emotional data from facial expressions and linguistic indicators.

[0516] The server integrates and analyzes the received communication data and sentiment data. In this analysis, a generative model understands the context, and the sentiment engine interprets the emotions. As a result of combining the data, the server understands the user's emotional state and reinterprets appropriate intentions based on that state.

[0517] Based on the analysis results, the server predicts communication misunderstandings and has the means to dynamically present intentions according to emotions. The terminal receives the analysis data obtained from the server and presents it in a format that is easy for the user to understand. This system provides more flexible and adaptive communication support by taking the user's emotional state into consideration.

[0518] As a concrete example, consider a scenario where members of a project team in remote locations are holding an online meeting. If the presenter is nervous, the emotion engine detects this, and the server clarifies the presenter's intentions to avoid misunderstandings, helping the audience accurately understand the presenter's intent. In this way, considering emotions improves the quality of communication.

[0519] This system allows users to objectively support their emotionally charged communication, thereby reducing misunderstandings. Furthermore, the accuracy improves over time through continuous refinement of the generative model using feedback.

[0520] The following describes the processing flow.

[0521] Step 1:

[0522] The user logs into the system using their device. The device collects past communication history, anonymizes the data to protect privacy, and then sends it to the server.

[0523] Step 2:

[0524] The device acquires real-time emotional data from the user. This utilizes an emotion engine that identifies emotions through facial expression analysis and voice tone analysis.

[0525] Step 3:

[0526] The server integrates received communication history data with real-time sentiment data. This data is centrally managed and prepared for analysis.

[0527] Step 4:

[0528] The server uses generative models to analyze the context of the communication content. Simultaneously, the emotion engine analyzes emotion data to identify the user's emotional state.

[0529] Step 5:

[0530] Based on contextual information and emotional states analyzed by the server, it predicts areas where misunderstandings are likely to occur. Considering emotional data enables more accurate intent analysis.

[0531] Step 6:

[0532] The server sends the results of the intent analysis to the terminal. The intent is dynamically adjusted based on the emotional situation and presented appropriately to the user.

[0533] Step 7:

[0534] The device displays the analysis results through an intuitive user interface. This allows users to easily understand their own emotions and intentions based on the content of the conversation.

[0535] Step 8:

[0536] Based on the information provided by the user, we will implement measures to improve communication. Users will provide feedback as needed, contributing to system improvements.

[0537] Step 9:

[0538] The server collects feedback and improves the generative model and sentiment engine. Based on the feedback, the model is retrained to improve the accuracy of the analysis.

[0539] (Example 2)

[0540] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0541] In user-to-user communication, misunderstandings of emotions and unclear intentions can hinder the smooth transmission of information. Furthermore, protecting privacy is difficult, and there is a lack of mechanisms to effectively utilize feedback and continuously improve generative models. These challenges need to be addressed to achieve emotionally sensitive communication.

[0542] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0543] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for detecting the emotional state of the information sender and presenting appropriate intentions based on that emotion, means for acquiring emotional data from the information sender's facial expressions and voice and analyzing that information with an emotion engine, means for presenting the analysis results to the information receiver in a visualized form, and means for processing user feedback and continuously improving the generative model. This enables the provision of flexible and intuitive communication that responds to emotions.

[0544] "Past communication information" refers to all messages and call history previously exchanged between users, and includes communication history data.

[0545] "Generative modeling" refers to the process of using machine learning models to understand the context and intent behind information, utilizing artificial intelligence technology.

[0546] "The emotional state of the information provider" refers to the mental and emotional state of the user who is disseminating the information, and this includes emotions such as joy, sadness, and anger.

[0547] An "emotion engine" refers to software or hardware technology that analyzes user data such as facial expressions and voice to determine emotions.

[0548] "Feedback" refers to opinions, reactions, or data collected from users, which are used to improve systems and generative models.

[0549] "Anonymization" refers to the process of removing or transforming personally identifiable elements from data in order to protect privacy.

[0550] "Visualizing analysis results" means presenting the results of data analysis in a visually easy-to-understand format, providing information in a way that is easy for users to comprehend.

[0551] This invention is a system that supports user communication and aims to reduce misunderstandings while taking user emotions into consideration. The system mainly consists of a server, a terminal, and a generative model equipped with an emotion engine.

[0552] The device first collects past communication information when the user logs in. This includes messages and call history. The collected information is anonymized to protect privacy and then sent to the server. Anonymization is a process that removes or transforms personally identifiable information.

[0553] The device also collects emotional data in real time. The device works in conjunction with the emotion engine to analyze the user's facial expressions and voice, and obtains emotional data from them. The emotion engine uses a specific algorithm to determine the user's emotional state from the acquired data.

[0554] The server receives communication information and sentiment data transmitted from the terminal. A generative AI model is used to analyze this data and present appropriate intentions based on the user's emotional state. This generative AI model simultaneously performs contextual understanding and sentiment interpretation, dynamically reinterpreting intentions to reduce misunderstandings.

[0555] The server then returns the analysis results to the terminal, which presents them to the user through an intuitive interface. This system can continuously improve the generated AI model using user feedback. Based on the feedback, the system evolves to improve the user experience.

[0556] As a concrete example, consider a scenario where a project team is holding an online meeting. If the presenter is nervous, the device detects the presenter's voice and facial expressions and analyzes their emotions via the server. Based on the analyzed information, the server provides the audience with information to clarify the presenter's intentions. The presenter can also use this information to give their presentation with greater confidence.

[0557] An example of a prompt might be, "Please tell me how the server detected the tension that user A showed during the online meeting and how it conveyed that to the audience." In this way, the system supports tackling realistic challenges.

[0558] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0559] Step 1:

[0560] When a user starts the system, the terminal displays a login screen where the user enters their authentication information. After the authentication information is entered, the terminal sends it to the server, and the login process is executed. This initiates the user's session. The inputs are a user ID and password, and the output is a message indicating login success or failure.

[0561] Step 2:

[0562] Once login is complete, the device collects past communication information. This information includes text messages and call history. The device encrypts and transforms this data to anonymize it. The input is raw communication data, and the output is anonymized communication data. The anonymized data is sent to the server.

[0563] Step 3:

[0564] The device also activates its camera and microphone to collect user emotion data. The collected facial and audio data is analyzed using an emotion engine and output as the user's real-time emotional state. The input for this step is facial expressions and audio, and the output is the analyzed emotion data.

[0565] Step 4:

[0566] The server receives anonymized communication data and sentiment datasets sent from the terminal. The server analyzes this data using a generating AI model and sentiment engine. This analysis enables an understanding of the user's context and emotional state. Raw data is the input, and the analysis results are the output.

[0567] Step 5:

[0568] The server uses a generative AI model to reinterpret the intent behind communication based on the analyzed emotional state and generates revised versions to avoid misunderstandings. In this step, the analysis results are used as input, and the revised intent is generated as output.

[0569] Step 6:

[0570] The server sends suggested revisions to the generated intent to the terminal, which then displays them. The user can then proceed with the conversation with confidence based on this information. The terminal visualizes the received data through an appropriate user interface and presents it to the user. The input is the suggested revision data from the server, and the output is the display on the user interface.

[0571] Step 7:

[0572] User feedback is collected through the device and sent to the server. The server uses this feedback to continuously improve the generated AI model. The input to this process is user feedback, and the output is an updated, improved model.

[0573] (Application Example 2)

[0574] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0575] In modern society, especially in family and daily communication, users' emotions can be a source of misunderstanding. In situations where direct dialogue is difficult, emotional misunderstandings can reduce the efficiency and quality of communication. Furthermore, emotional misunderstandings can negatively impact relationships with family and close friends. It is necessary to address this challenge and provide means to achieve smoother and more harmonious communication.

[0576] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0577] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for predicting misunderstandings between users and presenting appropriate intentions, means for processing user feedback and improving the generative model, means for using an emotion engine that analyzes the user's emotional state, and means for presenting feedback and support to the user in a parental voice based on the analysis results. This enables more accurate and harmonious communication while taking into account the user's emotional state.

[0578] "Past communication information" refers to data related to conversations and messages previously exchanged between users.

[0579] "Analysis of communication information" refers to the process of analyzing collected communication data in order to understand its content and context.

[0580] A "generative model" refers to an algorithm or program that uses artificial intelligence technology to analyze data and generate appropriate output.

[0581] "Predicting misunderstandings between users" refers to identifying potential misunderstandings and misperceptions that may arise in communication between users.

[0582] "Presenting the correct intent" means explicitly providing information so that users can understand the intended message without misunderstanding.

[0583] "Feedback processing" refers to the process of collecting responses and opinions provided by users and using them to improve systems and models.

[0584] An "emotion engine" refers to software or a device that detects and evaluates emotions from a user's facial expressions, tone of voice, and other factors.

[0585] "Providing feedback and support" means providing users with necessary information in a timely manner based on the analysis results and supporting smooth communication.

[0586] "Parental voice" refers to outputting a warm voice that gives the user a sense of security and familiarity.

[0587] The system implementing this invention is particularly envisioned as a consumer robot for use in the home. The server utilizes a generative model that understands context by collecting and analyzing the user's past communication information. Based on the results of this analysis, it predicts potential misunderstandings between users and presents appropriate intentions. The server also processes user feedback and continuously improves the generative model.

[0588] The terminal, or consumer robot, is equipped with the ability to collect the user's facial expressions and tone of voice in real time and acquire emotional data through an emotion engine. This emotional data is comprehensively evaluated in conjunction with the analysis results on the server, and feedback and support are presented to the user in a parental voice. This kind of feedback facilitates smooth communication with the user and minimizes misunderstandings.

[0589] As a concrete example of implementing this system, consider a scenario where a robot with home helper functionality detects that a child is struggling with their homework. The robot would then reduce the child's stress by suggesting to the user, i.e., the child, via voice, "Shall we take a break? Or do you need any help?"

[0590] This invention enables more accurate and harmonious communication that takes into account the user's emotional state through user-friendly feedback. The system design utilizes software such as OpenCV (image processing), TensorFlow (generative AI modeling), and NLTK (natural language processing), and is implemented on a consumer robot platform.

[0591] An example of a prompt message would be, "The user's stress level is high. Please provide some conversational tips to help them relax."

[0592] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0593] Step 1:

[0594] The device uses a camera and microphone to collect data on the user's facial expressions and voice. The input is real-time facial images and audio from the user, and the output is emotion data based on these. An emotion engine is used to process the data and analyze emotions from changes in facial expressions and voice tone. Specifically, OpenCV is used to extract facial feature points and audio data is analyzed to identify the emotional state.

[0595] Step 2:

[0596] The device transmits collected emotional data to the server while protecting privacy. The input is anonymized emotional data, and the output is a secure data transmission to the server. The data is transformed into a form that cannot be linked to a specific user and transmitted securely using protocols such as HTTPS. Specifically, the data is encrypted before being transmitted to the server using secure network protocols for medical or educational purposes.

[0597] Step 3:

[0598] The server integrates received sentiment data with past communication information and performs analysis using a generative AI model. The input is sentiment data and past communication information, and the output is the prediction of misunderstandings between users and the proposed intent. The generative AI model understands the context and performs data calculations to formulate the optimal intent for the user. Specifically, it uses TensorFlow for natural language processing and NLTK to reinforce the sentiment context.

[0599] Step 4:

[0600] The server sends the generated intent and feedback information to the terminal. The input is the communication strategy generated within the server, and the output is the information transmitted to the terminal. The processing includes practical feedback provided by a generative AI model based on a specific scenario. Specifically, the server formalizes the feedback statement, forms an appropriate prompt statement, and then sends it to the terminal.

[0601] Step 5:

[0602] The terminal presents feedback received from the server to the user both audibly and visually. The input is the generated intent sent from the server, and the output is the presentation of feedback to the user. Using speech synthesis technology, the feedback is provided in a friendly voice. Specifically, a voice synthesizer is used to convey information to the user in a gentle, questioning manner.

[0603] Through these steps, users receive feedback and support that reflects their emotions, enabling smoother communication.

[0604] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0605] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0606] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0607] [Fourth Embodiment]

[0608] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0609] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0610] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0611] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0612] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0613] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0614] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0615] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0616] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0617] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0618] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0619] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0620] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0621] This invention aims to realize a system for understanding the context of communication and preventing misunderstandings. The specific configuration and operation of this system are described below.

[0622] This system consists of three main elements: the user's terminal, a central server, and an AI-generated model. To begin using the system, the user must first log in by operating the terminal. The terminal extracts past communication history from its local storage and collects data to the extent agreed upon by the user. The collected data is anonymized for privacy protection and then sent to the server.

[0623] The server analyzes the received data and uses an AI generative model to analyze the context of the conversation. This process identifies points where misunderstandings are likely to occur from past conversation history. Based on the analysis results, the server predicts misunderstandings between users and provides interpretations to understand the correct intentions. These analysis results are sent back to the terminal and presented to the user in an easy-to-understand manner.

[0624] As a concrete example, suppose that in a project meeting held within a company, users A and B have different interpretations. The application on the terminal sends data to the server from past discussion records. Based on this information, the server predicts misunderstandings and uses an AI generative model to analyze and present the accurate intent. As a result, users A and B can proceed with the project based on a shared understanding.

[0625] This system allows users to prevent misunderstandings that may arise during communication and enables smooth transmission of intentions. The system also features a feedback processing function, which continuously improves the AI ​​generation model based on user feedback, providing more accurate and efficient analysis.

[0626] The following describes the processing flow.

[0627] Step 1:

[0628] The user launches the application and logs into the system. With consent, the device collects past communication history data from local storage.

[0629] Step 2:

[0630] The communication data collected by the device is anonymized for privacy protection and sent to the server using a secure protocol.

[0631] Step 3:

[0632] The server receives data, saves it to the database, and prepares the dataset. The data is organized and formatted into an easily accessible format.

[0633] Step 4:

[0634] The server uses an AI generative model to analyze data and understand the context of conversations between users. The analysis identifies the conversation's topic, emotions, and intentions.

[0635] Step 5:

[0636] The server analyzes the results to predict potential misunderstandings between users. It identifies the source of these misunderstandings and detects potential discrepancies in intent.

[0637] Step 6:

[0638] To prevent misunderstandings, the server reinterprets the intended meaning and sends it to the terminal. A selection of appropriate intentions and interpretations is then generated.

[0639] Step 7:

[0640] The analysis results received by the terminal are displayed to the user through the user interface in a format that is intuitively understandable to the user.

[0641] Step 8:

[0642] Based on the information provided by the user, the system adjusts communication to facilitate more effective dialogue.

[0643] Step 9:

[0644] Users provide feedback on their usage experience and the AI's intent analysis. This feedback is collected by the device and sent to the server.

[0645] Step 10:

[0646] The server analyzes the feedback it receives and uses it to improve the AI-generated model. Based on the feedback, the model is trained and its parameters are adjusted.

[0647] (Example 1)

[0648] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0649] In today's information and communication environment, communication between users takes place through a variety of means, but frequent misunderstandings hinder smooth communication. Furthermore, advanced data analysis and privacy protection are required to prevent these misunderstandings and accurately convey users' intentions. This invention aims to address these challenges and improve communication between users.

[0650] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0651] In this invention, the server includes means for collecting past communication information, means for anonymizing the communication information and protecting privacy, and means for analyzing the anonymized data and understanding contextual information using a generative AI model. This prevents misunderstandings between users and enables accurate communication of intentions.

[0652] "Past communication information" refers to digital records of previous communications such as messages and phone calls made by the user.

[0653] "Anonymization" is the process of removing or transforming personally identifiable information so that the data does not infringe on privacy.

[0654] A "secure protocol" is a set of communication rules or standards for ensuring the secure transmission of data over the internet or a network.

[0655] A "generative AI model" refers to an artificial intelligence algorithm that takes data as input and performs analysis or generation based on a specific purpose.

[0656] "Contextual information" refers to all relevant information that indicates the background, situation, and intentions behind statements and actions in communication.

[0657] A "user interface" is a collection of screens and display elements that a user uses when interacting with a system.

[0658] This system is implemented as a network-based platform using user terminals, a central server, and a generative AI model. Users first log in to the system using their own terminals. After logging in, the terminals collect past communication information from local data storage to the extent agreed upon by the user. Specifically, they filter and retrieve communication history for a specified period from email clients and messaging applications on the terminals.

[0659] The device anonymizes the collected data. The anonymization algorithm is designed to protect user privacy by removing or transforming personally identifiable information. During this process, the data is transmitted to the server using a strong, secure protocol.

[0660] The server stores the received data in a pre-configured database and applies a generative AI model to analyze the contextual information. The generative AI model utilizes natural language processing techniques to understand the intentions and emotions behind the conversation and identify areas where misunderstandings may occur between users. This analysis makes it possible to predict mismatches in intent between users.

[0661] The analysis results are sent back from the server to the user's terminal and presented to the user through the terminal's user interface. At this time, the analyzed information is provided in an easy-to-understand manner through graphical displays and notifications, allowing the user to receive information based on the analysis.

[0662] As a concrete example, consider a scenario where past communication data from a project team within a company is analyzed. The terminal application sends a prompt message to the AI ​​model stating, "Analyze the intent behind the meeting agenda," and the server visualizes the analysis results for the user. This supports the efficient progress of the project.

[0663] An example of a prompt message might be, "Use an AI model to analyze the likelihood of misunderstandings between User A and User B based on their past discussion history."

[0664] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0665] Step 1:

[0666] The user logs into the system using their own device. The device receives user authentication information (e.g., user ID and password) as input. Upon successful login, a user authentication token is generated as output, and session information is established.

[0667] Step 2:

[0668] The device collects past communication information related to logged-in users. The input is communication history obtained from the device's email client and messaging apps. For data processing, messages and emails from a specified period are filtered and collected. The output is a list of the relevant historical data.

[0669] Step 3:

[0670] The terminal anonymizes the collected communication information. The communication history data obtained in step 2 is used as input. An algorithm is applied to mask the data by removing personally identifiable information. Anonymized history data is generated as output.

[0671] Step 4:

[0672] The terminal sends anonymized data to the server using a secure protocol. Anonymized data and communication protocol information are used as input. The output is a status indicating successful transmission.

[0673] Step 5:

[0674] The server analyzes the received data and uses a generative AI model to understand contextual information. Anonymized data is supplied to the server as input. The data processing involves the AI ​​model evaluating the data and analyzing the intent and emotions of the conversation. The output is the result of the contextual analysis.

[0675] Step 6:

[0676] Based on the analysis results, the server predicts the possibility of misunderstandings between users and presents the appropriate intent. The analysis results generated in step 5 are used as input. As data processing, points where misunderstandings may occur are identified. As output, the interpretation of intent is generated.

[0677] Step 7:

[0678] The server returns the interpretation result of the generated intent to the user's terminal via the user interface. The inputs used are the interpretation result of the intent and the user ID. The output is the analysis result displayed on the user's terminal.

[0679] Step 8:

[0680] The user reviews the presented analysis results and provides feedback. As input, the user's evaluation and comments are entered on the device. As output, the feedback data is sent to the server.

[0681] Step 9:

[0682] The server processes user feedback and improves the generated AI model. Feedback data is used as input. The AI ​​model's learning process is updated as data computation, and adjustments are made to improve the model's accuracy. The improved AI model is obtained as output.

[0683] (Application Example 1)

[0684] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0685] In modern commercial facilities and service industries, communication among employees and with customers is becoming increasingly complex, leading to challenges such as misunderstandings and miscommunication. Such misunderstandings can result in decreased operational efficiency and customer satisfaction. Furthermore, protecting privacy is crucial when using communication history.

[0686] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0687] In this invention, the server includes means for collecting past communication content and dialogue history, means for analyzing the communication content to understand the context and using a generative model to predict responses in real time, and means for predicting misunderstandings between users or between users and customers, presenting appropriate intentions, and supporting the optimal response method. This prevents misunderstandings in communication and enables smoother and more effective information transmission.

[0688] "Past communication content and dialogue history" refers to the record of all previous messages and conversations that took place between users or between users and customers.

[0689] A "generative model" refers to an algorithm that uses artificial intelligence technology to analyze communication content, understand the context, and predict responses.

[0690] "Methods for predicting misunderstandings between users and customers" refers to technologies that detect potential misunderstandings and miscommunications that may occur during communication, based on past data and contextual analysis.

[0691] "A means of presenting appropriate intent and supporting the optimal response method" refers to a system that clarifies intent based on analysis results and guides users in selecting the most effective response.

[0692] "Information equipment" refers to all electronic devices used by users, such as smartphones, tablets, and computers.

[0693] "Means of anonymizing data to protect privacy and applying stronger encryption techniques" refers to the process of processing data so that individuals cannot be identified and then using encryption technology to further enhance data security.

[0694] The system requires user communication devices, a central information processing unit, and software including an artificial intelligence generative model. The user's communication devices, such as smartphones or tablets, are responsible for collecting past communication content and conversation history. This data is collected with the user's consent, anonymized for privacy protection, and further protected by strong encryption methods such as 256-bit AES.

[0695] When data is sent to a central information processing unit, a generative AI model activates, analyzing the communication content to understand the context and predicting responses in real time. This model uses AI libraries such as TensorFlow and leverages natural language processing techniques to predict misunderstandings. The analysis results are presented to the user's communication device via a user interface, assisting in facilitating communication between users or between users and customers.

[0696] As a concrete example, when a new staff member in a store answers a customer's question, this system provides the optimal response in real time based on past conversations. The analysis results displayed on the staff member's communication device help to present the appropriate intent, improving the accuracy and efficiency of responses.

[0697] An example of a prompt message to input into a generative AI model would be, "Predict the best response based on past conversation history. Example: Based on past responses to 'Do you have it in stock?'" By using this prompt message, the model can focus on the issue it needs to address and derive a highly accurate response.

[0698] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0699] Step 1:

[0700] The user's device begins collecting communication history. Within the scope agreed upon by the user, past conversation content and interaction history are selected, anonymized, and encrypted. The input is the user's past communication content, and the output is anonymized and encrypted data.

[0701] Step 2:

[0702] The collected data is sent to the server. The server decrypts the received encrypted data and extracts the information necessary for analysis. The input is anonymized encrypted data, and the output is communication data in an analyzable format.

[0703] Step 3:

[0704] The server processes analyzable data using a generative AI model. In this process, it analyzes the context of the communication, past patterns, and common points of misunderstanding. The input is communication data in an analyzable format, and the output is predictions of misunderstandings and suggestions for appropriate responses.

[0705] Step 4:

[0706] The analysis results are sent back to the terminal via the user interface. The terminal displays the results in a format that is easy for the user to understand. The input is a prediction of misunderstandings and a suggestion of appropriate responses, while the output is user-oriented information displayed on the terminal.

[0707] Step 5:

[0708] Users can choose actions based on the information presented and provide feedback as needed. This feedback is used to improve the generative AI model, increasing the accuracy of subsequent analyses. The input is user feedback, and the output is the improved result of analyzing the feedback and reflecting it in the generative AI model.

[0709] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0710] This invention is a system that supports communication between users, and through a new configuration that includes an emotion engine, it can also reflect the user's emotions in the analysis results. The system mainly consists of a user terminal, a server, and a generative model equipped with an emotion engine.

[0711] When a user starts the system, the terminal performs a login process, collects past communication history data, anonymizes it with privacy protection measures, and sends it to the server. At this time, the terminal collects the user's emotions in real time, using an emotion engine to obtain emotional data from facial expressions and linguistic indicators.

[0712] The server integrates and analyzes the received communication data and sentiment data. In this analysis, a generative model understands the context, and the sentiment engine interprets the emotions. As a result of combining the data, the server understands the user's emotional state and reinterprets appropriate intentions based on that state.

[0713] Based on the analysis results, the server predicts communication misunderstandings and has the means to dynamically present intentions according to emotions. The terminal receives the analysis data obtained from the server and presents it in a format that is easy for the user to understand. This system provides more flexible and adaptive communication support by taking the user's emotional state into consideration.

[0714] As a concrete example, consider a scenario where members of a project team in remote locations are holding an online meeting. If the presenter is nervous, the emotion engine detects this, and the server clarifies the presenter's intentions to avoid misunderstandings, helping the audience accurately understand the presenter's intent. In this way, considering emotions improves the quality of communication.

[0715] This system allows users to objectively support their emotionally charged communication, thereby reducing misunderstandings. Furthermore, the accuracy improves over time through continuous refinement of the generative model using feedback.

[0716] The following describes the processing flow.

[0717] Step 1:

[0718] The user logs into the system using their device. The device collects past communication history, anonymizes the data to protect privacy, and then sends it to the server.

[0719] Step 2:

[0720] The device acquires real-time emotional data from the user. This utilizes an emotion engine that identifies emotions through facial expression analysis and voice tone analysis.

[0721] Step 3:

[0722] The server integrates received communication history data with real-time sentiment data. This data is centrally managed and prepared for analysis.

[0723] Step 4:

[0724] The server uses generative models to analyze the context of the communication content. Simultaneously, the emotion engine analyzes emotion data to identify the user's emotional state.

[0725] Step 5:

[0726] Based on contextual information and emotional states analyzed by the server, it predicts areas where misunderstandings are likely to occur. Considering emotional data enables more accurate intent analysis.

[0727] Step 6:

[0728] The server sends the results of the intent analysis to the terminal. The intent is dynamically adjusted based on the emotional situation and presented appropriately to the user.

[0729] Step 7:

[0730] The device displays the analysis results through an intuitive user interface. This allows users to easily understand their own emotions and intentions based on the content of the conversation.

[0731] Step 8:

[0732] Based on the information provided by the user, we will implement measures to improve communication. Users will provide feedback as needed, contributing to system improvements.

[0733] Step 9:

[0734] The server collects feedback and improves the generative model and sentiment engine. Based on the feedback, the model is retrained to improve the accuracy of the analysis.

[0735] (Example 2)

[0736] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0737] In user-to-user communication, misunderstandings of emotions and unclear intentions can hinder the smooth transmission of information. Furthermore, protecting privacy is difficult, and there is a lack of mechanisms to effectively utilize feedback and continuously improve generative models. These challenges need to be addressed to achieve emotionally sensitive communication.

[0738] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0739] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for detecting the emotional state of the information sender and presenting appropriate intentions based on that emotion, means for acquiring emotional data from the information sender's facial expressions and voice and analyzing that information with an emotion engine, means for presenting the analysis results to the information receiver in a visualized form, and means for processing user feedback and continuously improving the generative model. This enables the provision of flexible and intuitive communication that responds to emotions.

[0740] "Past communication information" refers to all messages and call history previously exchanged between users, and includes communication history data.

[0741] "Generative modeling" refers to the process of using machine learning models to understand the context and intent behind information, utilizing artificial intelligence technology.

[0742] "The emotional state of the information provider" refers to the mental and emotional state of the user who is disseminating the information, and this includes emotions such as joy, sadness, and anger.

[0743] An "emotion engine" refers to software or hardware technology that analyzes user data such as facial expressions and voice to determine emotions.

[0744] "Feedback" refers to opinions, reactions, or data collected from users, which are used to improve systems and generative models.

[0745] "Anonymization" refers to the process of removing or transforming personally identifiable elements from data in order to protect privacy.

[0746] "Visualizing analysis results" means presenting the results of data analysis in a visually easy-to-understand format, providing information in a way that is easy for users to comprehend.

[0747] This invention is a system that supports user communication and aims to reduce misunderstandings while taking user emotions into consideration. The system mainly consists of a server, a terminal, and a generative model equipped with an emotion engine.

[0748] The device first collects past communication information when the user logs in. This includes messages and call history. The collected information is anonymized to protect privacy and then sent to the server. Anonymization is a process that removes or transforms personally identifiable information.

[0749] The device also collects emotional data in real time. The device works in conjunction with the emotion engine to analyze the user's facial expressions and voice, and obtains emotional data from them. The emotion engine uses a specific algorithm to determine the user's emotional state from the acquired data.

[0750] The server receives communication information and sentiment data transmitted from the terminal. A generative AI model is used to analyze this data and present appropriate intentions based on the user's emotional state. This generative AI model simultaneously performs contextual understanding and sentiment interpretation, dynamically reinterpreting intentions to reduce misunderstandings.

[0751] The server then returns the analysis results to the terminal, which presents them to the user through an intuitive interface. This system can continuously improve the generated AI model using user feedback. Based on the feedback, the system evolves to improve the user experience.

[0752] As a concrete example, consider a scenario where a project team is holding an online meeting. If the presenter is nervous, the device detects the presenter's voice and facial expressions and analyzes their emotions via the server. Based on the analyzed information, the server provides the audience with information to clarify the presenter's intentions. The presenter can also use this information to give their presentation with greater confidence.

[0753] An example of a prompt might be, "Please tell me how the server detected the tension that user A showed during the online meeting and how it conveyed that to the audience." In this way, the system supports tackling realistic challenges.

[0754] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0755] Step 1:

[0756] When a user starts the system, the terminal displays a login screen where the user enters their authentication information. After the authentication information is entered, the terminal sends it to the server, and the login process is executed. This initiates the user's session. The inputs are a user ID and password, and the output is a message indicating login success or failure.

[0757] Step 2:

[0758] Once login is complete, the device collects past communication information. This information includes text messages and call history. The device encrypts and transforms this data to anonymize it. The input is raw communication data, and the output is anonymized communication data. The anonymized data is sent to the server.

[0759] Step 3:

[0760] The device also activates its camera and microphone to collect user emotion data. The collected facial and audio data is analyzed using an emotion engine and output as the user's real-time emotional state. The input for this step is facial expressions and audio, and the output is the analyzed emotion data.

[0761] Step 4:

[0762] The server receives anonymized communication data and sentiment datasets sent from the terminal. The server analyzes this data using a generating AI model and sentiment engine. This analysis enables an understanding of the user's context and emotional state. Raw data is the input, and the analysis results are the output.

[0763] Step 5:

[0764] The server uses a generative AI model to reinterpret the intent behind communication based on the analyzed emotional state and generates revised versions to avoid misunderstandings. In this step, the analysis results are used as input, and the revised intent is generated as output.

[0765] Step 6:

[0766] The server sends suggested revisions to the generated intent to the terminal, which then displays them. The user can then proceed with the conversation with confidence based on this information. The terminal visualizes the received data through an appropriate user interface and presents it to the user. The input is the suggested revision data from the server, and the output is the display on the user interface.

[0767] Step 7:

[0768] User feedback is collected through the device and sent to the server. The server uses this feedback to continuously improve the generated AI model. The input to this process is user feedback, and the output is an updated, improved model.

[0769] (Application Example 2)

[0770] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0771] In modern society, especially in family and daily communication, users' emotions can be a source of misunderstanding. In situations where direct dialogue is difficult, emotional misunderstandings can reduce the efficiency and quality of communication. Furthermore, emotional misunderstandings can negatively impact relationships with family and close friends. It is necessary to address this challenge and provide means to achieve smoother and more harmonious communication.

[0772] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0773] In this invention, the server includes means for collecting past communication information, means for using a generative model that analyzes the communication information to understand the context, means for predicting misunderstandings between users and presenting appropriate intentions, means for processing user feedback and improving the generative model, means for using an emotion engine that analyzes the user's emotional state, and means for presenting feedback and support to the user in a parental voice based on the analysis results. This enables more accurate and harmonious communication while taking into account the user's emotional state.

[0774] "Past communication information" refers to data related to conversations and messages previously exchanged between users.

[0775] "Analysis of communication information" refers to the process of analyzing collected communication data in order to understand its content and context.

[0776] A "generative model" refers to an algorithm or program that uses artificial intelligence technology to analyze data and generate appropriate output.

[0777] "Predicting misunderstandings between users" refers to identifying potential misunderstandings and misperceptions that may arise in communication between users.

[0778] "Presenting the correct intent" means explicitly providing information so that users can understand the intended message without misunderstanding.

[0779] "Feedback processing" refers to the process of collecting responses and opinions provided by users and using them to improve systems and models.

[0780] An "emotion engine" refers to software or a device that detects and evaluates emotions from a user's facial expressions, tone of voice, and other factors.

[0781] "Providing feedback and support" means providing users with necessary information in a timely manner based on the analysis results and supporting smooth communication.

[0782] "Parental voice" refers to outputting a warm voice that gives the user a sense of security and familiarity.

[0783] The system implementing this invention is particularly envisioned as a consumer robot for use in the home. The server utilizes a generative model that understands context by collecting and analyzing the user's past communication information. Based on the results of this analysis, it predicts potential misunderstandings between users and presents appropriate intentions. The server also processes user feedback and continuously improves the generative model.

[0784] The terminal, or consumer robot, is equipped with the ability to collect the user's facial expressions and tone of voice in real time and acquire emotional data through an emotion engine. This emotional data is comprehensively evaluated in conjunction with the analysis results on the server, and feedback and support are presented to the user in a parental voice. This kind of feedback facilitates smooth communication with the user and minimizes misunderstandings.

[0785] As a concrete example of implementing this system, consider a scenario where a robot with home helper functionality detects that a child is struggling with their homework. The robot would then reduce the child's stress by suggesting to the user, i.e., the child, via voice, "Shall we take a break? Or do you need any help?"

[0786] This invention enables more accurate and harmonious communication that takes into account the user's emotional state through user-friendly feedback. The system design utilizes software such as OpenCV (image processing), TensorFlow (generative AI modeling), and NLTK (natural language processing), and is implemented on a consumer robot platform.

[0787] An example of a prompt message would be, "The user's stress level is high. Please provide some conversational tips to help them relax."

[0788] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0789] Step 1:

[0790] The device uses a camera and microphone to collect data on the user's facial expressions and voice. The input is real-time facial images and audio from the user, and the output is emotion data based on these. An emotion engine is used to process the data and analyze emotions from changes in facial expressions and voice tone. Specifically, OpenCV is used to extract facial feature points and audio data is analyzed to identify the emotional state.

[0791] Step 2:

[0792] The device transmits collected emotional data to the server while protecting privacy. The input is anonymized emotional data, and the output is a secure data transmission to the server. The data is transformed into a form that cannot be linked to a specific user and transmitted securely using protocols such as HTTPS. Specifically, the data is encrypted before being transmitted to the server using secure network protocols for medical or educational purposes.

[0793] Step 3:

[0794] The server integrates received sentiment data with past communication information and performs analysis using a generative AI model. The input is sentiment data and past communication information, and the output is the prediction of misunderstandings between users and the proposed intent. The generative AI model understands the context and performs data calculations to formulate the optimal intent for the user. Specifically, it uses TensorFlow for natural language processing and NLTK to reinforce the sentiment context.

[0795] Step 4:

[0796] The server sends the generated intent and feedback information to the terminal. The input is the communication strategy generated within the server, and the output is the information transmitted to the terminal. The processing includes practical feedback provided by a generative AI model based on a specific scenario. Specifically, the server formalizes the feedback statement, forms an appropriate prompt statement, and then sends it to the terminal.

[0797] Step 5:

[0798] The terminal presents feedback received from the server to the user both audibly and visually. The input is the generated intent sent from the server, and the output is the presentation of feedback to the user. Using speech synthesis technology, the feedback is provided in a friendly voice. Specifically, a voice synthesizer is used to convey information to the user in a gentle, questioning manner.

[0799] Through these steps, users receive feedback and support that reflects their emotions, enabling smoother communication.

[0800] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0801] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0802] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0803] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0804] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0805] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0806] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0807] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0808] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0809] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0810] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0811] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0812] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0813] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0814] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0815] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0816] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0817] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0818] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0819] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0820] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0821] The following is further disclosed regarding the embodiments described above.

[0822] (Claim 1)

[0823] Means for collecting past communication content,

[0824] A means of using a generative model to analyze the content of the communication and understand the context,

[0825] A means to anticipate misunderstandings among users and to present appropriate intent,

[0826] A means of processing user feedback and improving the generative model,

[0827] A system that includes this.

[0828] (Claim 2)

[0829] The system according to claim 1, comprising means for anonymizing data in order to protect user privacy when collecting communication content.

[0830] (Claim 3)

[0831] The system according to claim 1, comprising means for presenting the analyzed contextual information through an intuitive user interface.

[0832] "Example 1"

[0833] (Claim 1)

[0834] Means for collecting past communication information,

[0835] A means of anonymizing communication information and protecting privacy,

[0836] A means of analyzing anonymized data and understanding contextual information using a generative AI model,

[0837] A means to anticipate misunderstandings between users and to present appropriate intent,

[0838] A means of processing user feedback and improving the generated AI model,

[0839] A means of presenting the analyzed information in a user interface,

[0840] A system that includes this.

[0841] (Claim 2)

[0842] The system according to claim 1, characterized in that communication information is transmitted through a secure protocol.

[0843] (Claim 3)

[0844] The system according to claim 1, characterized in that the analysis results presented via the user interface have a function to accept user feedback.

[0845] "Application Example 1"

[0846] (Claim 1)

[0847] Means for collecting past communication content and dialogue history,

[0848] A means of using a generative model that analyzes the content of the communication to understand the context and predicts the response in real time,

[0849] A means to predict misunderstandings between users or between users and customers, to present appropriate intent, and to support the optimal response method.

[0850] A means of processing user feedback and continuously improving the generative model,

[0851] A means of presenting the analysis results to the staff's information devices via a user interface,

[0852] A system that includes this.

[0853] (Claim 2)

[0854] The system according to claim 1, further comprising means for anonymizing data and applying strong encryption methods in order to protect user privacy when collecting communication content.

[0855] (Claim 3)

[0856] The system according to claim 1, comprising means for presenting the analyzed contextual information to an information device in real time in a format that staff can intuitively understand.

[0857] "Example 2 of combining an emotion engine"

[0858] (Claim 1)

[0859] Means for collecting past communication information,

[0860] A means of using a generative model to analyze the communication information and understand the context,

[0861] A means of detecting the emotional state of information providers and presenting appropriate intentions based on those emotions,

[0862] A method for acquiring emotional data from the facial expressions and voice of information providers and analyzing that information using an emotion engine,

[0863] A means of presenting the analysis results to the information recipient in a visualized form,

[0864] A means of processing user feedback and continuously improving the generative model,

[0865] A system that includes this.

[0866] (Claim 2)

[0867] The system according to claim 1, comprising means for anonymizing information in order to protect user privacy when collecting communication information.

[0868] (Claim 3)

[0869] The system according to claim 1, comprising means for displaying the analyzed contextual information via an intuitive information presentation means.

[0870] "Application example 2 when combining with an emotional engine"

[0871] (Claim 1)

[0872] Means for collecting past communication information,

[0873] Methods that use generative models to analyze communication information and understand the context,

[0874] A means to anticipate misunderstandings among users and to present appropriate intent,

[0875] A means of processing user feedback and improving the generative model,

[0876] A method using an emotion engine to analyze the user's emotional state,

[0877] Based on the analysis results, a means of presenting feedback and support to the user in a parental voice,

[0878] A system that includes this.

[0879] (Claim 2)

[0880] The system according to claim 1, comprising means for anonymizing data in order to protect the user's personal information when collecting communication information.

[0881] (Claim 3)

[0882] The system according to claim 1, comprising means for presenting analyzed contextual information and sentiment information through an easy-to-understand user interface. [Explanation of symbols]

[0883] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting past communication content, A means of using a generative model to analyze the content of the communication and understand the context, A means to anticipate misunderstandings among users and to present appropriate intent, A means of processing user feedback and improving the generative model, A system that includes this.

2. The system according to claim 1, comprising means for anonymizing data in order to protect user privacy when collecting communication content.

3. The system according to claim 1, comprising means for presenting the analyzed contextual information through an intuitive user interface.

Images