system

Abstract

We provide the system. [Solution] A means of receiving message data, A means for analyzing the aforementioned message data and evaluating the possibility of misunderstanding of emotional components and content, A means for generating feedback and response proposals based on the analysis results, A means of presenting the generated feedback and suggested responses to the user, A means of sending reply data selected or entered by the user to the other party, A system that includes this.

Description

Technical Field

[0005] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

[0004]

Problems to be Solved by the Invention

[0005] When troubles, misunderstandings, especially emotional conflicts occur in human relationships, direct communication between the parties may exacerbate the problem. Conventional messaging applications lack a mechanism to arbitrate emotional arousal and misunderstandings, and as a result, they cause damage to personal relationships. There is a need for a neutral and effective method to solve such problems and promote smooth and constructive communication.

Means for Solving the Problems

[0005] This invention provides a system that uses AI within a messaging application to analyze the emotional components and potential for misunderstanding in messages, and to provide appropriate feedback and suggested replies. Specifically, it includes means for receiving message data and analyzing its content, means for presenting feedback and suggested replies generated based on the analysis results, and means for accepting and sending replies selected or entered by the user. This allows users to avoid direct conflict and communicate calmly with others from a neutral perspective. Furthermore, based on the analysis results, the system also provides necessary warning messages and mediation functions (on / off settings), enabling user-adapted support.

[0006] "Message data" refers to text and character information sent and received by users, and serves as a medium for communication.

[0007] "Analysis" refers to the process by which AI or programs analyze message data and evaluate its content, emotional tone, and whether there are any potential misunderstandings.

[0008] "Feedback" refers to information and advice provided to users based on analysis results, intended to facilitate or correct communication with others.

[0009] "Suggested replies" refer to the content of recommended responses generated by AI that users can use, helping them to engage in calm and constructive communication.

[0010] A "user" refers to a person who uses this system to send and receive messages, and is the entity that operates the system's functions.

[0011] "Sending" refers to the process of delivering a message or reply data created or selected by the user to the recipient's device.

[0012] "Warning messages" are warnings and additional advice provided to users based on AI analysis, intended to avoid misunderstandings and emotional conflicts.

[0013] The "arbitration function" is a feature in which AI acts as a neutral mediator in message exchanges, supporting users in continuing the conversation calmly. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of a data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0018] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is implemented as an AI arbitration system integrated into a messaging application. The system consists of a user's terminal, a central server, and an AI analysis engine. When a user sends a message, the terminal transmits this message data to the central server.

[0036] The server passes the received message data to an AI analysis engine, which analyzes the content and emotional tone. The AI ​​analysis engine detects emotional elements and potential misunderstandings within the message and generates a feedback message based on that. This process mitigates direct emotional conflicts that could exacerbate the problem.

[0037] The device presents the user with feedback and analysis results provided by the server. Furthermore, it suggests recommended replies generated by the AI ​​analysis engine, allowing the user to select the optimal reply or create their own.

[0038] For example, if user A sends an emotionally charged message to user B, the AI ​​analysis engine analyzes the message and determines that it contains feelings of anger or frustration. The server then provides user B with feedback that includes ways to alleviate these feelings and suggests a response that encourages calm dialogue. In this way, direct misunderstandings and emotional conflicts are prevented, and neutral and constructive communication is achieved.

[0039] This invention enables users to control their emotions during the communication process, leading to more effective and positive dialogue. The mediation function can be turned on or off at the user's discretion, allowing for flexible operation as needed.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user creates message data by typing a message and pressing the send button. The terminal retrieves this message data and instructs the central server to send it.

[0043] Step 2:

[0044] The server forwards the message data received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the text data and performs natural language processing to extract emotional tones and specific keywords.

[0045] Step 3:

[0046] The AI ​​analysis engine evaluates the message based on its analysis results, assessing whether there are emotional elements or potential misunderstandings, and generates corresponding feedback. In some cases, it prepares cautionary messages to alleviate tension.

[0047] Step 4:

[0048] The server receives the results from the AI ​​analysis engine and sends them to the user's device as a feedback message. If suggested replies exist, they are also provided.

[0049] Step 5:

[0050] The terminal displays feedback and suggested replies received from the server to the user. Based on this information, the user decides whether to select a suggested reply or to type a new message themselves.

[0051] Step 6:

[0052] Once the user decides on their reply and presses the send button, the device sends this reply data back to the server and begins the process of delivering it to the recipient's device.

[0053] Step 7:

[0054] The server receives the sent reply data, checks if the content requires re-analysis, and then sends the data to the recipient's device. This ensures that smooth communication between users continues.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] In recent years, with the increased use of messaging applications, emotional conflicts and misunderstandings have become more frequent. Such misunderstandings and emotional conflicts can hinder smooth communication and lead to a deterioration of interpersonal relationships. However, existing systems do not adequately analyze emotional elements or provide sufficient feedback, making it difficult to effectively resolve these communication problems. Against this backdrop, there is a need for a system that can prevent emotional conflicts and misunderstandings among users and promote constructive dialogue.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for receiving message information, means for analyzing the message information and evaluating the emotional elements and the possibility of misunderstanding the content, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to effectively analyze the emotional elements and the possibility of misunderstanding contained in the message and provide appropriate feedback and response suggestions to the user.

[0060] "Message information" refers to the content of communications in text or other data formats sent and received between users.

[0061] A "generative artificial intelligence model" is a type of AI technology used in natural language processing and other applications, referring to a machine learning model that aims to generate appropriate output based on input.

[0062] "Emotional elements" refer to indicators used to identify information that indicates human emotions or emotional tone contained within text or messages.

[0063] "Feedback" refers to information provided to users based on the analyzed results, for the purpose of improvement or appropriate action.

[0064] A "suggested reply" refers to a template or guide of responses that are recommended by the user based on AI analysis.

[0065] This invention is specifically implemented as an AI arbitration system integrated into a messaging application. This system consists of a user's terminal, a central server, and an AI analysis engine.

[0066] The terminal converts the messages sent by the user into a data format and transmits them to a central server over the network. The hardware used as a terminal is a common communication device such as a smartphone or computer, with a messaging application installed on it.

[0067] The server passes message information received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using natural language processing techniques. A generative AI model is used for this analysis. Specific examples include open-source natural language processing libraries and commercial AI platforms. The analysis results evaluate the emotional elements and potential for misunderstanding contained in the message.

[0068] Based on the analyzed results, the server sends back feedback and suggested replies generated by the AI ​​analysis engine to the terminal. This feedback includes advice to mitigate emotional conflict and specific suggested replies.

[0069] The device receives feedback from the server and presents it to the user. Based on this feedback, the user can choose the most appropriate response or create their own response based on their own judgment.

[0070] As a concrete example of its operation, if a user sends a message such as "I'm worried because the project is behind schedule," the AI ​​analysis engine will analyze this message and determine that it contains feelings of anxiety or worry. Based on this information, the server can then provide feedback to the user such as "Let's check the progress and think about the next steps."

[0071] The prompt text input to the generative AI model is in the format of, "Analyze the message, identify emotional elements, and generate appropriate feedback." In this way, the present invention provides users with a means to improve the quality of communication.

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The user creates and sends a message using a terminal. The terminal converts this message into a data format and sends it to a central server over the network. The input is the text message entered by the user, and the output is the message data sent over the network. The terminal takes in the raw data from the user, encodes it in the appropriate format, and sends it.

[0075] Step 2:

[0076] The server receives message data from the terminal. Next, the server prepares this data to be passed to the AI ​​analysis engine. The input is the message data received from the terminal, and the output is pre-processed data to be fed into the AI ​​analysis engine. The server verifies the data and prepares it for processing.

[0077] Step 3:

[0078] The server sends message data to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using a generative AI model. In this process, the message is tokenized and sentiment analysis is performed. The input is message data from the server, and the output is sentiment elements and potential misunderstandings as a result of the analysis. The analysis engine analyzes the message and quantifies the sentiment tone and potential misunderstandings.

[0079] Step 4:

[0080] The AI ​​analysis engine generates feedback messages and suggested replies based on the analysis results. This feedback includes advice to facilitate the conversation. The input is the analysis results, and the output is the feedback message and suggested replies. The engine applies a generative AI model to suggest appropriate communication guidance.

[0081] Step 5:

[0082] The server sends feedback messages and suggested replies from the AI ​​analysis engine back to the terminal. The input is feedback data from the analysis engine, and the output is feedback information sent to the terminal. The server formats this information appropriately and delivers it to the terminal.

[0083] Step 6:

[0084] The terminal displays the received feedback message and suggested replies to the user. The user can review the presented feedback and select or create an appropriate reply. The input is the feedback information from the server, and the output is what is displayed to the user. The terminal visually displays the information and assists the user in continuing the interaction.

[0085] (Application Example 1)

[0086] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] In modern businesses and social organizations, communication via electronic messages is widespread, but this process is prone to emotional conflict and misunderstandings. In particular, in emotionally charged situations, inappropriate expressions and misinterpretations can exacerbate problems. This can lead to deterioration of interpersonal relationships and friction within organizations, hindering efficient work performance. Therefore, systems are needed to prevent these issues from occurring.

[0088] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0089] In this invention, the server includes means for acquiring information, means for analyzing the information and evaluating the possibility of interpreting emotional elements and content, and means for generating response messages and proposed responses. This reduces emotional conflicts and misunderstandings that occur during the communication process, enabling neutral and constructive dialogue.

[0090] "Means of acquiring information" refers to system components that handle the initial input process for receiving messages and data from users and starting appropriate processing.

[0091] "Means for analyzing information and evaluating the possibility of interpreting emotional elements and content" refers to algorithms that analyze received information using natural language processing techniques to identify emotional components and potential misunderstandings contained in a message.

[0092] "Means for generating response messages and proposed responses" refers to system functions that provide users with appropriate feedback and proposed responses to confirm, based on the analysis results.

[0093] "Means of presenting to the user" refers to display methods that visually or audibly notify the user of the generated analysis results and feedback, and facilitate their understanding.

[0094] "Means for forwarding response information selected or entered by the user to the recipient" refers to a communication process that accurately delivers the user's selected reply to the recipient.

[0095] "Means for enabling / disabling neutral functions" refers to an interface that allows users to choose to turn the AI-based sentiment analysis function on or off as needed.

[0096] "Means for evaluating and improving communication safety" refer to system functions that identify potential risk factors in communication and prevent misunderstandings and conflicts between users.

[0097] This invention is a system for realizing an AI arbitration system in messaging applications. It mainly consists of a server, a user terminal, and an AI analysis engine. The server receives messages exchanged between users and extracts information from them. The extracted information is sent to the AI ​​analysis engine. The AI ​​analysis engine uses machine learning libraries such as TENSORFLOW® and PyTorch to perform natural language processing and evaluate the emotional elements of the messages and the possibility of misinterpretation. Based on the analysis results, the server automatically generates response messages and specific response suggestions and presents them to the user.

[0098] Users can view the analysis results through applications on their smartphones or desktop PCs. Here, users are presented with generated response suggestions and can select or edit their own responses as needed. The generated responses are then forwarded back to the recipient via the server.

[0099] As a concrete example, suppose a discussion about the progress of a project in the workplace is taking place via email. If an employee sends a message such as, "This project is a complete waste of time," the AI ​​analysis engine will analyze the negative emotions contained in the message and generate feedback such as, "This message may contain dissatisfaction. Please review it." In this way, it is possible to prevent emotional conflict and misunderstandings with the other party.

[0100] An example of a prompt for a generative AI model would be: "Analyze the sentiment expressed in this message and generate feedback and constructive response suggestions. Message: 'I am dissatisfied with the progress of this project.'" This prompt serves as the basic input for instructing the AI ​​analysis engine to perform sentiment analysis and generate response suggestions. This system allows users to reduce the risks in message communication and maintain healthy dialogue.

[0101] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0102] Step 1:

[0103] The server receives message data from the user's terminal. The input is the message sent by the user, and the output is the message data itself. This data is stored in a clear format in data storage for use in subsequent processing.

[0104] Step 2:

[0105] The server sends the received message data to the AI ​​analysis engine. The input is the message data acquired in step 1, and the output is the analysis result from the AI ​​analysis engine. The AI ​​analysis engine evaluates the emotional elements and potential for misunderstanding of the message using natural language processing techniques. This process includes, for example, sentiment analysis of the text and tone detection.

[0106] Step 3:

[0107] The AI ​​analysis engine generates response suggestions based on the analysis results. The input is the analysis results obtained in step 2, and the output is the generated response suggestions and feedback. Specifically, the generating AI model devises constructive responses based on the prompt text and creates data to provide to the user.

[0108] Step 4:

[0109] The server forwards the generated response and feedback to the user's terminal. The input is the data generated in step 3, and the output is the response and feedback the user receives. The terminal displays this information in the user interface for the user to review.

[0110] Step 5:

[0111] The user reviews the suggested replies presented on the device, selects or edits them as needed, and creates their reply. The input is the information presented on the device in step 4, and the output is the reply data confirmed by the user. Through this interface, the user can choose the reply that best maintains communication.

[0112] Step 6:

[0113] The terminal sends the reply data selected or edited by the user to the server. The input is the user's reply data created in step 5, and the output is the final reply message sent to the server. This reply is then ready to be delivered to the recipient's terminal again via the server.

[0114] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0115] This invention is implemented in a messaging application by combining an AI arbitration system with an emotion engine. This system recognizes the user's emotions and provides feedback based on that emotional state, thereby supporting more human-like and effective communication.

[0116] The system components include a user terminal, a central server, an AI analysis engine, and an emotion engine. When a user sends a message, the terminal receives the message data and also sends the user's facial expression image or voice tone to the emotion engine.

[0117] The emotion engine uses image recognition technology and voice analysis to recognize the user's emotional state in real time. This recognition result is sent to the server along with the message data, where the AI ​​analysis engine analyzes the message content and emotional tone.

[0118] Based on these analysis results, the server generates a feedback message. The generated feedback is adjusted according to the user's emotional state, as determined by the emotion engine. For example, if the user is angry, the feedback will include calming expressions, and if the user is sad, the feedback will emphasize comforting elements.

[0119] The device displays generated feedback and suggested responses to the user. The user can then use this information to select or create an appropriate response.

[0120] For example, if user A receives a message from user B expressing dissatisfaction, the emotion engine will recognize that user A's facial expression indicates anxiety. Based on this information, the AI ​​analysis engine will generate a response suggestion that offers comfort and reassurance to user A, and the device will present it to help user A deal with the situation calmly.

[0121] This system ensures that message exchanges take users' emotions into account, enabling more empathetic communication and helping to avoid unnecessary conflicts.

[0122] The following describes the processing flow.

[0123] Step 1:

[0124] The user types a message into the device and presses the send button. At this time, the device uses its camera or microphone to collect the user's facial expressions or voice.

[0125] Step 2:

[0126] The terminal sends the collected facial expression images or voice data along with the entered message data to the server.

[0127] Step 3:

[0128] The server sends the received message data to the AI ​​analysis engine and facial expression images or voice data to the emotion engine. The AI ​​analysis engine analyzes the message content, and the emotion engine analyzes the user's emotional state in real time.

[0129] Step 4:

[0130] The AI ​​analysis engine evaluates the emotional tone and potential for misunderstanding in a message, while the emotion engine determines the user's emotional state. Based on these results, the server generates appropriate feedback and suggested replies.

[0131] Step 5:

[0132] The server considers the emotion recognition results from the emotion engine and adjusts the feedback and suggested replies. The adjusted content is then sent to the terminal.

[0133] Step 6:

[0134] The device presents the user with tailored feedback and suggested responses. The user then selects or creates a response based on this information.

[0135] Step 7:

[0136] Once the user decides on a reply and sends it, the device sends this reply data to the server, and the normal message transmission process is executed.

[0137] This process allows users to receive feedback that takes their emotional state into account, enabling them to communicate more effectively.

[0138] (Example 2)

[0139] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0140] In the exchange of information involving emotions, misunderstandings of users' intentions and feelings can lead to conflict and unnecessary stress. Traditional messaging systems have difficulty providing feedback that takes into account the emotional state of users, and have limitations in supporting empathetic and effective communication.

[0141] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0142] In this invention, the server includes means for receiving information, means for analyzing the information and emotional data and evaluating the possibility of misunderstandings regarding emotional state and intentions, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to provide appropriate feedback based on the user's emotional state, avoid conflict through neutral mediation, and promote more empathetic communication.

[0143] "Means of receiving information" refers to the function of acquiring data and messages from users in order to transmit them to other system components.

[0144] "Means for analyzing information and emotional data to assess the possibility of misunderstandings regarding emotional states and intentions" refers to a process that identifies users' emotions and communication intentions based on acquired data and assesses the risk of misunderstandings.

[0145] "Means for generating feedback and response suggestions using a generative artificial intelligence model" refers to a mechanism that uses natural language processing technology to automatically generate responses that are appropriate to the user's emotions and intentions.

[0146] "User" refers to an individual or organization that communicates through the system.

[0147] A "processing device" is a computer hardware or software system designed to perform a specific task.

[0148] "Means for setting the enablement or disablement of the neutral arbitration function" refers to a function that allows users to arbitrarily activate or deactivate the arbitration function.

[0149] This invention is an emotion-aware messaging system that reduces misunderstandings of intent and emotions between users and supports smooth communication. The system primarily consists of a terminal, a server, an emotion analysis engine, and a generative AI model.

[0150] The user sends messages through the device. The device is equipped with a camera to capture the user's facial expressions and a microphone to collect their voice tone. The device uses these devices to collect the user's emotional data along with the message data and sends it to the emotion analysis engine.

[0151] The emotion analysis engine uses image processing technologies such as OpenCV and speech analysis technologies such as LibROSA to identify the user's emotional state in real time. This emotion data is sent to a server, which uses a generative AI model to analyze the content of the received message and the user's emotional state.

[0152] The server uses generative AI models such as BERT and GPT to analyze the information and generate appropriate feedback and response suggestions. In doing so, it uses prompts to the generative AI model such as, "If the user is showing signs of anxiety, suggest a comforting feedback message to provide." This generated feedback is tailored to the user's emotional state and presented to the user via the device.

[0153] As a concrete example, suppose user A receives a message from their friend, user B. If the system detects dissatisfaction in the message, user A's device sends an expression of anxiety to an emotion analysis engine. Based on this data, the server performs an AI analysis and provides user A with appropriate feedback. This allows user A to calmly consider their reply and avoid conflict. This system enables more intimate communication between users.

[0154] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0155] Step 1:

[0156] The user composes a message using a messaging application and presses the send button. The input here is the text or voice message created by the user. The device captures this message data and simultaneously uses its built-in camera and microphone to acquire images of the user's facial expressions and voice tone. As a result, data indicating the user's emotions is output.

[0157] Step 2:

[0158] The device sends the acquired message data and emotion data to the emotion analysis engine. The emotion analysis engine uses image processing software and speech analysis software to analyze the user's emotional state in real time based on the input image and audio data. At this stage, the results of the analyzed emotional state are output. Specifically, OpenCV analyzes facial feature points, and LibROSA detects changes in voice tone.

[0159] Step 3:

[0160] The device sends the analyzed sentiment data and message content to the server. The server receives this data and analyzes it using a generative AI model. It receives the sentiment state and message content as input and evaluates the intent and emotional tone from them. The generative AI model (e.g., GPT) also generates a feedback message based on the input information and creates appropriate reply suggestions. The output consists of a feedback message and suggested replies.

[0161] Step 4:

[0162] The server sends back a generated feedback message and suggested replies to the terminal. The terminal has a means of displaying this to the user, who then refers to the displayed feedback and suggested replies. The user can then consider the options and prepare a calm response. The resulting action is that the terminal's display shows the user multiple suggested replies.

[0163] Step 5:

[0164] The user selects a suggested reply or creates and sends a new reply. The device then sends the user's chosen or created reply data back to the server and sends the message to the recipient's device. At this point, the final data is output as the sent reply message. In this case, the user's decision is the final action.

[0165] (Application Example 2)

[0166] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0167] In today's service delivery environment, improving communication between customers and service providers is extremely important. However, a problem arises when the quality of service deteriorates and customer satisfaction is diminished due to the inability to properly understand the customer's emotions and intentions. This invention aims to enable the provision of higher quality, more humane services by analyzing the customer's emotional state in real time and providing appropriate feedback to the service provider.

[0168] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0169] In this invention, the server includes means for receiving message data, means for analyzing the message data and evaluating the emotional components and the possibility of misunderstanding of the content, and means for analyzing the customer's emotional state and generating information to display to the service provider. This enables the service provider to respond appropriately to the customer's emotional state.

[0170] "Message data" refers to information in the form of text, audio, or images that is exchanged through communication.

[0171] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0172] "Emotional components" are characteristic elements that indicate a user's emotional state.

[0173] "Potential for misunderstanding" refers to assessing the risk that the content of a message may be misunderstood by the recipient.

[0174] "Feedback" refers to information and advice provided based on the analysis results.

[0175] A "suggested reply" is a proposed message that would be appropriate as a response to the user.

[0176] "Generation" is the process of creating new information or data.

[0177] "Presentation" means showing information to the user.

[0178] "Customer emotional state" refers to the psychological or emotional condition exhibited by a customer.

[0179] A "service provider" is an individual or organization that provides services to a customer.

[0180] The system for implementing this invention mainly consists of a user terminal, a central server, an AI analysis engine, and an emotion analysis engine. The user terminal employs a device such as smart glasses and has the function of recognizing the customer's facial expressions and acquiring voice. Specific hardware examples include Google® Glass® and Vuzix Blade.

[0181] The server receives data sent from user terminals and processes it in real time. It uses OpenCV for image recognition and the Google Cloud Speech-to-Text API for speech analysis. Furthermore, it analyzes the data using a TensorFlow-based sentiment analysis model to understand the customer's emotional state. Based on these results, it generates appropriate information to display to service providers. Here, a generative AI model is utilized to create optimal feedback.

[0182] For example, if a customer shows signs of anxiety while asking a question about a product, the AI ​​analysis engine will generate advice such as "explain gently and carefully" and display it on the smart glasses' screen. In this way, service providers can respond in a way that takes customer emotions into consideration.

[0183] Example prompt: "This customer seems anxious. How can I reassure them?" This prompt is used by the AI ​​to provide appropriate feedback based on their emotional state.

[0184] This system enables personalized customer service and improves the quality of communication.

[0185] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0186] Step 1:

[0187] The smart glasses, which act as the terminal, capture the customer's facial expressions and voice. This generates image data and voice data as physical input data.

[0188] Step 2:

[0189] The device sends the acquired image and audio data to a central server. This step involves data encoding and secure transfer over a network.

[0190] Step 3:

[0191] The server analyzes the received image data using OpenCV and extracts emotional elements from facial expressions. Audio data is converted to text using the Google Cloud Speech-to-Text API, and emotions are analyzed from the tone. These processes generate analysis results that indicate the customer's emotional state.

[0192] Step 4:

[0193] The server uses a generated AI model to produce appropriate feedback and advice based on the analysis results. This prompt includes specific instructions such as, "This customer seems anxious. How can we reassure them?"

[0194] Step 5:

[0195] The server sends the generated feedback and advice information to the terminal. The server then reformats the data so that the information is displayed correctly on the terminal.

[0196] Step 6:

[0197] The device displays feedback and advice information on the smart glasses' display so that service providers can visually confirm it. Based on this display, service providers can take appropriate action for customers.

[0198] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0199] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0200] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0201] [Second Embodiment]

[0202] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0203] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0204] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0205] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0206] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0207] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0208] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0209] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0210] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0211] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0212] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0213] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0214] This invention is implemented as an AI arbitration system integrated into a messaging application. The system consists of a user's terminal, a central server, and an AI analysis engine. When a user sends a message, the terminal transmits this message data to the central server.

[0215] The server passes the received message data to an AI analysis engine, which analyzes the content and emotional tone. The AI ​​analysis engine detects emotional elements and potential misunderstandings within the message and generates a feedback message based on that. This process mitigates direct emotional conflicts that could exacerbate the problem.

[0216] The device presents the user with feedback and analysis results provided by the server. Furthermore, it suggests recommended replies generated by the AI ​​analysis engine, allowing the user to select the optimal reply or create their own.

[0217] For example, if user A sends an emotionally charged message to user B, the AI ​​analysis engine analyzes the message and determines that it contains feelings of anger or frustration. The server then provides user B with feedback that includes ways to alleviate these feelings and suggests a response that encourages calm dialogue. In this way, direct misunderstandings and emotional conflicts are prevented, and neutral and constructive communication is achieved.

[0218] This invention enables users to control their emotions during the communication process, leading to more effective and positive dialogue. The mediation function can be turned on or off at the user's discretion, allowing for flexible operation as needed.

[0219] The following describes the processing flow.

[0220] Step 1:

[0221] The user creates message data by typing a message and pressing the send button. The terminal retrieves this message data and instructs the central server to send it.

[0222] Step 2:

[0223] The server forwards the message data received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the text data and performs natural language processing to extract emotional tones and specific keywords.

[0224] Step 3:

[0225] The AI ​​analysis engine evaluates the message based on its analysis results, assessing whether there are emotional elements or potential misunderstandings, and generates corresponding feedback. In some cases, it prepares cautionary messages to alleviate tension.

[0226] Step 4:

[0227] The server receives the results from the AI ​​analysis engine and sends them to the user's device as a feedback message. If suggested replies exist, they are also provided.

[0228] Step 5:

[0229] The terminal displays feedback and suggested replies received from the server to the user. Based on this information, the user decides whether to select a suggested reply or to type a new message themselves.

[0230] Step 6:

[0231] Once the user decides on their reply and presses the send button, the device sends this reply data back to the server and begins the process of delivering it to the recipient's device.

[0232] Step 7:

[0233] The server receives the sent reply data, checks if the content requires re-analysis, and then sends the data to the recipient's device. This ensures that smooth communication between users continues.

[0234] (Example 1)

[0235] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0236] In recent years, with the increased use of messaging applications, emotional conflicts and misunderstandings have become more frequent. Such misunderstandings and emotional conflicts can hinder smooth communication and lead to a deterioration of interpersonal relationships. However, existing systems do not adequately analyze emotional elements or provide sufficient feedback, making it difficult to effectively resolve these communication problems. Against this backdrop, there is a need for a system that can prevent emotional conflicts and misunderstandings among users and promote constructive dialogue.

[0237] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0238] In this invention, the server includes means for receiving message information, means for analyzing the message information and evaluating the emotional elements and the possibility of misunderstanding the content, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to effectively analyze the emotional elements and the possibility of misunderstanding contained in the message and provide appropriate feedback and response suggestions to the user.

[0239] "Message information" refers to the content of communications in text or other data formats sent and received between users.

[0240] A "generative artificial intelligence model" is a type of AI technology used in natural language processing and other applications, referring to a machine learning model that aims to generate appropriate output based on input.

[0241] "Emotional elements" refer to indicators used to identify information that indicates human emotions or emotional tone contained within text or messages.

[0242] "Feedback" refers to information provided to users based on the analyzed results, for the purpose of improvement or appropriate action.

[0243] A "suggested reply" refers to a template or guide of responses that are recommended by the user based on AI analysis.

[0244] This invention is specifically implemented as an AI arbitration system integrated into a messaging application. This system consists of a user's terminal, a central server, and an AI analysis engine.

[0245] The terminal converts the messages sent by the user into a data format and transmits them to a central server over the network. The hardware used as a terminal is a common communication device such as a smartphone or computer, with a messaging application installed on it.

[0246] The server passes message information received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using natural language processing techniques. A generative AI model is used for this analysis. Specific examples include open-source natural language processing libraries and commercial AI platforms. The analysis results evaluate the emotional elements and potential for misunderstanding contained in the message.

[0247] Based on the analyzed results, the server sends back feedback and suggested replies generated by the AI ​​analysis engine to the terminal. This feedback includes advice to mitigate emotional conflict and specific suggested replies.

[0248] The device receives feedback from the server and presents it to the user. Based on this feedback, the user can choose the most appropriate response or create their own response based on their own judgment.

[0249] As a concrete example of its operation, if a user sends a message such as "I'm worried because the project is behind schedule," the AI ​​analysis engine will analyze this message and determine that it contains feelings of anxiety or worry. Based on this information, the server can then provide feedback to the user such as "Let's check the progress and think about the next steps."

[0250] The prompt text input to the generative AI model is in the format of, "Analyze the message, identify emotional elements, and generate appropriate feedback." In this way, the present invention provides users with a means to improve the quality of communication.

[0251] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0252] Step 1:

[0253] The user creates and sends a message using a terminal. The terminal converts this message into a data format and sends it to a central server over the network. The input is the text message entered by the user, and the output is the message data sent over the network. The terminal takes in the raw data from the user, encodes it in the appropriate format, and sends it.

[0254] Step 2:

[0255] The server receives message data from the terminal. Next, the server prepares this data to be passed to the AI ​​analysis engine. The input is the message data received from the terminal, and the output is pre-processed data to be fed into the AI ​​analysis engine. The server verifies the data and prepares it for processing.

[0256] Step 3:

[0257] The server sends message data to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using a generative AI model. In this process, the message is tokenized and sentiment analysis is performed. The input is message data from the server, and the output is sentiment elements and potential misunderstandings as a result of the analysis. The analysis engine analyzes the message and quantifies the sentiment tone and potential misunderstandings.

[0258] Step 4:

[0259] The AI ​​analysis engine generates feedback messages and suggested replies based on the analysis results. This feedback includes advice to facilitate the conversation. The input is the analysis results, and the output is the feedback message and suggested replies. The engine applies a generative AI model to suggest appropriate communication guidance.

[0260] Step 5:

[0261] The server sends feedback messages and suggested replies from the AI ​​analysis engine back to the terminal. The input is feedback data from the analysis engine, and the output is feedback information sent to the terminal. The server formats this information appropriately and delivers it to the terminal.

[0262] Step 6:

[0263] The terminal displays the received feedback message and suggested replies to the user. The user can review the presented feedback and select or create an appropriate reply. The input is the feedback information from the server, and the output is what is displayed to the user. The terminal visually displays the information and assists the user in continuing the interaction.

[0264] (Application Example 1)

[0265] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] In modern businesses and social organizations, communication via electronic messages is widespread, but this process is prone to emotional conflict and misunderstandings. In particular, in emotionally charged situations, inappropriate expressions and misinterpretations can exacerbate problems. This can lead to deterioration of interpersonal relationships and friction within organizations, hindering efficient work performance. Therefore, systems are needed to prevent these issues from occurring.

[0267] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0268] In this invention, the server includes means for acquiring information, means for analyzing the information and evaluating the possibility of interpreting emotional elements and content, and means for generating response messages and proposed responses. This reduces emotional conflicts and misunderstandings that occur during the communication process, enabling neutral and constructive dialogue.

[0269] "Means of acquiring information" refers to system components that handle the initial input process for receiving messages and data from users and starting appropriate processing.

[0270] "Means for analyzing information and evaluating the possibility of interpreting emotional elements and content" refers to algorithms that analyze received information using natural language processing techniques to identify emotional components and potential misunderstandings contained in a message.

[0271] "Means for generating response messages and proposed responses" refers to system functions that provide users with appropriate feedback and proposed responses to confirm, based on the analysis results.

[0272] "Means of presenting to the user" refers to display methods that visually or audibly notify the user of the generated analysis results and feedback, and facilitate their understanding.

[0273] "Means for forwarding response information selected or entered by the user to the recipient" refers to a communication process that accurately delivers the user's selected reply to the recipient.

[0274] "Means for enabling / disabling neutral functions" refers to an interface that allows users to choose to turn the AI-based sentiment analysis function on or off as needed.

[0275] "Means for evaluating and improving communication safety" refer to system functions that identify potential risk factors in communication and prevent misunderstandings and conflicts between users.

[0276] This invention is a system for realizing an AI arbitration system in messaging applications. It mainly consists of a server, a user's terminal, and an AI analysis engine. The server receives messages exchanged between users and extracts information from them. The extracted information is sent to the AI ​​analysis engine. The AI ​​analysis engine uses machine learning libraries such as TensorFlow and PyTorch to perform natural language processing and evaluate the emotional elements of the messages and the possibility of misinterpretation. Based on the analysis results, the server automatically generates response messages and specific response suggestions and presents them to the user.

[0277] Users can view the analysis results through applications on their smartphones or desktop PCs. Here, users are presented with generated response suggestions and can select or edit their own responses as needed. The generated responses are then forwarded back to the recipient via the server.

[0278] As a specific example, assume that discussions about the progress of a project at work are being conducted via email. If an employee sends a message saying, "This project is really a waste of time," the AI analysis engine analyzes the negative sentiment contained in that message and generates feedback such as, "There may be dissatisfaction in this message. Please confirm." In this way, it is possible to prevent emotional conflicts and misunderstandings with the other party in advance.

[0279] As an example of a prompt sentence for the generation AI model, it takes the form of, "Analyze the sentiment indicated by this message and generate feedback and constructive reply proposals. Message: 'There is dissatisfaction with the progress of this project.'" This prompt sentence serves as the basic input sentence for instructing the AI analysis engine to perform sentiment analysis and generate reply proposals. With this system, users can reduce the risks in message communication and maintain a healthy dialogue.

[0280] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0281] Step 1:

[0282] The server receives message data from the user's terminal. The input is the message sent by the user, and the output is that message data. This data is stored in the data storage in a clear form for use in the following processes.

[0283] Step 2:

[0284] The server sends the received message data to the AI analysis engine. The input is the message data obtained in Step 1, and the output is the analysis result by the AI analysis engine. The AI analysis engine evaluates the sentiment elements and the possibility of misunderstanding in the message using natural language processing technology. In this process, for example, text sentiment analysis and tone detection are performed.

[0285] Step 3:

[0286] Based on the analysis results, the AI analysis engine generates a reply proposal. The input is the analysis results obtained in Step 2, and the output is the generated reply proposal and feedback. Specifically, the generation AI model devises a constructive reply based on the prompt text and creates data for providing to the user.

[0287] Step 4:

[0288] The server transfers the generated reply proposal and feedback to the user's terminal. The input is the data generated in Step 3, and the output is the reply proposal and feedback received by the user. The terminal displays this information on the user interface so that the user can view it.

[0289] Step 5:

[0290] The user checks the reply proposal presented on the terminal, selects or edits it as necessary to create a reply. The input is the information presented to the terminal in Step 4, and the output is the reply data determined by the user. Through this interface, the user can select a reply for maintaining optimal communication.

[0291] Step 6:

[0292] The terminal sends the reply data selected or edited by the user to the server. The input is the user's reply data created in Step 5, and the output is the final reply message sent to the server side. This reply is then ready to be delivered to the other party's terminal through the server again.

[0293] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0294] This invention is implemented in a messaging application by combining an AI arbitration system with an emotion engine. This system recognizes the user's emotions and provides feedback based on that emotional state, thereby supporting more human-like and effective communication.

[0295] The system components include a user terminal, a central server, an AI analysis engine, and an emotion engine. When a user sends a message, the terminal receives the message data and also sends the user's facial expression image or voice tone to the emotion engine.

[0296] The emotion engine uses image recognition technology and voice analysis to recognize the user's emotional state in real time. This recognition result is sent to the server along with the message data, where the AI ​​analysis engine analyzes the message content and emotional tone.

[0297] Based on these analysis results, the server generates a feedback message. The generated feedback is adjusted according to the user's emotional state, as determined by the emotion engine. For example, if the user is angry, the feedback will include calming expressions, and if the user is sad, the feedback will emphasize comforting elements.

[0298] The device displays generated feedback and suggested responses to the user. The user can then use this information to select or create an appropriate response.

[0299] For example, if user A receives a message from user B expressing dissatisfaction, the emotion engine will recognize that user A's facial expression indicates anxiety. Based on this information, the AI ​​analysis engine will generate a response suggestion that offers comfort and reassurance to user A, and the device will present it to help user A deal with the situation calmly.

[0300] With this system, the exchange of messages takes into account the user's emotions, enabling more personal communication and helping to avoid unnecessary conflicts.

[0301] The following is an explanation of the processing flow.

[0302] Step 1:

[0303] The user inputs a message into the terminal and presses the send button. At this time, the terminal uses the camera or microphone to collect the user's facial expression or voice.

[0304] Step 2:

[0305] The terminal sends the collected facial expression image or voice data together with the input message data to the server.

[0306] Step 3:

[0307] The server sends the received message data to the AI analysis engine and the facial expression image or voice data to the emotion engine. The AI analysis engine analyzes the message content, and the emotion engine analyzes the user's emotional state in real time.

[0308] Step 4:

[0309] The AI analysis engine evaluates the emotional tone and potential for misunderstanding of the message, and the emotion engine determines the user's emotional state. Based on this result, the server generates appropriate feedback and reply proposals.

[0310] Step 5:

[0311] The server considers the emotion recognition result from the emotion engine and adjusts the feedback and reply proposals. The adjusted content is sent to the terminal.

[0312] Step 6:

[0313] The device presents the user with tailored feedback and suggested responses. The user then selects or creates a response based on this information.

[0314] Step 7:

[0315] Once the user decides on a reply and sends it, the device sends this reply data to the server, and the normal message transmission process is executed.

[0316] This process allows users to receive feedback that takes their emotional state into account, enabling them to communicate more effectively.

[0317] (Example 2)

[0318] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0319] In the exchange of information involving emotions, misunderstandings of users' intentions and feelings can lead to conflict and unnecessary stress. Traditional messaging systems have difficulty providing feedback that takes into account the emotional state of users, and have limitations in supporting empathetic and effective communication.

[0320] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0321] In this invention, the server includes means for receiving information, means for analyzing the information and emotional data and evaluating the possibility of misunderstandings regarding emotional state and intentions, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to provide appropriate feedback based on the user's emotional state, avoid conflict through neutral mediation, and promote more empathetic communication.

[0322] "Means of receiving information" refers to the function of acquiring data and messages from users in order to transmit them to other system components.

[0323] "Means for analyzing information and emotional data to assess the possibility of misunderstandings regarding emotional states and intentions" refers to a process that identifies users' emotions and communication intentions based on acquired data and assesses the risk of misunderstandings.

[0324] "Means for generating feedback and response suggestions using a generative artificial intelligence model" refers to a mechanism that uses natural language processing technology to automatically generate responses that are appropriate to the user's emotions and intentions.

[0325] "User" refers to an individual or organization that communicates through the system.

[0326] A "processing device" is a computer hardware or software system designed to perform a specific task.

[0327] "Means for setting the enablement or disablement of the neutral arbitration function" refers to a function that allows users to arbitrarily activate or deactivate the arbitration function.

[0328] This invention is an emotion-aware messaging system that reduces misunderstandings of intent and emotions between users and supports smooth communication. The system primarily consists of a terminal, a server, an emotion analysis engine, and a generative AI model.

[0329] The user sends messages through the device. The device is equipped with a camera to capture the user's facial expressions and a microphone to collect their voice tone. The device uses these devices to collect the user's emotional data along with the message data and sends it to the emotion analysis engine.

[0330] The emotion analysis engine uses image processing technologies such as OpenCV and speech analysis technologies such as LibROSA to identify the user's emotional state in real time. This emotion data is sent to a server, which uses a generative AI model to analyze the content of the received message and the user's emotional state.

[0331] The server uses generative AI models such as BERT and GPT to analyze the information and generate appropriate feedback and response suggestions. In doing so, it uses prompts to the generative AI model such as, "If the user is showing signs of anxiety, suggest a comforting feedback message to provide." This generated feedback is tailored to the user's emotional state and presented to the user via the device.

[0332] As a concrete example, suppose user A receives a message from their friend, user B. If the system detects dissatisfaction in the message, user A's device sends an expression of anxiety to an emotion analysis engine. Based on this data, the server performs an AI analysis and provides user A with appropriate feedback. This allows user A to calmly consider their reply and avoid conflict. This system enables more intimate communication between users.

[0333] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0334] Step 1:

[0335] The user composes a message using a messaging application and presses the send button. The input here is the text or voice message created by the user. The device captures this message data and simultaneously uses its built-in camera and microphone to acquire images of the user's facial expressions and voice tone. As a result, data indicating the user's emotions is output.

[0336] Step 2:

[0337] The device sends the acquired message data and emotion data to the emotion analysis engine. The emotion analysis engine uses image processing software and speech analysis software to analyze the user's emotional state in real time based on the input image and audio data. At this stage, the results of the analyzed emotional state are output. Specifically, OpenCV analyzes facial feature points, and LibROSA detects changes in voice tone.

[0338] Step 3:

[0339] The device sends the analyzed sentiment data and message content to the server. The server receives this data and analyzes it using a generative AI model. It receives the sentiment state and message content as input and evaluates the intent and emotional tone from them. The generative AI model (e.g., GPT) also generates a feedback message based on the input information and creates appropriate reply suggestions. The output consists of a feedback message and suggested replies.

[0340] Step 4:

[0341] The server sends back a generated feedback message and suggested replies to the terminal. The terminal has a means of displaying this to the user, who then refers to the displayed feedback and suggested replies. The user can then consider the options and prepare a calm response. The resulting action is that the terminal's display shows the user multiple suggested replies.

[0342] Step 5:

[0343] The user selects a suggested reply or creates and sends a new reply. The device then sends the user's chosen or created reply data back to the server and sends the message to the recipient's device. At this point, the final data is output as the sent reply message. In this case, the user's decision is the final action.

[0344] (Application Example 2)

[0345] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0346] In today's service delivery environment, improving communication between customers and service providers is extremely important. However, a problem arises when the quality of service deteriorates and customer satisfaction is diminished due to the inability to properly understand the customer's emotions and intentions. This invention aims to enable the provision of higher quality, more humane services by analyzing the customer's emotional state in real time and providing appropriate feedback to the service provider.

[0347] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0348] In this invention, the server includes means for receiving message data, means for analyzing the message data and evaluating the emotional components and the possibility of misunderstanding of the content, and means for analyzing the customer's emotional state and generating information to display to the service provider. This enables the service provider to respond appropriately to the customer's emotional state.

[0349] "Message data" refers to information in the form of text, audio, or images that is exchanged through communication.

[0350] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0351] "Emotional components" are characteristic elements that indicate a user's emotional state.

[0352] "Potential for misunderstanding" refers to assessing the risk that the content of a message may be misunderstood by the recipient.

[0353] "Feedback" refers to information and advice provided based on the analysis results.

[0354] A "suggested reply" is a proposed message that would be appropriate as a response to the user.

[0355] "Generation" is the process of creating new information or data.

[0356] "Presentation" means showing information to the user.

[0357] "Customer emotional state" refers to the psychological or emotional condition exhibited by a customer.

[0358] A "service provider" is an individual or organization that provides services to a customer.

[0359] The system for implementing this invention mainly consists of a user terminal, a central server, an AI analysis engine, and an emotion analysis engine. The user terminal employs a device such as smart glasses and has the function of recognizing the customer's facial expressions and acquiring voice. Specific hardware examples include Google Glass and Vuzix Blade.

[0360] The server receives data sent from user terminals and processes it in real time. It uses OpenCV for image recognition and the Google Cloud Speech-to-Text API for speech analysis. Furthermore, it analyzes the data using a TensorFlow-based sentiment analysis model to understand the customer's emotional state. Based on these results, it generates appropriate information to display to service providers. Here, a generative AI model is utilized to create optimal feedback.

[0361] For example, if a customer shows signs of anxiety while asking a question about a product, the AI ​​analysis engine will generate advice such as "explain gently and carefully" and display it on the smart glasses' screen. In this way, service providers can respond in a way that takes customer emotions into consideration.

[0362] Example prompt: "This customer seems anxious. How can I reassure them?" This prompt is used by the AI ​​to provide appropriate feedback based on their emotional state.

[0363] This system enables personalized customer service and improves the quality of communication.

[0364] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0365] Step 1:

[0366] The smart glasses, which act as the terminal, capture the customer's facial expressions and voice. This generates image data and voice data as physical input data.

[0367] Step 2:

[0368] The device sends the acquired image and audio data to a central server. This step involves data encoding and secure transfer over a network.

[0369] Step 3:

[0370] The server analyzes the received image data using OpenCV and extracts emotional elements from facial expressions. Audio data is converted to text using the Google Cloud Speech-to-Text API, and emotions are analyzed from the tone. These processes generate analysis results that indicate the customer's emotional state.

[0371] Step 4:

[0372] The server uses a generated AI model to produce appropriate feedback and advice based on the analysis results. This prompt includes specific instructions such as, "This customer seems anxious. How can we reassure them?"

[0373] Step 5:

[0374] The server sends the generated feedback and advice information to the terminal. The server then reformats the data so that the information is displayed correctly on the terminal.

[0375] Step 6:

[0376] The device displays feedback and advice information on the smart glasses' display so that service providers can visually confirm it. Based on this display, service providers can take appropriate action for customers.

[0377] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0378] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0379] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0380] [Third Embodiment]

[0381] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0382] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0383] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0384] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0385] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0386] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0387] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0388] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0389] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0390] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0391] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0392] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0393] This invention is implemented as an AI arbitration system integrated into a messaging application. The system consists of a user's terminal, a central server, and an AI analysis engine. When a user sends a message, the terminal transmits this message data to the central server.

[0394] The server passes the received message data to an AI analysis engine, which analyzes the content and emotional tone. The AI ​​analysis engine detects emotional elements and potential misunderstandings within the message and generates a feedback message based on that. This process mitigates direct emotional conflicts that could exacerbate the problem.

[0395] The device presents the user with feedback and analysis results provided by the server. Furthermore, it suggests recommended replies generated by the AI ​​analysis engine, allowing the user to select the optimal reply or create their own.

[0396] For example, if user A sends an emotionally charged message to user B, the AI ​​analysis engine analyzes the message and determines that it contains feelings of anger or frustration. The server then provides user B with feedback that includes ways to alleviate these feelings and suggests a response that encourages calm dialogue. In this way, direct misunderstandings and emotional conflicts are prevented, and neutral and constructive communication is achieved.

[0397] This invention enables users to control their emotions during the communication process, leading to more effective and positive dialogue. The mediation function can be turned on or off at the user's discretion, allowing for flexible operation as needed.

[0398] The following describes the processing flow.

[0399] Step 1:

[0400] The user creates message data by typing a message and pressing the send button. The terminal retrieves this message data and instructs the central server to send it.

[0401] Step 2:

[0402] The server forwards the message data received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the text data and performs natural language processing to extract emotional tones and specific keywords.

[0403] Step 3:

[0404] The AI ​​analysis engine evaluates the message based on its analysis results, assessing whether there are emotional elements or potential misunderstandings, and generates corresponding feedback. In some cases, it prepares cautionary messages to alleviate tension.

[0405] Step 4:

[0406] The server receives the results from the AI ​​analysis engine and sends them to the user's device as a feedback message. If suggested replies exist, they are also provided.

[0407] Step 5:

[0408] The terminal displays feedback and suggested replies received from the server to the user. Based on this information, the user decides whether to select a suggested reply or to type a new message themselves.

[0409] Step 6:

[0410] Once the user decides on their reply and presses the send button, the device sends this reply data back to the server and begins the process of delivering it to the recipient's device.

[0411] Step 7:

[0412] The server receives the sent reply data, checks if the content requires re-analysis, and then sends the data to the recipient's device. This ensures that smooth communication between users continues.

[0413] (Example 1)

[0414] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0415] In recent years, with the increased use of messaging applications, emotional conflicts and misunderstandings have become more frequent. Such misunderstandings and emotional conflicts can hinder smooth communication and lead to a deterioration of interpersonal relationships. However, existing systems do not adequately analyze emotional elements or provide sufficient feedback, making it difficult to effectively resolve these communication problems. Against this backdrop, there is a need for a system that can prevent emotional conflicts and misunderstandings among users and promote constructive dialogue.

[0416] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0417] In this invention, the server includes means for receiving message information, means for analyzing the message information and evaluating the emotional elements and the possibility of misunderstanding the content, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to effectively analyze the emotional elements and the possibility of misunderstanding contained in the message and provide appropriate feedback and response suggestions to the user.

[0418] "Message information" refers to the content of communications in text or other data formats sent and received between users.

[0419] A "generative artificial intelligence model" is a type of AI technology used in natural language processing and other applications, referring to a machine learning model that aims to generate appropriate output based on input.

[0420] "Emotional elements" refer to indicators used to identify information that indicates human emotions or emotional tone contained within text or messages.

[0421] "Feedback" refers to information provided to users based on the analyzed results, for the purpose of improvement or appropriate action.

[0422] A "suggested reply" refers to a template or guide of responses that are recommended by the user based on AI analysis.

[0423] This invention is specifically implemented as an AI arbitration system integrated into a messaging application. This system consists of a user's terminal, a central server, and an AI analysis engine.

[0424] The terminal converts the messages sent by the user into a data format and transmits them to a central server over the network. The hardware used as a terminal is a common communication device such as a smartphone or computer, with a messaging application installed on it.

[0425] The server passes message information received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using natural language processing techniques. A generative AI model is used for this analysis. Specific examples include open-source natural language processing libraries and commercial AI platforms. The analysis results evaluate the emotional elements and potential for misunderstanding contained in the message.

[0426] Based on the analyzed results, the server sends back feedback and suggested replies generated by the AI ​​analysis engine to the terminal. This feedback includes advice to mitigate emotional conflict and specific suggested replies.

[0427] The device receives feedback from the server and presents it to the user. Based on this feedback, the user can choose the most appropriate response or create their own response based on their own judgment.

[0428] As a concrete example of its operation, if a user sends a message such as "I'm worried because the project is behind schedule," the AI ​​analysis engine will analyze this message and determine that it contains feelings of anxiety or worry. Based on this information, the server can then provide feedback to the user such as "Let's check the progress and think about the next steps."

[0429] The prompt text input to the generative AI model is in the format of, "Analyze the message, identify emotional elements, and generate appropriate feedback." In this way, the present invention provides users with a means to improve the quality of communication.

[0430] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0431] Step 1:

[0432] The user creates and sends a message using a terminal. The terminal converts this message into a data format and sends it to a central server over the network. The input is the text message entered by the user, and the output is the message data sent over the network. The terminal takes in the raw data from the user, encodes it in the appropriate format, and sends it.

[0433] Step 2:

[0434] The server receives message data from the terminal. Next, the server prepares this data to be passed to the AI ​​analysis engine. The input is the message data received from the terminal, and the output is pre-processed data to be fed into the AI ​​analysis engine. The server verifies the data and prepares it for processing.

[0435] Step 3:

[0436] The server sends message data to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using a generative AI model. In this process, the message is tokenized and sentiment analysis is performed. The input is message data from the server, and the output is sentiment elements and potential misunderstandings as a result of the analysis. The analysis engine analyzes the message and quantifies the sentiment tone and potential misunderstandings.

[0437] Step 4:

[0438] The AI ​​analysis engine generates feedback messages and suggested replies based on the analysis results. This feedback includes advice to facilitate the conversation. The input is the analysis results, and the output is the feedback message and suggested replies. The engine applies a generative AI model to suggest appropriate communication guidance.

[0439] Step 5:

[0440] The server sends feedback messages and suggested replies from the AI ​​analysis engine back to the terminal. The input is feedback data from the analysis engine, and the output is feedback information sent to the terminal. The server formats this information appropriately and delivers it to the terminal.

[0441] Step 6:

[0442] The terminal displays the received feedback message and suggested replies to the user. The user can review the presented feedback and select or create an appropriate reply. The input is the feedback information from the server, and the output is what is displayed to the user. The terminal visually displays the information and assists the user in continuing the interaction.

[0443] (Application Example 1)

[0444] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] In modern businesses and social organizations, communication via electronic messages is widespread, but this process is prone to emotional conflict and misunderstandings. In particular, in emotionally charged situations, inappropriate expressions and misinterpretations can exacerbate problems. This can lead to deterioration of interpersonal relationships and friction within organizations, hindering efficient work performance. Therefore, systems are needed to prevent these issues from occurring.

[0446] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0447] In this invention, the server includes means for acquiring information, means for analyzing the information and evaluating the possibility of interpreting emotional elements and content, and means for generating response messages and proposed responses. This reduces emotional conflicts and misunderstandings that occur during the communication process, enabling neutral and constructive dialogue.

[0448] "Means of acquiring information" refers to system components that handle the initial input process for receiving messages and data from users and starting appropriate processing.

[0449] "Means for analyzing information and evaluating the possibility of interpreting emotional elements and content" refers to algorithms that analyze received information using natural language processing techniques to identify emotional components and potential misunderstandings contained in a message.

[0450] "Means for generating response messages and proposed responses" refers to system functions that provide users with appropriate feedback and proposed responses to confirm, based on the analysis results.

[0451] "Means of presenting to the user" refers to display methods that visually or audibly notify the user of the generated analysis results and feedback, and facilitate their understanding.

[0452] "Means for forwarding response information selected or entered by the user to the recipient" refers to a communication process that accurately delivers the user's selected reply to the recipient.

[0453] "Means for enabling / disabling neutral functions" refers to an interface that allows users to choose to turn the AI-based sentiment analysis function on or off as needed.

[0454] "Means for evaluating and improving communication safety" refer to system functions that identify potential risk factors in communication and prevent misunderstandings and conflicts between users.

[0455] This invention is a system for realizing an AI arbitration system in messaging applications. It mainly consists of a server, a user's terminal, and an AI analysis engine. The server receives messages exchanged between users and extracts information from them. The extracted information is sent to the AI ​​analysis engine. The AI ​​analysis engine uses machine learning libraries such as TensorFlow and PyTorch to perform natural language processing and evaluate the emotional elements of the messages and the possibility of misinterpretation. Based on the analysis results, the server automatically generates response messages and specific response suggestions and presents them to the user.

[0456] Users can view the analysis results through applications on their smartphones or desktop PCs. Here, users are presented with generated response suggestions and can select or edit their own responses as needed. The generated responses are then forwarded back to the recipient via the server.

[0457] As a concrete example, suppose a discussion about the progress of a project in the workplace is taking place via email. If an employee sends a message such as, "This project is a complete waste of time," the AI ​​analysis engine will analyze the negative emotions contained in the message and generate feedback such as, "This message may contain dissatisfaction. Please review it." In this way, it is possible to prevent emotional conflict and misunderstandings with the other party.

[0458] An example of a prompt for a generative AI model would be: "Analyze the sentiment expressed in this message and generate feedback and constructive response suggestions. Message: 'I am dissatisfied with the progress of this project.'" This prompt serves as the basic input for instructing the AI ​​analysis engine to perform sentiment analysis and generate response suggestions. This system allows users to reduce the risks in message communication and maintain healthy dialogue.

[0459] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0460] Step 1:

[0461] The server receives message data from the user's terminal. The input is the message sent by the user, and the output is the message data itself. This data is stored in a clear format in data storage for use in subsequent processing.

[0462] Step 2:

[0463] The server sends the received message data to the AI ​​analysis engine. The input is the message data acquired in step 1, and the output is the analysis result from the AI ​​analysis engine. The AI ​​analysis engine evaluates the emotional elements and potential for misunderstanding of the message using natural language processing techniques. This process includes, for example, sentiment analysis of the text and tone detection.

[0464] Step 3:

[0465] The AI ​​analysis engine generates response suggestions based on the analysis results. The input is the analysis results obtained in step 2, and the output is the generated response suggestions and feedback. Specifically, the generating AI model devises constructive responses based on the prompt text and creates data to provide to the user.

[0466] Step 4:

[0467] The server forwards the generated response and feedback to the user's terminal. The input is the data generated in step 3, and the output is the response and feedback the user receives. The terminal displays this information in the user interface for the user to review.

[0468] Step 5:

[0469] The user reviews the suggested replies presented on the device, selects or edits them as needed, and creates their reply. The input is the information presented on the device in step 4, and the output is the reply data confirmed by the user. Through this interface, the user can choose the reply that best maintains communication.

[0470] Step 6:

[0471] The terminal sends the user-selected or edited reply data to the server. The input is the user's reply data created in step 5, and the output is the final reply message sent to the server. This reply is then ready to be delivered to the recipient's terminal via the server.

[0472] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0473] This invention is implemented in a messaging application by combining an AI arbitration system with an emotion engine. This system recognizes the user's emotions and provides feedback based on that emotional state, thereby supporting more human-like and effective communication.

[0474] The system components include a user terminal, a central server, an AI analysis engine, and an emotion engine. When a user sends a message, the terminal receives the message data and also sends the user's facial expression image or voice tone to the emotion engine.

[0475] The emotion engine uses image recognition technology and voice analysis to recognize the user's emotional state in real time. This recognition result is sent to the server along with the message data, where the AI ​​analysis engine analyzes the message content and emotional tone.

[0476] Based on these analysis results, the server generates a feedback message. The generated feedback is adjusted according to the user's emotional state, as determined by the emotion engine. For example, if the user is angry, the feedback will include calming expressions, and if the user is sad, the feedback will emphasize comforting elements.

[0477] The device displays generated feedback and suggested responses to the user. The user can then use this information to select or create an appropriate response.

[0478] For example, if user A receives a message from user B expressing dissatisfaction, the emotion engine will recognize that user A's facial expression indicates anxiety. Based on this information, the AI ​​analysis engine will generate a response suggestion that offers comfort and reassurance to user A, and the device will present it to help user A deal with the situation calmly.

[0479] This system ensures that message exchanges take users' emotions into account, enabling more empathetic communication and helping to avoid unnecessary conflicts.

[0480] The following describes the processing flow.

[0481] Step 1:

[0482] The user types a message into the device and presses the send button. At this time, the device uses its camera or microphone to collect the user's facial expressions or voice.

[0483] Step 2:

[0484] The terminal sends the collected facial expression images or voice data along with the entered message data to the server.

[0485] Step 3:

[0486] The server sends the received message data to the AI ​​analysis engine and facial expression images or voice data to the emotion engine. The AI ​​analysis engine analyzes the message content, and the emotion engine analyzes the user's emotional state in real time.

[0487] Step 4:

[0488] The AI ​​analysis engine evaluates the emotional tone and potential for misunderstanding in a message, while the emotion engine determines the user's emotional state. Based on these results, the server generates appropriate feedback and suggested replies.

[0489] Step 5:

[0490] The server considers the emotion recognition results from the emotion engine and adjusts the feedback and suggested replies. The adjusted content is then sent to the terminal.

[0491] Step 6:

[0492] The device presents the user with tailored feedback and suggested responses. The user then selects or creates a response based on this information.

[0493] Step 7:

[0494] Once the user decides on a reply and sends it, the device sends this reply data to the server, and the normal message transmission process is executed.

[0495] This process allows users to receive feedback that takes their emotional state into account, enabling them to communicate more effectively.

[0496] (Example 2)

[0497] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0498] In the exchange of information involving emotions, misunderstandings of users' intentions and feelings can lead to conflict and unnecessary stress. Traditional messaging systems have difficulty providing feedback that takes into account the emotional state of users, and have limitations in supporting empathetic and effective communication.

[0499] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0500] In this invention, the server includes means for receiving information, means for analyzing the information and emotional data and evaluating the possibility of misunderstandings regarding emotional state and intentions, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to provide appropriate feedback based on the user's emotional state, avoid conflict through neutral mediation, and promote more empathetic communication.

[0501] "Means of receiving information" refers to the function of acquiring data and messages from users in order to transmit them to other system components.

[0502] "Means for analyzing information and emotional data to assess the possibility of misunderstandings regarding emotional states and intentions" refers to a process that identifies users' emotions and communication intentions based on acquired data and assesses the risk of misunderstandings.

[0503] "Means for generating feedback and response suggestions using a generative artificial intelligence model" refers to a mechanism that uses natural language processing technology to automatically generate responses that are appropriate to the user's emotions and intentions.

[0504] "User" refers to an individual or organization that communicates through the system.

[0505] A "processing device" is a computer hardware or software system designed to perform a specific task.

[0506] "Means for setting the enablement or disablement of the neutral arbitration function" refers to a function that allows users to arbitrarily activate or deactivate the arbitration function.

[0507] This invention is an emotion-aware messaging system that reduces misunderstandings of intent and emotions between users and supports smooth communication. The system primarily consists of a terminal, a server, an emotion analysis engine, and a generative AI model.

[0508] The user sends messages through the device. The device is equipped with a camera to capture the user's facial expressions and a microphone to collect their voice tone. The device uses these devices to collect the user's emotional data along with the message data and sends it to the emotion analysis engine.

[0509] The emotion analysis engine uses image processing technologies such as OpenCV and speech analysis technologies such as LibROSA to identify the user's emotional state in real time. This emotion data is sent to a server, which uses a generative AI model to analyze the content of the received message and the user's emotional state.

[0510] The server uses generative AI models such as BERT and GPT to analyze the information and generate appropriate feedback and response suggestions. In doing so, it uses prompts to the generative AI model such as, "If the user is showing signs of anxiety, suggest a comforting feedback message to provide." This generated feedback is tailored to the user's emotional state and presented to the user via the device.

[0511] As a concrete example, suppose user A receives a message from their friend, user B. If the system detects dissatisfaction in the message, user A's device sends an expression of anxiety to an emotion analysis engine. Based on this data, the server performs an AI analysis and provides user A with appropriate feedback. This allows user A to calmly consider their reply and avoid conflict. This system enables more intimate communication between users.

[0512] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0513] Step 1:

[0514] The user composes a message using a messaging application and presses the send button. The input here is the text or voice message created by the user. The device captures this message data and simultaneously uses its built-in camera and microphone to acquire images of the user's facial expressions and voice tone. As a result, data indicating the user's emotions is output.

[0515] Step 2:

[0516] The device sends the acquired message data and emotion data to the emotion analysis engine. The emotion analysis engine uses image processing software and speech analysis software to analyze the user's emotional state in real time based on the input image and audio data. At this stage, the results of the analyzed emotional state are output. Specifically, OpenCV analyzes facial feature points, and LibROSA detects changes in voice tone.

[0517] Step 3:

[0518] The device sends the analyzed sentiment data and message content to the server. The server receives this data and analyzes it using a generative AI model. It receives the sentiment state and message content as input and evaluates the intent and emotional tone from them. The generative AI model (e.g., GPT) also generates a feedback message based on the input information and creates appropriate reply suggestions. The output consists of a feedback message and suggested replies.

[0519] Step 4:

[0520] The server sends back a generated feedback message and suggested replies to the terminal. The terminal has a means of displaying this to the user, who then refers to the displayed feedback and suggested replies. The user can then consider the options and prepare a calm response. The resulting action is that the terminal's display shows the user multiple suggested replies.

[0521] Step 5:

[0522] The user selects a suggested reply or creates and sends a new reply. The device then sends the user's chosen or created reply data back to the server and sends the message to the recipient's device. At this point, the final data is output as the sent reply message. In this case, the user's decision is the final action.

[0523] (Application Example 2)

[0524] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0525] In today's service delivery environment, improving communication between customers and service providers is extremely important. However, a problem arises when the quality of service deteriorates and customer satisfaction is diminished due to the inability to properly understand the customer's emotions and intentions. This invention aims to enable the provision of higher quality, more humane services by analyzing the customer's emotional state in real time and providing appropriate feedback to the service provider.

[0526] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0527] In this invention, the server includes means for receiving message data, means for analyzing the message data and evaluating the emotional components and the possibility of misunderstanding of the content, and means for analyzing the customer's emotional state and generating information to display to the service provider. This enables the service provider to respond appropriately to the customer's emotional state.

[0528] "Message data" refers to information in the form of text, audio, or images that is exchanged through communication.

[0529] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0530] "Emotional components" are characteristic elements that indicate a user's emotional state.

[0531] "Potential for misunderstanding" refers to assessing the risk that the content of a message may be misunderstood by the recipient.

[0532] "Feedback" refers to information and advice provided based on the analysis results.

[0533] A "suggested reply" is a proposed message that would be appropriate as a response to the user.

[0534] "Generation" is the process of creating new information or data.

[0535] "Presentation" means showing information to the user.

[0536] "Customer emotional state" refers to the psychological or emotional condition exhibited by a customer.

[0537] A "service provider" is an individual or organization that provides services to a customer.

[0538] The system for implementing this invention mainly consists of a user terminal, a central server, an AI analysis engine, and an emotion analysis engine. The user terminal employs a device such as smart glasses and has the function of recognizing the customer's facial expressions and acquiring voice. Specific hardware examples include Google Glass and Vuzix Blade.

[0539] The server receives data sent from user terminals and processes it in real time. It uses OpenCV for image recognition and the Google Cloud Speech-to-Text API for speech analysis. Furthermore, it analyzes the data using a TensorFlow-based sentiment analysis model to understand the customer's emotional state. Based on these results, it generates appropriate information to display to service providers. Here, a generative AI model is utilized to create optimal feedback.

[0540] For example, if a customer shows signs of anxiety while asking a question about a product, the AI ​​analysis engine will generate advice such as "explain gently and carefully" and display it on the smart glasses' screen. In this way, service providers can respond in a way that takes customer emotions into consideration.

[0541] Example prompt: "This customer seems anxious. How can I reassure them?" This prompt is used by the AI ​​to provide appropriate feedback based on their emotional state.

[0542] This system enables personalized customer service and improves the quality of communication.

[0543] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0544] Step 1:

[0545] The smart glasses, which act as the terminal, capture the customer's facial expressions and voice. This generates image data and voice data as physical input data.

[0546] Step 2:

[0547] The device sends the acquired image and audio data to a central server. This step involves data encoding and secure transfer over a network.

[0548] Step 3:

[0549] The server analyzes the received image data using OpenCV and extracts emotional elements from facial expressions. Audio data is converted to text using the Google Cloud Speech-to-Text API, and emotions are analyzed from the tone. These processes generate analysis results that indicate the customer's emotional state.

[0550] Step 4:

[0551] The server uses a generated AI model to produce appropriate feedback and advice based on the analysis results. This prompt includes specific instructions such as, "This customer seems anxious. How can we reassure them?"

[0552] Step 5:

[0553] The server sends the generated feedback and advice information to the terminal. The server then reformats the data so that the information is displayed correctly on the terminal.

[0554] Step 6:

[0555] The device displays feedback and advice information on the smart glasses' display so that service providers can visually confirm it. Based on this display, service providers can take appropriate action for customers.

[0556] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0557] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0558] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0559] [Fourth Embodiment]

[0560] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0561] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0562] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0563] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0564] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0565] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0566] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0567] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0568] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0569] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0570] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0571] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0572] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0573] This invention is implemented as an AI arbitration system integrated into a messaging application. The system consists of a user's terminal, a central server, and an AI analysis engine. When a user sends a message, the terminal transmits this message data to the central server.

[0574] The server passes the received message data to an AI analysis engine, which analyzes the content and emotional tone. The AI ​​analysis engine detects emotional elements and potential misunderstandings within the message and generates a feedback message based on that. This process mitigates direct emotional conflicts that could exacerbate the problem.

[0575] The device presents the user with feedback and analysis results provided by the server. Furthermore, it suggests recommended replies generated by the AI ​​analysis engine, allowing the user to select the optimal reply or create their own.

[0576] For example, if user A sends an emotionally charged message to user B, the AI ​​analysis engine analyzes the message and determines that it contains feelings of anger or frustration. The server then provides user B with feedback that includes ways to alleviate these feelings and suggests a response that encourages calm dialogue. In this way, direct misunderstandings and emotional conflicts are prevented, and neutral and constructive communication is achieved.

[0577] This invention enables users to control their emotions during the communication process, leading to more effective and positive dialogue. The mediation function can be turned on or off at the user's discretion, allowing for flexible operation as needed.

[0578] The following describes the processing flow.

[0579] Step 1:

[0580] The user creates message data by typing a message and pressing the send button. The terminal retrieves this message data and instructs the central server to send it.

[0581] Step 2:

[0582] The server forwards the message data received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the text data and performs natural language processing to extract emotional tones and specific keywords.

[0583] Step 3:

[0584] The AI ​​analysis engine evaluates the message based on its analysis results, assessing whether there are emotional elements or potential misunderstandings, and generates corresponding feedback. In some cases, it prepares cautionary messages to alleviate tension.

[0585] Step 4:

[0586] The server receives the results from the AI ​​analysis engine and sends them to the user's device as a feedback message. If suggested replies exist, they are also provided.

[0587] Step 5:

[0588] The terminal displays feedback and suggested replies received from the server to the user. Based on this information, the user decides whether to select a suggested reply or to type a new message themselves.

[0589] Step 6:

[0590] Once the user decides on their reply and presses the send button, the device sends this reply data back to the server and begins the process of delivering it to the recipient's device.

[0591] Step 7:

[0592] The server receives the sent reply data, checks if the content requires re-analysis, and then sends the data to the recipient's device. This ensures that smooth communication between users continues.

[0593] (Example 1)

[0594] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0595] In recent years, with the increased use of messaging applications, emotional conflicts and misunderstandings have become more frequent. Such misunderstandings and emotional conflicts can hinder smooth communication and lead to a deterioration of interpersonal relationships. However, existing systems do not adequately analyze emotional elements or provide sufficient feedback, making it difficult to effectively resolve these communication problems. Against this backdrop, there is a need for a system that can prevent emotional conflicts and misunderstandings among users and promote constructive dialogue.

[0596] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0597] In this invention, the server includes means for receiving message information, means for analyzing the message information and evaluating the emotional elements and the possibility of misunderstanding the content, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to effectively analyze the emotional elements and the possibility of misunderstanding contained in the message and provide appropriate feedback and response suggestions to the user.

[0598] "Message information" refers to the content of communications in text or other data formats sent and received between users.

[0599] A "generative artificial intelligence model" is a type of AI technology used in natural language processing and other applications, referring to a machine learning model that aims to generate appropriate output based on input.

[0600] "Emotional elements" refer to indicators used to identify information that indicates human emotions or emotional tone contained within text or messages.

[0601] "Feedback" refers to information provided to users based on the analyzed results, for the purpose of improvement or appropriate action.

[0602] A "suggested reply" refers to a template or guide of responses that are recommended by the user based on AI analysis.

[0603] This invention is specifically implemented as an AI arbitration system integrated into a messaging application. This system consists of a user's terminal, a central server, and an AI analysis engine.

[0604] The terminal converts the messages sent by the user into a data format and transmits them to a central server over the network. The hardware used as a terminal is a common communication device such as a smartphone or computer, with a messaging application installed on it.

[0605] The server passes message information received from the terminal to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using natural language processing techniques. A generative AI model is used for this analysis. Specific examples include open-source natural language processing libraries and commercial AI platforms. The analysis results evaluate the emotional elements and potential for misunderstanding contained in the message.

[0606] Based on the analyzed results, the server sends back feedback and suggested replies generated by the AI ​​analysis engine to the terminal. This feedback includes advice to mitigate emotional conflict and specific suggested replies.

[0607] The device receives feedback from the server and presents it to the user. Based on this feedback, the user can choose the most appropriate response or create their own response based on their own judgment.

[0608] As a concrete example of its operation, if a user sends a message such as "I'm worried because the project is behind schedule," the AI ​​analysis engine will analyze this message and determine that it contains feelings of anxiety or worry. Based on this information, the server can then provide feedback to the user such as "Let's check the progress and think about the next steps."

[0609] The prompt text input to the generative AI model is in the format of, "Analyze the message, identify emotional elements, and generate appropriate feedback." In this way, the present invention provides users with a means to improve the quality of communication.

[0610] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0611] Step 1:

[0612] The user creates and sends a message using a terminal. The terminal converts this message into a data format and sends it to a central server over the network. The input is the text message entered by the user, and the output is the message data sent over the network. The terminal takes in the raw data from the user, encodes it in the appropriate format, and sends it.

[0613] Step 2:

[0614] The server receives message data from the terminal. Next, the server prepares this data to be passed to the AI ​​analysis engine. The input is the message data received from the terminal, and the output is pre-processed data to be fed into the AI ​​analysis engine. The server verifies the data and prepares it for processing.

[0615] Step 3:

[0616] The server sends message data to the AI ​​analysis engine. The AI ​​analysis engine analyzes the message content using a generative AI model. In this process, the message is tokenized and sentiment analysis is performed. The input is message data from the server, and the output is sentiment elements and potential misunderstandings as a result of the analysis. The analysis engine analyzes the message and quantifies the sentiment tone and potential misunderstandings.

[0617] Step 4:

[0618] The AI ​​analysis engine generates feedback messages and suggested replies based on the analysis results. This feedback includes advice to facilitate the conversation. The input is the analysis results, and the output is the feedback message and suggested replies. The engine applies a generative AI model to suggest appropriate communication guidance.

[0619] Step 5:

[0620] The server sends feedback messages and suggested replies from the AI ​​analysis engine back to the terminal. The input is feedback data from the analysis engine, and the output is feedback information sent to the terminal. The server formats this information appropriately and delivers it to the terminal.

[0621] Step 6:

[0622] The terminal displays the received feedback message and suggested replies to the user. The user can review the presented feedback and select or create an appropriate reply. The input is the feedback information from the server, and the output is what is displayed to the user. The terminal visually displays the information and assists the user in continuing the interaction.

[0623] (Application Example 1)

[0624] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0625] In modern businesses and social organizations, communication via electronic messages is widespread, but this process is prone to emotional conflict and misunderstandings. In particular, in emotionally charged situations, inappropriate expressions and misinterpretations can exacerbate problems. This can lead to deterioration of interpersonal relationships and friction within organizations, hindering efficient work performance. Therefore, systems are needed to prevent these issues from occurring.

[0626] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0627] In this invention, the server includes means for acquiring information, means for analyzing the information and evaluating the possibility of interpreting emotional elements and content, and means for generating response messages and proposed responses. This reduces emotional conflicts and misunderstandings that occur during the communication process, enabling neutral and constructive dialogue.

[0628] "Means of acquiring information" refers to system components that handle the initial input process for receiving messages and data from users and starting appropriate processing.

[0629] "Means for analyzing information and evaluating the possibility of interpreting emotional elements and content" refers to algorithms that analyze received information using natural language processing techniques to identify emotional components and potential misunderstandings contained in a message.

[0630] "Means for generating response messages and proposed responses" refers to system functions that provide users with appropriate feedback and proposed responses to confirm, based on the analysis results.

[0631] "Means of presenting to the user" refers to display methods that visually or audibly notify the user of the generated analysis results and feedback, and facilitate their understanding.

[0632] "Means for forwarding response information selected or entered by the user to the recipient" refers to a communication process that accurately delivers the user's selected reply to the recipient.

[0633] "Means for enabling / disabling neutral functions" refers to an interface that allows users to choose to turn the AI-based sentiment analysis function on or off as needed.

[0634] "Means for evaluating and improving communication safety" refer to system functions that identify potential risk factors in communication and prevent misunderstandings and conflicts between users.

[0635] This invention is a system for realizing an AI arbitration system in messaging applications. It mainly consists of a server, a user's terminal, and an AI analysis engine. The server receives messages exchanged between users and extracts information from them. The extracted information is sent to the AI ​​analysis engine. The AI ​​analysis engine uses machine learning libraries such as TensorFlow and PyTorch to perform natural language processing and evaluate the emotional elements of the messages and the possibility of misinterpretation. Based on the analysis results, the server automatically generates response messages and specific response suggestions and presents them to the user.

[0636] Users can view the analysis results through applications on their smartphones or desktop PCs. Here, users are presented with generated response suggestions and can select or edit their own responses as needed. The generated responses are then forwarded back to the recipient via the server.

[0637] As a concrete example, suppose a discussion about the progress of a project in the workplace is taking place via email. If an employee sends a message such as, "This project is a complete waste of time," the AI ​​analysis engine will analyze the negative emotions contained in the message and generate feedback such as, "This message may contain dissatisfaction. Please review it." In this way, it is possible to prevent emotional conflict and misunderstandings with the other party.

[0638] An example of a prompt for a generative AI model would be: "Analyze the sentiment expressed in this message and generate feedback and constructive response suggestions. Message: 'I am dissatisfied with the progress of this project.'" This prompt serves as the basic input for instructing the AI ​​analysis engine to perform sentiment analysis and generate response suggestions. This system allows users to reduce the risks in message communication and maintain healthy dialogue.

[0639] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0640] Step 1:

[0641] The server receives message data from the user's terminal. The input is the message sent by the user, and the output is the message data itself. This data is stored in a clear format in data storage for use in subsequent processing.

[0642] Step 2:

[0643] The server sends the received message data to the AI ​​analysis engine. The input is the message data acquired in step 1, and the output is the analysis result from the AI ​​analysis engine. The AI ​​analysis engine evaluates the emotional elements and potential for misunderstanding of the message using natural language processing techniques. This process includes, for example, sentiment analysis of the text and tone detection.

[0644] Step 3:

[0645] The AI ​​analysis engine generates response suggestions based on the analysis results. The input is the analysis results obtained in step 2, and the output is the generated response suggestions and feedback. Specifically, the generating AI model devises constructive responses based on the prompt text and creates data to provide to the user.

[0646] Step 4:

[0647] The server forwards the generated response and feedback to the user's terminal. The input is the data generated in step 3, and the output is the response and feedback the user receives. The terminal displays this information in the user interface for the user to review.

[0648] Step 5:

[0649] The user reviews the suggested replies presented on the device, selects or edits them as needed, and creates their reply. The input is the information presented on the device in step 4, and the output is the reply data confirmed by the user. Through this interface, the user can choose the reply that best maintains communication.

[0650] Step 6:

[0651] The terminal sends the user-selected or edited reply data to the server. The input is the user's reply data created in step 5, and the output is the final reply message sent to the server. This reply is then ready to be delivered to the recipient's terminal via the server.

[0652] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0653] This invention is implemented in a messaging application by combining an AI arbitration system with an emotion engine. This system recognizes the user's emotions and provides feedback based on that emotional state, thereby supporting more human-like and effective communication.

[0654] The system components include a user terminal, a central server, an AI analysis engine, and an emotion engine. When a user sends a message, the terminal receives the message data and also sends the user's facial expression image or voice tone to the emotion engine.

[0655] The emotion engine uses image recognition technology and voice analysis to recognize the user's emotional state in real time. This recognition result is sent to the server along with the message data, where the AI ​​analysis engine analyzes the message content and emotional tone.

[0656] Based on these analysis results, the server generates a feedback message. The generated feedback is adjusted according to the user's emotional state, as determined by the emotion engine. For example, if the user is angry, the feedback will include calming expressions, and if the user is sad, the feedback will emphasize comforting elements.

[0657] The device displays generated feedback and suggested responses to the user. The user can then use this information to select or create an appropriate response.

[0658] For example, if user A receives a message from user B expressing dissatisfaction, the emotion engine will recognize that user A's facial expression indicates anxiety. Based on this information, the AI ​​analysis engine will generate a response suggestion that offers comfort and reassurance to user A, and the device will present it to help user A deal with the situation calmly.

[0659] This system ensures that message exchanges take users' emotions into account, enabling more empathetic communication and helping to avoid unnecessary conflicts.

[0660] The following describes the processing flow.

[0661] Step 1:

[0662] The user types a message into the device and presses the send button. At this time, the device uses its camera or microphone to collect the user's facial expressions or voice.

[0663] Step 2:

[0664] The terminal sends the collected facial expression images or voice data along with the entered message data to the server.

[0665] Step 3:

[0666] The server sends the received message data to the AI ​​analysis engine and facial expression images or voice data to the emotion engine. The AI ​​analysis engine analyzes the message content, and the emotion engine analyzes the user's emotional state in real time.

[0667] Step 4:

[0668] The AI ​​analysis engine evaluates the emotional tone and potential for misunderstanding in a message, while the emotion engine determines the user's emotional state. Based on these results, the server generates appropriate feedback and suggested replies.

[0669] Step 5:

[0670] The server considers the emotion recognition results from the emotion engine and adjusts the feedback and suggested replies. The adjusted content is then sent to the terminal.

[0671] Step 6:

[0672] The device presents the user with tailored feedback and suggested responses. The user then selects or creates a response based on this information.

[0673] Step 7:

[0674] Once the user decides on a reply and sends it, the device sends this reply data to the server, and the normal message transmission process is executed.

[0675] This process allows users to receive feedback that takes their emotional state into account, enabling them to communicate more effectively.

[0676] (Example 2)

[0677] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In the exchange of information involving emotions, misunderstandings of users' intentions and feelings can lead to conflict and unnecessary stress. Traditional messaging systems have difficulty providing feedback that takes into account the emotional state of users, and have limitations in supporting empathetic and effective communication.

[0679] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0680] In this invention, the server includes means for receiving information, means for analyzing the information and emotional data and evaluating the possibility of misunderstandings regarding emotional state and intentions, and means for generating feedback and response suggestions using a generative artificial intelligence model. This makes it possible to provide appropriate feedback based on the user's emotional state, avoid conflict through neutral mediation, and promote more empathetic communication.

[0681] "Means of receiving information" refers to the function of acquiring data and messages from users in order to transmit them to other system components.

[0682] "Means for analyzing information and emotional data to assess the possibility of misunderstandings regarding emotional states and intentions" refers to a process that identifies users' emotions and communication intentions based on acquired data and assesses the risk of misunderstandings.

[0683] "Means for generating feedback and response suggestions using a generative artificial intelligence model" refers to a mechanism that uses natural language processing technology to automatically generate responses that are appropriate to the user's emotions and intentions.

[0684] "User" refers to an individual or organization that communicates through the system.

[0685] A "processing device" is a computer hardware or software system designed to perform a specific task.

[0686] "Means for setting the enablement or disablement of the neutral arbitration function" refers to a function that allows users to arbitrarily activate or deactivate the arbitration function.

[0687] This invention is an emotion-aware messaging system that reduces misunderstandings of intent and emotions between users and supports smooth communication. The system primarily consists of a terminal, a server, an emotion analysis engine, and a generative AI model.

[0688] The user sends messages through the device. The device is equipped with a camera to capture the user's facial expressions and a microphone to collect their voice tone. The device uses these devices to collect the user's emotional data along with the message data and sends it to the emotion analysis engine.

[0689] The emotion analysis engine uses image processing technologies such as OpenCV and speech analysis technologies such as LibROSA to identify the user's emotional state in real time. This emotion data is sent to a server, which uses a generative AI model to analyze the content of the received message and the user's emotional state.

[0690] The server uses generative AI models such as BERT and GPT to analyze the information and generate appropriate feedback and response suggestions. In doing so, it uses prompts to the generative AI model such as, "If the user is showing signs of anxiety, suggest a comforting feedback message to provide." This generated feedback is tailored to the user's emotional state and presented to the user via the device.

[0691] As a concrete example, suppose user A receives a message from their friend, user B. If the system detects dissatisfaction in the message, user A's device sends an expression of anxiety to an emotion analysis engine. Based on this data, the server performs an AI analysis and provides user A with appropriate feedback. This allows user A to calmly consider their reply and avoid conflict. This system enables more intimate communication between users.

[0692] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0693] Step 1:

[0694] The user composes a message using a messaging application and presses the send button. The input here is the text or voice message created by the user. The device captures this message data and simultaneously uses its built-in camera and microphone to acquire images of the user's facial expressions and voice tone. As a result, data indicating the user's emotions is output.

[0695] Step 2:

[0696] The device sends the acquired message data and emotion data to the emotion analysis engine. The emotion analysis engine uses image processing software and speech analysis software to analyze the user's emotional state in real time based on the input image and audio data. At this stage, the results of the analyzed emotional state are output. Specifically, OpenCV analyzes facial feature points, and LibROSA detects changes in voice tone.

[0697] Step 3:

[0698] The device sends the analyzed sentiment data and message content to the server. The server receives this data and analyzes it using a generative AI model. It receives the sentiment state and message content as input and evaluates the intent and emotional tone from them. The generative AI model (e.g., GPT) also generates a feedback message based on the input information and creates appropriate reply suggestions. The output consists of a feedback message and suggested replies.

[0699] Step 4:

[0700] The server sends back a generated feedback message and suggested replies to the terminal. The terminal has a means of displaying this to the user, who then refers to the displayed feedback and suggested replies. The user can then consider the options and prepare a calm response. The resulting action is that the terminal's display shows the user multiple suggested replies.

[0701] Step 5:

[0702] The user selects a suggested reply or creates and sends a new reply. The device then sends the user's chosen or created reply data back to the server and sends the message to the recipient's device. At this point, the final data is output as the sent reply message. In this case, the user's decision is the final action.

[0703] (Application Example 2)

[0704] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0705] In today's service delivery environment, improving communication between customers and service providers is extremely important. However, a problem arises when the quality of service deteriorates and customer satisfaction is diminished due to the inability to properly understand the customer's emotions and intentions. This invention aims to enable the provision of higher quality, more humane services by analyzing the customer's emotional state in real time and providing appropriate feedback to the service provider.

[0706] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0707] In this invention, the server includes means for receiving message data, means for analyzing the message data and evaluating the emotional components and the possibility of misunderstanding of the content, and means for analyzing the customer's emotional state and generating information to display to the service provider. This enables the service provider to respond appropriately to the customer's emotional state.

[0708] "Message data" refers to information in the form of text, audio, or images that is exchanged through communication.

[0709] "Analysis" is the process of breaking down data and understanding its structure and meaning.

[0710] "Emotional components" are characteristic elements that indicate a user's emotional state.

[0711] "Potential for misunderstanding" refers to assessing the risk that the content of a message may be misunderstood by the recipient.

[0712] "Feedback" refers to information and advice provided based on the analysis results.

[0713] A "suggested reply" is a proposed message that would be appropriate as a response to the user.

[0714] "Generation" is the process of creating new information or data.

[0715] "Presentation" means showing information to the user.

[0716] "Customer emotional state" refers to the psychological or emotional condition exhibited by a customer.

[0717] A "service provider" is an individual or organization that provides services to a customer.

[0718] The system for implementing this invention mainly consists of a user terminal, a central server, an AI analysis engine, and an emotion analysis engine. The user terminal employs a device such as smart glasses and has the function of recognizing the customer's facial expressions and acquiring voice. Specific hardware examples include Google Glass and Vuzix Blade.

[0719] The server receives data sent from user terminals and processes it in real time. It uses OpenCV for image recognition and the Google Cloud Speech-to-Text API for speech analysis. Furthermore, it analyzes the data using a TensorFlow-based sentiment analysis model to understand the customer's emotional state. Based on these results, it generates appropriate information to display to service providers. Here, a generative AI model is utilized to create optimal feedback.

[0720] For example, if a customer shows signs of anxiety while asking a question about a product, the AI ​​analysis engine will generate advice such as "explain gently and carefully" and display it on the smart glasses' screen. In this way, service providers can respond in a way that takes customer emotions into consideration.

[0721] Example prompt: "This customer seems anxious. How can I reassure them?" This prompt is used by the AI ​​to provide appropriate feedback based on their emotional state.

[0722] This system enables personalized customer service and improves the quality of communication.

[0723] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0724] Step 1:

[0725] The smart glasses, which act as the terminal, capture the customer's facial expressions and voice. This generates image data and voice data as physical input data.

[0726] Step 2:

[0727] The device sends the acquired image and audio data to a central server. This step involves data encoding and secure transfer over a network.

[0728] Step 3:

[0729] The server analyzes the received image data using OpenCV and extracts emotional elements from facial expressions. Audio data is converted to text using the Google Cloud Speech-to-Text API, and emotions are analyzed from the tone. These processes generate analysis results that indicate the customer's emotional state.

[0730] Step 4:

[0731] The server uses a generated AI model to produce appropriate feedback and advice based on the analysis results. This prompt includes specific instructions such as, "This customer seems anxious. How can we reassure them?"

[0732] Step 5:

[0733] The server sends the generated feedback and advice information to the terminal. The server then reformats the data so that the information is displayed correctly on the terminal.

[0734] Step 6:

[0735] The device displays feedback and advice information on the smart glasses' display so that service providers can visually confirm it. Based on this display, service providers can take appropriate action for customers.

[0736] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0737] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include those described above. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions shown by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0738] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0739] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0740] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0741] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0742] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0743] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0744] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0745] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0746] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0747] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0748] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0749] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0750] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0751] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0752] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0753] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0754] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0755] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0756] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0757] The following is further disclosed regarding the embodiments described above.

[0758] (Claim 1)

[0759] A means of receiving message data,

[0760] A means for analyzing the aforementioned message data and evaluating the possibility of misunderstanding of emotional components and content,

[0761] A means for generating feedback and response proposals based on the analysis results,

[0762] A means of presenting the generated feedback and suggested responses to the user,

[0763] A means of sending reply data selected or entered by the user to the other party,

[0764] A system that includes this.

[0765] (Claim 2)

[0766] The system according to claim 1, further comprising means for generating a warning message to the message sender based on the analysis results.

[0767] (Claim 3)

[0768] The system according to claim 1, further comprising means for the user to turn a neutral AI arbitration function on or off.

[0769] "Example 1"

[0770] (Claim 1)

[0771] A means of receiving message information,

[0772] A means for analyzing the aforementioned message information and evaluating the possibility of misunderstanding of emotional elements and content,

[0773] A means for generating feedback and response proposals based on the analysis results using a generative artificial intelligence model,

[0774] A means of presenting the generated feedback and proposed responses to the user,

[0775] A means of sending reply information selected or entered by the user to the other party,

[0776] A system that includes this.

[0777] (Claim 2)

[0778] The system according to claim 1, further comprising means for generating cautionary information for the message sender based on the analysis results.

[0779] (Claim 3)

[0780] The system according to claim 1, further comprising means for the user to set the neutral artificial intelligence arbitration function on or off.

[0781] "Application Example 1"

[0782] (Claim 1)

[0783] Means of obtaining information,

[0784] A means for analyzing the aforementioned information and evaluating the possibility of interpreting emotional elements and content,

[0785] A means for generating a response message and a proposed response based on the analysis results,

[0786] A means for presenting the generated response message and proposed response to the user,

[0787] A means for forwarding response information selected or entered by the user to the destination,

[0788] A means for users to enable or disable the neutral function,

[0789] Means for evaluating and improving the safety of communication,

[0790] A system that includes this.

[0791] (Claim 2)

[0792] The system according to claim 1, further comprising means for generating cautionary information for information providers based on the aforementioned analysis results.

[0793] (Claim 3)

[0794] The system according to claim 1, further comprising means for providing information to prevent internal conflicts based on the aforementioned analysis results.

[0795] "Example 2 of combining an emotion engine"

[0796] (Claim 1)

[0797] Means of receiving information,

[0798] A means for analyzing the aforementioned information and emotional data to evaluate the possibility of misunderstandings regarding emotional state and intentions,

[0799] Based on the aforementioned analysis results, means for generating feedback and response suggestions using a generative artificial intelligence model,

[0800] A means of presenting the generated feedback and suggested replies to the user,

[0801] A means of transmitting response data selected or entered by the user to the recipient,

[0802] A processing device that includes a processing device.

[0803] (Claim 2)

[0804] The processing apparatus according to claim 1, further comprising means for generating a warning to the information sender based on the analysis results.

[0805] (Claim 3)

[0806] The processing apparatus according to claim 1, further comprising means for a user to enable or disable a neutral arbitration function.

[0807] "Application example 2 when combining with an emotional engine"

[0808] (Claim 1)

[0809] A means of receiving message data,

[0810] A means for analyzing the aforementioned message data and evaluating the possibility of misunderstanding of emotional components and content,

[0811] A means for generating feedback and response proposals based on the analysis results,

[0812] A means of presenting the generated feedback and suggested responses to the user,

[0813] A means of sending reply data selected or entered by the user to the other party,

[0814] A means for analyzing the emotional state of customers and generating information to display to service providers,

[0815] A system that includes this.

[0816] (Claim 2)

[0817] The system according to claim 1, further comprising means for generating a warning message to the message sender based on the analysis results.

[0818] (Claim 3)

[0819] The system according to claim 1, further comprising means for the user to turn a neutral AI arbitration function on or off. [Explanation of Symbols]

[0820] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving message data, A means for analyzing the aforementioned message data and evaluating the possibility of misunderstanding of emotional components and content, A means for generating feedback and response proposals based on the analysis results, A means of presenting the generated feedback and suggested responses to the user, A means of sending reply data selected or entered by the user to the other party, A system that includes this.

2. The system according to claim 1, further comprising means for generating a warning message to the message sender based on the analysis results.

3. The system according to claim 1, further comprising means for the user to turn a neutral AI arbitration function on or off.

Images