system

Abstract

We provide the system. [Solution] A means of obtaining communication content entered by the user, A means of evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content, A means of notifying the user of warnings and suggestions for correcting communication content based on the evaluation results, A means of performing the appropriate action when the user selects to modify or cancel communication, A system that performs technical processing in real time while providing means to offer corrective suggestions for workers to communicate with users in an appropriate and user-friendly manner.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the spread of text-based communication means such as SNS, chat, and mail, situations where slander and harassment are likely to occur among users have emerged. Such situations may particularly have a profound impact on the younger generation and enterprises, etc., and have become a major social problem. Conventional methods generally respond after these problems occur, and there is a lack of effective means for pre-suppression. Therefore, it is required to detect inappropriate expressions before message transmission and prevent their occurrence.

Means for Solving the Problems

[0005] This invention provides a system that acquires messages entered by users, analyzes them using a generative model, and evaluates the potential for inappropriate language and harassment. Based on the evaluation results, the system notifies the user in real time with warnings and correction suggestions, enabling the user to modify or cancel the message. Furthermore, the system is designed to perform a more accurate risk assessment by considering the time of day the message is sent and the attributes of the recipient during message analysis. This contributes to maintaining an appropriate and healthy environment in text-based communication.

[0006] A "user" refers to a person who creates and sends messages using this system.

[0007] A "message" refers to text-based information, specifically content sent via social media, chat, email, etc.

[0008] "Generative models" refer to artificial intelligence technology used to analyze message content and evaluate the possibility of inappropriate language or harassment.

[0009] A "warning" refers to a notification that alerts users to messages containing inappropriate language or potentially harassing content.

[0010] "Revision suggestions" refers to a function that provides specific editing proposals to improve a message created by a user.

[0011] Considering the "level of risk" refers to determining the degree of inappropriateness based on the message's content, recipient, time of day, etc.

[0012] "Device" refers to a computer, smartphone, or other digital device that a user uses to compose and send messages.

[0013] A "server" refers to a network-based computer system that provides generative model analysis capabilities and returns message evaluation results to the user's terminal.

[0014] "Notification" refers to the action of informing the user of the results of message analysis, and is the information that is visually displayed on the user interface. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The system according to the present invention mainly consists of a terminal used by the user, a server that performs data processing, and an interface for communication between them. Specific examples of its operation and implementation are shown below.

[0037] The user first enters a message using an SNS, chat, or email application on their device. At this point, the message the user attempts to send is temporarily held on the device and prepared for transmission to the server. When the send button is pressed, the device sends the message content to the server, and message analysis begins.

[0038] The server inputs received messages into a generating AI model, which analyzes the text's context and content to determine if it contains inappropriate language or harassment. This analysis utilizes keyword extraction, sentiment analysis, and sentiment analysis techniques. Furthermore, the time of message transmission and recipient information are also considered to calculate an overall risk score.

[0039] Based on the analysis results, the server generates a warning message for messages deemed high-risk. It also simultaneously generates correction suggestions to encourage the user to make improvements, and sends these back to the terminal.

[0040] The terminal immediately notifies the user of any warnings and correction suggestions received from the server. This allows the user to understand the potential impact of the message and choose to correct the message content or cancel sending it as needed. The corrected message can be sent again by pressing the send button, after which it will be analyzed by the server once more.

[0041] For example, if a user tries to send a message to a colleague in a business setting saying, "There are too many problems with your work," the server will deem the message inappropriate and send a warning to the user's device with a suggestion to revise it, such as, "Why not try to provide more specific guidance?" As a result, the user has the opportunity to revise the message to make it more constructive.

[0042] This system allows users to re-evaluate messages before sending them, thereby preventing harassment and inappropriate communication.

[0043] The following describes the processing flow.

[0044] Step 1:

[0045] When a user types a message on their device and presses the send button, the device captures the entered message.

[0046] Step 2:

[0047] The terminal prepares to send the captured message to the server using a pre-configured communication protocol.

[0048] Step 3:

[0049] When the server receives a message, it initiates a request for analysis to the generative AI model. At this point, the message is first tokenized for natural language processing.

[0050] Step 4:

[0051] The server passes tokenized message data to a generating AI model, which evaluates the potential for inappropriate language and harassment while considering the context.

[0052] Step 5:

[0053] The server calculates a risk score based on the evaluation results, and if the risk level is high, it generates a warning message and suggested corrective actions.

[0054] Step 6:

[0055] Warnings and suggestions generated by the server are sent to the user's terminal in real time, and the terminal displays them to the user as alerts.

[0056] Step 7:

[0057] The user reviews the displayed warning and chooses to modify the message as needed or cancel sending it. If modifications are made, the message will attempt to be sent again.

[0058] Step 8:

[0059] If the terminal resends the corrected message, the server re-evaluates it and, if there are no problems, proceeds to deliver the message to the recipient.

[0060] Through these steps, the system attempts to effectively suppress inappropriate communication.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] In modern communication methods, there is a problem in that emotionally or unintentionally inappropriate expressions by users can easily spread. This often leads to deterioration of relationships and social trouble. Furthermore, because message content is transmitted instantly, there is a challenge in that there is no time to re-evaluate the content before sending it.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for acquiring information input from an information processing device, means for evaluating the possibility of inappropriate expressions and aggressive behavior using a generative AI model to analyze the acquired information, and means for evaluating the degree of risk taking into account time information and recipient attribute information at the time of information transmission. This allows users to re-evaluate the message content in advance and improve the content as necessary.

[0066] An "information processing device" is a device that has the function of inputting and processing information such as messages.

[0067] A "generative AI model" is an artificial intelligence model that performs natural language processing to understand and analyze the context and content of input text.

[0068] "Inappropriate language" refers to language that may cause offense or aggression to the recipient or a third party.

[0069] "Aggressive behavior" refers to any action that is intended to hurt or offend others, or has the potential to do so.

[0070] "Risk level" is an indicator that shows the degree to which a message being sent is inappropriate or likely to cause social problems.

[0071] "Communication software" refers to programs used by users to exchange messages and information.

[0072] "Attribute information" refers to data that indicates the characteristics and profile information of the recipient.

[0073] This invention primarily utilizes an information processing device (hereinafter referred to as a terminal), a remote device for information processing (hereinafter referred to as a server), and means for exchanging information between them to configure a system for pre-evaluating messages. The user inputs a message using communication software on the terminal and attempts to send it. At this time, the information is temporarily stored on the terminal.

[0074] Upon receiving information from a terminal, the server begins analysis using a generative AI model. This model employs natural language processing techniques to understand the context of the information and assess the potential for inappropriate language or aggressive behavior. The analysis is carried out using keyword extraction, sentiment analysis, and other similar methods. Furthermore, the server also considers the time information of the transmitted information and the recipient's attributes to comprehensively evaluate the level of risk.

[0075] Based on the analysis results, the server generates a warning message and correction suggestions, and sends them back to the terminal. The terminal notifies the user of the received warnings and correction suggestions, and allows the user to choose to correct the information or stop sending it as needed.

[0076] For example, if a user tries to send a message to a colleague saying, "There are too many problems with your work," the server may deem this message inappropriate and suggest a revised version, such as, "Please consider giving more specific and constructive feedback." This system allows users to re-evaluate information and prevent harassment and inappropriate communication.

[0077] An example of a prompt message could be something like, "Is this message inappropriate? How should it be corrected?" which could be passed to the server. In this way, the generative AI model sequentially evaluates the appropriateness of the information.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user enters a message using the terminal's communication software and presses the send button. The entered message is saved on the terminal and enters a temporary waiting state. At this stage, the message's text data is output as ready for transmission.

[0081] Step 2:

[0082] After the send button is pressed, the terminal sends the saved message to the server. The input on the server is the message data from the terminal, and the output is an acknowledgment of receipt of this data. The server passes the message content to the generating AI model and inputs prompts for text analysis.

[0083] Step 3:

[0084] The server inputs the received message into a generating AI model, which then begins contextual understanding and keyword extraction. The model uses natural language processing techniques to analyze the emotions and sentiments within the text and calculate the risk level. The input here is the message text, and the output is a data object containing the analysis results.

[0085] Step 4:

[0086] The server evaluates the potential for inappropriate language and offensive behavior based on the generated analysis results. Based on this evaluation, the server generates appropriate warning messages and correction suggestions. At this stage, the input is the analysis results, and the output is the text data of the warnings and correction suggestions.

[0087] Step 5:

[0088] The server sends the generated warning message and correction suggestions back to the terminal. The terminal receives this data and notifies the user. The input is the suggested data from the server, and the output is a visual or auditory notification to the user.

[0089] Step 6:

[0090] The user reviews the received warnings and correction suggestions and chooses to revise the message or cancel sending it as needed. If corrections are made, the user re-edits the message and proceeds with the sending process again. At this point, the input is the suggestions received by the user, and the output is the revised message.

[0091] Step 7:

[0092] Once the corrected message is sent via the resend button, the server parses the message again to determine whether to send it or not. In this step, the input is the corrected message, and the output is feedback on the final sending result.

[0093] (Application Example 1)

[0094] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In electronic payment services, it is crucial for customer support staff to provide appropriate and friendly service to users. However, especially in situations requiring immediate responses, inappropriate or misleading language is often unavoidable. This can damage the user experience and risk lowering the service's rating. This invention aims to prevent inappropriate language in such communication situations and achieve a comfortable user experience.

[0096] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0097] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content, and means for notifying the user of warnings and suggestions for correcting the communication content based on the evaluation results. This ensures that the communications the user receives through customer support are always appropriate and user-friendly, and as a result, enables the provision of a better user experience.

[0098] A "user" is an individual or group that uses the system and is the entity that inputs the communication content.

[0099] "Communication content" refers to messages and conversations that a user attempts to input or send, and is subject to analysis.

[0100] A "knowledge model" is a machine learning or artificial intelligence algorithm used to evaluate the inappropriateness or harmfulness of communication content using natural language processing techniques.

[0101] "Inappropriate language" refers to language that may offend others or is considered socially undesirable.

[0102] "Harassment" refers to actions or expressions that are intentionally made to offend someone, and is usually done intentionally.

[0103] "Evaluation results" refer to judgments regarding the inappropriateness or risk level of communication content, based on analysis using a knowledge model.

[0104] A "warning" is a notification to a user indicating that the content of their communication may be inappropriate.

[0105] A "correction suggestion" refers to alternative expressions or methods of improvement presented to the user to correct inappropriate parts of the communication content.

[0106] The system implementing this invention consists of a user, a server, and a terminal. When a user inputs communication content and attempts to send it through the terminal, the terminal first acquires the communication content. The terminal sends the communication content to the server, which analyzes the content using a knowledge model. In this analysis, natural language processing techniques are used to evaluate whether the communication content is inappropriate or contains harassment. In this process, the server utilizes NLTK and TENSORFLOW®, which are natural language processing libraries implemented in Python. Based on the analysis results, the server generates warnings and correction suggestions, which are then sent to the terminal.

[0107] The terminal immediately notifies the user of any received warnings and corrective suggestions. Based on these notifications, the user re-evaluates the communication content and makes corrections or cancels the transmission. For example, if a customer support user is about to send a message such as "We apologize for the inconvenience," the server can provide a corrective suggestion such as "Why not offer more specific solutions?" This function helps the server provide contextually appropriate responses on behalf of the user.

[0108] As a concrete example, the AI ​​generation model can be instructed with a prompt message such as, "Evaluate whether this message is easy for the customer to understand, and generate improvement suggestions if necessary." This allows the system to efficiently check the communication content and make appropriate suggestions.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The user inputs the communication content into the device. The device captures this communication content and stores it in temporary storage. This communication content will be used as input for analysis in the next step.

[0112] Step 2:

[0113] The terminal sends the communication content to the server. The server uses a generative AI model to analyze the received communication content. First, it extracts keywords and analyzes the context based on natural language processing techniques to generate data necessary to evaluate whether the communication content is inappropriate. The output of this analysis is an inappropriateness score.

[0114] Step 3:

[0115] The server uses a generative AI model to generate warning messages and corrective action suggestions based on the inappropriateness score. This process uses the prompt "Evaluate whether this message is easy for customers to understand and generate improvement suggestions if necessary." The generated warnings and corrective action suggestions are then output for transmission to the terminal.

[0116] Step 4:

[0117] Warnings and suggested corrections are sent from the server to the terminal. The terminal immediately displays these notifications to the user. In this step, the terminal is responsible for providing the user with the output data from the server.

[0118] Step 5:

[0119] The user reviews the warnings and correction suggestions and chooses to correct the communication content or cancel transmission. If the user chooses to correct, the terminal can resend the corrected communication content to the server and repeat the analysis process. Since the user's choice determines the final content of the communication, the output at this step reflects the user's decision.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] The system according to the present invention is intended for a messaging application equipped with emotion recognition functionality used by users on their terminals. This system provides more advanced message evaluation by using a server that works in conjunction with an emotion engine that receives message input from the user and performs real-time emotion analysis on it.

[0122] Users open social media, chat, or email applications on their devices and type messages. During this process, a sentiment engine analyzes the user's input patterns and wording in the background, evaluating the user's emotional state in real time. This information is sent from the device to the server, where general contextual analysis and sentiment analysis of the message are performed simultaneously.

[0123] The server not only evaluates the potential for inappropriate language and harassment in messages through a generative AI model, but also considers emotional data obtained by the emotion engine to generate warnings and corrective suggestions tailored to the user's emotions. For example, if the user is expressing stress or negative emotions, the server generates a more emphatic message and sends it back to the device.

[0124] The device promptly presents the user with warnings, suggestions, and even additional, emotion-based advice received from the server. The user can then modify their messages based on this information, facilitating more appropriate communication.

[0125] For example, if a user enters a message expressing strong dissatisfaction with workplace communication, the emotion engine detects the high stress level. Based on this emotion information and the message content, the server suggests less misleading expressions and provides feedback in a gentle tone, giving the user an opportunity to reconsider their message.

[0126] By implementing this system, users can receive immediate, emotionally sensitive feedback, which in turn helps prevent harassment and inappropriate language.

[0127] The following describes the processing flow.

[0128] Step 1:

[0129] The user enters a message using the messaging application on their device. The emotion engine analyzes the user's keystrokes and input patterns in the background as they type, and evaluates the user's emotional state.

[0130] Step 2:

[0131] The terminal packages the input message and the emotion data output by the emotion engine, and sends them to the server.

[0132] Step 3:

[0133] The server first sends the received message and sentiment data to a generative AI model to begin the process of evaluating the possibility of inappropriate language or harassment.

[0134] Step 4:

[0135] The server simultaneously uses sentiment data to analyze whether the user's emotional state is influencing the message content, and incorporates this into the contextual evaluation.

[0136] Step 5:

[0137] The server integrates the results of the generated AI model and sentiment analysis to calculate a risk score. Based on this score, it generates appropriate warning messages, corrective action suggestions, and even emotionally sensitive advice.

[0138] Step 6:

[0139] The server compiles the generated warnings, suggestions, and advice and sends them back to the terminal.

[0140] Step 7:

[0141] The device immediately notifies the user of any received warnings or suggestions. The notifications also include additional advice based on sentiment analysis, which the user can use to decide whether to modify the message or cancel sending it.

[0142] Step 8:

[0143] If a user modifies and resends a message, the device sends the modified message back to the server for evaluation. If it determines that there are no problems, the message is delivered to the recipient.

[0144] This approach aims to achieve effective communication by providing more human-like feedback from the system.

[0145] (Example 2)

[0146] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0147] In today's communication environment, users are required to send emotionally charged messages quickly and appropriately. However, because emotional judgments cannot be made instantaneously, there is a risk that inappropriate language or potentially human rights-violating messages may be accidentally sent. Therefore, there is a need for a system that allows users to receive appropriate feedback immediately when sending emotionally charged messages.

[0148] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0149] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate expression and human rights violations using a statistical model to analyze the acquired communication content, and means for notifying the user of warnings and correction suggestions based on the acquired emotional state. This enables users to take their emotions into consideration, prevent inappropriate expression during communication, and send safer and more appropriate messages.

[0150] "Communication content" refers to the text messages and information that a user intends to send, which may include emotions and intentions.

[0151] A "statistical model" is a computational method that uses statistical techniques to analyze data and evaluate the inappropriateness of communication content and the risks related to human rights.

[0152] "Potential human rights violation" refers to the possibility that the message content infringes upon the fundamental rights of others, and problematic expressions are identified by assessing this risk.

[0153] An "emotion analysis device" is a system that detects the emotions and tone contained in a user's communication content and quantifies or categorizes them.

[0154] A "pattern detection algorithm" is a method for identifying certain patterns within data and generating evaluations or warnings that correspond to the user's emotions and intentions.

[0155] An embodiment of the present invention is a communication support system using an emotion analysis and generation AI model for use by users on a communication terminal. This system analyzes the communication content entered by the user in real time and provides feedback to prevent inappropriate expressions and human rights violations.

[0156] The terminal first acquires the text entered by the user. The acquired content is then analyzed by an emotion analysis device. This device uses natural language processing technology to quantify the emotions contained in the communication content and evaluates the user's emotional state.

[0157] Emotional data and communication content are sent together to the server. The server consists of hardware with advanced computing power and is equipped with statistical models. Based on the received data, this server uses a generative AI model to perform contextual analysis. Through contextual analysis, it evaluates whether the message is inappropriate or poses a risk of human rights violations. Furthermore, to enable quick decision-making, the server generates feedback tailored to the user's emotional state, providing warnings and correction suggestions.

[0158] The generated feedback is immediately sent to the device. The device then presents this information to the user, giving the user the opportunity to modify their communication based on the feedback.

[0159] For example, if a user types "This meeting is a complete waste of time," the sentiment analyzer will detect frustration. The server will then generate and provide a calmer suggestion to the user, such as "How about reconfirming the purpose of the meeting?"

[0160] An example of a prompt might be, "Generate suggestions for improving the expression based on emotion." Based on this prompt, the generation AI model generates appropriate feedback on the communication content.

[0161] By utilizing this system, users will be able to express their emotions in a more appropriate way, resulting in smoother communication.

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The user inputs the communication content via a terminal. The terminal receives this input, converts it into a data format, and sends it to the sentiment analysis device. The input is the text entered by the user, and the output is data ready for analysis. Specifically, the text undergoes initial formatting and is formatted in a way that is suitable for analysis.

[0165] Step 2:

[0166] The terminal analyzes communication content using an emotion analysis device. The device uses natural language processing technology to analyze keywords and context from the input text and generate emotion data. The input is formatted text data, and the output is numerical or categorical data indicating the emotional state. Specific analysis operations include keyword detection and sentiment score calculation.

[0167] Step 3:

[0168] The terminal sends the generated sentiment data and communication content to the server. The server receives this data and performs a detailed analysis using statistical models and generative AI models. The input is a set of sentiment data and text data, and the output is an evaluation result that takes context and sentiment into account. Specifically, the model performs pattern detection and risk assessment simultaneously.

[0169] Step 4:

[0170] The server generates warnings and correction suggestions based on the evaluation results. The generation AI model creates appropriate feedback according to the given prompt. The input is the evaluation results and prompt, and the output is a feedback message to notify the user. Specifically, the process involves generating the suggested content and formatting the message.

[0171] Step 5:

[0172] The server sends the generated feedback to the terminal, which then displays it to the user. The user re-evaluates the communication based on the provided feedback and makes corrections as needed. The input is the feedback data received from the server, and the output is the user's corresponding action. Specific actions include displaying notification pop-ups and presenting feedback through the user interface.

[0173] (Application Example 2)

[0174] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0175] In recent years, online communication has been steadily increasing, but inappropriate language and harassment have become a problem. Furthermore, there are many cases where such content significantly impacts users' mental state and emotions. In response, there is a need for the development of systems that can detect inappropriate content in real time and provide feedback that takes users' feelings into consideration.

[0176] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0177] In this invention, the server includes means for acquiring information input from the user, means for evaluating the potential for inappropriate expressions and harassment using a generative model to analyze the acquired information, and means for providing warnings, correction suggestions, and sentiment-sensitive feedback based on the evaluation results and sentiment analysis. This enables the user to improve their message expression in real time and obtain a safe and comfortable communication environment.

[0178] A "user" is the entity that inputs information and receives analysis results and feedback from the system.

[0179] "Inputted information" refers to the content of text and messages provided by the user to the system.

[0180] A "generative model" is an algorithm or program used for natural language processing, specifically for detecting inappropriate expressions and harassment.

[0181] "Inappropriate language" refers to expressions or phrases that may cause misunderstandings or conflicts in online communication.

[0182] "Potential harassment" refers to harassment or the risk of harassment that can be inferred from the information entered by the user.

[0183] "Evaluation results" refer to the identification of inappropriate expressions and harassment behaviors derived from the generative model, along with detailed information about them.

[0184] "Emotional analysis" is the process of inferring and analyzing a user's emotional state from the input information.

[0185] A "warning" is a message or notification that alerts the user based on the evaluation results.

[0186] A "revision suggestion" is a specific proposal provided to the user for correcting content deemed inappropriate.

[0187] "Emotionally sensitive feedback" refers to information and suggestions provided while taking into account the user's mental state and emotions.

[0188] This invention consists of a system that performs real-time sentiment analysis and evaluation of the appropriateness of expression on information input by the user, and provides feedback. The system mainly consists of an emotion engine for recognizing and evaluating emotions, a generative AI model for analyzing messages, a server for aggregating and processing information, and a terminal for notifying the user of the evaluation results.

[0189] First, the user enters a message on their device. This device can be a smartphone or computer and is integrated with a communication platform. The entered information is sent directly to the server. On the server, an application built using Python or Django analyzes this information using a generative AI model (for example, OpenAI's GPT-3). The analysis uses natural language processing libraries (NLTK, spaCy, TextBlob) to evaluate the possibility of inappropriate language or harassment, and simultaneously performs sentiment analysis using a sentiment engine.

[0190] The server determines the user's mental state based on the generated evaluation results and sentiment analysis data. This information is fed back to the device as warnings and correction suggestions, and the device notifies the user of this information without delay. This allows the user to check whether their message is inappropriate and correct it to a more appropriate expression.

[0191] For example, when a user at work tries to send a team message to a colleague that contains strong emotions, the emotion engine detects the high stress level. The server then uses a generative model to suggest revisions to adjust the tone of the message, offering advice such as, "How about phrasing it like this?"

[0192] An example of a prompt message is, "If the entered message is offensive, please suggest a way to rephrase it in a more positive and constructive tone."

[0193] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0194] Step 1:

[0195] The user enters a message from their device. The entered information is immediately sent to the server via the communication platform. The input here is raw text data entered by the user.

[0196] Step 2:

[0197] The server passes the received information to the emotion engine and the generative AI model. The emotion engine uses natural language processing libraries (such as NLTK and spaCy) to analyze the user's emotional state from the input text. The type and intensity of the emotion are output and used for the next processing step.

[0198] Step 3:

[0199] The generative AI model evaluates the potential for inappropriate expressions and harassment based on analyzed sentiment information and text data. The generative AI model (such as OpenAI's GPT-3) identifies inappropriate elements from the input data and calculates their likelihood. The evaluation results are generated on the server side and used as feedback.

[0200] Step 4:

[0201] The server integrates the generated evaluation results and sentiment analysis data to produce appropriate correction suggestions and warnings. The output here is a specific feedback message indicating what correction suggestions or warnings are best for the user.

[0202] Step 5:

[0203] A feedback message is sent to the device and notified to the user. The user receives this notification and, if necessary, modifies or cancels the message. The final revision of the message is made based on the user's actions.

[0204] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0205] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0206] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0207] [Second Embodiment]

[0208] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0209] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0210] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0211] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0212] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0213] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0214] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0215] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0216] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0217] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0218] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0219] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0220] The system according to the present invention mainly consists of a terminal used by the user, a server that performs data processing, and an interface for communication between them. Specific examples of its operation and implementation are shown below.

[0221] The user first enters a message using an SNS, chat, or email application on their device. At this point, the message the user attempts to send is temporarily held on the device and prepared for transmission to the server. When the send button is pressed, the device sends the message content to the server, and message analysis begins.

[0222] The server inputs received messages into a generating AI model, which analyzes the text's context and content to determine if it contains inappropriate language or harassment. This analysis utilizes keyword extraction, sentiment analysis, and sentiment analysis techniques. Furthermore, the time of message transmission and recipient information are also considered to calculate an overall risk score.

[0223] Based on the analysis results, the server generates a warning message for messages deemed high-risk. It also simultaneously generates correction suggestions to encourage the user to make improvements, and sends all of these back to the terminal.

[0224] The terminal immediately notifies the user of any warnings and correction suggestions received from the server. This allows the user to understand the potential impact of the message and choose to correct the message content or cancel sending it as needed. The corrected message can be sent again by pressing the send button, after which it will be analyzed by the server once more.

[0225] For example, if a user tries to send a message to a colleague in a business setting saying, "There are too many problems with your work," the server will deem the message inappropriate and send a warning to the user's device with a suggestion to revise it, such as, "Why not try to provide more specific guidance?" As a result, the user has the opportunity to revise the message to make it more constructive.

[0226] This system allows users to re-evaluate messages before sending them, thereby preventing harassment and inappropriate communication.

[0227] The following describes the processing flow.

[0228] Step 1:

[0229] When a user types a message on their device and presses the send button, the device captures the entered message.

[0230] Step 2:

[0231] The terminal prepares to send the captured message to the server using a pre-configured communication protocol.

[0232] Step 3:

[0233] When the server receives a message, it initiates a request for analysis to the generative AI model. At this point, the message is first tokenized for natural language processing.

[0234] Step 4:

[0235] The server passes tokenized message data to a generating AI model, which evaluates the potential for inappropriate language and harassment while considering the context.

[0236] Step 5:

[0237] The server calculates a risk score based on the evaluation results, and if the risk level is high, it generates a warning message and suggested corrective actions.

[0238] Step 6:

[0239] Warnings and suggestions generated by the server are sent to the user's terminal in real time, and the terminal displays them to the user as alerts.

[0240] Step 7:

[0241] The user reviews the displayed warning and chooses to modify the message as needed or cancel sending it. If modifications are made, the message will attempt to be sent again.

[0242] Step 8:

[0243] If the terminal resends the corrected message, the server re-evaluates it and, if there are no problems, proceeds to deliver the message to the recipient.

[0244] Through these steps, the system attempts to effectively suppress inappropriate communication.

[0245] (Example 1)

[0246] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0247] In modern communication methods, there is a problem in that emotionally or unintentionally inappropriate expressions by users can easily spread. This often leads to deterioration of relationships and social trouble. Furthermore, because message content is transmitted instantly, there is a challenge in that there is no time to re-evaluate the content before sending it.

[0248] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0249] In this invention, the server includes means for acquiring information input from an information processing device, means for evaluating the possibility of inappropriate expressions and aggressive behavior using a generative AI model to analyze the acquired information, and means for evaluating the degree of risk taking into account time information and recipient attribute information at the time of information transmission. This allows users to re-evaluate the message content in advance and improve the content as necessary.

[0250] An "information processing device" is a device that has the function of inputting and processing information such as messages.

[0251] A "generative AI model" is an artificial intelligence model that performs natural language processing to understand and analyze the context and content of input text.

[0252] "Inappropriate language" refers to language that may cause offense or aggression to the recipient or a third party.

[0253] "Aggressive behavior" refers to any action that is intended to hurt or offend others, or has the potential to do so.

[0254] "Risk level" is an indicator that shows the degree to which a message being sent is inappropriate or likely to cause social problems.

[0255] "Communication software" refers to programs used by users to exchange messages and information.

[0256] "Attribute information" refers to data that indicates the characteristics and profile information of the recipient.

[0257] This invention primarily utilizes an information processing device (hereinafter referred to as a terminal), a remote device for information processing (hereinafter referred to as a server), and means for exchanging information between them to configure a system for pre-evaluating messages. The user inputs a message using communication software on the terminal and attempts to send it. At this time, the information is temporarily stored on the terminal.

[0258] Upon receiving information from a terminal, the server begins analysis using a generative AI model. This model employs natural language processing techniques to understand the context of the information and assess the potential for inappropriate language or aggressive behavior. The analysis is carried out using keyword extraction, sentiment analysis, and other similar methods. Furthermore, the server also considers the time information of the transmitted information and the recipient's attributes to comprehensively evaluate the level of risk.

[0259] Based on the analysis results, the server generates a warning message and correction suggestions, and sends them back to the terminal. The terminal notifies the user of the received warnings and correction suggestions, and allows the user to choose to correct the information or stop sending it as needed.

[0260] For example, if a user tries to send a message to a colleague saying, "There are too many problems with your work," the server may deem this message inappropriate and suggest a revised version, such as, "Please consider giving more specific and constructive feedback." This system allows users to re-evaluate information and prevent harassment and inappropriate communication.

[0261] An example of a prompt message could be something like, "Is this message inappropriate? How should it be corrected?" which could be passed to the server. In this way, the generative AI model sequentially evaluates the appropriateness of the information.

[0262] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0263] Step 1:

[0264] The user enters a message using the terminal's communication software and presses the send button. The entered message is saved on the terminal and enters a temporary waiting state. At this stage, the message's text data is output as ready for transmission.

[0265] Step 2:

[0266] After the send button is pressed, the terminal sends the saved message to the server. The input on the server is the message data from the terminal, and the output is an acknowledgment of receipt of this data. The server passes the message content to the generating AI model and inputs prompts for text analysis.

[0267] Step 3:

[0268] The server inputs the received message into a generating AI model, which then begins contextual understanding and keyword extraction. The model uses natural language processing techniques to analyze the emotions and sentiments within the text and calculate the risk level. The input here is the message text, and the output is a data object containing the analysis results.

[0269] Step 4:

[0270] The server evaluates the potential for inappropriate language and offensive behavior based on the generated analysis results. Based on this evaluation, the server generates appropriate warning messages and correction suggestions. At this stage, the input is the analysis results, and the output is the text data of the warnings and correction suggestions.

[0271] Step 5:

[0272] The server sends the generated warning message and correction suggestions back to the terminal. The terminal receives this data and notifies the user. The input is the suggested data from the server, and the output is a visual or auditory notification to the user.

[0273] Step 6:

[0274] The user reviews the received warnings and correction suggestions and chooses to revise the message or cancel sending it as needed. If corrections are made, the user re-edits the message and proceeds with the sending process again. At this point, the input is the suggestions received by the user, and the output is the revised message.

[0275] Step 7:

[0276] Once the corrected message is sent via the resend button, the server parses the message again to determine whether to send it or not. In this step, the input is the corrected message, and the output is feedback on the final sending result.

[0277] (Application Example 1)

[0278] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0279] In electronic payment services, it is important for customer support staff to provide appropriate and friendly responses to users. In particular, in situations where immediate responses are required, inappropriate expressions or expressions that may cause misunderstandings are often inevitable. However, this may damage the user experience and there is a risk of a decrease in service evaluation. The object of this invention is to prevent inappropriate expressions in such communication scenarios and realize a comfortable response for users.

[0280] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0281] In this invention, the server includes means for acquiring the communication content input by the user, means for evaluating inappropriate expressions and the possibility of harassment using a knowledge model for analyzing the acquired communication content, and means for notifying the user of warnings and modification proposals for the communication content based on the evaluation results. This ensures that the communication received by the user through customer support is always appropriate and friendly, and as a result, it becomes possible to provide a better user experience.

[0282] A "user" is an individual or group that uses the system and is the subject that inputs communication content.

[0283] "Communication content" is a message or conversation that the user inputs or intends to send, and is the object of analysis.

[0284] A "knowledge model" is a machine learning or artificial intelligence algorithm used to evaluate the inappropriateness and harmfulness of communication content using natural language processing technology.

[0285] An "inappropriate expression" is an expression that may give discomfort to the other party or an expression that is considered socially undesirable.

[0286] "Annoyance" refers to an act or expression that intentionally makes others uncomfortable, usually intentional.

[0287] "Evaluation result" refers to a judgment regarding the inappropriateness or risk level of the communication content analyzed using a knowledge model.

[0288] "Warning" refers to a notification indicating the possibility that the communication content is inappropriate to the user.

[0289] "Suggestion for correction" refers to an alternative expression or improvement method presented to the user to improve inappropriate parts of the communication content.

[0290] The system for implementing this invention consists of a user, a server, and a terminal. When a user attempts to input communication content and transmit it through the terminal, first the terminal acquires the communication content. The terminal transmits the communication content to the server, and the server analyzes the communication content using a knowledge model. In this analysis, natural language processing technology is used to evaluate whether the communication content is inappropriate or contains annoyance. At that time, the server utilizes NLTK or TensorFlow, which are natural language processing libraries implemented in Python. Based on the analysis result, the server generates a warning and a suggestion for correction and transmits them to the terminal.

[0291] The terminal immediately notifies the user of the received warning and suggestion for correction. Based on this notification, the user re-evaluates the communication content and makes corrections or cancels the transmission. For example, when attempting to send a message such as "We apologize for any inconvenience" in customer support, the server can provide a suggestion for correction such as "How about presenting more specific improvement measures?" This function supports the server in assisting the user to provide an appropriate response according to the context on behalf of the user.

[0292] As a concrete example, the AI ​​generation model can be instructed with a prompt message such as, "Evaluate whether this message is easy for the customer to understand, and generate improvement suggestions if necessary." This allows the system to efficiently check the communication content and make appropriate suggestions.

[0293] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0294] Step 1:

[0295] The user inputs the communication content into the device. The device captures this communication content and stores it in temporary storage. This communication content will be used as input for analysis in the next step.

[0296] Step 2:

[0297] The terminal sends the communication content to the server. The server uses a generative AI model to analyze the received communication content. First, it extracts keywords and analyzes the context based on natural language processing techniques to generate data necessary to evaluate whether the communication content is inappropriate. The output of this analysis is an inappropriateness score.

[0298] Step 3:

[0299] The server uses a generative AI model to generate warning messages and corrective action suggestions based on the inappropriateness score. This process uses the prompt "Evaluate whether this message is easy for customers to understand and generate improvement suggestions if necessary." The generated warnings and corrective action suggestions are then output for transmission to the terminal.

[0300] Step 4:

[0301] Warnings and suggested corrections are sent from the server to the terminal. The terminal immediately displays these notifications to the user. In this step, the terminal is responsible for providing the user with the output data from the server.

[0302] Step 5:

[0303] The user checks the warnings and correction suggestions and selects whether to correct the communication content or cancel the transmission. If the user selects to correct, the terminal can resend the corrected communication content to the server and repeat the analysis process. Since the user's selection determines the final communication content, the output at this step is the user's decision.

[0304] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0305] The system according to the present invention assumes a messaging application having an emotion recognition function used by the user on the terminal. This system uses a server linked with an emotion engine that receives a message input from the user and performs real-time emotion analysis on it, thereby providing a more advanced message evaluation.

[0306] The user opens an SNS, chat, or email application on the terminal and inputs a message. At that time, the emotion engine analyzes the user's input pattern and words in the background and evaluates the user's emotional state in real time. This information is transmitted from the terminal to the server, and general context analysis and emotion analysis of the message are performed simultaneously on the server side.

[0307] The server not only evaluates the inappropriate expressions and the possibility of harassment of the message through the generative AI model, but also considers the emotion data obtained by the emotion engine and generates warnings and correction suggestions according to the user's emotion. For example, when the user expresses stress or negative emotions, the server generates a message that strengthens the alert and returns it to the terminal.

[0308] The device promptly presents the user with warnings, suggestions, and even additional, emotion-based advice received from the server. Users can use this information to modify their messages, facilitating more appropriate communication.

[0309] For example, if a user enters a message expressing strong dissatisfaction with workplace communication, the emotion engine detects the high stress level. Based on this emotion information and the message content, the server suggests less misleading expressions and provides feedback in a gentle tone, giving the user an opportunity to reconsider their message.

[0310] By implementing this system, users can receive immediate, emotionally sensitive feedback, which in turn helps prevent harassment and inappropriate language.

[0311] The following describes the processing flow.

[0312] Step 1:

[0313] The user enters a message using the messaging application on their device. The emotion engine analyzes the user's keystrokes and input patterns in the background as they type, and evaluates the user's emotional state.

[0314] Step 2:

[0315] The terminal packages the input message and the emotion data output by the emotion engine, and sends them to the server.

[0316] Step 3:

[0317] The server first sends the received message and sentiment data to a generative AI model to begin the process of evaluating the possibility of inappropriate language or harassment.

[0318] Step 4:

[0319] The server simultaneously uses sentiment data to analyze whether the user's emotional state is influencing the message content, and incorporates this into the contextual evaluation.

[0320] Step 5:

[0321] The server integrates the results of the generated AI model and sentiment analysis to calculate a risk score. Based on this score, it generates appropriate warning messages, corrective action suggestions, and even emotionally sensitive advice.

[0322] Step 6:

[0323] The server compiles the generated warnings, suggestions, and advice and sends them back to the terminal.

[0324] Step 7:

[0325] The device immediately notifies the user of any received warnings or suggestions. The notifications also include additional advice based on sentiment analysis, which the user can use to decide whether to modify the message or cancel sending it.

[0326] Step 8:

[0327] If a user modifies and resends a message, the device sends the modified message back to the server for evaluation. If it determines that there are no problems, the message is delivered to the recipient.

[0328] This approach aims to achieve effective communication by providing more human-like feedback from the system.

[0329] (Example 2)

[0330] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0331] In today's communication environment, users are required to send emotionally charged messages quickly and appropriately. However, because emotional judgments cannot be made instantaneously, there is a risk that inappropriate language or potentially human rights-violating messages may be accidentally sent. Therefore, there is a need for a system that allows users to receive appropriate feedback immediately when sending emotionally charged messages.

[0332] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0333] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate expression and human rights violations using a statistical model to analyze the acquired communication content, and means for notifying the user of warnings and correction suggestions based on the acquired emotional state. This enables users to take their emotions into consideration, prevent inappropriate expression during communication, and send safer and more appropriate messages.

[0334] "Communication content" refers to the text messages and information that a user intends to send, which may include emotions and intentions.

[0335] A "statistical model" is a computational method that uses statistical techniques to analyze data and evaluate the inappropriateness of communication content and the risks related to human rights.

[0336] "Potential human rights violation" refers to the possibility that the message content infringes upon the fundamental rights of others, and problematic expressions are identified by assessing this risk.

[0337] An "emotion analysis device" is a system that detects the emotions and tone contained in a user's communication content and quantifies or categorizes them.

[0338] A "pattern detection algorithm" is a method for identifying certain patterns within data and generating evaluations or warnings that correspond to the user's emotions and intentions.

[0339] An embodiment of the present invention is a communication support system using an emotion analysis and generation AI model for use by users on a communication terminal. This system analyzes the communication content entered by the user in real time and provides feedback to prevent inappropriate expressions and human rights violations.

[0340] The terminal first acquires the text entered by the user. The acquired content is then analyzed by an emotion analysis device. This device uses natural language processing technology to quantify the emotions contained in the communication content and evaluates the user's emotional state.

[0341] Emotional data and communication content are sent together to the server. The server consists of hardware with advanced computing power and is equipped with statistical models. Based on the received data, this server uses a generative AI model to perform contextual analysis. Through contextual analysis, it evaluates whether the message is inappropriate or poses a risk of human rights violations. Furthermore, to enable quick decision-making, the server generates feedback tailored to the user's emotional state, providing warnings and correction suggestions.

[0342] The generated feedback is immediately sent to the device. The device then presents this information to the user, giving the user the opportunity to modify their communication based on the feedback.

[0343] For example, if a user types "This meeting is a complete waste of time," the sentiment analyzer will detect frustration. The server will then generate and provide a calmer suggestion to the user, such as "How about reconfirming the purpose of the meeting?"

[0344] An example of a prompt might be, "Generate suggestions for improving the expression based on emotion." Based on this prompt, the generation AI model generates appropriate feedback on the communication content.

[0345] By utilizing this system, users will be able to express their emotions in a more appropriate way, resulting in smoother communication.

[0346] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0347] Step 1:

[0348] The user inputs the communication content via a terminal. The terminal receives this input, converts it into a data format, and sends it to the sentiment analysis device. The input is the text entered by the user, and the output is data ready for analysis. Specifically, the text undergoes initial formatting and is formatted in a way that is suitable for analysis.

[0349] Step 2:

[0350] The terminal analyzes communication content using an emotion analysis device. The device uses natural language processing technology to analyze keywords and context from the input text and generate emotion data. The input is formatted text data, and the output is numerical or categorical data indicating the emotional state. Specific analysis operations include keyword detection and sentiment score calculation.

[0351] Step 3:

[0352] The terminal sends the generated sentiment data and communication content to the server. The server receives this data and performs a detailed analysis using statistical models and generative AI models. The input is a set of sentiment data and text data, and the output is an evaluation result that takes context and sentiment into account. Specifically, the model performs pattern detection and risk assessment simultaneously.

[0353] Step 4:

[0354] The server generates warnings and correction suggestions based on the evaluation results. The generation AI model creates appropriate feedback according to the given prompt. The input is the evaluation results and prompt, and the output is a feedback message to notify the user. Specifically, the process involves generating the suggested content and formatting the message.

[0355] Step 5:

[0356] The server sends the generated feedback to the terminal, which then displays it to the user. The user re-evaluates the communication based on the provided feedback and makes corrections as needed. The input is the feedback data received from the server, and the output is the user's corresponding action. Specific actions include displaying notification pop-ups and presenting feedback through the user interface.

[0357] (Application Example 2)

[0358] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0359] In recent years, online communication has been steadily increasing, but inappropriate language and harassment have become a problem. Furthermore, there are many cases where such content significantly impacts users' mental state and emotions. In response, there is a need for the development of systems that can detect inappropriate content in real time and provide feedback that takes users' feelings into consideration.

[0360] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0361] In this invention, the server includes means for acquiring information input from the user, means for evaluating the potential for inappropriate expressions and harassment using a generative model to analyze the acquired information, and means for providing warnings, correction suggestions, and sentiment-sensitive feedback based on the evaluation results and sentiment analysis. This enables the user to improve their message expression in real time and obtain a safe and comfortable communication environment.

[0362] A "user" is the entity that inputs information and receives analysis results and feedback from the system.

[0363] "Inputted information" refers to the content of text and messages provided by the user to the system.

[0364] A "generative model" is an algorithm or program used for natural language processing, specifically for detecting inappropriate expressions and harassment.

[0365] "Inappropriate language" refers to expressions or phrases that may cause misunderstandings or conflicts in online communication.

[0366] "Potential harassment" refers to harassment or the risk of harassment that can be inferred from the information entered by the user.

[0367] "Evaluation results" refer to the identification of inappropriate expressions and harassment behaviors derived from the generative model, along with detailed information about them.

[0368] "Emotional analysis" is the process of inferring and analyzing a user's emotional state from the input information.

[0369] A "warning" is a message or notification that alerts the user based on the evaluation results.

[0370] A "revision suggestion" is a specific proposal provided to the user for correcting content deemed inappropriate.

[0371] "Emotionally sensitive feedback" refers to information and suggestions provided while taking into account the user's mental state and emotions.

[0372] This invention consists of a system that performs real-time sentiment analysis and evaluation of the appropriateness of expression on information input by the user, and provides feedback. The system mainly consists of an emotion engine for recognizing and evaluating emotions, a generative AI model for analyzing messages, a server for aggregating and processing information, and a terminal for notifying the user of the evaluation results.

[0373] First, the user enters a message on their device. This device can be a smartphone or computer and is integrated with a communication platform. The entered information is sent directly to the server. On the server, an application built using Python or Django analyzes this information using a generative AI model (for example, OpenAI's GPT-3). The analysis uses natural language processing libraries (NLTK, spaCy, TextBlob) to evaluate the possibility of inappropriate language or harassment, and simultaneously performs sentiment analysis using a sentiment engine.

[0374] The server determines the user's mental state based on the generated evaluation results and sentiment analysis data. This information is fed back to the device as warnings and correction suggestions, and the device notifies the user of this information without delay. This allows the user to check whether their message is inappropriate and correct it to a more appropriate expression.

[0375] For example, when a user at work tries to send a team message to a colleague that contains strong emotions, the emotion engine detects the high stress level. The server then uses a generative model to suggest revisions to adjust the tone of the message, offering advice such as, "How about phrasing it like this?"

[0376] An example of a prompt message is, "If the entered message is offensive, please suggest a way to rephrase it in a more positive and constructive tone."

[0377] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0378] Step 1:

[0379] The user enters a message from their device. The entered information is immediately sent to the server via the communication platform. The input here is raw text data entered by the user.

[0380] Step 2:

[0381] The server passes the received information to the emotion engine and the generative AI model. The emotion engine uses natural language processing libraries (such as NLTK and spaCy) to analyze the user's emotional state from the input text. The type and intensity of the emotion are output and used for the next processing step.

[0382] Step 3:

[0383] The generative AI model evaluates the potential for inappropriate expressions and harassment based on analyzed sentiment information and text data. The generative AI model (such as OpenAI's GPT-3) identifies inappropriate elements from the input data and calculates their likelihood. The evaluation results are generated on the server side and used as feedback.

[0384] Step 4:

[0385] The server integrates the generated evaluation results and sentiment analysis data to produce appropriate correction suggestions and warnings. The output here is a specific feedback message indicating what correction suggestions or warnings are best for the user.

[0386] Step 5:

[0387] A feedback message is sent to the device and notified to the user. The user receives this notification and, if necessary, modifies or cancels the message. The final revision of the message is made based on the user's actions.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0391] [Third Embodiment]

[0392] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0393] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0404] The system according to the present invention mainly consists of a terminal used by the user, a server that performs data processing, and an interface for communication between them. Specific examples of its operation and implementation are shown below.

[0405] The user first enters a message using an SNS, chat, or email application on their device. At this point, the message the user attempts to send is temporarily held on the device and prepared for transmission to the server. When the send button is pressed, the device sends the message content to the server, and message analysis begins.

[0406] The server inputs received messages into a generating AI model, which analyzes the text's context and content to determine if it contains inappropriate language or harassment. This analysis utilizes keyword extraction, sentiment analysis, and sentiment analysis techniques. Furthermore, the time of message transmission and recipient information are also considered to calculate an overall risk score.

[0407] Based on the analysis results, the server generates a warning message for messages deemed high-risk. It also simultaneously generates correction suggestions to encourage the user to make improvements, and sends all of these back to the terminal.

[0408] The terminal immediately notifies the user of any warnings and correction suggestions received from the server. This allows the user to understand the potential impact of the message and choose to correct the message content or cancel sending it as needed. The corrected message can be sent again by pressing the send button, after which it will be analyzed by the server once more.

[0409] For example, if a user tries to send a message to a colleague in a business setting saying, "There are too many problems with your work," the server will deem the message inappropriate and send a warning to the user's device with a suggestion to revise it, such as, "Why not try to provide more specific guidance?" As a result, the user has the opportunity to revise the message to make it more constructive.

[0410] This system allows users to re-evaluate messages before sending them, thereby preventing harassment and inappropriate communication.

[0411] The following describes the processing flow.

[0412] Step 1:

[0413] When a user types a message on their device and presses the send button, the device captures the entered message.

[0414] Step 2:

[0415] The terminal prepares to send the captured message to the server using a pre-configured communication protocol.

[0416] Step 3:

[0417] When the server receives a message, it initiates a request for analysis to the generative AI model. At this point, the message is first tokenized for natural language processing.

[0418] Step 4:

[0419] The server passes tokenized message data to a generating AI model, which evaluates the potential for inappropriate language and harassment while considering the context.

[0420] Step 5:

[0421] The server calculates a risk score based on the evaluation results, and if the risk level is high, it generates a warning message and suggested corrective actions.

[0422] Step 6:

[0423] Warnings and suggestions generated by the server are sent to the user's terminal in real time, and the terminal displays them to the user as alerts.

[0424] Step 7:

[0425] The user reviews the displayed warning and chooses to modify the message as needed or cancel sending it. If modifications are made, the message will attempt to be sent again.

[0426] Step 8:

[0427] If the terminal resends the corrected message, the server re-evaluates it and, if there are no problems, proceeds to deliver the message to the recipient.

[0428] Through these steps, the system attempts to effectively suppress inappropriate communication.

[0429] (Example 1)

[0430] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0431] In modern communication methods, there is a problem in that emotionally or unintentionally inappropriate expressions by users can easily spread. This often leads to deterioration of relationships and social trouble. Furthermore, because message content is transmitted instantly, there is a challenge in that there is no time to re-evaluate the content before sending it.

[0432] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0433] In this invention, the server includes means for acquiring information input from an information processing device, means for evaluating the possibility of inappropriate expressions and aggressive behavior using a generative AI model to analyze the acquired information, and means for evaluating the degree of risk taking into account time information and recipient attribute information at the time of information transmission. This allows users to re-evaluate the message content in advance and improve the content as necessary.

[0434] An "information processing device" is a device that has the function of inputting and processing information such as messages.

[0435] A "generative AI model" is an artificial intelligence model that performs natural language processing to understand and analyze the context and content of input text.

[0436] "Inappropriate language" refers to language that may cause offense or aggression to the recipient or a third party.

[0437] "Aggressive behavior" refers to any action that is intended to hurt or offend others, or has the potential to do so.

[0438] "Risk level" is an indicator that shows the degree to which a message being sent is inappropriate or likely to cause social problems.

[0439] "Communication software" refers to programs used by users to exchange messages and information.

[0440] "Attribute information" refers to data that indicates the characteristics and profile information of the recipient.

[0441] This invention primarily utilizes an information processing device (hereinafter referred to as a terminal), a remote device for information processing (hereinafter referred to as a server), and means for exchanging information between them to configure a system for pre-evaluating messages. The user inputs a message using communication software on the terminal and attempts to send it. At this time, the information is temporarily stored on the terminal.

[0442] Upon receiving information from a terminal, the server begins analysis using a generative AI model. This model employs natural language processing techniques to understand the context of the information and assess the potential for inappropriate language or aggressive behavior. The analysis is carried out using keyword extraction, sentiment analysis, and other similar methods. Furthermore, the server also considers the time information of the transmitted information and the recipient's attributes to comprehensively evaluate the level of risk.

[0443] Based on the analysis results, the server generates a warning message and correction suggestions, and sends them back to the terminal. The terminal notifies the user of the received warnings and correction suggestions, and allows the user to choose to correct the information or stop sending it as needed.

[0444] For example, if a user tries to send a message to a colleague saying, "There are too many problems with your work," the server may deem this message inappropriate and suggest a revised version, such as, "Please consider giving more specific and constructive feedback." This system allows users to re-evaluate information and prevent harassment and inappropriate communication.

[0445] An example of a prompt message could be something like, "Is this message inappropriate? How should it be corrected?" which could be passed to the server. In this way, the generative AI model sequentially evaluates the appropriateness of the information.

[0446] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0447] Step 1:

[0448] The user enters a message using the terminal's communication software and presses the send button. The entered message is saved on the terminal and enters a temporary waiting state. At this stage, the message's text data is output as ready for transmission.

[0449] Step 2:

[0450] After the send button is pressed, the terminal sends the saved message to the server. The input on the server is the message data from the terminal, and the output is an acknowledgment of receipt of this data. The server passes the message content to the generating AI model and inputs prompts for text analysis.

[0451] Step 3:

[0452] The server inputs the received message into a generating AI model, which then begins contextual understanding and keyword extraction. The model uses natural language processing techniques to analyze the emotions and sentiments within the text and calculate the risk level. The input here is the message text, and the output is a data object containing the analysis results.

[0453] Step 4:

[0454] The server evaluates the potential for inappropriate language and offensive behavior based on the generated analysis results. Based on this evaluation, the server generates appropriate warning messages and correction suggestions. At this stage, the input is the analysis results, and the output is the text data of the warnings and correction suggestions.

[0455] Step 5:

[0456] The server sends the generated warning message and correction suggestions back to the terminal. The terminal receives this data and notifies the user. The input is the suggested data from the server, and the output is a visual or auditory notification to the user.

[0457] Step 6:

[0458] The user reviews the received warnings and correction suggestions and chooses to revise the message or cancel sending it as needed. If corrections are made, the user re-edits the message and proceeds with the sending process again. At this point, the input is the suggestions received by the user, and the output is the revised message.

[0459] Step 7:

[0460] Once the corrected message is sent via the resend button, the server parses the message again to determine whether to send it or not. In this step, the input is the corrected message, and the output is feedback on the final sending result.

[0461] (Application Example 1)

[0462] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0463] In electronic payment services, it is crucial for customer support staff to provide appropriate and friendly service to users. However, especially in situations requiring immediate responses, inappropriate or misleading language is often unavoidable. This can damage the user experience and risk lowering the service's rating. This invention aims to prevent inappropriate language in such communication situations and achieve a comfortable user experience.

[0464] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0465] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content, and means for notifying the user of warnings and suggestions for correcting the communication content based on the evaluation results. This ensures that the communications the user receives through customer support are always appropriate and user-friendly, and as a result, enables the provision of a better user experience.

[0466] A "user" is an individual or group that uses the system and is the entity that inputs the communication content.

[0467] "Communication content" refers to messages and conversations that a user attempts to input or send, and is subject to analysis.

[0468] A "knowledge model" is a machine learning or artificial intelligence algorithm used to evaluate the inappropriateness or harmfulness of communication content using natural language processing techniques.

[0469] "Inappropriate language" refers to language that may offend others or is considered socially undesirable.

[0470] "Harassment" refers to actions or expressions that are intentionally made to offend someone, and is usually done intentionally.

[0471] "Evaluation results" refer to judgments regarding the inappropriateness or risk level of communication content, based on analysis using a knowledge model.

[0472] A "warning" is a notification to a user indicating that the content of their communication may be inappropriate.

[0473] A "correction suggestion" refers to alternative expressions or methods of improvement presented to the user to correct inappropriate parts of the communication content.

[0474] The system implementing this invention consists of a user, a server, and a terminal. When a user inputs communication content and attempts to send it through the terminal, the terminal first acquires the communication content. The terminal sends the communication content to the server, which analyzes the content using a knowledge model. In this analysis, natural language processing techniques are used to evaluate whether the communication content is inappropriate or contains harassment. In this process, the server utilizes NLTK or TensorFlow, which are natural language processing libraries implemented in Python. Based on the analysis results, the server generates warnings and correction suggestions, which are then sent to the terminal.

[0475] The terminal immediately notifies the user of any received warnings and corrective suggestions. Based on these notifications, the user re-evaluates the communication content and makes corrections or cancels the transmission. For example, if a customer support user is about to send a message such as "We apologize for the inconvenience," the server can provide a corrective suggestion such as "Why not offer more specific solutions?" This function helps the server provide contextually appropriate responses on behalf of the user.

[0476] As a concrete example, the AI ​​generation model can be instructed with a prompt message such as, "Evaluate whether this message is easy for the customer to understand, and generate improvement suggestions if necessary." This allows the system to efficiently check the communication content and make appropriate suggestions.

[0477] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0478] Step 1:

[0479] The user inputs the communication content into the device. The device captures this communication content and stores it in temporary storage. This communication content will be used as input for analysis in the next step.

[0480] Step 2:

[0481] The terminal sends the communication content to the server. The server uses a generative AI model to analyze the received communication content. First, it extracts keywords and analyzes the context based on natural language processing techniques to generate data necessary to evaluate whether the communication content is inappropriate. The output of this analysis is an inappropriateness score.

[0482] Step 3:

[0483] The server uses a generative AI model to generate warning messages and corrective action suggestions based on the inappropriateness score. This process uses the prompt "Evaluate whether this message is easy for customers to understand and generate improvement suggestions if necessary." The generated warnings and corrective action suggestions are then output for transmission to the terminal.

[0484] Step 4:

[0485] Warnings and suggested corrections are sent from the server to the terminal. The terminal immediately displays these notifications to the user. In this step, the terminal is responsible for providing the user with the output data from the server.

[0486] Step 5:

[0487] The user reviews the warnings and correction suggestions and chooses to correct the communication content or cancel transmission. If the user chooses to correct, the terminal can resend the corrected communication content to the server and repeat the analysis process. Since the user's choice determines the final content of the communication, the output at this step reflects the user's decision.

[0488] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0489] The system according to the present invention is intended for a messaging application equipped with emotion recognition functionality used by users on their terminals. This system provides more advanced message evaluation by using a server that works in conjunction with an emotion engine that receives message input from the user and performs real-time emotion analysis on it.

[0490] Users open social media, chat, or email applications on their devices and type messages. During this process, a sentiment engine analyzes the user's input patterns and wording in the background, evaluating the user's emotional state in real time. This information is sent from the device to the server, where general contextual analysis and sentiment analysis of the message are performed simultaneously.

[0491] The server not only evaluates the potential for inappropriate language and harassment in messages through a generative AI model, but also considers emotional data obtained by the emotion engine to generate warnings and corrective suggestions tailored to the user's emotions. For example, if the user is expressing stress or negative emotions, the server generates a more emphatic message and sends it back to the device.

[0492] The device promptly presents the user with warnings, suggestions, and even additional, emotion-based advice received from the server. Users can use this information to modify their messages, facilitating more appropriate communication.

[0493] For example, if a user enters a message expressing strong dissatisfaction with workplace communication, the emotion engine detects the high stress level. Based on this emotion information and the message content, the server suggests less misleading expressions and provides feedback in a gentle tone, giving the user an opportunity to reconsider their message.

[0494] By implementing this system, users can receive immediate, emotionally sensitive feedback, which in turn helps prevent harassment and inappropriate language.

[0495] The following describes the processing flow.

[0496] Step 1:

[0497] The user enters a message using the messaging application on their device. The emotion engine analyzes the user's keystrokes and input patterns in the background as they type, and evaluates the user's emotional state.

[0498] Step 2:

[0499] The terminal packages the input message and the emotion data output by the emotion engine, and sends them to the server.

[0500] Step 3:

[0501] The server first sends the received message and sentiment data to a generative AI model to begin the process of evaluating the possibility of inappropriate language or harassment.

[0502] Step 4:

[0503] The server simultaneously uses sentiment data to analyze whether the user's emotional state is influencing the message content, and incorporates this into the contextual evaluation.

[0504] Step 5:

[0505] The server integrates the results of the generated AI model and sentiment analysis to calculate a risk score. Based on this score, it generates appropriate warning messages, corrective action suggestions, and even emotionally sensitive advice.

[0506] Step 6:

[0507] The server compiles the generated warnings, suggestions, and advice and sends them back to the terminal.

[0508] Step 7:

[0509] The device immediately notifies the user of any received warnings or suggestions. The notifications also include additional advice based on sentiment analysis, which the user can use to decide whether to modify the message or cancel sending it.

[0510] Step 8:

[0511] If a user modifies and resends a message, the device sends the modified message back to the server for evaluation. If it determines that there are no problems, the message is delivered to the recipient.

[0512] This approach aims to achieve effective communication by providing more human-like feedback from the system.

[0513] (Example 2)

[0514] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0515] In today's communication environment, users are required to send emotionally charged messages quickly and appropriately. However, because emotional judgments cannot be made instantaneously, there is a risk that inappropriate language or potentially human rights-violating messages may be accidentally sent. Therefore, there is a need for a system that allows users to receive appropriate feedback immediately when sending emotionally charged messages.

[0516] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0517] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate expression and human rights violations using a statistical model to analyze the acquired communication content, and means for notifying the user of warnings and correction suggestions based on the acquired emotional state. This enables users to take their emotions into consideration, prevent inappropriate expression during communication, and send safer and more appropriate messages.

[0518] "Communication content" refers to the text messages and information that a user intends to send, which may include emotions and intentions.

[0519] A "statistical model" is a computational method that uses statistical techniques to analyze data and evaluate the inappropriateness of communication content and the risks related to human rights.

[0520] "Potential human rights violation" refers to the possibility that the message content infringes upon the fundamental rights of others, and problematic expressions are identified by assessing this risk.

[0521] An "emotion analysis device" is a system that detects the emotions and tone contained in a user's communication content and quantifies or categorizes them.

[0522] A "pattern detection algorithm" is a method for identifying certain patterns within data and generating evaluations or warnings that correspond to the user's emotions and intentions.

[0523] An embodiment of the present invention is a communication support system using an emotion analysis and generation AI model for use by users on a communication terminal. This system analyzes the communication content entered by the user in real time and provides feedback to prevent inappropriate expressions and human rights violations.

[0524] The terminal first acquires the text entered by the user. The acquired content is then analyzed by an emotion analysis device. This device uses natural language processing technology to quantify the emotions contained in the communication content and evaluates the user's emotional state.

[0525] Emotional data and communication content are sent together to the server. The server consists of hardware with advanced computing power and is equipped with statistical models. Based on the received data, this server uses a generative AI model to perform contextual analysis. Through contextual analysis, it evaluates whether the message is inappropriate or poses a risk of human rights violations. Furthermore, to enable quick decision-making, the server generates feedback tailored to the user's emotional state, providing warnings and correction suggestions.

[0526] The generated feedback is immediately sent to the device. The device then presents this information to the user, giving the user the opportunity to modify their communication based on the feedback.

[0527] For example, if a user types "This meeting is a complete waste of time," the sentiment analyzer will detect frustration. The server will then generate and provide a calmer suggestion to the user, such as "How about reconfirming the purpose of the meeting?"

[0528] An example of a prompt might be, "Generate suggestions for improving the expression based on emotion." Based on this prompt, the generation AI model generates appropriate feedback on the communication content.

[0529] By utilizing this system, users will be able to express their emotions in a more appropriate way, resulting in smoother communication.

[0530] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0531] Step 1:

[0532] The user inputs the communication content via a terminal. The terminal receives this input, converts it into a data format, and sends it to the sentiment analysis device. The input is the text entered by the user, and the output is data ready for analysis. Specifically, the text undergoes initial formatting and is formatted in a way that is suitable for analysis.

[0533] Step 2:

[0534] The terminal analyzes communication content using an emotion analysis device. The device uses natural language processing technology to analyze keywords and context from the input text and generate emotion data. The input is formatted text data, and the output is numerical or categorical data indicating the emotional state. Specific analysis operations include keyword detection and sentiment score calculation.

[0535] Step 3:

[0536] The terminal sends the generated sentiment data and communication content to the server. The server receives this data and performs a detailed analysis using statistical models and generative AI models. The input is a set of sentiment data and text data, and the output is an evaluation result that takes context and sentiment into account. Specifically, the model performs pattern detection and risk assessment simultaneously.

[0537] Step 4:

[0538] The server generates warnings and correction suggestions based on the evaluation results. The generation AI model creates appropriate feedback according to the given prompt. The input is the evaluation results and prompt, and the output is a feedback message to notify the user. Specifically, the process involves generating the suggested content and formatting the message.

[0539] Step 5:

[0540] The server sends the generated feedback to the terminal, which then displays it to the user. The user re-evaluates the communication based on the provided feedback and makes corrections as needed. The input is the feedback data received from the server, and the output is the user's corresponding action. Specific actions include displaying notification pop-ups and presenting feedback through the user interface.

[0541] (Application Example 2)

[0542] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0543] In recent years, online communication has been steadily increasing, but inappropriate language and harassment have become a problem. Furthermore, there are many cases where such content significantly impacts users' mental state and emotions. In response, there is a need for the development of systems that can detect inappropriate content in real time and provide feedback that takes users' feelings into consideration.

[0544] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0545] In this invention, the server includes means for acquiring information input from the user, means for evaluating the potential for inappropriate expressions and harassment using a generative model to analyze the acquired information, and means for providing warnings, correction suggestions, and sentiment-sensitive feedback based on the evaluation results and sentiment analysis. This enables the user to improve their message expression in real time and obtain a safe and comfortable communication environment.

[0546] A "user" is the entity that inputs information and receives analysis results and feedback from the system.

[0547] "Inputted information" refers to the content of text and messages provided by the user to the system.

[0548] A "generative model" is an algorithm or program used for natural language processing, specifically for detecting inappropriate expressions and harassment.

[0549] "Inappropriate language" refers to expressions or phrases that may cause misunderstandings or conflicts in online communication.

[0550] "Potential harassment" refers to harassment or the risk of harassment that can be inferred from the information entered by the user.

[0551] "Evaluation results" refer to the identification of inappropriate expressions and harassment behaviors derived from the generative model, along with detailed information about them.

[0552] "Emotional analysis" is the process of inferring and analyzing a user's emotional state from the input information.

[0553] A "warning" is a message or notification that alerts the user based on the evaluation results.

[0554] A "revision suggestion" is a specific proposal provided to the user for correcting content deemed inappropriate.

[0555] "Emotionally sensitive feedback" refers to information and suggestions provided while taking into account the user's mental state and emotions.

[0556] This invention consists of a system that performs real-time sentiment analysis and evaluation of the appropriateness of expression on information input by the user, and provides feedback. The system mainly consists of an emotion engine for recognizing and evaluating emotions, a generative AI model for analyzing messages, a server for aggregating and processing information, and a terminal for notifying the user of the evaluation results.

[0557] First, the user enters a message on their device. This device can be a smartphone or computer and is integrated with a communication platform. The entered information is sent directly to the server. On the server, an application built using Python or Django analyzes this information using a generative AI model (for example, OpenAI's GPT-3). The analysis uses natural language processing libraries (NLTK, spaCy, TextBlob) to evaluate the possibility of inappropriate language or harassment, and simultaneously performs sentiment analysis using a sentiment engine.

[0558] The server determines the user's mental state based on the generated evaluation results and sentiment analysis data. This information is fed back to the device as warnings and correction suggestions, and the device notifies the user of this information without delay. This allows the user to check whether their message is inappropriate and correct it to a more appropriate expression.

[0559] For example, when a user at work tries to send a team message to a colleague that contains strong emotions, the emotion engine detects the high stress level. The server then uses a generative model to suggest revisions to adjust the tone of the message, offering advice such as, "How about phrasing it like this?"

[0560] An example of a prompt message is, "If the entered message is offensive, please suggest a way to rephrase it in a more positive and constructive tone."

[0561] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0562] Step 1:

[0563] The user enters a message from their device. The entered information is immediately sent to the server via the communication platform. The input here is raw text data entered by the user.

[0564] Step 2:

[0565] The server passes the received information to the emotion engine and the generative AI model. The emotion engine uses natural language processing libraries (such as NLTK and spaCy) to analyze the user's emotional state from the input text. The type and intensity of the emotion are output and used for the next processing step.

[0566] Step 3:

[0567] The generative AI model evaluates the potential for inappropriate expressions and harassment based on analyzed sentiment information and text data. The generative AI model (such as OpenAI's GPT-3) identifies inappropriate elements from the input data and calculates their likelihood. The evaluation results are generated on the server side and used as feedback.

[0568] Step 4:

[0569] The server integrates the generated evaluation results and sentiment analysis data to produce appropriate correction suggestions and warnings. The output here is a specific feedback message indicating what correction suggestions or warnings are best for the user.

[0570] Step 5:

[0571] A feedback message is sent to the device and notified to the user. The user receives this notification and, if necessary, modifies or cancels the message. The final revision of the message is made based on the user's actions.

[0572] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0573] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0574] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0575] [Fourth Embodiment]

[0576] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0577] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0578] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0579] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0580] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0581] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0582] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0583] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0584] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0585] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0586] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0587] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0588] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0589] The system according to the present invention mainly consists of a terminal used by the user, a server that performs data processing, and an interface for communication between them. Specific examples of its operation and implementation are shown below.

[0590] The user first enters a message using an SNS, chat, or email application on their device. At this point, the message the user attempts to send is temporarily held on the device and prepared for transmission to the server. When the send button is pressed, the device sends the message content to the server, and message analysis begins.

[0591] The server inputs received messages into a generating AI model, which analyzes the text's context and content to determine if it contains inappropriate language or harassment. This analysis utilizes keyword extraction, sentiment analysis, and sentiment analysis techniques. Furthermore, the time of message transmission and recipient information are also considered to calculate an overall risk score.

[0592] Based on the analysis results, the server generates a warning message for messages deemed high-risk. It also simultaneously generates correction suggestions to encourage the user to make improvements, and sends all of these back to the terminal.

[0593] The terminal immediately notifies the user of any warnings and correction suggestions received from the server. This allows the user to understand the potential impact of the message and choose to correct the message content or cancel sending it as needed. The corrected message can be sent again by pressing the send button, after which it will be analyzed by the server once more.

[0594] For example, if a user tries to send a message to a colleague in a business setting saying, "There are too many problems with your work," the server will deem the message inappropriate and send a warning to the user's device with a suggestion to revise it, such as, "Why not try to provide more specific guidance?" As a result, the user has the opportunity to revise the message to make it more constructive.

[0595] This system allows users to re-evaluate messages before sending them, thereby preventing harassment and inappropriate communication.

[0596] The following describes the processing flow.

[0597] Step 1:

[0598] When a user types a message on their device and presses the send button, the device captures the entered message.

[0599] Step 2:

[0600] The terminal prepares to send the captured message to the server using a pre-configured communication protocol.

[0601] Step 3:

[0602] When the server receives a message, it initiates a request for analysis to the generative AI model. At this point, the message is first tokenized for natural language processing.

[0603] Step 4:

[0604] The server passes tokenized message data to a generating AI model, which evaluates the potential for inappropriate language and harassment while considering the context.

[0605] Step 5:

[0606] The server calculates a risk score based on the evaluation results, and if the risk level is high, it generates a warning message and suggested corrective actions.

[0607] Step 6:

[0608] Warnings and suggestions generated by the server are sent to the user's terminal in real time, and the terminal displays them to the user as alerts.

[0609] Step 7:

[0610] The user reviews the displayed warning and chooses to modify the message as needed or cancel sending it. If modifications are made, the message will attempt to be sent again.

[0611] Step 8:

[0612] If the terminal resends the corrected message, the server re-evaluates it and, if there are no problems, proceeds to deliver the message to the recipient.

[0613] Through these steps, the system attempts to effectively suppress inappropriate communication.

[0614] (Example 1)

[0615] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0616] In modern communication methods, there is a problem in that emotionally or unintentionally inappropriate expressions by users can easily spread. This often leads to deterioration of relationships and social trouble. Furthermore, because message content is transmitted instantly, there is a challenge in that there is no time to re-evaluate the content before sending it.

[0617] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0618] In this invention, the server includes means for acquiring information input from an information processing device, means for evaluating the possibility of inappropriate expressions and aggressive behavior using a generative AI model to analyze the acquired information, and means for evaluating the degree of risk taking into account time information and recipient attribute information at the time of information transmission. This allows users to re-evaluate the message content in advance and improve the content as necessary.

[0619] An "information processing device" is a device that has the function of inputting and processing information such as messages.

[0620] A "generative AI model" is an artificial intelligence model that performs natural language processing to understand and analyze the context and content of input text.

[0621] "Inappropriate language" refers to language that may cause offense or aggression to the recipient or a third party.

[0622] "Aggressive behavior" refers to any action that is intended to hurt or offend others, or has the potential to do so.

[0623] "Risk level" is an indicator that shows the degree to which a message being sent is inappropriate or likely to cause social problems.

[0624] "Communication software" refers to programs used by users to exchange messages and information.

[0625] "Attribute information" refers to data that indicates the characteristics and profile information of the recipient.

[0626] This invention primarily utilizes an information processing device (hereinafter referred to as a terminal), a remote device for information processing (hereinafter referred to as a server), and means for exchanging information between them to configure a system for pre-evaluating messages. The user inputs a message using communication software on the terminal and attempts to send it. At this time, the information is temporarily stored on the terminal.

[0627] Upon receiving information from a terminal, the server begins analysis using a generative AI model. This model employs natural language processing techniques to understand the context of the information and assess the potential for inappropriate language or aggressive behavior. The analysis is carried out using keyword extraction, sentiment analysis, and other similar methods. Furthermore, the server also considers the time information of the transmitted information and the recipient's attributes to comprehensively evaluate the level of risk.

[0628] Based on the analysis results, the server generates a warning message and correction suggestions, and sends them back to the terminal. The terminal notifies the user of the received warnings and correction suggestions, and allows the user to choose to correct the information or stop sending it as needed.

[0629] For example, if a user tries to send a message to a colleague saying, "There are too many problems with your work," the server may deem this message inappropriate and suggest a revised version, such as, "Please consider giving more specific and constructive feedback." This system allows users to re-evaluate information and prevent harassment and inappropriate communication.

[0630] An example of a prompt message could be something like, "Is this message inappropriate? How should it be corrected?" which could be passed to the server. In this way, the generative AI model sequentially evaluates the appropriateness of the information.

[0631] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0632] Step 1:

[0633] The user enters a message using the terminal's communication software and presses the send button. The entered message is saved on the terminal and enters a temporary waiting state. At this stage, the message's text data is output as ready for transmission.

[0634] Step 2:

[0635] After the send button is pressed, the terminal sends the saved message to the server. The input on the server is the message data from the terminal, and the output is an acknowledgment of receipt of this data. The server passes the message content to the generating AI model and inputs prompts for text analysis.

[0636] Step 3:

[0637] The server inputs the received message into a generating AI model, which then begins contextual understanding and keyword extraction. The model uses natural language processing techniques to analyze the emotions and sentiments within the text and calculate the risk level. The input here is the message text, and the output is a data object containing the analysis results.

[0638] Step 4:

[0639] The server evaluates the potential for inappropriate language and offensive behavior based on the generated analysis results. Based on this evaluation, the server generates appropriate warning messages and correction suggestions. At this stage, the input is the analysis results, and the output is the text data of the warnings and correction suggestions.

[0640] Step 5:

[0641] The server sends the generated warning message and correction suggestions back to the terminal. The terminal receives this data and notifies the user. The input is the suggested data from the server, and the output is a visual or auditory notification to the user.

[0642] Step 6:

[0643] The user reviews the received warnings and correction suggestions and chooses to revise the message or cancel sending it as needed. If corrections are made, the user re-edits the message and proceeds with the sending process again. At this point, the input is the suggestions received by the user, and the output is the revised message.

[0644] Step 7:

[0645] Once the corrected message is sent via the resend button, the server parses the message again to determine whether to send it or not. In this step, the input is the corrected message, and the output is feedback on the final sending result.

[0646] (Application Example 1)

[0647] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0648] In electronic payment services, it is crucial for customer support staff to provide appropriate and friendly service to users. However, especially in situations requiring immediate responses, inappropriate or misleading language is often unavoidable. This can damage the user experience and risk lowering the service's rating. This invention aims to prevent inappropriate language in such communication situations and achieve a comfortable user experience.

[0649] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0650] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content, and means for notifying the user of warnings and suggestions for correcting the communication content based on the evaluation results. This ensures that the communications the user receives through customer support are always appropriate and user-friendly, and as a result, enables the provision of a better user experience.

[0651] A "user" is an individual or group that uses the system and is the entity that inputs the communication content.

[0652] "Communication content" refers to messages and conversations that a user attempts to input or send, and is subject to analysis.

[0653] A "knowledge model" is a machine learning or artificial intelligence algorithm used to evaluate the inappropriateness or harmfulness of communication content using natural language processing techniques.

[0654] "Inappropriate language" refers to language that may offend others or is considered socially undesirable.

[0655] "Harassment" refers to actions or expressions that are intentionally made to offend someone, and is usually done intentionally.

[0656] "Evaluation results" refer to judgments regarding the inappropriateness or risk level of communication content, based on analysis using a knowledge model.

[0657] A "warning" is a notification to a user indicating that the content of their communication may be inappropriate.

[0658] A "correction suggestion" refers to alternative expressions or methods of improvement presented to the user to correct inappropriate parts of the communication content.

[0659] The system implementing this invention consists of a user, a server, and a terminal. When a user inputs communication content and attempts to send it through the terminal, the terminal first acquires the communication content. The terminal sends the communication content to the server, which analyzes the content using a knowledge model. In this analysis, natural language processing techniques are used to evaluate whether the communication content is inappropriate or contains harassment. In this process, the server utilizes NLTK or TensorFlow, which are natural language processing libraries implemented in Python. Based on the analysis results, the server generates warnings and correction suggestions, which are then sent to the terminal.

[0660] The terminal immediately notifies the user of any received warnings and corrective suggestions. Based on these notifications, the user re-evaluates the communication content and makes corrections or cancels the transmission. For example, if a customer support user is about to send a message such as "We apologize for the inconvenience," the server can provide a corrective suggestion such as "Why not offer more specific solutions?" This function helps the server provide contextually appropriate responses on behalf of the user.

[0661] As a concrete example, the AI ​​generation model can be instructed with a prompt message such as, "Evaluate whether this message is easy for the customer to understand, and generate improvement suggestions if necessary." This allows the system to efficiently check the communication content and make appropriate suggestions.

[0662] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0663] Step 1:

[0664] The user inputs the communication content into the device. The device captures this communication content and stores it in temporary storage. This communication content will be used as input for analysis in the next step.

[0665] Step 2:

[0666] The terminal sends the communication content to the server. The server uses a generative AI model to analyze the received communication content. First, it extracts keywords and analyzes the context based on natural language processing techniques to generate data necessary to evaluate whether the communication content is inappropriate. The output of this analysis is an inappropriateness score.

[0667] Step 3:

[0668] The server uses a generative AI model to generate warning messages and corrective action suggestions based on the inappropriateness score. This process uses the prompt "Evaluate whether this message is easy for customers to understand and generate improvement suggestions if necessary." The generated warnings and corrective action suggestions are then output for transmission to the terminal.

[0669] Step 4:

[0670] Warnings and suggested corrections are sent from the server to the terminal. The terminal immediately displays these notifications to the user. In this step, the terminal is responsible for providing the user with the output data from the server.

[0671] Step 5:

[0672] The user reviews the warnings and correction suggestions and chooses to correct the communication content or cancel transmission. If the user chooses to correct, the terminal can resend the corrected communication content to the server and repeat the analysis process. Since the user's choice determines the final content of the communication, the output at this step reflects the user's decision.

[0673] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0674] The system according to the present invention is intended for a messaging application equipped with emotion recognition functionality used by users on their terminals. This system provides more advanced message evaluation by using a server that works in conjunction with an emotion engine that receives message input from the user and performs real-time emotion analysis on it.

[0675] Users open social media, chat, or email applications on their devices and type messages. During this process, a sentiment engine analyzes the user's input patterns and wording in the background, evaluating the user's emotional state in real time. This information is sent from the device to the server, where general contextual analysis and sentiment analysis of the message are performed simultaneously.

[0676] The server not only evaluates the potential for inappropriate language and harassment in messages through a generative AI model, but also considers emotional data obtained by the emotion engine to generate warnings and corrective suggestions tailored to the user's emotions. For example, if the user is expressing stress or negative emotions, the server generates a more emphatic message and sends it back to the device.

[0677] The device promptly presents the user with warnings, suggestions, and even additional, emotion-based advice received from the server. Users can use this information to modify their messages, facilitating more appropriate communication.

[0678] For example, if a user enters a message expressing strong dissatisfaction with workplace communication, the emotion engine detects the high stress level. Based on this emotion information and the message content, the server suggests less misleading expressions and provides feedback in a gentle tone, giving the user an opportunity to reconsider their message.

[0679] By implementing this system, users can receive immediate, emotionally sensitive feedback, which in turn helps prevent harassment and inappropriate language.

[0680] The following describes the processing flow.

[0681] Step 1:

[0682] The user enters a message using the messaging application on their device. The emotion engine analyzes the user's keystrokes and input patterns in the background as they type, and evaluates the user's emotional state.

[0683] Step 2:

[0684] The terminal packages the input message and the emotion data output by the emotion engine, and sends them to the server.

[0685] Step 3:

[0686] The server first sends the received message and sentiment data to a generative AI model to begin the process of evaluating the possibility of inappropriate language or harassment.

[0687] Step 4:

[0688] The server simultaneously uses sentiment data to analyze whether the user's emotional state is influencing the message content, and incorporates this into the contextual evaluation.

[0689] Step 5:

[0690] The server integrates the results of the generated AI model and sentiment analysis to calculate a risk score. Based on this score, it generates appropriate warning messages, corrective action suggestions, and even emotionally sensitive advice.

[0691] Step 6:

[0692] The server compiles the generated warnings, suggestions, and advice and sends them back to the terminal.

[0693] Step 7:

[0694] The device immediately notifies the user of any received warnings or suggestions. The notifications also include additional advice based on sentiment analysis, which the user can use to decide whether to modify the message or cancel sending it.

[0695] Step 8:

[0696] If a user modifies and resends a message, the device sends the modified message back to the server for evaluation. If it determines that there are no problems, the message is delivered to the recipient.

[0697] This approach aims to achieve effective communication by providing more human-like feedback from the system.

[0698] (Example 2)

[0699] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0700] In today's communication environment, users are required to send emotionally charged messages quickly and appropriately. However, because emotional judgments cannot be made instantaneously, there is a risk that inappropriate language or potentially human rights-violating messages may be accidentally sent. Therefore, there is a need for a system that allows users to receive appropriate feedback immediately when sending emotionally charged messages.

[0701] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0702] In this invention, the server includes means for acquiring communication content input by the user, means for evaluating the possibility of inappropriate expression and human rights violations using a statistical model to analyze the acquired communication content, and means for notifying the user of warnings and correction suggestions based on the acquired emotional state. This enables users to take their emotions into consideration, prevent inappropriate expression during communication, and send safer and more appropriate messages.

[0703] "Communication content" refers to the text messages and information that a user intends to send, which may include emotions and intentions.

[0704] A "statistical model" is a computational method that uses statistical techniques to analyze data and evaluate the inappropriateness of communication content and the risks related to human rights.

[0705] "Potential human rights violation" refers to the possibility that the message content infringes upon the fundamental rights of others, and problematic expressions are identified by assessing this risk.

[0706] An "emotion analysis device" is a system that detects the emotions and tone contained in a user's communication content and quantifies or categorizes them.

[0707] A "pattern detection algorithm" is a method for identifying certain patterns within data and generating evaluations or warnings that correspond to the user's emotions and intentions.

[0708] An embodiment of the present invention is a communication support system using an emotion analysis and generation AI model for use by users on a communication terminal. This system analyzes the communication content entered by the user in real time and provides feedback to prevent inappropriate expressions and human rights violations.

[0709] The terminal first acquires the text entered by the user. The acquired content is then analyzed by an emotion analysis device. This device uses natural language processing technology to quantify the emotions contained in the communication content and evaluates the user's emotional state.

[0710] Emotional data and communication content are sent together to the server. The server consists of hardware with advanced computing power and is equipped with statistical models. Based on the received data, this server uses a generative AI model to perform contextual analysis. Through contextual analysis, it evaluates whether the message is inappropriate or poses a risk of human rights violations. Furthermore, to enable quick decision-making, the server generates feedback tailored to the user's emotional state, providing warnings and correction suggestions.

[0711] The generated feedback is immediately sent to the device. The device then presents this information to the user, giving the user the opportunity to modify their communication based on the feedback.

[0712] For example, if a user types "This meeting is a complete waste of time," the sentiment analyzer will detect frustration. The server will then generate and provide a calmer suggestion to the user, such as "How about reconfirming the purpose of the meeting?"

[0713] An example of a prompt might be, "Generate suggestions for improving the expression based on emotion." Based on this prompt, the generation AI model generates appropriate feedback on the communication content.

[0714] By utilizing this system, users will be able to express their emotions in a more appropriate way, resulting in smoother communication.

[0715] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0716] Step 1:

[0717] The user inputs the communication content via a terminal. The terminal receives this input, converts it into a data format, and sends it to the sentiment analysis device. The input is the text entered by the user, and the output is data ready for analysis. Specifically, the text undergoes initial formatting and is formatted in a way that is suitable for analysis.

[0718] Step 2:

[0719] The terminal analyzes communication content using an emotion analysis device. The device uses natural language processing technology to analyze keywords and context from the input text and generate emotion data. The input is formatted text data, and the output is numerical or categorical data indicating the emotional state. Specific analysis operations include keyword detection and sentiment score calculation.

[0720] Step 3:

[0721] The terminal sends the generated sentiment data and communication content to the server. The server receives this data and performs a detailed analysis using statistical models and generative AI models. The input is a set of sentiment data and text data, and the output is an evaluation result that takes context and sentiment into account. Specifically, the model performs pattern detection and risk assessment simultaneously.

[0722] Step 4:

[0723] The server generates warnings and correction suggestions based on the evaluation results. The generation AI model creates appropriate feedback according to the given prompt. The input is the evaluation results and prompt, and the output is a feedback message to notify the user. Specifically, the process involves generating the suggested content and formatting the message.

[0724] Step 5:

[0725] The server sends the generated feedback to the terminal, which then displays it to the user. The user re-evaluates the communication based on the provided feedback and makes corrections as needed. The input is the feedback data received from the server, and the output is the user's corresponding action. Specific actions include displaying notification pop-ups and presenting feedback through the user interface.

[0726] (Application Example 2)

[0727] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0728] In recent years, online communication has been steadily increasing, but inappropriate language and harassment have become a problem. Furthermore, there are many cases where such content significantly impacts users' mental state and emotions. In response, there is a need for the development of systems that can detect inappropriate content in real time and provide feedback that takes users' feelings into consideration.

[0729] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0730] In this invention, the server includes means for acquiring information input from the user, means for evaluating the potential for inappropriate expressions and harassment using a generative model to analyze the acquired information, and means for providing warnings, correction suggestions, and sentiment-sensitive feedback based on the evaluation results and sentiment analysis. This enables the user to improve their message expression in real time and obtain a safe and comfortable communication environment.

[0731] A "user" is the entity that inputs information and receives analysis results and feedback from the system.

[0732] "Inputted information" refers to the content of text and messages provided by the user to the system.

[0733] A "generative model" is an algorithm or program used for natural language processing, specifically for detecting inappropriate expressions and harassment.

[0734] "Inappropriate language" refers to expressions or phrases that may cause misunderstandings or conflicts in online communication.

[0735] "Potential harassment" refers to harassment or the risk of harassment that can be inferred from the information entered by the user.

[0736] "Evaluation results" refer to the identification of inappropriate expressions and harassment behaviors derived from the generative model, along with detailed information about them.

[0737] "Emotional analysis" is the process of inferring and analyzing a user's emotional state from the input information.

[0738] A "warning" is a message or notification that alerts the user based on the evaluation results.

[0739] A "revision suggestion" is a specific proposal provided to the user for correcting content deemed inappropriate.

[0740] "Emotionally sensitive feedback" refers to information and suggestions provided while taking into account the user's mental state and emotions.

[0741] This invention consists of a system that performs real-time sentiment analysis and evaluation of the appropriateness of expression on information input by the user, and provides feedback. The system mainly consists of an emotion engine for recognizing and evaluating emotions, a generative AI model for analyzing messages, a server for aggregating and processing information, and a terminal for notifying the user of the evaluation results.

[0742] First, the user enters a message on their device. This device can be a smartphone or computer and is integrated with a communication platform. The entered information is sent directly to the server. On the server, an application built using Python or Django analyzes this information using a generative AI model (for example, OpenAI's GPT-3). The analysis uses natural language processing libraries (NLTK, spaCy, TextBlob) to evaluate the possibility of inappropriate language or harassment, and simultaneously performs sentiment analysis using a sentiment engine.

[0743] The server determines the user's mental state based on the generated evaluation results and sentiment analysis data. This information is fed back to the device as warnings and correction suggestions, and the device notifies the user of this information without delay. This allows the user to check whether their message is inappropriate and correct it to a more appropriate expression.

[0744] For example, when a user at work tries to send a team message to a colleague that contains strong emotions, the emotion engine detects the high stress level. The server then uses a generative model to suggest revisions to adjust the tone of the message, offering advice such as, "How about phrasing it like this?"

[0745] An example of a prompt message is, "If the entered message is offensive, please suggest a way to rephrase it in a more positive and constructive tone."

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The user enters a message from their device. The entered information is immediately sent to the server via the communication platform. The input here is raw text data entered by the user.

[0749] Step 2:

[0750] The server passes the received information to the emotion engine and the generative AI model. The emotion engine uses natural language processing libraries (such as NLTK and spaCy) to analyze the user's emotional state from the input text. The type and intensity of the emotion are output and used for the next processing step.

[0751] Step 3:

[0752] The generative AI model evaluates the potential for inappropriate expressions and harassment based on analyzed sentiment information and text data. The generative AI model (such as OpenAI's GPT-3) identifies inappropriate elements from the input data and calculates their likelihood. The evaluation results are generated on the server side and used as feedback.

[0753] Step 4:

[0754] The server integrates the generated evaluation results and sentiment analysis data to produce appropriate correction suggestions and warnings. The output here is a specific feedback message indicating what correction suggestions or warnings are best for the user.

[0755] Step 5:

[0756] A feedback message is sent to the device and notified to the user. The user receives this notification and, if necessary, modifies or cancels the message. The final revision of the message is made based on the user's actions.

[0757] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0758] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0759] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0760] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0761] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0762] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0763] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0764] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0765] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0766] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0767] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0768] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0769] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0770] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0771] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0772] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0773] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0774] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0775] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0776] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0777] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0778] The following is further disclosed regarding the embodiments described above.

[0779] (Claim 1)

[0780] A means of obtaining a message entered by the user,

[0781] A means of evaluating the potential for inappropriate language and harassment using a generative model to analyze the acquired messages,

[0782] A means of notifying the user of warnings and suggested corrections to messages based on the evaluation results,

[0783] A system that includes means for performing actions in response to a user selecting to modify or cancel a transmission.

[0784] (Claim 2)

[0785] The system according to claim 1, which, in analyzing a message, evaluates the degree of risk by taking into account the time of day and the attributes of the recipient.

[0786] (Claim 3)

[0787] The system according to claim 1, which is integrated with a communication application on the user's terminal and performs message checking and notification in real time.

[0788] "Example 1"

[0789] (Claim 1)

[0790] A means for acquiring information input from an information processing device,

[0791] A means of evaluating the possibility of inappropriate expressions and aggressive behavior using a generative AI model to analyze the acquired information,

[0792] A means of notifying users of warnings and suggestions for correcting information based on evaluation results,

[0793] A means for performing a corresponding action when the user selects to modify or cancel transmission,

[0794] A system that includes means for evaluating the degree of risk by taking into account time information and recipient attribute information at the time of information transmission.

[0795] (Claim 2)

[0796] The system according to claim 1, which is integrated with communication software in an information processing device, and which performs information checking and notification immediately.

[0797] (Claim 3)

[0798] The system according to claim 1, which provides an opportunity to re-evaluate and improve the potential impact of information transmitted from an information processing device.

[0799] "Application Example 1"

[0800] (Claim 1)

[0801] A means of obtaining communication content entered by the user,

[0802] A means of evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content,

[0803] A means of notifying the user of warnings and suggestions for correcting communication content based on the evaluation results,

[0804] A means of performing the appropriate action when the user selects to modify or cancel communication,

[0805] A system that performs technical processing in real time while providing means to offer corrective suggestions for workers to communicate with users in an appropriate and user-friendly manner.

[0806] (Claim 2)

[0807] The system according to claim 1, which, in analyzing the content of communications, evaluates the degree of risk by taking into account the time of day and the characteristics of the recipient.

[0808] (Claim 3)

[0809] The system according to claim 1, which is integrated with a communication application on a user's information terminal and performs real-time inspection and notification of communications.

[0810] "Example 2 of combining an emotion engine"

[0811] (Claim 1)

[0812] A means of obtaining communication content entered by the user,

[0813] A means of using statistical models to evaluate the possibility of inappropriate expression and human rights violations in order to analyze the acquired communication content,

[0814] A means of notifying the user of warnings and suggestions for correcting communication content based on the analysis results,

[0815] A means of taking appropriate action if the user chooses to modify or cancel the transmission,

[0816] A means for evaluating the user's emotional state using an emotion analysis device and transmitting the results,

[0817] A system that includes a means of generating appropriate warnings and suggestions based on the user's emotional state using a pattern detection algorithm.

[0818] (Claim 2)

[0819] The system according to claim 1, which evaluates the degree of risk in analyzing communication content, taking into account the time of day and the attributes of the recipient.

[0820] (Claim 3)

[0821] The system according to claim 1, which is integrated with a communication program in the user's information processing device, and which allows for immediate confirmation and notification of communication content.

[0822] "Application example 2 when combining with an emotional engine"

[0823] (Claim 1)

[0824] Means for obtaining information entered by the user,

[0825] A means of evaluating the possibility of inappropriate expressions and harassment using a generative model to analyze the acquired information,

[0826] A means of providing warnings, corrective suggestions, and emotionally sensitive feedback based on evaluation results and sentiment analysis,

[0827] A system that includes means for performing actions in response to a user selecting to modify, confirm sentiment, or cancel transmission.

[0828] (Claim 2)

[0829] The system according to claim 1, which evaluates the degree of risk in the analysis of information by taking into account the time of day, the attributes of the recipient, and the emotional state of the user.

[0830] (Claim 3)

[0831] The system according to claim 1, which is integrated with a communication platform on the user's terminal and performs information checking, sentiment analysis, and notification in real time. [Explanation of Symbols]

[0832] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of obtaining communication content entered by the user, A means of evaluating the possibility of inappropriate language and harassment using a knowledge model to analyze the acquired communication content, A means of notifying the user of warnings and suggestions for correcting communication content based on the evaluation results, A means of performing the appropriate action when the user selects to modify or cancel communication, A system that performs technical processing in real time while providing means to offer corrective suggestions for workers to communicate with users in an appropriate and user-friendly manner.

2. The system according to claim 1, which, in analyzing the content of communications, evaluates the degree of risk by taking into account the time of day and the characteristics of the recipient.

3. The system according to claim 1, which is integrated with a communication application on the user's information terminal and performs communication inspection and notification in real time.

Images