system

Abstract

We provide the system. [Solution] A means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, A means for automatically generating an email subject line based on extracted information elements, A method for generating email subject lines related to events such as electronic payments using a generative AI model, A means of displaying the generated email subject on the user interface of a mobile information terminal device, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The subject of an email is usually manually created by the sender based on the body of the email. As a result, there are many cases where the subject inadequately conveys the content and importance to the recipient. This problem is likely to have an impact such as important emails being overlooked or responses being delayed, especially in business. Furthermore, efficient creation of the subject is a work burden and a factor that reduces the productivity of users. To solve this problem, there is a need for a method that can accurately grasp the information in the email body and automatically generate a subject that effectively conveys the content and importance to the recipient immediately.

Means for Solving the Problems

[0005] This invention provides a means for analyzing the body of an email using a natural language processing algorithm and extracting predetermined important elements from its content. Based on these extracted important elements, it incorporates a means for automatically generating an email subject line and suggesting an appropriate subject line to the user. Furthermore, it presents the generated subject line through a user interface, allowing the user to modify it as needed. The system also allows for the incorporation of phrases indicating urgency and importance into the email subject line, enabling recipients to immediately grasp the email's significance. In this way, it realizes a system that improves the efficiency of email communication and ensures the rapid and accurate transmission of important information.

[0006] A "natural language processing algorithm" is a technology that enables computers to understand and process the language that humans use on a daily basis.

[0007] "Key elements" are specific keywords or phrases analyzed from the information within an email, and are used to indicate the content and priority of the email.

[0008] The "email subject line" is the title displayed at the top of an email to give the recipient a quick overview of its content and importance.

[0009] A "user interface" refers to the screens and operating environments that allow a system and a user to interact with each other. [Brief explanation of the drawing]

[0010] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4]This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0011] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0012] First, let's explain the terminology used in the following explanation.

[0013] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0014] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0015] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0016] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0017] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0018] [First Embodiment]

[0019] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0020] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0021] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0022] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0023] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0024] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0025] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0026] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0027] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0028] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0029] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0030] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0031] This invention relates to a system for automatically generating appropriate subject lines from email bodies. This system mainly consists of a server, a terminal, and a user interface, and utilizes natural language processing technology.

[0032] When the server receives an email sent by a user, it uses a natural language processing algorithm to analyze the email body. During the analysis, key elements of the email, such as content, deadline, and task, are extracted. Based on this information, the server generates an appropriate email subject line.

[0033] The generated subject line is sent to the terminal and displayed in the user interface. The user interface not only presents the generated subject line to the user but also provides an opportunity for the user to modify it. The user can add to or correct the suggested subject line as needed, including phrases that reflect the urgency or importance they wish to convey to the recipient.

[0034] For example, if the email body contains the message "New project starting, deadline is next Wednesday, everyone must participate," the server will use natural language processing to understand this and suggest a subject line to the terminal such as "Important: New project starting - Deadline: next Wednesday (everyone must participate)."

[0035] The system implemented in this manner aims to enable email recipients to instantly understand the importance of an email based on its subject line, thereby improving work efficiency and ensuring the proper sharing of important information.

[0036] The following describes the processing flow.

[0037] Step 1:

[0038] The terminal sends the email body to the server once the user has finished typing it. This happens when the user requests "automatic subject line generation."

[0039] Step 2:

[0040] The server applies a natural language processing algorithm to the received email body to perform analysis. This analysis process tokenizes the text within the email, picks out information such as nouns, verbs, numbers, and dates, and tags them by part of speech.

[0041] Step 3:

[0042] The server extracts key elements necessary for subject line generation from the analyzed data. These include important keywords that indicate the purpose and urgency of the email, such as the project name, deadline, and action items.

[0043] Step 4:

[0044] Based on the extracted key elements, the server automatically generates a subject line according to a template. For example, it applies the elements to the template "Important {Project Name} - Deadline: {Date}" to prepare a completed subject line.

[0045] Step 5:

[0046] The server sends the generated subject to the terminal. The terminal immediately displays this subject in the user interface and presents it to the user.

[0047] Step 6:

[0048] The user reviews the presented subject line. If necessary, the user can modify the subject line through the interface. For example, they can add information or highlight important points.

[0049] Step 7:

[0050] The subject line is finalized and the email is sent once the user is satisfied with it. This allows the recipient to immediately recognize important information and convey the importance of the email.

[0051] (Example 1)

[0052] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0053] Traditionally, titles for electronic communications were often created manually, which was time-consuming and laborious. In particular, efficiently generating titles that accurately reflected urgency or importance was difficult, sometimes leading to decreased information transmission efficiency. Furthermore, if a title did not intuitively convey importance to the recipient, crucial information could be overlooked.

[0054] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0055] In this invention, the server includes means for extracting predetermined information using natural language processing technology for analyzing text, means for using a generative model for automatically generating a title based on the extracted information, means for displaying the generated title on a display device, and means for providing an interface that allows the user to edit the title. This enables the accurate and rapid generation of a text title, and by allowing the user to edit the displayed title, it becomes possible to more effectively convey urgency and importance.

[0056] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0057] In electronic communication, "information" refers to important data and elements extracted from text.

[0058] A "generative model" refers to an algorithm or program that creates a new output from given data, and in this invention, it is used for generating titles.

[0059] "Display device" refers to a device or screen used to visually present the generated title to the user.

[0060] An "interface" refers to a point of contact or means for a user to interact with a system, and in this invention, it specifically refers to one that enables the editing of titles.

[0061] This invention is a system that improves the efficiency of information transmission by automatically generating a title from the content of a document and presenting it to the user. The system mainly consists of a server, a terminal, and a user interface.

[0062] When the server receives emails or other text content, it analyzes the text using natural language processing techniques. The server utilizes Python and employs the natural language processing library spaCy and the deep learning framework TENSORFLOW® to tokenize information within the text and identify key elements. This analysis process extracts information such as content, deadlines, and tasks. For example, if the text reads, "New project starting, deadline next Wednesday, everyone must participate," the analysis will identify these elements.

[0063] The server then uses a generative AI model to automatically generate a title based on the extracted information. This generative model is based on machine learning algorithms and applies natural language generation technology. An example of a prompt message is: "Generate a title based on the following information: Event = New project start, Due date = Next Wednesday, Participant requirements = All participants must attend."

[0064] The generated title is sent to the device and presented to the user via a user interface. The user interface on the device is built using React.js or similar technologies, allowing the user to visually confirm the title. Furthermore, the user can edit the suggested title through this interface. For example, the user can modify the title to something like "Very Important: New Project Starts - Absolute Participation (Deadline: Next Wednesday)" to further emphasize the urgency and importance of the information. This feature allows important information to be conveyed quickly and accurately to recipients.

[0065] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0066] Step 1:

[0067] The server retrieves text data received from the client. The input is the body of an email or message. The server then prepares this text for analysis.

[0068] Step 2:

[0069] The server analyzes the text content using natural language processing techniques. It uses a Python natural language processing library to tokenize the text and perform syntactic analysis. The input is the body of an email, and the output extracts important information elements (e.g., tasks, deadlines, participants, etc.).

[0070] Step 3:

[0071] The server automatically generates a title using a generative AI model based on the extracted information. The input is important information obtained by the server through analysis, and the data is provided to the model as a prompt in the format of "Event = New project start, Due date = Next Wednesday, Participant conditions = All participants must attend". The output is the generated title.

[0072] Step 4:

[0073] The server sends the generated title to the terminal. The input is the title generated by the server, and the output is the data ready to be displayed in the user interface.

[0074] Step 5:

[0075] The device presents the received title information to the user through the user interface. The device uses React.js to visually display the title on the screen. The user confirms the title displayed on the device as visual information.

[0076] Step 6:

[0077] The user reviews the displayed title and edits it if necessary. Through the user interface, the user enters a new title and completes the revision. The system records the edited result, and the final title decided by the user is output.

[0078] (Application Example 1)

[0079] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0080] In email communication, there is a problem where important information regarding payments and other important events is not properly conveyed to recipients, leading to delays in information verification and response. Furthermore, users have the added burden of individually setting email subject lines. Therefore, there is a need for a method to quickly and reliably transmit important information to users, particularly in electronic payment services.

[0081] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0082] In this invention, the server includes means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, means for automatically generating an email subject based on the extracted information elements, and means for generating an email subject related to an event such as an electronic payment using a generation AI model. This makes it possible to automatically and appropriately generate email subjects containing important information.

[0083] "Email" refers to digital messages sent and received via networks such as the internet.

[0084] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language.

[0085] An "information element" is a unit of particularly important data or information extracted from the body of an email.

[0086] A "generative AI model" is a collection of algorithms or programs that utilize artificial intelligence to produce a specific output.

[0087] A "portable information terminal device" is a portable computing device such as a smartphone or tablet.

[0088] This invention relates to a system for automatically generating email subject lines. The system mainly consists of a server, a terminal, and a user interface operated by the user.

[0089] When the server receives the body of an email sent over the internet, it analyzes the text using a natural language processing algorithm. This analysis extracts predetermined information elements such as the payee, amount, and date. The Spacy library of the Python programming language is used for natural language processing.

[0090] Subsequently, the server utilizes a generative AI model, specifically a model such as GPT-2, to generate an appropriate email subject line based on the extracted information elements. The Transformers library is used in this process, and the generated subject line appropriately reflects the electronic payment event.

[0091] The terminal displays the generated email subject line in the user interface. On this interface, the user can edit the subject line and, if necessary, add expressions indicating urgency or importance. This allows the recipient to quickly determine the importance of the email.

[0092] As a concrete example, consider a payment confirmation email for a meal with a friend. If the body of this email is something like, "I paid ¥5,000 for a meal at a restaurant on October 10, 2023," then using the appropriate prompt "Generate subject from email body: Restaurant, ¥5,000, October 10, 2023," the model will automatically generate a subject like "Receipt Restaurant - ¥5,000, Payment Date: October 10, 2023."

[0093] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0094] Step 1:

[0095] The server receives the email body. The received email body is used as input data, and the natural language processing library Spacy is used to begin analysis. This analysis extracts information elements such as the payee, amount, and date from the email body. The extracted information elements are then output.

[0096] Step 2:

[0097] The server generates a prompt based on the extracted information elements. Here, the extracted information elements (for example, "payee," "amount," and "date") are combined to create a prompt in the format "Generate subject from email body: payee, amount, date." This prompt becomes the input for the next AI model.

[0098] Step 3:

[0099] The server inputs a prompt message into the GPT-2 AI model. Based on the input prompt message, the AI ​​model generates the optimal email subject line. In this process, natural language generation technology is used to create a subject line suitable for a payment event, and it is output.

[0100] Step 4:

[0101] The terminal displays the generated email subject on the user interface. Here, the generated subject is displayed on the screen so that the user can check it. This display is the output to the user.

[0102] Step 5:

[0103] The user can modify the subject line as needed. The user can add to or modify the presented subject line through the interface. The modified subject line will be finalized as the final output name.

[0104] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0105] This invention integrates an emotion engine into an email system to generate email subject lines that not only extract important elements from the email body but also take into account the user's emotions. This system consists of a server, a terminal, and a user interface that includes the emotion engine.

[0106] Upon receiving an email, the server analyzes its content using natural language processing algorithms and extracts key elements. Furthermore, it analyzes the user's emotions through an emotion engine. This emotion analysis is performed using general emotion recognition techniques and by analyzing the user's typing speed and patterns. This allows the server to identify whether the user was experiencing emotions such as relief, joy, or anxiety when composing the email.

[0107] Based on the extracted key elements and sentiment data, the server generates a subject line. The sentiment data is used to add emojis and appropriate emotional phrases to the subject line. Furthermore, the subject line is provided with expressions that resonate more emotionally with the recipient.

[0108] The generated subject line is sent to the device and displayed in the user interface. The user can review the displayed subject line and modify it as needed. This allows the user's emotions to be strongly reflected in the email subject line, resulting in more personalized information being conveyed to the recipient.

[0109] For example, if a user is in a cheerful emotional state when sending an email with the theme "New Project Launch," the server will generate a subject line like "🎉 New Project Launch - Deadline: Next Wednesday 😊." By combining this with an emotion engine, email subject lines become more human-like and more clearly convey the intent. The goal is to allow recipients to grasp the mood and nuance of an email just from the subject line, leading to smoother communication.

[0110] The following describes the processing flow.

[0111] Step 1:

[0112] The device simultaneously records the user's typing speed and patterns as they type the email body. This data is sent to the emotion engine and used to estimate the user's emotional state.

[0113] Step 2:

[0114] Once the user has finished entering their email address, the device sends the email body and sentiment data to the server. The server receives this data and begins the next analysis process.

[0115] Step 3:

[0116] The server uses a natural language processing algorithm to analyze the email body and extract important elements such as the project name, deadline, and action items.

[0117] Step 4:

[0118] The emotion engine utilizes typing data and facial recognition technology to primarily analyze the user's emotions during input. The results of this analysis are generated as data specifying emotional states such as relief, joy, anxiety, and excitement.

[0119] Step 5:

[0120] The server integrates extracted key elements and sentiment data to generate email subject lines. Based on the sentiment data, it includes emojis and phrases that convey emotions to the recipient.

[0121] Step 6:

[0122] The generated email subject line is sent to the device and displayed in the user interface. The user can review the generated subject line and, if necessary, make revisions to reflect nuances of emotion or information.

[0123] Step 7:

[0124] After the user confirms the subject line, they send the email through their device. The recipient of the email can immediately understand the content of the email and the sender's emotional state based on this subject line.

[0125] (Example 2)

[0126] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0127] Traditional email systems have struggled to adequately reflect the user's emotions and the importance of the email body in the subject line, making it difficult to effectively communicate the email's intent and urgency to the recipient. Furthermore, the process of reflecting users' emotions in the subject line is often done manually and is time-consuming, creating a need for more efficient email communication.

[0128] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0129] In this invention, the server includes language processing means, means for analyzing the user's emotions, and means for adding specific symbols or phrases that represent emotions to the subject line based on the emotion data. This makes it possible to accurately reflect not only importance and urgency but also the user's emotions in the subject line of an email, resulting in more empathetic and effective email communication.

[0130] "Language processing means" refers to algorithms and technologies used to analyze text data and extract important information.

[0131] "Means of analyzing user emotions" refers to technologies and methods for identifying a user's emotional state by analyzing input data and user behavior patterns.

[0132] "Emotional data" refers to information that quantifies or categorizes a user's emotional state.

[0133] "Specific symbols and words" refer to emojis and specific words used to convey emotions or intentions.

[0134] A "display device" refers to a device or interface used to visually present generated information to a user.

[0135] This invention is a system comprising language processing means, means for analyzing user emotions, and means for adding symbols or phrases representing emotions to the title using emotion data. Specific embodiments are described below.

[0136] When the server receives an email sent by a user, it first analyzes the text using natural language processing (NLP) techniques. This analysis uses a programming language like Python and leverages libraries such as NLTK and SpaCy to extract important information. The extracted information is then used to summarize the email content and as keywords.

[0137] Next, the server performs sentiment analysis techniques to analyze the user's emotions. This typically involves using tools such as TextBlob or SentimentIntensityAnalyzer. The server considers typing speed and patterns during email composition to quantify or categorize the user's emotional state. The resulting emotional data is then classified into emotional categories such as relief, joy, and anxiety.

[0138] Based on this data, the server uses a generative AI model to automatically generate email subject lines. During the generation process, prompts are used to instruct the AI ​​model to create appropriate subject lines. Specific prompts can be used, such as, "Analyze the email body sent by the user and generate a subject line that reflects the user's emotional state. However, the subject line must include emojis and emotionally expressive words." The generated subject lines will include emojis and emotionally expressive terms, aiming to deliver an emotionally resonant message to the recipient.

[0139] Finally, the server sends the generated title to the terminal, which visually displays this information on its user interface. The user can review this title and make corrections as needed, resulting in smoother and more personalized communication.

[0140] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0141] Step 1:

[0142] The server receives the email body sent by the user. Using this received text data as input, it first performs analysis using natural language processing (NLP) techniques. Using NLTK and SpaCy libraries in Python, the server tokenizes the email body and assigns part-of-speech tags to extract important words such as nouns and verbs. This process outputs a summary and key information from the email content.

[0143] Step 2:

[0144] The server analyzes the user's emotions based on the key keywords extracted in Step 1. This process considers the user's typing data and input patterns, and generates emotional data using tools such as TextBlob and SentimentIntensityAnalyzer. The speed and rhythm of key input during email composition are used to determine the emotional state, and the output obtained from this data is categorized into emotional categories such as "joy," "relief," and "anxiety."

[0145] Step 3:

[0146] The server uses a generative AI model to synthesize the sentiment data obtained in Step 2 with the key keywords from Step 1 to generate an email subject line. In this process, a prompt is provided to the generative AI model, instructing it to "create a subject line that resonates emotionally with the recipient, based on the extracted key elements and sentiment data." This generates a subject line that includes expressions and emojis reflecting emotions, and the result is obtained as output.

[0147] Step 4:

[0148] The server sends the generated subject line to the user's terminal, which then displays it on the user interface. The subject line is displayed on the terminal and visually adjusted, allowing the user to intuitively understand the content. The user reviews the outputted subject line, makes any necessary corrections, and sends the email after final confirmation.

[0149] (Application Example 2)

[0150] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0151] Traditional email systems primarily generate subject lines based on the importance of the text, making it difficult to reflect user emotions. As a result, the content and nuances of emails can be unclear to recipients. Furthermore, smooth communication with recipients requires flexible responses that adapt to user emotions and circumstances, but current systems lack the means to achieve this. Technology is needed to solve this problem.

[0152] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0153] In this invention, the server includes means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email; means for automatically generating an email subject line that takes emotional elements into consideration based on the extracted important elements and sentiment analysis; and means for presenting the generated email subject line to a user interface and providing the user with an interactive response. This makes it possible to generate a subject line that reflects the user's emotions, and to convey clearer and more emotional nuances to the recipient.

[0154] A "natural language processing algorithm" is a technology that enables computers to understand human language and extract meaning from text.

[0155] "Sentiment analysis" is a technology used to estimate a person's emotions from text and user data.

[0156] "Emotional elements" refer to information or expressions based on emotions that convey a specific emotional nuance to the recipient or user.

[0157] A "user interface" is a mechanism that allows a user to interact with a system and input or output information.

[0158] "Interactive responses" are responses generated based on dialogue with the user, primarily intended to facilitate smoother interaction between computers and humans.

[0159] This embodiment of the invention is a system for realizing natural, emotion-reflecting dialogue when a user interacts with a household robot. The specific implementation method of this system is described below.

[0160] The server first receives the user's voice and converts it into text data using speech recognition software. Technologies such as Google® Cloud Speech-to-Text can be used for this speech recognition. Then, a natural language processing algorithm is executed to extract important elements from the input text. TensorFlow or PyTorch are preferred for this process.

[0161] Next, services such as Watson® Natural Language Understanding are used to perform sentiment analysis. This allows the system to determine the user's emotional state. Based on the extracted key elements and sentiment data, the server utilizes a generative AI model (e.g., GPT-3®) to generate an appropriate response for the user.

[0162] The generated responses are automatically presented to the user interface. For example, Google Cloud Text-to-Speech can be used for speech synthesis, allowing the robot to respond in voice. This process enables natural interaction with the user.

[0163] As a concrete example, consider a scenario where a user says to a home robot, "Today was a busy day." If the emotion analysis engine determines that the user is stressed, the server can generate a response such as, "That sounds tough. Is there anything I can do to help you relax?" This response can include humor or suggestions that take the user's emotions into consideration.

[0164] Examples of prompt messages include the following:

[0165] "User said: 'Today was a busy day.' Information: Stress, emotional burden. Generate an appropriate robot response."

[0166] This makes it possible to create more personalized and emotionally sensitive interactions with home robots.

[0167] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0168] Step 1:

[0169] The server receives voice input from the user. This voice input is transmitted to a home robot via a microphone, and the server converts it into text data using speech recognition software (e.g., Google Cloud Speech-to-Text). The input is the user's voice data, and the output is the converted text data. This conversion prepares the data for subsequent text analysis.

[0170] Step 2:

[0171] The server passes the text data to a natural language processing algorithm to extract important elements. Here, TensorFlow or PyTorch is used to analyze sentence structure and extract necessary keywords and the main point of the sentence. The input is the text data generated in step 1, and the output is a dataset containing the important elements. This allows for the identification of the intent of the utterance.

[0172] Step 3:

[0173] The server inputs text data containing important elements into an emotion analysis engine (e.g., Watson Natural Language Understanding) to analyze the user's emotions. This analysis examines emotional expressions within the text and estimates the user's emotional state. The input is the dataset obtained in step 2, and the output is data indicating the user's emotional state. This allows for an understanding of the user's emotional condition.

[0174] Step 4:

[0175] The server provides prompt sentences to a generative AI model (e.g., GPT-3) based on extracted key elements and sentiment data. The generative AI model uses this data to generate an appropriate response to the user. The input is sentiment data and key elements, and the output is an interactive response to the user. This response is required to be empathetic to the user's emotions.

[0176] Step 5:

[0177] The server sends the generated response to the home robot's interface, and uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to output it as speech. This allows the user to hear the voice response from the robot. The input is the response sentence generated in step 4, and the output is the voice response message. This process enables natural interaction with the robot.

[0178] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0179] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0180] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0181] [Second Embodiment]

[0182] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0183] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0184] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0185] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0186] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0187] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0188] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0189] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0190] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0191] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0192] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0193] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0194] This invention relates to a system for automatically generating appropriate subject lines from email bodies. This system mainly consists of a server, a terminal, and a user interface, and utilizes natural language processing technology.

[0195] When the server receives an email sent by a user, it uses a natural language processing algorithm to analyze the email body. During the analysis, key elements of the email, such as content, deadline, and task, are extracted. Based on this information, the server generates an appropriate email subject line.

[0196] The generated subject line is sent to the terminal and displayed in the user interface. The user interface not only presents the generated subject line to the user but also provides an opportunity for the user to modify it. The user can add to or correct the suggested subject line as needed, including phrases that reflect the urgency or importance they wish to convey to the recipient.

[0197] For example, if the email body contains the message "New project starting, deadline is next Wednesday, everyone must participate," the server will use natural language processing to understand this and suggest a subject line to the terminal such as "Important: New project starting - Deadline: next Wednesday (everyone must participate)."

[0198] The system implemented in this manner aims to enable email recipients to instantly understand the importance of an email based on its subject line, thereby improving work efficiency and ensuring the proper sharing of important information.

[0199] The following describes the processing flow.

[0200] Step 1:

[0201] The terminal sends the email body to the server once the user has finished typing it. This happens when the user requests "automatic subject line generation."

[0202] Step 2:

[0203] The server applies a natural language processing algorithm to the received email body to perform analysis. This analysis process tokenizes the text within the email, picks out information such as nouns, verbs, numbers, and dates, and tags them by part of speech.

[0204] Step 3:

[0205] The server extracts key elements necessary for subject line generation from the analyzed data. These include important keywords that indicate the purpose and urgency of the email, such as the project name, deadline, and action items.

[0206] Step 4:

[0207] Based on the extracted key elements, the server automatically generates a subject line according to a template. For example, it applies the elements to the template "Important {Project Name} - Deadline: {Date}" to prepare a completed subject line.

[0208] Step 5:

[0209] The server sends the generated subject to the terminal. The terminal immediately displays this subject in the user interface and presents it to the user.

[0210] Step 6:

[0211] The user reviews the presented subject line. If necessary, the user can modify the subject line through the interface. For example, they can add information or highlight important points.

[0212] Step 7:

[0213] The subject line is finalized and the email is sent once the user is satisfied with it. This allows the recipient to immediately recognize important information and convey the importance of the email.

[0214] (Example 1)

[0215] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0216] Traditionally, titles for electronic communications were often created manually, which was time-consuming and laborious. In particular, efficiently generating titles that accurately reflected urgency or importance was difficult, sometimes leading to decreased information transmission efficiency. Furthermore, if a title did not intuitively convey importance to the recipient, crucial information could be overlooked.

[0217] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0218] In this invention, the server includes means for extracting predetermined information using natural language processing technology for analyzing text, means for using a generative model for automatically generating a title based on the extracted information, means for displaying the generated title on a display device, and means for providing an interface that allows the user to edit the title. This enables the accurate and rapid generation of a text title, and by allowing the user to edit the displayed title, it becomes possible to more effectively convey urgency and importance.

[0219] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0220] In electronic communication, "information" refers to important data and elements extracted from text.

[0221] A "generative model" refers to an algorithm or program that creates a new output from given data, and in this invention, it is used for generating titles.

[0222] "Display device" refers to a device or screen used to visually present the generated title to the user.

[0223] An "interface" refers to a point of contact or means for a user to interact with a system, and in this invention, it specifically refers to one that enables the editing of titles.

[0224] This invention is a system that improves the efficiency of information transmission by automatically generating a title from the content of a document and presenting it to the user. The system mainly consists of a server, a terminal, and a user interface.

[0225] When the server receives emails or other text content, it analyzes the text using natural language processing techniques. The server utilizes Python, employing the natural language processing library spaCy and the deep learning framework TensorFlow to tokenize information within the text and identify key elements. This analysis process extracts information such as content, deadlines, and tasks. For example, if the text reads, "New project starting, deadline next Wednesday, everyone must participate," the analysis will identify these elements.

[0226] The server then uses a generative AI model to automatically generate a title based on the extracted information. This generative model is based on machine learning algorithms and applies natural language generation technology. An example of a prompt message is: "Generate a title based on the following information: Event = New project start, Due date = Next Wednesday, Participant requirements = All participants must attend."

[0227] The generated title is sent to the device and presented to the user via a user interface. The user interface on the device is built using React.js or similar technologies, allowing the user to visually confirm the title. Furthermore, the user can edit the suggested title through this interface. For example, the user can modify the title to something like "Very Important: New Project Starts - Absolute Participation (Deadline: Next Wednesday)" to further emphasize the urgency and importance of the information. This feature allows important information to be conveyed quickly and accurately to recipients.

[0228] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0229] Step 1:

[0230] The server retrieves text data received from the client. The input is the body of an email or message. The server then prepares this text for analysis.

[0231] Step 2:

[0232] The server analyzes the text content using natural language processing techniques. It uses a Python natural language processing library to tokenize the text and perform syntactic analysis. The input is the body of an email, and the output extracts important information elements (e.g., tasks, deadlines, participants, etc.).

[0233] Step 3:

[0234] The server automatically generates a title using a generative AI model based on the extracted information. The input is important information obtained by the server through analysis, and the data is provided to the model as a prompt in the format of "Event = New project start, Due date = Next Wednesday, Participant conditions = All participants must attend". The output is the generated title.

[0235] Step 4:

[0236] The server sends the generated title to the terminal. The input is the title generated by the server, and the output is the data ready to be displayed in the user interface.

[0237] Step 5:

[0238] The device presents the received title information to the user through the user interface. The device uses React.js to visually display the title on the screen. The user confirms the title displayed on the device as visual information.

[0239] Step 6:

[0240] The user reviews the displayed title and edits it if necessary. Through the user interface, the user enters a new title and completes the revision. The system records the edited result, and the final title decided by the user is output.

[0241] (Application Example 1)

[0242] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0243] In email communication, there is a problem where important information regarding payments and other important events is not properly conveyed to recipients, leading to delays in information verification and response. Furthermore, users have the added burden of individually setting email subject lines. Therefore, there is a need for a method to quickly and reliably transmit important information to users, particularly in electronic payment services.

[0244] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0245] In this invention, the server includes means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, means for automatically generating an email subject based on the extracted information elements, and means for generating an email subject related to an event such as an electronic payment using a generation AI model. This makes it possible to automatically and appropriately generate email subjects containing important information.

[0246] "Email" refers to digital messages sent and received via networks such as the internet.

[0247] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language.

[0248] An "information element" is a unit of particularly important data or information extracted from the body of an email.

[0249] A "generative AI model" is a collection of algorithms or programs that utilize artificial intelligence to produce a specific output.

[0250] A "portable information terminal device" is a portable computing device such as a smartphone or tablet.

[0251] This invention relates to a system for automatically generating email subject lines. The system mainly consists of a server, a terminal, and a user interface operated by the user.

[0252] When the server receives the body of an email sent over the internet, it analyzes the text using a natural language processing algorithm. This analysis extracts predetermined information elements such as the payee, amount, and date. The Spacy library of the Python programming language is used for natural language processing.

[0253] Subsequently, the server utilizes a generative AI model, specifically a model such as GPT-2, to generate an appropriate email subject line based on the extracted information elements. The Transformers library is used in this process, and the generated subject line appropriately reflects the electronic payment event.

[0254] The terminal displays the generated email subject line in the user interface. On this interface, the user can edit the subject line and, if necessary, add expressions indicating urgency or importance. This allows the recipient to quickly determine the importance of the email.

[0255] As a concrete example, consider a payment confirmation email for a meal with a friend. If the body of this email is something like, "I paid ¥5,000 for a meal at a restaurant on October 10, 2023," then using the appropriate prompt "Generate subject from email body: Restaurant, ¥5,000, October 10, 2023," the model will automatically generate a subject like "Receipt Restaurant - ¥5,000, Payment Date: October 10, 2023."

[0256] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0257] Step 1:

[0258] The server receives the email body. The received email body is used as input data, and the natural language processing library Spacy is used to begin analysis. This analysis extracts information elements such as the payee, amount, and date from the email body. The extracted information elements are then output.

[0259] Step 2:

[0260] The server generates a prompt based on the extracted information elements. Here, the extracted information elements (for example, "payee," "amount," and "date") are combined to create a prompt in the format "Generate subject from email body: payee, amount, date." This prompt becomes the input for the next AI model.

[0261] Step 3:

[0262] The server inputs a prompt message into the GPT-2 AI model. Based on the input prompt message, the AI ​​model generates the optimal email subject line. In this process, natural language generation technology is used to create a subject line suitable for a payment event, and it is output.

[0263] Step 4:

[0264] The terminal displays the generated email subject on the user interface. Here, the generated subject is displayed on the screen so that the user can check it. This display is the output to the user.

[0265] Step 5:

[0266] The user can modify the subject line as needed. The user can add to or modify the presented subject line through the interface. The modified subject line will be finalized as the final output name.

[0267] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0268] This invention integrates an emotion engine into an email system to generate email subject lines that not only extract important elements from the email body but also take into account the user's emotions. This system consists of a server, a terminal, and a user interface that includes the emotion engine.

[0269] Upon receiving an email, the server analyzes its content using natural language processing algorithms and extracts key elements. Furthermore, it analyzes the user's emotions through an emotion engine. This emotion analysis is performed using general emotion recognition techniques and by analyzing the user's typing speed and patterns. This allows the server to identify whether the user was experiencing emotions such as relief, joy, or anxiety when composing the email.

[0270] Based on the extracted key elements and sentiment data, the server generates a subject line. The sentiment data is used to add emojis and appropriate emotional phrases to the subject line. Furthermore, the subject line is provided with expressions that resonate more emotionally with the recipient.

[0271] The generated subject line is sent to the device and displayed in the user interface. The user can review the displayed subject line and modify it as needed. This allows the user's emotions to be strongly reflected in the email subject line, resulting in more personalized information being conveyed to the recipient.

[0272] For example, if a user is in a cheerful emotional state when sending an email with the theme "New Project Launch," the server will generate a subject line like "🎉 New Project Launch - Deadline: Next Wednesday 😊." By combining this with an emotion engine, email subject lines become more human-like and more clearly convey the intent. The goal is to allow recipients to grasp the mood and nuance of an email just from the subject line, leading to smoother communication.

[0273] The following describes the processing flow.

[0274] Step 1:

[0275] The device simultaneously records the user's typing speed and patterns as they type the email body. This data is sent to the emotion engine and used to estimate the user's emotional state.

[0276] Step 2:

[0277] Once the user has finished entering their email address, the device sends the email body and sentiment data to the server. The server receives this data and begins the next analysis process.

[0278] Step 3:

[0279] The server uses a natural language processing algorithm to analyze the email body and extract important elements such as the project name, deadline, and action items.

[0280] Step 4:

[0281] The emotion engine utilizes typing data and emotion recognition technology to mainly analyze the emotions of the user during input. The analysis results are generated as data specifying emotional states such as peace of mind, joy, anxiety, excitement, etc.

[0282] Step 5:

[0283] The server integrates the extracted key elements and emotion data to generate the email subject. At this time, based on the emotion data, emojis and phrases containing emotional elements conveyed to the recipient are included in the subject.

[0284] Step 6:

[0285] The generated email subject is sent to the terminal and displayed on the user interface. The user can check the generated subject and make corrections reflecting the nuances of emotions and information if necessary.

[0286] Step 7:

[0287] After the user confirms the subject, the email is sent through the terminal. The recipient of the email can immediately understand the content of the email and the emotional state of the sender based on this subject.

[0288] (Example 2)

[0289] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0290] In the conventional email system, it is difficult to fully reflect the user's emotions and the importance of the text in the email subject, and there is a problem that the intention and urgency of the email cannot be effectively conveyed to the recipient. Furthermore, the work of the user reflecting their own emotions in the subject is often done manually and takes time, so it has been required to realize efficient email communication.

[0291] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0292] In this invention, the server includes language processing means, means for analyzing the user's emotions, and means for adding specific symbols or phrases that represent emotions to the subject line based on the emotion data. This makes it possible to accurately reflect not only importance and urgency but also the user's emotions in the subject line of an email, resulting in more empathetic and effective email communication.

[0293] "Language processing means" refers to algorithms and technologies used to analyze text data and extract important information.

[0294] "Means of analyzing user emotions" refers to technologies and methods for identifying a user's emotional state by analyzing input data and user behavior patterns.

[0295] "Emotional data" refers to information that quantifies or categorizes a user's emotional state.

[0296] "Specific symbols and words" refer to emojis and specific words used to convey emotions or intentions.

[0297] A "display device" refers to a device or interface used to visually present generated information to a user.

[0298] This invention is a system comprising language processing means, means for analyzing user emotions, and means for adding symbols or phrases representing emotions to the title using emotion data. Specific embodiments are described below.

[0299] When the server receives an email sent by a user, it first analyzes the text using natural language processing (NLP) techniques. This analysis uses a programming language like Python and leverages libraries such as NLTK and SpaCy to extract important information. The extracted information is then used to summarize the email content and as keywords.

[0300] Next, the server performs sentiment analysis techniques to analyze the user's emotions. This typically involves using tools such as TextBlob or SentimentIntensityAnalyzer. The server considers typing speed and patterns during email composition to quantify or categorize the user's emotional state. The resulting emotional data is then classified into emotional categories such as relief, joy, and anxiety.

[0301] Based on this data, the server uses a generative AI model to automatically generate email subject lines. During the generation process, prompts are used to instruct the AI ​​model to create appropriate subject lines. Specific prompts can be used, such as, "Analyze the email body sent by the user and generate a subject line that reflects the user's emotional state. However, the subject line must include emojis and emotionally expressive words." The generated subject lines will include emojis and emotionally expressive terms, aiming to deliver an emotionally resonant message to the recipient.

[0302] Finally, the server sends the generated title to the terminal, which visually displays this information on its user interface. The user can review this title and make corrections as needed, resulting in smoother and more personalized communication.

[0303] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0304] Step 1:

[0305] The server receives the email body sent by the user. Using this received text data as input, it first performs analysis using natural language processing (NLP) techniques. With NLTK and SpaCy libraries in Python, the server tokenizes the email body and assigns part-of-speech tags to extract important phrases such as nouns and verbs. Through this process, it outputs the summary and key information in the email content.

[0306] Step 2:

[0307] Based on the important phrases extracted in Step 1, the server analyzes the user's sentiment. In this process, considering the user's typing data and input patterns, tools such as TextBlob and SentimentIntensityAnalyzer are used to generate sentiment data. The speed and rhythm of key input when creating the email serve as materials for judging the emotional state, and the output obtained from this data is in categories of emotions such as "joy", "relief", and "anxiety".

[0308] Step 3:

[0309] The server uses a generative AI model to synthesize the sentiment data obtained in Step 2 and the important phrases in Step 1 to generate the subject line of the email. In this step, a prompt sentence is provided to the generative AI model, giving instructions to the model such as "Create a subject line that emotionally resonates with the recipient based on the extracted important elements and sentiment data". As a result, a subject line containing expressions and emojis that reflect the sentiment is generated, and the result is obtained as the output.

[0310] Step 4:

[0311] The server sends the generated subject line to the user's terminal, and the terminal presents it on the user interface. On the terminal, the subject line is displayed, and through visual adjustments, the user can intuitively confirm the content. The user checks the output subject line, makes corrections if necessary, and finally sends the email after final confirmation.

[0312] (Application Example 2)

[0313] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0314] Traditional email systems primarily generate subject lines based on the importance of the text, making it difficult to reflect user emotions. As a result, the content and nuances of emails can be unclear to recipients. Furthermore, smooth communication with recipients requires flexible responses that adapt to user emotions and circumstances, but current systems lack the means to achieve this. Technology is needed to solve this problem.

[0315] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0316] In this invention, the server includes means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email; means for automatically generating an email subject line that takes emotional elements into consideration based on the extracted important elements and sentiment analysis; and means for presenting the generated email subject line to a user interface and providing the user with an interactive response. This makes it possible to generate a subject line that reflects the user's emotions, and to convey clearer and more emotional nuances to the recipient.

[0317] A "natural language processing algorithm" is a technology that enables computers to understand human language and extract meaning from text.

[0318] "Sentiment analysis" is a technology used to estimate a person's emotions from text and user data.

[0319] "Emotional elements" refer to information or expressions based on emotions that convey a specific emotional nuance to the recipient or user.

[0320] A "user interface" is a mechanism that allows a user to interact with a system and input or output information.

[0321] "Interactive responses" are responses generated based on dialogue with the user, primarily intended to facilitate smoother interaction between computers and humans.

[0322] This embodiment of the invention is a system for realizing natural, emotion-reflecting dialogue when a user interacts with a household robot. The specific implementation method of this system is described below.

[0323] The server first receives the user's voice and converts it into text data using speech recognition software. Technologies such as Google Cloud Speech-to-Text can be used for this speech recognition. Then, a natural language processing algorithm is executed to extract important elements from the input text. TensorFlow or PyTorch are preferred for this process.

[0324] Next, services such as Watson Natural Language Understanding are used to perform sentiment analysis. This allows the system to determine the user's emotional state. Based on the extracted key elements and sentiment data, the server utilizes a generative AI model (e.g., GPT-3) to generate an appropriate response for the user.

[0325] The generated responses are automatically presented to the user interface. For example, Google Cloud Text-to-Speech can be used for speech synthesis, allowing the robot to respond in voice. This process enables natural interaction with the user.

[0326] As a concrete example, consider a scenario where a user says to a home robot, "Today was a busy day." If the emotion analysis engine determines that the user is stressed, the server can generate a response such as, "That sounds tough. Is there anything I can do to help you relax?" This response can include humor or suggestions that take the user's emotions into consideration.

[0327] Examples of prompt messages include the following:

[0328] "User said: 'Today was a busy day.' Information: Stress, emotional burden. Generate an appropriate robot response."

[0329] This makes it possible to create more personalized and emotionally sensitive interactions with home robots.

[0330] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0331] Step 1:

[0332] The server receives voice input from the user. This voice input is transmitted to a home robot via a microphone, and the server converts it into text data using speech recognition software (e.g., Google Cloud Speech-to-Text). The input is the user's voice data, and the output is the converted text data. This conversion prepares the data for subsequent text analysis.

[0333] Step 2:

[0334] The server passes the text data to a natural language processing algorithm to extract important elements. Here, TensorFlow or PyTorch is used to analyze sentence structure and extract necessary keywords and the main point of the sentence. The input is the text data generated in step 1, and the output is a dataset containing the important elements. This allows for the identification of the intent of the utterance.

[0335] Step 3:

[0336] The server inputs text data containing important elements into an emotion analysis engine (e.g., Watson Natural Language Understanding) to analyze the user's emotions. This analysis examines emotional expressions within the text and estimates the user's emotional state. The input is the dataset obtained in step 2, and the output is data indicating the user's emotional state. This allows for an understanding of the user's emotional condition.

[0337] Step 4:

[0338] The server provides prompt sentences to a generative AI model (e.g., GPT-3) based on extracted key elements and sentiment data. The generative AI model uses this data to generate an appropriate response to the user. The input is sentiment data and key elements, and the output is an interactive response to the user. This response is required to be empathetic to the user's emotions.

[0339] Step 5:

[0340] The server sends the generated response to the home robot's interface, and uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to output it as speech. This allows the user to hear the voice response from the robot. The input is the response sentence generated in step 4, and the output is the voice response message. This process enables natural interaction with the robot.

[0341] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0342] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0343] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0344] [Third Embodiment]

[0345] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0346] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0347] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0348] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0349] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0350] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0351] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0352] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0353] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0354] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0355] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0356] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0357] This invention relates to a system for automatically generating appropriate subject lines from email bodies. This system mainly consists of a server, a terminal, and a user interface, and utilizes natural language processing technology.

[0358] When the server receives an email sent by a user, it uses a natural language processing algorithm to analyze the email body. During the analysis, key elements of the email, such as content, deadline, and task, are extracted. Based on this information, the server generates an appropriate email subject line.

[0359] The generated subject line is sent to the terminal and displayed in the user interface. The user interface not only presents the generated subject line to the user but also provides an opportunity for the user to modify it. The user can add to or correct the suggested subject line as needed, including phrases that reflect the urgency or importance they wish to convey to the recipient.

[0360] For example, if the email body contains the message "New project starting, deadline is next Wednesday, everyone must participate," the server will use natural language processing to understand this and suggest a subject line to the terminal such as "Important: New project starting - Deadline: next Wednesday (everyone must participate)."

[0361] The system implemented in this manner aims to enable email recipients to instantly understand the importance of an email based on its subject line, thereby improving work efficiency and ensuring the proper sharing of important information.

[0362] The following describes the processing flow.

[0363] Step 1:

[0364] The terminal sends the email body to the server once the user has finished typing it. This happens when the user requests "automatic subject line generation."

[0365] Step 2:

[0366] The server applies a natural language processing algorithm to the received email body to perform analysis. This analysis process tokenizes the text within the email, picks out information such as nouns, verbs, numbers, and dates, and tags them by part of speech.

[0367] Step 3:

[0368] The server extracts key elements necessary for subject line generation from the analyzed data. These include important keywords that indicate the purpose and urgency of the email, such as the project name, deadline, and action items.

[0369] Step 4:

[0370] Based on the extracted key elements, the server automatically generates a subject line according to a template. For example, it applies the elements to the template "Important {Project Name} - Deadline: {Date}" to prepare a completed subject line.

[0371] Step 5:

[0372] The server sends the generated subject to the terminal. The terminal immediately displays this subject in the user interface and presents it to the user.

[0373] Step 6:

[0374] The user reviews the presented subject line. If necessary, the user can modify the subject line through the interface. For example, they can add information or highlight important points.

[0375] Step 7:

[0376] The subject line is finalized and the email is sent once the user is satisfied with it. This allows the recipient to immediately recognize important information and convey the importance of the email.

[0377] (Example 1)

[0378] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0379] Traditionally, titles for electronic communications were often created manually, which was time-consuming and laborious. In particular, efficiently generating titles that accurately reflected urgency or importance was difficult, sometimes leading to decreased information transmission efficiency. Furthermore, if a title did not intuitively convey importance to the recipient, crucial information could be overlooked.

[0380] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0381] In this invention, the server includes means for extracting predetermined information using natural language processing technology for analyzing text, means for using a generative model for automatically generating a title based on the extracted information, means for displaying the generated title on a display device, and means for providing an interface that allows the user to edit the title. This enables the accurate and rapid generation of a text title, and by allowing the user to edit the displayed title, it becomes possible to more effectively convey urgency and importance.

[0382] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0383] In electronic communication, "information" refers to important data and elements extracted from text.

[0384] A "generative model" refers to an algorithm or program that creates a new output from given data, and in this invention, it is used for generating titles.

[0385] "Display device" refers to a device or screen used to visually present the generated title to the user.

[0386] An "interface" refers to a point of contact or means for a user to interact with a system, and in this invention, it specifically refers to one that enables the editing of titles.

[0387] This invention is a system that improves the efficiency of information transmission by automatically generating a title from the content of a document and presenting it to the user. The system mainly consists of a server, a terminal, and a user interface.

[0388] When the server receives emails or other text content, it analyzes the text using natural language processing techniques. The server utilizes Python, employing the natural language processing library spaCy and the deep learning framework TensorFlow to tokenize information within the text and identify key elements. This analysis process extracts information such as content, deadlines, and tasks. For example, if the text reads, "New project starting, deadline next Wednesday, everyone must participate," the analysis will identify these elements.

[0389] The server then uses a generative AI model to automatically generate a title based on the extracted information. This generative model is based on machine learning algorithms and applies natural language generation technology. An example of a prompt message is: "Generate a title based on the following information: Event = New project start, Due date = Next Wednesday, Participant requirements = All participants must attend."

[0390] The generated title is sent to the device and presented to the user via a user interface. The user interface on the device is built using React.js or similar technologies, allowing the user to visually confirm the title. Furthermore, the user can edit the suggested title through this interface. For example, the user can modify the title to something like "Very Important: New Project Starts - Absolute Participation (Deadline: Next Wednesday)" to further emphasize the urgency and importance of the information. This feature allows important information to be conveyed quickly and accurately to recipients.

[0391] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0392] Step 1:

[0393] The server retrieves text data received from the client. The input is the body of an email or message. The server then prepares this text for analysis.

[0394] Step 2:

[0395] The server analyzes the text content using natural language processing techniques. It uses a Python natural language processing library to tokenize the text and perform syntactic analysis. The input is the body of an email, and the output extracts important information elements (e.g., tasks, deadlines, participants, etc.).

[0396] Step 3:

[0397] The server automatically generates a title using a generative AI model based on the extracted information. The input is important information obtained by the server through analysis, and the data is provided to the model as a prompt in the format of "Event = New project start, Due date = Next Wednesday, Participant conditions = All participants must attend". The output is the generated title.

[0398] Step 4:

[0399] The server sends the generated title to the terminal. The input is the title generated by the server, and the output is the data ready to be displayed in the user interface.

[0400] Step 5:

[0401] The device presents the received title information to the user through the user interface. The device uses React.js to visually display the title on the screen. The user confirms the title displayed on the device as visual information.

[0402] Step 6:

[0403] The user reviews the displayed title and edits it if necessary. Through the user interface, the user enters a new title and completes the revision. The system records the edited result, and the final title decided by the user is output.

[0404] (Application Example 1)

[0405] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0406] In email communication, there is a problem where important information regarding payments and other important events is not properly conveyed to recipients, leading to delays in information verification and response. Furthermore, users have the added burden of individually setting email subject lines. Therefore, there is a need for a method to quickly and reliably transmit important information to users, particularly in electronic payment services.

[0407] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0408] In this invention, the server includes means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, means for automatically generating an email subject based on the extracted information elements, and means for generating an email subject related to an event such as an electronic payment using a generation AI model. This makes it possible to automatically and appropriately generate email subjects containing important information.

[0409] "Email" refers to digital messages sent and received via networks such as the internet.

[0410] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language.

[0411] An "information element" is a unit of particularly important data or information extracted from the body of an email.

[0412] A "generative AI model" is a collection of algorithms or programs that utilize artificial intelligence to produce a specific output.

[0413] A "portable information terminal device" is a portable computing device such as a smartphone or tablet.

[0414] This invention relates to a system for automatically generating email subject lines. The system mainly consists of a server, a terminal, and a user interface operated by the user.

[0415] When the server receives the body of an email sent over the internet, it analyzes the text using a natural language processing algorithm. This analysis extracts predetermined information elements such as the payee, amount, and date. The Spacy library of the Python programming language is used for natural language processing.

[0416] Subsequently, the server utilizes a generative AI model, specifically a model such as GPT-2, to generate an appropriate email subject line based on the extracted information elements. The Transformers library is used in this process, and the generated subject line appropriately reflects the electronic payment event.

[0417] The terminal displays the generated email subject line in the user interface. On this interface, the user can edit the subject line and, if necessary, add expressions indicating urgency or importance. This allows the recipient to quickly determine the importance of the email.

[0418] As a concrete example, consider a payment confirmation email for a meal with a friend. If the body of this email is something like, "I paid ¥5,000 for a meal at a restaurant on October 10, 2023," then using the appropriate prompt "Generate subject from email body: Restaurant, ¥5,000, October 10, 2023," the model will automatically generate a subject like "Receipt Restaurant - ¥5,000, Payment Date: October 10, 2023."

[0419] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0420] Step 1:

[0421] The server receives the email body. The received email body is used as input data, and the natural language processing library Spacy is used to begin analysis. This analysis extracts information elements such as the payee, amount, and date from the email body. The extracted information elements are then output.

[0422] Step 2:

[0423] The server generates a prompt based on the extracted information elements. Here, the extracted information elements (for example, "payee," "amount," and "date") are combined to create a prompt in the format "Generate subject from email body: payee, amount, date." This prompt becomes the input for the next AI model.

[0424] Step 3:

[0425] The server inputs a prompt message into the GPT-2 AI model. Based on the input prompt message, the AI ​​model generates the optimal email subject line. In this process, natural language generation technology is used to create a subject line suitable for a payment event, and it is output.

[0426] Step 4:

[0427] The terminal displays the generated email subject on the user interface. Here, the generated subject is displayed on the screen so that the user can check it. This display is the output to the user.

[0428] Step 5:

[0429] The user can modify the subject line as needed. The user can add to or modify the presented subject line through the interface. The modified subject line will be finalized as the final output name.

[0430] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0431] This invention integrates an emotion engine into an email system to generate email subject lines that not only extract important elements from the email body but also take into account the user's emotions. This system consists of a server, a terminal, and a user interface that includes the emotion engine.

[0432] Upon receiving an email, the server analyzes its content using natural language processing algorithms and extracts key elements. Furthermore, it analyzes the user's emotions through an emotion engine. This emotion analysis is performed using general emotion recognition techniques and by analyzing the user's typing speed and patterns. This allows the server to identify whether the user was experiencing emotions such as relief, joy, or anxiety when composing the email.

[0433] Based on the extracted key elements and sentiment data, the server generates a subject line. The sentiment data is used to add emojis and appropriate emotional phrases to the subject line. Furthermore, the subject line is provided with expressions that resonate more emotionally with the recipient.

[0434] The generated subject line is sent to the device and displayed in the user interface. The user can review the displayed subject line and modify it as needed. This allows the user's emotions to be strongly reflected in the email subject line, resulting in more personalized information being conveyed to the recipient.

[0435] For example, if a user is in a cheerful emotional state when sending an email with the theme "New Project Launch," the server will generate a subject line like "🎉 New Project Launch - Deadline: Next Wednesday 😊." By combining this with an emotion engine, email subject lines become more human-like and more clearly convey the intent. The goal is to allow recipients to grasp the mood and nuance of an email just from the subject line, leading to smoother communication.

[0436] The following describes the processing flow.

[0437] Step 1:

[0438] The device simultaneously records the user's typing speed and patterns as they type the email body. This data is sent to the emotion engine and used to estimate the user's emotional state.

[0439] Step 2:

[0440] Once the user has finished entering their email address, the device sends the email body and sentiment data to the server. The server receives this data and begins the next analysis process.

[0441] Step 3:

[0442] The server uses a natural language processing algorithm to analyze the email body and extract important elements such as the project name, deadline, and action items.

[0443] Step 4:

[0444] The emotion engine utilizes typing data and facial recognition technology to primarily analyze the user's emotions during input. The results of this analysis are generated as data specifying emotional states such as relief, joy, anxiety, and excitement.

[0445] Step 5:

[0446] The server integrates extracted key elements and sentiment data to generate email subject lines. Based on the sentiment data, it includes emojis and phrases that convey emotions to the recipient.

[0447] Step 6:

[0448] The generated email subject line is sent to the device and displayed in the user interface. The user can review the generated subject line and, if necessary, make revisions to reflect nuances of emotion or information.

[0449] Step 7:

[0450] After the user confirms the subject line, they send the email through their device. The recipient of the email can immediately understand the content of the email and the sender's emotional state based on this subject line.

[0451] (Example 2)

[0452] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0453] Traditional email systems have struggled to adequately reflect the user's emotions and the importance of the email body in the subject line, making it difficult to effectively communicate the email's intent and urgency to the recipient. Furthermore, the process of reflecting users' emotions in the subject line is often done manually and is time-consuming, creating a need for more efficient email communication.

[0454] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0455] In this invention, the server includes language processing means, means for analyzing the user's emotions, and means for adding specific symbols or phrases that represent emotions to the subject line based on the emotion data. This makes it possible to accurately reflect not only importance and urgency but also the user's emotions in the subject line of an email, resulting in more empathetic and effective email communication.

[0456] "Language processing means" refers to algorithms and technologies used to analyze text data and extract important information.

[0457] "Means of analyzing user emotions" refers to technologies and methods for identifying a user's emotional state by analyzing input data and user behavior patterns.

[0458] "Emotional data" refers to information that quantifies or categorizes a user's emotional state.

[0459] "Specific symbols and words" refer to emojis and specific words used to convey emotions or intentions.

[0460] A "display device" refers to a device or interface used to visually present generated information to a user.

[0461] This invention is a system comprising language processing means, means for analyzing user emotions, and means for adding symbols or phrases representing emotions to the title using emotion data. Specific embodiments are described below.

[0462] When the server receives an email sent by a user, it first analyzes the text using natural language processing (NLP) techniques. This analysis uses a programming language like Python and leverages libraries such as NLTK and SpaCy to extract important information. The extracted information is then used to summarize the email content and as keywords.

[0463] Next, the server performs sentiment analysis techniques to analyze the user's emotions. This typically involves using tools such as TextBlob or SentimentIntensityAnalyzer. The server considers typing speed and patterns during email composition to quantify or categorize the user's emotional state. The resulting emotional data is then classified into emotional categories such as relief, joy, and anxiety.

[0464] Based on this data, the server uses a generative AI model to automatically generate email subject lines. During the generation process, prompts are used to instruct the AI ​​model to create appropriate subject lines. Specific prompts can be used, such as, "Analyze the email body sent by the user and generate a subject line that reflects the user's emotional state. However, the subject line must include emojis and emotionally expressive words." The generated subject lines will include emojis and emotionally expressive terms, aiming to deliver an emotionally resonant message to the recipient.

[0465] Finally, the server sends the generated title to the terminal, which visually displays this information on its user interface. The user can review this title and make corrections as needed, resulting in smoother and more personalized communication.

[0466] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0467] Step 1:

[0468] The server receives the email body sent by the user. Using this received text data as input, it first performs analysis using natural language processing (NLP) techniques. Using NLTK and SpaCy libraries in Python, the server tokenizes the email body and assigns part-of-speech tags to extract important words such as nouns and verbs. This process outputs a summary and key information from the email content.

[0469] Step 2:

[0470] The server analyzes the user's emotions based on the key keywords extracted in Step 1. This process considers the user's typing data and input patterns, and generates emotional data using tools such as TextBlob and SentimentIntensityAnalyzer. The speed and rhythm of key input during email composition are used to determine the emotional state, and the output obtained from this data is categorized into emotional categories such as "joy," "relief," and "anxiety."

[0471] Step 3:

[0472] The server uses a generative AI model to synthesize the sentiment data obtained in Step 2 with the key keywords from Step 1 to generate an email subject line. In this process, a prompt is provided to the generative AI model, instructing it to "create a subject line that resonates emotionally with the recipient, based on the extracted key elements and sentiment data." This generates a subject line that includes expressions and emojis reflecting emotions, and the result is obtained as output.

[0473] Step 4:

[0474] The server sends the generated subject line to the user's terminal, which then displays it on the user interface. The subject line is displayed on the terminal and visually adjusted, allowing the user to intuitively understand the content. The user reviews the outputted subject line, makes any necessary corrections, and sends the email after final confirmation.

[0475] (Application Example 2)

[0476] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0477] Traditional email systems primarily generate subject lines based on the importance of the text, making it difficult to reflect user emotions. As a result, the content and nuances of emails can be unclear to recipients. Furthermore, smooth communication with recipients requires flexible responses that adapt to user emotions and circumstances, but current systems lack the means to achieve this. Technology is needed to solve this problem.

[0478] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0479] In this invention, the server includes means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email; means for automatically generating an email subject line that takes emotional elements into consideration based on the extracted important elements and sentiment analysis; and means for presenting the generated email subject line to a user interface and providing the user with an interactive response. This makes it possible to generate a subject line that reflects the user's emotions, and to convey clearer and more emotional nuances to the recipient.

[0480] A "natural language processing algorithm" is a technology that enables computers to understand human language and extract meaning from text.

[0481] "Sentiment analysis" is a technology used to estimate a person's emotions from text and user data.

[0482] "Emotional elements" refer to information or expressions based on emotions that convey a specific emotional nuance to the recipient or user.

[0483] A "user interface" is a mechanism that allows a user to interact with a system and input or output information.

[0484] "Interactive responses" are responses generated based on dialogue with the user, primarily intended to facilitate smoother interaction between computers and humans.

[0485] This embodiment of the invention is a system for realizing natural, emotion-reflecting dialogue when a user interacts with a household robot. The specific implementation method of this system is described below.

[0486] The server first receives the user's voice and converts it into text data using speech recognition software. Technologies such as Google Cloud Speech-to-Text can be used for this speech recognition. Then, a natural language processing algorithm is executed to extract important elements from the input text. TensorFlow or PyTorch are preferred for this process.

[0487] Next, services such as Watson Natural Language Understanding are used to perform sentiment analysis. This allows the system to determine the user's emotional state. Based on the extracted key elements and sentiment data, the server utilizes a generative AI model (e.g., GPT-3) to generate an appropriate response for the user.

[0488] The generated responses are automatically presented to the user interface. For example, Google Cloud Text-to-Speech can be used for speech synthesis, allowing the robot to respond in voice. This process enables natural interaction with the user.

[0489] As a concrete example, consider a scenario where a user says to a home robot, "Today was a busy day." If the emotion analysis engine determines that the user is stressed, the server can generate a response such as, "That sounds tough. Is there anything I can do to help you relax?" This response can include humor or suggestions that take the user's emotions into consideration.

[0490] Examples of prompt messages include the following:

[0491] "User said: 'Today was a busy day.' Information: Stress, emotional burden. Generate an appropriate robot response."

[0492] This makes it possible to create more personalized and emotionally sensitive interactions with home robots.

[0493] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0494] Step 1:

[0495] The server receives voice input from the user. This voice input is transmitted to a home robot via a microphone, and the server converts it into text data using speech recognition software (e.g., Google Cloud Speech-to-Text). The input is the user's voice data, and the output is the converted text data. This conversion prepares the data for subsequent text analysis.

[0496] Step 2:

[0497] The server passes the text data to a natural language processing algorithm to extract important elements. Here, TensorFlow or PyTorch is used to analyze sentence structure and extract necessary keywords and the main point of the sentence. The input is the text data generated in step 1, and the output is a dataset containing the important elements. This allows for the identification of the intent of the utterance.

[0498] Step 3:

[0499] The server inputs text data containing important elements into an emotion analysis engine (e.g., Watson Natural Language Understanding) to analyze the user's emotions. This analysis examines emotional expressions within the text and estimates the user's emotional state. The input is the dataset obtained in step 2, and the output is data indicating the user's emotional state. This allows for an understanding of the user's emotional condition.

[0500] Step 4:

[0501] The server provides prompt sentences to a generative AI model (e.g., GPT-3) based on extracted key elements and sentiment data. The generative AI model uses this data to generate an appropriate response to the user. The input is sentiment data and key elements, and the output is an interactive response to the user. This response is required to be empathetic to the user's emotions.

[0502] Step 5:

[0503] The server sends the generated response to the home robot's interface, and uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to output it as speech. This allows the user to hear the voice response from the robot. The input is the response sentence generated in step 4, and the output is the voice response message. This process enables natural interaction with the robot.

[0504] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0505] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0506] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0507] [Fourth Embodiment]

[0508] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0509] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0510] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0511] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0512] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0513] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0514] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0515] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0516] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0517] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0518] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0519] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0520] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0521] This invention relates to a system for automatically generating appropriate subject lines from email bodies. This system mainly consists of a server, a terminal, and a user interface, and utilizes natural language processing technology.

[0522] When the server receives an email sent by a user, it uses a natural language processing algorithm to analyze the email body. During the analysis, key elements of the email, such as content, deadline, and task, are extracted. Based on this information, the server generates an appropriate email subject line.

[0523] The generated subject line is sent to the terminal and displayed in the user interface. The user interface not only presents the generated subject line to the user but also provides an opportunity for the user to modify it. The user can add to or correct the suggested subject line as needed, including phrases that reflect the urgency or importance they wish to convey to the recipient.

[0524] For example, if the email body contains the message "New project starting, deadline is next Wednesday, everyone must participate," the server will use natural language processing to understand this and suggest a subject line to the terminal such as "Important: New project starting - Deadline: next Wednesday (everyone must participate)."

[0525] The system implemented in this manner aims to enable email recipients to instantly understand the importance of an email based on its subject line, thereby improving work efficiency and ensuring the proper sharing of important information.

[0526] The following describes the processing flow.

[0527] Step 1:

[0528] The terminal sends the email body to the server once the user has finished typing it. This happens when the user requests "automatic subject line generation."

[0529] Step 2:

[0530] The server applies a natural language processing algorithm to the received email body to perform analysis. This analysis process tokenizes the text within the email, picks out information such as nouns, verbs, numbers, and dates, and tags them by part of speech.

[0531] Step 3:

[0532] The server extracts key elements necessary for subject line generation from the analyzed data. These include important keywords that indicate the purpose and urgency of the email, such as the project name, deadline, and action items.

[0533] Step 4:

[0534] Based on the extracted key elements, the server automatically generates a subject line according to a template. For example, it applies the elements to the template "Important {Project Name} - Deadline: {Date}" to prepare a completed subject line.

[0535] Step 5:

[0536] The server sends the generated subject to the terminal. The terminal immediately displays this subject in the user interface and presents it to the user.

[0537] Step 6:

[0538] The user reviews the presented subject line. If necessary, the user can modify the subject line through the interface. For example, they can add information or highlight important points.

[0539] Step 7:

[0540] The subject line is finalized and the email is sent once the user is satisfied with it. This allows the recipient to immediately recognize important information and convey the importance of the email.

[0541] (Example 1)

[0542] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0543] Traditionally, titles for electronic communications were often created manually, which was time-consuming and laborious. In particular, efficiently generating titles that accurately reflected urgency or importance was difficult, sometimes leading to decreased information transmission efficiency. Furthermore, if a title did not intuitively convey importance to the recipient, crucial information could be overlooked.

[0544] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0545] In this invention, the server includes means for extracting predetermined information using natural language processing technology for analyzing text, means for using a generative model for automatically generating a title based on the extracted information, means for displaying the generated title on a display device, and means for providing an interface that allows the user to edit the title. This enables the accurate and rapid generation of a text title, and by allowing the user to edit the displayed title, it becomes possible to more effectively convey urgency and importance.

[0546] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0547] In electronic communication, "information" refers to important data and elements extracted from text.

[0548] A "generative model" refers to an algorithm or program that creates a new output from given data, and in this invention, it is used for generating titles.

[0549] "Display device" refers to a device or screen used to visually present the generated title to the user.

[0550] An "interface" refers to a point of contact or means for a user to interact with a system, and in this invention, it specifically refers to one that enables the editing of titles.

[0551] This invention is a system that improves the efficiency of information transmission by automatically generating a title from the content of a document and presenting it to the user. The system mainly consists of a server, a terminal, and a user interface.

[0552] When the server receives emails or other text content, it analyzes the text using natural language processing techniques. The server utilizes Python, employing the natural language processing library spaCy and the deep learning framework TensorFlow to tokenize information within the text and identify key elements. This analysis process extracts information such as content, deadlines, and tasks. For example, if the text reads, "New project starting, deadline next Wednesday, everyone must participate," the analysis will identify these elements.

[0553] The server then uses a generative AI model to automatically generate a title based on the extracted information. This generative model is based on machine learning algorithms and applies natural language generation technology. An example of a prompt message is: "Generate a title based on the following information: Event = New project start, Due date = Next Wednesday, Participant requirements = All participants must attend."

[0554] The generated title is sent to the device and presented to the user via a user interface. The user interface on the device is built using React.js or similar technologies, allowing the user to visually confirm the title. Furthermore, the user can edit the suggested title through this interface. For example, the user can modify the title to something like "Very Important: New Project Starts - Absolute Participation (Deadline: Next Wednesday)" to further emphasize the urgency and importance of the information. This feature allows important information to be conveyed quickly and accurately to recipients.

[0555] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0556] Step 1:

[0557] The server retrieves text data received from the client. The input is the body of an email or message. The server then prepares this text for analysis.

[0558] Step 2:

[0559] The server analyzes the text content using natural language processing techniques. It uses a Python natural language processing library to tokenize the text and perform syntactic analysis. The input is the body of an email, and the output extracts important information elements (e.g., tasks, deadlines, participants, etc.).

[0560] Step 3:

[0561] The server automatically generates a title using a generative AI model based on the extracted information. The input is important information obtained by the server through analysis, and the data is provided to the model as a prompt in the format of "Event = New project start, Due date = Next Wednesday, Participant conditions = All participants must attend". The output is the generated title.

[0562] Step 4:

[0563] The server sends the generated title to the terminal. The input is the title generated by the server, and the output is the data ready to be displayed in the user interface.

[0564] Step 5:

[0565] The device presents the received title information to the user through the user interface. The device uses React.js to visually display the title on the screen. The user confirms the title displayed on the device as visual information.

[0566] Step 6:

[0567] The user reviews the displayed title and edits it if necessary. Through the user interface, the user enters a new title and completes the revision. The system records the edited result, and the final title decided by the user is output.

[0568] (Application Example 1)

[0569] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0570] In email communication, there is a problem where important information regarding payments and other important events is not properly conveyed to recipients, leading to delays in information verification and response. Furthermore, users have the added burden of individually setting email subject lines. Therefore, there is a need for a method to quickly and reliably transmit important information to users, particularly in electronic payment services.

[0571] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0572] In this invention, the server includes means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, means for automatically generating an email subject based on the extracted information elements, and means for generating an email subject related to an event such as an electronic payment using a generation AI model. This makes it possible to automatically and appropriately generate email subjects containing important information.

[0573] "Email" refers to digital messages sent and received via networks such as the internet.

[0574] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language.

[0575] An "information element" is a unit of particularly important data or information extracted from the body of an email.

[0576] A "generative AI model" is a collection of algorithms or programs that utilize artificial intelligence to produce a specific output.

[0577] A "portable information terminal device" is a portable computing device such as a smartphone or tablet.

[0578] This invention relates to a system for automatically generating email subject lines. The system mainly consists of a server, a terminal, and a user interface operated by the user.

[0579] When the server receives the body of an email sent over the internet, it analyzes the text using a natural language processing algorithm. This analysis extracts predetermined information elements such as the payee, amount, and date. The Spacy library of the Python programming language is used for natural language processing.

[0580] Subsequently, the server utilizes a generative AI model, specifically a model such as GPT-2, to generate an appropriate email subject line based on the extracted information elements. The Transformers library is used in this process, and the generated subject line appropriately reflects the electronic payment event.

[0581] The terminal displays the generated email subject line in the user interface. On this interface, the user can edit the subject line and, if necessary, add expressions indicating urgency or importance. This allows the recipient to quickly determine the importance of the email.

[0582] As a concrete example, consider a payment confirmation email for a meal with a friend. If the body of this email is something like, "I paid ¥5,000 for a meal at a restaurant on October 10, 2023," then using the appropriate prompt "Generate subject from email body: Restaurant, ¥5,000, October 10, 2023," the model will automatically generate a subject like "Receipt Restaurant - ¥5,000, Payment Date: October 10, 2023."

[0583] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0584] Step 1:

[0585] The server receives the email body. The received email body is used as input data, and the natural language processing library Spacy is used to begin analysis. This analysis extracts information elements such as the payee, amount, and date from the email body. The extracted information elements are then output.

[0586] Step 2:

[0587] The server generates a prompt based on the extracted information elements. Here, the extracted information elements (for example, "payee," "amount," and "date") are combined to create a prompt in the format "Generate subject from email body: payee, amount, date." This prompt becomes the input for the next AI model.

[0588] Step 3:

[0589] The server inputs a prompt message into the GPT-2 AI model. Based on the input prompt message, the AI ​​model generates the optimal email subject line. In this process, natural language generation technology is used to create a subject line suitable for a payment event, and it is output.

[0590] Step 4:

[0591] The terminal displays the generated email subject on the user interface. Here, the generated subject is displayed on the screen so that the user can check it. This display is the output to the user.

[0592] Step 5:

[0593] The user can modify the subject line as needed. The user can add to or modify the presented subject line through the interface. The modified subject line will be finalized as the final output name.

[0594] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0595] This invention integrates an emotion engine into an email system to generate email subject lines that not only extract important elements from the email body but also take into account the user's emotions. This system consists of a server, a terminal, and a user interface that includes the emotion engine.

[0596] Upon receiving an email, the server analyzes its content using natural language processing algorithms and extracts key elements. Furthermore, it analyzes the user's emotions through an emotion engine. This emotion analysis is performed using general emotion recognition techniques and by analyzing the user's typing speed and patterns. This allows the server to identify whether the user was experiencing emotions such as relief, joy, or anxiety when composing the email.

[0597] Based on the extracted key elements and sentiment data, the server generates a subject line. The sentiment data is used to add emojis and appropriate emotional phrases to the subject line. Furthermore, the subject line is provided with expressions that resonate more emotionally with the recipient.

[0598] The generated subject line is sent to the device and displayed in the user interface. The user can review the displayed subject line and modify it as needed. This allows the user's emotions to be strongly reflected in the email subject line, resulting in more personalized information being conveyed to the recipient.

[0599] For example, if a user is in a cheerful emotional state when sending an email with the theme "New Project Launch," the server will generate a subject line like "🎉 New Project Launch - Deadline: Next Wednesday 😊." By combining this with an emotion engine, email subject lines become more human-like and more clearly convey the intent. The goal is to allow recipients to grasp the mood and nuance of an email just from the subject line, leading to smoother communication.

[0600] The following describes the processing flow.

[0601] Step 1:

[0602] The device simultaneously records the user's typing speed and patterns as they type the email body. This data is sent to the emotion engine and used to estimate the user's emotional state.

[0603] Step 2:

[0604] Once the user has finished entering their email address, the device sends the email body and sentiment data to the server. The server receives this data and begins the next analysis process.

[0605] Step 3:

[0606] The server uses a natural language processing algorithm to analyze the email body and extract important elements such as the project name, deadline, and action items.

[0607] Step 4:

[0608] The emotion engine utilizes typing data and facial recognition technology to primarily analyze the user's emotions during input. The results of this analysis are generated as data specifying emotional states such as relief, joy, anxiety, and excitement.

[0609] Step 5:

[0610] The server integrates extracted key elements and sentiment data to generate email subject lines. Based on the sentiment data, it includes emojis and phrases that convey emotions to the recipient.

[0611] Step 6:

[0612] The generated email subject line is sent to the device and displayed in the user interface. The user can review the generated subject line and, if necessary, make revisions to reflect nuances of emotion or information.

[0613] Step 7:

[0614] After the user confirms the subject line, they send the email through their device. The recipient of the email can immediately understand the content of the email and the sender's emotional state based on this subject line.

[0615] (Example 2)

[0616] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0617] Traditional email systems have struggled to adequately reflect the user's emotions and the importance of the email body in the subject line, making it difficult to effectively communicate the email's intent and urgency to the recipient. Furthermore, the process of reflecting users' emotions in the subject line is often done manually and is time-consuming, creating a need for more efficient email communication.

[0618] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0619] In this invention, the server includes language processing means, means for analyzing the user's emotions, and means for adding specific symbols or phrases that represent emotions to the subject line based on the emotion data. This makes it possible to accurately reflect not only importance and urgency but also the user's emotions in the subject line of an email, resulting in more empathetic and effective email communication.

[0620] "Language processing means" refers to algorithms and technologies used to analyze text data and extract important information.

[0621] "Means of analyzing user emotions" refers to technologies and methods for identifying a user's emotional state by analyzing input data and user behavior patterns.

[0622] "Emotional data" refers to information that quantifies or categorizes a user's emotional state.

[0623] "Specific symbols and words" refer to emojis and specific words used to convey emotions or intentions.

[0624] A "display device" refers to a device or interface used to visually present generated information to a user.

[0625] This invention is a system comprising language processing means, means for analyzing user emotions, and means for adding symbols or phrases representing emotions to the title using emotion data. Specific embodiments are described below.

[0626] When the server receives an email sent by a user, it first analyzes the text using natural language processing (NLP) techniques. This analysis uses a programming language like Python and leverages libraries such as NLTK and SpaCy to extract important information. The extracted information is then used to summarize the email content and as keywords.

[0627] Next, the server performs sentiment analysis techniques to analyze the user's emotions. This typically involves using tools such as TextBlob or SentimentIntensityAnalyzer. The server considers typing speed and patterns during email composition to quantify or categorize the user's emotional state. The resulting emotional data is then classified into emotional categories such as relief, joy, and anxiety.

[0628] Based on this data, the server uses a generative AI model to automatically generate email subject lines. During the generation process, prompts are used to instruct the AI ​​model to create appropriate subject lines. Specific prompts can be used, such as, "Analyze the email body sent by the user and generate a subject line that reflects the user's emotional state. However, the subject line must include emojis and emotionally expressive words." The generated subject lines will include emojis and emotionally expressive terms, aiming to deliver an emotionally resonant message to the recipient.

[0629] Finally, the server sends the generated title to the terminal, which visually displays this information on its user interface. The user can review this title and make corrections as needed, resulting in smoother and more personalized communication.

[0630] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0631] Step 1:

[0632] The server receives the email body sent by the user. Using this received text data as input, it first performs analysis using natural language processing (NLP) techniques. Using NLTK and SpaCy libraries in Python, the server tokenizes the email body and assigns part-of-speech tags to extract important words such as nouns and verbs. This process outputs a summary and key information from the email content.

[0633] Step 2:

[0634] The server analyzes the user's emotions based on the key keywords extracted in Step 1. This process considers the user's typing data and input patterns, and generates emotional data using tools such as TextBlob and SentimentIntensityAnalyzer. The speed and rhythm of key input during email composition are used to determine the emotional state, and the output obtained from this data is categorized into emotional categories such as "joy," "relief," and "anxiety."

[0635] Step 3:

[0636] The server uses a generative AI model to synthesize the sentiment data obtained in Step 2 with the key keywords from Step 1 to generate an email subject line. In this process, a prompt is provided to the generative AI model, instructing it to "create a subject line that resonates emotionally with the recipient, based on the extracted key elements and sentiment data." This generates a subject line that includes expressions and emojis reflecting emotions, and the result is obtained as output.

[0637] Step 4:

[0638] The server sends the generated subject line to the user's terminal, which then displays it on the user interface. The subject line is displayed on the terminal and visually adjusted, allowing the user to intuitively understand the content. The user reviews the outputted subject line, makes any necessary corrections, and sends the email after final confirmation.

[0639] (Application Example 2)

[0640] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0641] Traditional email systems primarily generate subject lines based on the importance of the text, making it difficult to reflect user emotions. As a result, the content and nuances of emails can be unclear to recipients. Furthermore, smooth communication with recipients requires flexible responses that adapt to user emotions and circumstances, but current systems lack the means to achieve this. Technology is needed to solve this problem.

[0642] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0643] In this invention, the server includes means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email; means for automatically generating an email subject line that takes emotional elements into consideration based on the extracted important elements and sentiment analysis; and means for presenting the generated email subject line to a user interface and providing the user with an interactive response. This makes it possible to generate a subject line that reflects the user's emotions, and to convey clearer and more emotional nuances to the recipient.

[0644] A "natural language processing algorithm" is a technology that enables computers to understand human language and extract meaning from text.

[0645] "Sentiment analysis" is a technology used to estimate a person's emotions from text and user data.

[0646] "Emotional elements" refer to information or expressions based on emotions that convey a specific emotional nuance to the recipient or user.

[0647] A "user interface" is a mechanism that allows a user to interact with a system and input or output information.

[0648] "Interactive responses" are responses generated based on dialogue with the user, primarily intended to facilitate smoother interaction between computers and humans.

[0649] This embodiment of the invention is a system for realizing natural, emotion-reflecting dialogue when a user interacts with a household robot. The specific implementation method of this system is described below.

[0650] The server first receives the user's voice and converts it into text data using speech recognition software. Technologies such as Google Cloud Speech-to-Text can be used for this speech recognition. Then, a natural language processing algorithm is executed to extract important elements from the input text. TensorFlow or PyTorch are preferred for this process.

[0651] Next, services such as Watson Natural Language Understanding are used to perform sentiment analysis. This allows the system to determine the user's emotional state. Based on the extracted key elements and sentiment data, the server utilizes a generative AI model (e.g., GPT-3) to generate an appropriate response for the user.

[0652] The generated responses are automatically presented to the user interface. For example, Google Cloud Text-to-Speech can be used for speech synthesis, allowing the robot to respond in voice. This process enables natural interaction with the user.

[0653] As a concrete example, consider a scenario where a user says to a home robot, "Today was a busy day." If the emotion analysis engine determines that the user is stressed, the server can generate a response such as, "That sounds tough. Is there anything I can do to help you relax?" This response can include humor or suggestions that take the user's emotions into consideration.

[0654] Examples of prompt messages include the following:

[0655] "User said: 'Today was a busy day.' Information: Stress, emotional burden. Generate an appropriate robot response."

[0656] This makes it possible to create more personalized and emotionally sensitive interactions with home robots.

[0657] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0658] Step 1:

[0659] The server receives voice input from the user. This voice input is transmitted to a home robot via a microphone, and the server converts it into text data using speech recognition software (e.g., Google Cloud Speech-to-Text). The input is the user's voice data, and the output is the converted text data. This conversion prepares the data for subsequent text analysis.

[0660] Step 2:

[0661] The server passes the text data to a natural language processing algorithm to extract important elements. Here, TensorFlow or PyTorch is used to analyze sentence structure and extract necessary keywords and the main point of the sentence. The input is the text data generated in step 1, and the output is a dataset containing the important elements. This allows for the identification of the intent of the utterance.

[0662] Step 3:

[0663] The server inputs text data containing important elements into an emotion analysis engine (e.g., Watson Natural Language Understanding) to analyze the user's emotions. This analysis examines emotional expressions within the text and estimates the user's emotional state. The input is the dataset obtained in step 2, and the output is data indicating the user's emotional state. This allows for an understanding of the user's emotional condition.

[0664] Step 4:

[0665] The server provides prompt sentences to a generative AI model (e.g., GPT-3) based on extracted key elements and sentiment data. The generative AI model uses this data to generate an appropriate response to the user. The input is sentiment data and key elements, and the output is an interactive response to the user. This response is required to be empathetic to the user's emotions.

[0666] Step 5:

[0667] The server sends the generated response to the home robot's interface, and uses speech synthesis software (e.g., Google Cloud Text-to-Speech) to output it as speech. This allows the user to hear the voice response from the robot. The input is the response sentence generated in step 4, and the output is the voice response message. This process enables natural interaction with the robot.

[0668] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0669] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0670] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0671] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0672] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0673] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0674] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0675] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0676] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0677] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0678] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0679] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0680] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0681] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0682] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0683] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0684] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0685] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0686] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0687] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0688] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0689] The following is further disclosed regarding the embodiments described above.

[0690] (Claim 1)

[0691] A means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email,

[0692] A means for automatically generating email subject lines based on extracted key elements,

[0693] A means of displaying the generated email subject line in the user interface,

[0694] A system that includes this.

[0695] (Claim 2)

[0696] The system according to claim 1, including an interface for accepting user modifications to the email subject line.

[0697] (Claim 3)

[0698] The system according to claim 1, further comprising means for incorporating phrases indicating urgency or importance in generating email subject lines.

[0699] "Example 1"

[0700] (Claim 1)

[0701] A means for extracting predetermined information using natural language processing techniques for analyzing text,

[0702] A means of using a generative model to automatically generate titles based on extracted information,

[0703] A means for displaying the generated title on a display device,

[0704] A means of providing an interface that allows users to edit the title,

[0705] A system that includes this.

[0706] (Claim 2)

[0707] The system according to claim 1, configured to include an expression indicating urgency or importance in the title, depending on the results of the text analysis.

[0708] (Claim 3)

[0709] The system according to claim 1, further comprising means for using information technology to record user-edited titles and utilize them for subsequent improvement of the generative model.

[0710] "Application Example 1"

[0711] (Claim 1)

[0712] A means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email,

[0713] A means for automatically generating an email subject line based on extracted information elements,

[0714] A method for generating email subject lines related to events such as electronic payments using a generative AI model,

[0715] A means of displaying the generated email subject on the user interface of a mobile information terminal device,

[0716] A system that includes this.

[0717] (Claim 2)

[0718] The system according to claim 1, including an interface for accepting user modifications to the email subject line.

[0719] (Claim 3)

[0720] The system according to claim 1, further comprising means for incorporating expressions indicating urgency or importance in generating email subject lines.

[0721] "Example 2 of combining an emotion engine"

[0722] (Claim 1)

[0723] A language processing tool for analyzing the content of emails,

[0724] A means for automatically creating an email subject line based on extracted elements,

[0725] A means of analyzing user emotions,

[0726] A method for adding specific symbols or words that represent emotions to the title based on emotional data,

[0727] A means for displaying the subject of the generated email on a display device,

[0728] A system that includes this.

[0729] (Claim 2)

[0730] The system according to claim 1, including an interface for accepting changes to email subject lines by the user.

[0731] (Claim 3)

[0732] The system according to claim 1, further comprising means for using sentiment data to incorporate expressions that resonate emotionally with the recipient when creating an email subject line.

[0733] "Application example 2 when combining with an emotional engine"

[0734] (Claim 1)

[0735] A means for extracting predetermined important elements using a natural language processing algorithm for analyzing the body of an email,

[0736] A means for automatically generating email subject lines that take emotional elements into consideration, based on extracted key elements and sentiment analysis,

[0737] A means of presenting the generated email subject line in the user interface and providing the user with an interactive response,

[0738] A system that includes this.

[0739] (Claim 2)

[0740] The system according to claim 1, comprising an interface for accepting user modifications to email subject lines and means for generating personalized conversations based on sentiment data.

[0741] (Claim 3)

[0742] The system according to claim 1, further comprising means for generating email subject lines, incorporating phrases indicating urgency or importance, and adding expressions that include emotional nuances. [Explanation of symbols]

[0743] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for extracting predetermined information elements using a natural language processing algorithm for analyzing the body of an email, A means for automatically generating an email subject line based on extracted information elements, A method for generating email subject lines related to events such as electronic payments using a generative AI model, A means of displaying the generated email subject on the user interface of a mobile information terminal device, A system that includes this.

2. The system according to claim 1, including an interface for accepting user modifications to the email subject line.

3. The system according to claim 1, further comprising means for incorporating expressions indicating urgency or importance in the generation of email subject lines.

Images