System

Abstract

A system is provided.SOLUTION: The system includes a means for registering a picture or a character that a user likes, a means for monitoring conversation contents and analyzing the contents, a means for generating an image by using the picture or the character registered by the user based on the conversation contents, and a means for suggesting the generated image to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern communication tools, users want to enjoy more individual and expressive conversations by using stamps that suit their preferences. However, the types of stamps available are limited, and they cannot satisfy the types of characters and designs that users truly desire. This creates a problem where users cannot use stamps that suit their preferences. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a system that allows users to register their favorite pictures and characters, and generates and suggests stamps that match the content of the conversation. This system includes a means for users to register their favorite pictures and characters, a means for monitoring and analyzing the content of the conversation, a means for generating images using the user's registered pictures and characters based on the content of the conversation, and a means for suggesting the generated images to the user. This means allows users to use stamps that suit the flow of the conversation and suit their preferences in real time.

[0006] "User" refers to any individual or entity that uses the System.

[0007] "Pictures" refers to visual designs or artwork, images with a particular theme or style.

[0008] A "character" refers to a specific person, animal, object, etc. that a user likes, and is an image or icon that the user uses to express their emotions.

[0009] "Means for registration" refers to a function that allows users to save and register their favorite pictures and characters in the system.

[0010] "Means for monitoring" refers to a function for monitoring and recording the content of user conversations in real time.

[0011] "Means for analysis" refers to the function for analyzing collected conversation content and understanding the user's intentions and emotions.

[0012] "Means for generating images" refers to the function of creating new images based on the images and characters registered by the user and the results of conversation analysis.

[0013] The "means for suggesting" refers to a function for suggesting and displaying the generated image to the user.

[0014] A "system" refers to a series of processes in which multiple means operate in an integrated manner to provide services to users. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. The program processing is explained in detail below.

[0037] First, the user performs initial setup to use the system. The user registers their favorite designs and characters in the system, and this information is sent from the terminal to the server. The server stores the received information in a database and holds data according to the preferences of each individual user.

[0038] Once a user starts chatting, the device periodically monitors the conversation and sends the data to the server, which then analyzes the received conversation using a natural language processing engine to determine whether a stamp matching the conversation is needed.

[0039] If a stamp is deemed necessary, the server uses image generation AI to generate a new stamp based on the analysis results and the user's registered favorite designs and characters. The generated stamp is then sent from the server to the device.

[0040] The device displays the received stamps on the user interface and suggests them to the user. By sending the suggested stamps, the user can communicate more uniquely and expressively.

[0041] Specific examples

[0042] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp using the "cat" registered by the user to express a "tired look." The generated stamp is then suggested to the user, who then sends it in the chat.

[0043] This allows users to use a variety of stamps featuring their favorite characters and send emotionally rich messages to others.The purpose of the present invention is to improve the user's communication experience by generating stamps based on the user's preferences and the context of the conversation.

[0044] The processing flow will be explained below.

[0045] Step 1:

[0046] Users use the system to register their favorite designs and characters, and then input the names of each character and design through the application interface.

[0047] Step 2:

[0048] The device acquires the user's input information and sends it to the server, which includes the user ID and registered image and character information.

[0049] Step 3:

[0050] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0051] Step 4:

[0052] A user initiates a chat. The user chats by sending regular messages.

[0053] Step 5:

[0054] The device will periodically monitor the chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0055] Step 6:

[0056] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0057] Step 7:

[0058] The server uses a natural language processing engine (NLP) to analyze the chat content it receives, and determines whether the message needs an appropriate stamp based on its intent and sentiment.

[0059] Step 8:

[0060] If the server determines that a stamp needs to be generated based on the results of the NLP analysis, it obtains information about the design and character registered by the user.

[0061] Step 9:

[0062] The server activates the image generation AI and generates a stamp image based on the user's preferences and the results of NLP analysis. For example, it generates a "cat" stamp with the theme of "tired."

[0063] Step 10:

[0064] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0065] Step 11:

[0066] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it if necessary.

[0067] Example 1

[0068] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0069] In conventional chat systems, users had to manually select stamps, making it difficult to quickly use appropriate stamps that fit the flow of the conversation. Furthermore, because users had to choose from existing stamps, it was difficult to easily find stamps that matched individual users' preferences and the context of the conversation. This limited the user's communication experience and made it difficult to have expressive conversations.

[0070] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0071] In this invention, the server includes means for users to register their favorite images and characters, means for monitoring and analyzing conversation content, means for generating stamps using the images and characters registered by the user based on the conversation content, and means for suggesting the generated stamps to the user. This enables users to quickly use unique and appropriate stamps in line with the flow of conversation, enabling more expressive communication.

[0072] A "user" is an entity that uses the system to register their favorite images and characters and chat.

[0073] "Images and characters" are visual representations such as favorite pictures and characters that users register in the system.

[0074] The "means for monitoring the content of conversations" is a function for continuously observing the content of chat conversations and extracting important information.

[0075] "Means for analyzing conversation content" refers to a function that uses natural language processing technology to analyze the conversation content of a user and understand their emotions and intentions.

[0076] A "stamp" is a type of image generated by the system to visually express the user's emotions and intentions.

[0077] "Means of generation" is a function that creates stamps suitable for conversation based on registered images and characters and analysis results.

[0078] The "means of suggesting" is a function of displaying the generated stamp to the user and encouraging them to use it.

[0079] A "server" is a central computer that processes and manages data for the entire system.

[0080] A "natural language processing engine" is a software technology that analyzes text data and understands its meaning and emotions.

[0081] The "generative AI model" is an artificial intelligence model that generates new stamps based on user registration information and conversation analysis results.

[0082] The present invention is a system that automatically generates and suggests stamps that match the flow of conversation using images or characters that users like. The specific processing of this system is as follows.

[0083] First, the user performs initial setup to use the system. The user registers their favorite images and characters in the system. This information is sent from the user's device to the server. The server stores the received information in a database and holds data according to the preferences of each individual user. This data includes image data and keywords uploaded by the user.

[0084] Next, when a user starts chatting, the device periodically monitors the conversation content and sends the data to the server. The server then analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and determines whether a stamp appropriate for the conversation is needed. The natural language processing engine extracts emotions and intentions from the user's conversation content and performs analysis based on this.

[0085] If a stamp is deemed necessary, the server uses the analysis results and the user's favorite images and characters to generate a new stamp using an image generation AI (e.g., a standard image generation algorithm). The image generation AI is provided with a prompt that reflects the user's preferences and the context of the conversation.

[0086] As a concrete example, consider a case where a user has registered "cats" and "dogs" as their favorite characters in the system. When the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp that expresses "tiredness" using the "cat" registered by the user. The generated stamp is then suggested to the user, who then sends the stamp in the chat.

[0087] An example prompt is:

[0088] "The user's favorite character is a cat. The conversation is, 'I'm tired today.' Please generate an image of a cat that expresses this conversation."

[0089] This system allows users to use a variety of stamps featuring their favorite characters and send emotive messages to others. As described above, the purpose of this invention is to enrich users' communication experiences by automatically generating and suggesting stamps that match the user's preferences and the context of the conversation.

[0090] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0091] Step 1:

[0092] Users register their favorite images and characters.

[0093] Specific behavior:

[0094] The user opens the system's settings screen and registers their favorite image or character by entering the image file and keywords.

[0095] Input: Image files and keywords entered by the user

[0096] Output: Registration information is generated

[0097] Step 2:

[0098] The terminal sends the registration information to the server.

[0099] Specific behavior:

[0100] The terminal (for example, a smartphone or PC) sends the data entered by the user to the server using the HTTPS protocol.

[0101] Input: Image files and keywords registered by the user

[0102] Output: Registration information sent to the server

[0103] Step 3:

[0104] The server stores the information in a database.

[0105] Specific behavior:

[0106] The server analyzes the received data, organizes it by user, and stores it in a database (e.g., MySQL).

[0107] Input: Registration information received by the server

[0108] Output: User registration information saved in the database

[0109] Step 4:

[0110] The device monitors the conversation.

[0111] Specific behavior:

[0112] When a user launches a chat app and starts a conversation, the device monitors messages entered every few seconds.

[0113] Input: The chat message the user types

[0114] Output: Monitored conversation

[0115] Step 5:

[0116] The terminal transmits the conversation data to the server.

[0117] Specific behavior:

[0118] The device sends text data of the conversation to the server at regular intervals, and this communication also uses the HTTPS protocol.

[0119] Input: Monitored conversation

[0120] Output: Conversation data sent to the server

[0121] Step 6:

[0122] The server analyzes the conversation using a natural language processing engine.

[0123] Specific behavior:

[0124] The server analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and extracts emotions and intentions from the text.

[0125] Input: Conversation data sent to the server

[0126] Output: Analyzed conversation content (emotions, intentions, etc.)

[0127] Step 7:

[0128] The server determines whether a stamp is needed.

[0129] Specific behavior:

[0130] Based on the results of the natural language processing engine, the server uses logic to determine whether a stamp is necessary depending on the context and emotion of the conversation.

[0131] Input: Parsed conversation

[0132] Output: Stamp necessity determination result

[0133] Step 8:

[0134] The server generates stamps using image generation AI.

[0135] Specific behavior:

[0136] If it is determined that a stamp is necessary, the server sends a prompt message to the image generation AI based on the analysis results and the image or character registered by the user. The image generation AI then generates a stamp based on the prompt message.

[0137] Input: The result of determining the necessity of the stamp, the image or character registered by the user, and the prompt text

[0138] Output: Generated stamp image

[0139] Step 9:

[0140] The server sends the generated stamp to the terminal.

[0141] Specific behavior:

[0142] The server sends the generated stamp image to the terminal. The image data is sent in binary format.

[0143] Input: Generated stamp image

[0144] Output: Stamp image sent to the device

[0145] Step 10:

[0146] The terminal displays the stamp on the user interface.

[0147] Specific behavior:

[0148] The terminal decodes the received stamp and displays it on the user interface.

[0149] Input: Stamp image sent to the device

[0150] Output: Stamp displayed in the user interface

[0151] Step 11:

[0152] A user sends a stamp in a chat.

[0153] Specific behavior:

[0154] The user checks the suggested stamp and presses the send button to send the stamp to the chat partner.

[0155] Input: Stamp displayed in the user interface

[0156] Output: Stamp sent to chat partner

[0157] (Application example 1)

[0158] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0159] Conventional electronic payment services have had the problem that communication between users is monotonous and does not allow for rich expression of emotions or intentions. The present invention aims to solve this problem and improve the user experience in electronic payment services.

[0160] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0161] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring and analyzing conversation content, means for generating images using the user's registered pictures and characters based on the conversation content, means for suggesting the generated images to the user, means for including a chat function when the user transfers money or makes a payment, and means for automatically generating and suggesting stamps suitable for the chat function. This allows for richer communication during electronic payments and makes it possible to accurately convey the user's feelings and intentions.

[0162] - "Pictures and characters" refers to graphics, images, or specific characters that a user registers as their preference.

[0163] "Conversation content" refers to the text content exchanged via chat or messaging apps.

[0164] "Monitoring" means the process of watching and recording conversations in real time.

[0165] "Means of analysis" refers to the technical means of analyzing the content of a conversation and understanding its intentions and emotions.

[0166] "Means for generating images" refers to the process of creating new images using pictures and characters registered by the user based on the content of the conversation.

[0167] "Means for suggesting" refers to a process for suggesting generated images to a user.

[0168] "Chat function" refers to a means for users to exchange text messages with each other.

[0169] "Means for automatic generation and suggestion" means a process for automatically generating stamps based on the content of a user's chat and suggesting them to the user.

[0170] A "natural language processing engine" refers to software that analyzes text data and understands its meaning and sentiment.

[0171] "Electronic payment service" refers to a system for transferring money and making payments via the Internet.

[0172] The present invention provides a system that allows users to realize emotionally rich communication through electronic payment services. Specific embodiments of the system are described below.

[0173] The system mainly consists of a user terminal, a server, a natural language processing engine, and an image generation AI.

[0174] First, the user performs the initial setup and registers their favorite designs and characters in the system. The information the user enters into the device is sent to the server and saved in a database. This is used to record the user's preferences.

[0175] Next, when a user transfers or makes a payment as part of an electronic payment service, they use the chat function. The chat content is periodically sent to a server. The server then analyzes the received conversation content using a natural language processing engine. This analysis allows the intent and sentiment of the conversation to be understood. The natural language processing engine uses software such as Hugging Face's Transformers.

[0176] Furthermore, based on the analysis results and the images and characters registered by the user, the server uses image generation AI to generate stamps. The generated stamps are based on the content of the user's chat and are intended to enrich the conversation. A specific generation AI model is used for the image generation AI.

[0177] The generated stamps are instantly sent to the user's device and displayed on the user interface. Users can easily send emotive messages by sending them in chats.

[0178] Specific examples

[0179] For example, if a user has registered "cats" and "dogs" as their favorite characters in the system, when they write "I'm tired today" in a chat, the chat content is sent to the server. The server analyzes the emotion "tired" using a natural language processing engine and generates a stamp of a cat that shows "tiredness." The generated stamp is then suggested to the user, who can then send it in the chat.

[0180] Prompt Sentence Examples

[0181] Based on the text below, please create a stamp image of a cat character that matches the emotion.

[0182] Text: "I'm tired today"

[0183] Character: "Cat"

[0184] Emotion: "Tired"

[0185] Thus, the present invention aims to improve the user's communication experience by generating stamps according to the user's preferences and conversation context.

[0186] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0187] Step 1:

[0188] The user registers their favorite designs and characters on the device. The device receives the registration information (designs and characters) entered by the user and sends it to the server. The server stores the received information in a database. This records the user's preferences and makes them available for later processing.

[0189] Input: Images or characters registered by the user

[0190] Data processing: The terminal converts the input information into a structured data format.

[0191] Output: Structured image and character data is sent to the server

[0192] Step 2:

[0193] A user uses the chat function within an electronic payment service to have a conversation. The device periodically sends the contents of the conversation to the server. The server receives the content and analyzes it using a natural language processing engine. The analysis results identify the intent and sentiment of the conversation.

[0194] Input: User chat content

[0195] Data processing: The device sends chat content in text format to the server

[0196] Output: The conversation is analyzed by a natural language processing engine to obtain sentiment and intent data.

[0197] Step 3:

[0198] The server uses the analyzed conversation content and the images and character information registered by the user to generate stamps using an image generation AI model. Specifically, it inputs prompts according to emotions and intentions into the generation AI model to generate stamp images.

[0199] Input: Analyzed emotions and intentions, registered images and characters

[0200] Data processing: The server generates prompts according to emotions and intentions, and inputs them into the image generation AI model.

[0201] Output: A new stamp image is generated based on the prompt.

[0202] Step 4:

[0203] The generated stamp image is sent from the server to the device. The device receives this information and displays it on the user interface. The user can then check the proposed stamp and send it to the chat if necessary.

[0204] Input: Generated stamp image

[0205] Data processing: The server sends the stamp image data to the terminal.

[0206] Output: The stamp image is displayed on the device's user interface.

[0207] Step 5:

[0208] Users can send suggested stamps in chat, which allows for richer emotional expression and enriches conversations.

[0209] Input: User confirmation and stamp sending operation

[0210] Data processing: The device receives the user's sending operation and inserts the stamp image into the chat.

[0211] Output: The stamp is sent in the chat and reflected in the conversation.

[0212] In this way, the system of the present invention generates and suggests emotive stamps in real time based on the user's registration information and conversational context, improving the user experience in electronic payment services.

[0213] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0214] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to suggest more appropriate stamps. The program processing is explained in detail below.

[0215] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[0216] Next, when a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server. The server then analyzes the received data using a natural language processing engine (NLP) and simultaneously analyzes the user's emotional state using an emotion engine. This analysis process identifies emotions such as positive, negative, and neutral from the conversation content.

[0217] Based on the analysis results of the emotion engine and NLP, the server generates new stamps based on the images and characters registered by the user. For example, if a user sends a message expressing the negative emotion of "tired," the server will generate a stamp expressing "tiredness" using a "cat" character.

[0218] The generated stamps are sent from the server to the device. The device displays the received stamps on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[0219] Specific examples

[0220] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. The server analyzes the context of "tired" using an NLP engine and simultaneously identifies "negative" emotions using an emotion engine. It then generates stamps that express the feeling of "tired" using the "cat" registered by the user. The generated stamps are then sent to the device and suggested to the user. The user can continue chatting using the suggested stamps.

[0221] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[0222] The processing flow will be explained below.

[0223] Step 1:

[0224] Users use the system to register their favorite designs and characters, and then input the desired character name or design name through the application interface.

[0225] Step 2:

[0226] The device receives the user's input information and sends it to the server, including the user ID and registered image and character information.

[0227] Step 3:

[0228] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0229] Step 4:

[0230] A user starts a chat. The user communicates with other users by sending messages as usual.

[0231] Step 5:

[0232] Your device will periodically monitor chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0233] Step 6:

[0234] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0235] Step 7:

[0236] The server uses a natural language processing engine (NLP) to analyze the chat content received and understand the user's intentions and emotions. At the same time, it uses an emotion engine to analyze the user's emotional state and identify whether it is positive, negative, or neutral.

[0237] Step 8:

[0238] If the server determines that a stamp needs to be generated based on the analysis results of the NLP and emotion engine, it obtains information about the design and character registered by the user.

[0239] Step 9:

[0240] The server activates the image generation AI and generates stamp images based on the user's preferences, conversation analysis results, and emotional state. For example, it generates a "cat" stamp with the theme of "tired."

[0241] Step 10:

[0242] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0243] Step 11:

[0244] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it via chat if necessary.

[0245] Example 2

[0246] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0247] When users chat, it takes time and effort to select and use appropriate stamps that match the content of the conversation and their own emotions. Furthermore, the stress and time required to search and select the appropriate stamps each time can be a problem. To solve this problem, a system is needed that can automatically generate and suggest stamps that match the user's emotions and conversation content.

[0248] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0249] In this invention, the server includes a means for registering characters and images that users like, a means for monitoring and analyzing conversation content, and a means for analyzing conversation content and emotions and generating images using the characters and images registered by the user. This allows users to be quickly and appropriately suggested stamps based on the chat content and emotions.

[0250] "User" refers to someone who uses the system to register their favorite characters or images and chat.

[0251] "Character" refers to visual elements such as characters, pictures, and images that users can register in the system.

[0252] "Picture" refers to a specific design or image that a user can register in the system.

[0253] "Conversation content" refers to text messages and statements entered into the chat by users.

[0254] "Monitoring" refers to the act of the terminal monitoring the user's chat content at regular intervals and obtaining the latest messages.

[0255] "Analyzing" refers to the act of analyzing the content and emotions of messages acquired by the server using a natural language processing engine or emotion engine.

[0256] "Emotion engine" refers to software or algorithms for identifying a user's emotion from the text of a message.

[0257] A "natural language processing engine" refers to software or algorithms that analyze the text of a conversation and understand its meaning and intent.

[0258] "Generative AI model" refers to an artificial intelligence model for generating new stamp images based on analyzed data.

[0259] "Generating an image" refers to the act of creating an image such as a stamp using a specified character or design based on the analysis results.

[0260] "Suggesting" refers to the act of displaying the generated image or stamp to the user and recommending its use.

[0261] "Terminal" refers to a device used by a user to chat, such as a smartphone or PC.

[0262] "Server" refers to a computer system that receives data sent by users and performs analysis and image generation.

[0263] "Database" refers to a data management system for storing information on characters and designs registered by users.

[0264] "Stamps" refer to images or emoticons that are generated based on the user's emotions or conversation content.

[0265] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using characters and images that users like. Furthermore, by combining it with an emotion engine, it is possible to suggest even more appropriate stamps.

[0266] First, the user opens the application interface and registers the name of their favorite character or design. The registered information is sent from the device to the server, which then stores this data in a database.

[0267] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. The retrieved messages are then sent to the server, which uses a natural language processing engine (NLP) to analyze the message content and an emotion engine to identify the user's emotions. This analysis process identifies emotions such as positive, negative, and neutral from the message content.

[0268] Based on the analysis results, the server generates new stamps using characters and images registered by the user. Using a generative AI model, stamps are created based on emotions and the content of the conversation. For example, if a user sends a message saying, "I'm tired today," the server analyzes the emotion "tired" using an NLP engine and identifies it as a negative emotion using an emotion engine. Based on this, a stamp expressing a "tired look" is generated using a "cat" registered by the user.

[0269] The generated stamps are sent from the server to the terminal, and the terminal displays them on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[0270] Specific examples

[0271] Let us take as an example the process when a user registers "cat" and "dog" as their favorite characters in the system.

[0272] If a user types "I'm tired today" in a chat, the device sends the chat content to the server. The server uses an NLP engine to analyze the context of "tired" and an emotion engine to identify "negative emotions." It then generates a stamp that expresses the feeling of "tired" using the "cat" registered by the user. The generated stamp is then sent to the device and suggested to the user. The user can continue chatting using the suggested stamp.

[0273] Prompt Sentence Examples

[0274] The following prompt will generate a stamp using a generative AI model:

[0275] "Brief description: A system that allows users to register their favorite characters and automatically generates and suggests stamps based on what they say in chat. It combines an emotion engine and natural language processing to provide stamps that reflect the user's emotions. For example, if a user types "I'm tired today," a stamp expressing "tiredness" will be generated using the "cat" character registered by the user."

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1: User character registration

[0278] Users open the application interface, enter the name of their favorite character or image, and register it.

[0279] Input: Character name or picture name entered by the user.

[0280] Operation: The user enters the character name "cat" and the picture name "dog" into the input form and presses the registration button.

[0281] Output: The terminal sends the entered information to the server.

[0282] Step 2: Save your registration information

[0283] The server receives the character names and design names sent from the terminal and stores them in a database.

[0284] Input: Character name or picture name sent from the device.

[0285] Operation: The server stores the received data in a database.

[0286] Output: Character and picture information stored in the database.

[0287] Step 3: Monitoring the chat

[0288] A user initiates a chat.

[0289] The device periodically monitors the chat content and retrieves the latest messages.

[0290] Input: The chat message typed by the user.

[0291] Operation: Periodically checks and retrieves messages entered on the chat screen, such as "I'm tired today."

[0292] Output: The most recent message data.

[0293] Step 4: Sending the message to the server

[0294] The terminal transmits the acquired chat message to the server.

[0295] Input: The most recent chat message.

[0296] Action: The device sends a message to the server.

[0297] Output: The message data received by the server.

[0298] Step 5: Parsing the message

[0299] The server analyzes the received message using a natural language processing engine (NLP).

[0300] Input: Received chat messages.

[0301] How it works: The NLP engine analyzes the context of "tired."

[0302] Output: Analysis results (e.g., context information such as "tired").

[0303] Step 6: Identify the emotion

[0304] The server uses an emotion engine to identify the emotion of the message.

[0305] Input: Analysis results of the NLP engine.

[0306] How it works: The emotion engine identifies negative emotions based on "tired."

[0307] Output: Identified emotion information (e.g., "negative").

[0308] Step 7: Generate stamps

[0309] The server is a generative AI model that generates stamps using characters and images registered by the user.

[0310] Input: Identified emotion information and user registration information.

[0311] How it works: A generative AI model creates a "Tired Cat" stamp.

[0312] Output: The generated stamp image.

[0313] Step 8: Send your stamp

[0314] The server transmits the generated stamp image to the terminal.

[0315] Input: The generated stamp image.

[0316] Operation: The server sends the stamp image to the device.

[0317] Output: Stamp image data for display on the device.

[0318] Step 9: Propose and use stamps

[0319] The terminal displays the received stamps on the user interface and suggests them to the user.

[0320] Input: The stamp image sent to the device.

[0321] What it does: A stamp appears on the chat screen.

[0322] Output: A user interface for the user to review and send the stamp.

[0323] (Application example 2)

[0324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0325] Conventional chat applications lack a mechanism for automatically suggesting appropriate stickers based on the user's emotions or the flow of the conversation, forcing users to select stickers themselves. Furthermore, the emotions and nuances that users want to convey through chat messages may not be conveyed properly, making personalized communication difficult. Furthermore, technology for understanding users' emotions and providing appropriate content in real time based on those emotions remains immature, and there is a need for improved accuracy.

[0326] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0327] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring conversation content and analyzing the conversation content and the user's emotions, means for generating images using the user's registered pictures and characters based on the conversation content and the user's emotions, and means for suggesting the generated images to the user. This enables more intuitive communication by automatically suggesting personalized stamps according to the user's emotions and the flow of the conversation.

[0328] A "user" is a person who uses the system to chat.

[0329] "Pictures and characters" are images or illustrations registered by users and used as stamps.

[0330] "Means of registration" refers to a method by which a user inputs and saves their favorite designs or characters into the system.

[0331] "Means for monitoring conversation content and analyzing the conversation content and user emotions" refers to the process of monitoring chat text data and understanding its content and emotions using natural language processing and emotion analysis techniques.

[0332] "Means for generating images using user-registered pictures and characters based on the content of the conversation and the user's emotions" refers to a technology that generates stamps using pre-registered pictures and characters, taking into account the context of the conversation and the user's emotions.

[0333] The "means for suggesting the generated image to the user" is a method for suggesting the generated stamp to the user and displaying it on the chat interface.

[0334] The "means for periodically sending chat content and analysis results to the server" is a mechanism for sending users' chat text and the analysis results to the server at regular intervals.

[0335] A "natural language processing engine" is software that analyzes text data and understands its context and meaning.

[0336] An "emotion analysis engine" is an algorithm or software that identifies a user's emotions from text data.

[0337] "Stamps" are images or icons used to convey emotions or messages within chat.

[0338] This invention is a system that automatically generates and suggests stamps that match the flow of conversation and emotions using the user's favorite designs and characters. Specifically, it can be implemented as follows.

[0339] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[0340] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server, which then analyzes the received data using a natural language processing engine (NLP) and a sentiment analysis engine. This analysis process identifies sentiments, such as negative, positive, or neutral, from the conversation and stores the results in a database.

[0341] Based on the analysis results, the server generates new stamps using the images and characters registered by the user. A generative AI model is used to generate stamps, so for example, if a user sends a message expressing a negative emotion such as "I'm tired," the server will generate a stamp that expresses a "tired look" using a "cat" character.

[0342] The generated stamps are sent from the server to the terminal, and the terminal displays the received stamps on the user interface and suggests them to the user. The user can then check the suggested stamps and send them in chat.

[0343] The hardware used is a smartphone (e.g., iPhone, Android device), and the software can be Python, the Transformers library (Hugging Face), and a REST API (backend server).The natural language processing engine used is, for example, the Hugging Face NLP engine, and the Hugging Face model is used for emotion analysis.

[0344] As a specific example, if a user registers a "cat" and a "dog" character in the system and writes "I feel tired today" in a chat, the device will send the chat content to the server. The server will analyze the context of "tired" using an NLP engine and simultaneously identify "negative emotions" using an emotion engine. It will then use a generative AI model to generate a stamp that expresses a "tired appearance" according to the prompt "Generate a stamp showing cat feeling tired." The generated stamp will then be sent to the device and suggested to the user.

[0345] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[0346] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0347] Step 1:

[0348] The user uses a means for registering a favorite picture or character to input the desired character name or picture name through the interface of the application.

[0349] Input: Character name or picture name

[0350] Data processing: The terminal collects and formats input data

[0351] Output: Formatted user registration information

[0352] Step 2:

[0353] The terminal transmits the formatted user registration information to the server.

[0354] Input: Formatted user registration information

[0355] Data processing: Sending data from the device to the server

[0356] Output: User registration information stored on the server

[0357] Step 3:

[0358] Once a user starts chatting, the terminal periodically monitors the conversation and retrieves the latest messages.

[0359] Input: Chat message

[0360] Data processing: Collection and temporary storage of chat messages

[0361] Output: Latest chat messages

[0362] Step 4:

[0363] The latest chat message acquired by the device is sent to the server.

[0364] Input: Latest chat message

[0365] Data processing: Sending data from the device to the server

[0366] Output: Chat messages sent to the server

[0367] Step 5:

[0368] The server analyzes the chat messages it receives using a natural language processing engine (NLP) and sentiment analysis engine.

[0369] Input: Chat message

[0370] Data processing: Context analysis using NLP and sentiment analysis using a sentiment analysis engine

[0371] Output: Analysis results (emotions and contextual information)

[0372] Step 6:

[0373] Based on the analysis results, the server uses a generative AI model to generate new stamps based on the images and characters registered by the user.

[0374] Input: Analysis results (emotions and context information), user registration information

[0375] Data processing: Prompt sentence generation and stamp generation using generative AI models

[0376] Output: Generated stamp (image data)

[0377] As a concrete example, a stamp is generated according to the prompt sentence "Generate a stamp showing a cat feeling tired."

[0378] Step 7:

[0379] The server sends the generated stamp to the terminal.

[0380] Input: Generated stamp (image data)

[0381] Data processing: Sending generated stamp data

[0382] Output: Generated stamp sent to the device

[0383] Step 8:

[0384] The terminal displays the received stamp on the user interface and suggests it to the user.

[0385] Input: Generated stamp (image data)

[0386] Data processing: Stamp display in user interface

[0387] Output: Stamps suggested to the user

[0388] Step 9:

[0389] The user can check the suggested stamps and send them via chat if necessary.

[0390] Input: Suggested stamp

[0391] Data processing: Selecting and sending stamps

[0392] Output: Stickers sent in chat

[0393] In this way, a system is constructed that automatically generates appropriate stamps and suggests them to users based on the user's emotions and conversation content.

[0394] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0395] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0396] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0397] [Second embodiment]

[0398] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0399] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0400] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0401] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0402] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0403] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0404] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0405] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0406] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0407] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0408] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0409] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0410] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. The program processing is explained in detail below.

[0411] First, the user performs initial setup to use the system. The user registers their favorite designs and characters in the system, and this information is sent from the terminal to the server. The server stores the received information in a database and holds data according to the preferences of each individual user.

[0412] Once a user starts chatting, the device periodically monitors the conversation and sends the data to the server, which then analyzes the received conversation using a natural language processing engine to determine whether a stamp matching the conversation is needed.

[0413] If a stamp is deemed necessary, the server uses image generation AI to generate a new stamp based on the analysis results and the user's registered favorite designs and characters. The generated stamp is then sent from the server to the device.

[0414] The device displays the received stamps on the user interface and suggests them to the user. By sending the suggested stamps, the user can communicate more uniquely and expressively.

[0415] Specific examples

[0416] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp using the "cat" registered by the user to express a "tired look." The generated stamp is then suggested to the user, who then sends it in the chat.

[0417] This allows users to use a variety of stamps featuring their favorite characters and send emotionally rich messages to others.The purpose of the present invention is to improve the user's communication experience by generating stamps based on the user's preferences and the context of the conversation.

[0418] The processing flow will be explained below.

[0419] Step 1:

[0420] Users use the system to register their favorite designs and characters, and then input the names of each character and design through the application interface.

[0421] Step 2:

[0422] The device acquires the user's input information and sends it to the server, including the user ID and registered image and character information.

[0423] Step 3:

[0424] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0425] Step 4:

[0426] A user initiates a chat. The user chats by sending regular messages.

[0427] Step 5:

[0428] The device will periodically monitor the chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0429] Step 6:

[0430] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0431] Step 7:

[0432] The server uses a natural language processing engine (NLP) to analyze the chat content it receives, and determines whether the message needs an appropriate stamp based on its intent and sentiment.

[0433] Step 8:

[0434] If the server determines that a stamp needs to be generated based on the results of the NLP analysis, it obtains information about the design and character registered by the user.

[0435] Step 9:

[0436] The server activates the image generation AI and generates a stamp image based on the user's preferences and the results of NLP analysis. For example, it generates a "cat" stamp with the theme of "tired."

[0437] Step 10:

[0438] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0439] Step 11:

[0440] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it if necessary.

[0441] Example 1

[0442] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0443] In conventional chat systems, users had to manually select stamps, making it difficult to quickly use appropriate stamps that fit the flow of the conversation. Furthermore, because users had to choose from existing stamps, it was difficult to easily find stamps that matched individual users' preferences and the context of the conversation. This limited the user's communication experience and made it difficult to have expressive conversations.

[0444] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0445] In this invention, the server includes means for users to register their favorite images and characters, means for monitoring and analyzing conversation content, means for generating stamps using the images and characters registered by the user based on the conversation content, and means for suggesting the generated stamps to the user. This enables users to quickly use unique and appropriate stamps in line with the flow of conversation, enabling more expressive communication.

[0446] A "user" is an entity that uses the system to register their favorite images and characters and chat.

[0447] "Images and characters" are visual representations such as favorite pictures and characters that users register in the system.

[0448] The "means for monitoring the content of conversations" is a function for continuously observing the content of chat conversations and extracting important information.

[0449] "Means for analyzing conversation content" refers to a function that uses natural language processing technology to analyze the conversation content of a user and understand their emotions and intentions.

[0450] A "stamp" is a type of image generated by the system to visually express the user's emotions and intentions.

[0451] "Means of generation" is a function that creates stamps suitable for conversation based on registered images and characters and analysis results.

[0452] The "means of suggesting" is a function of displaying the generated stamp to the user and encouraging them to use it.

[0453] A "server" is a central computer that processes and manages data for the entire system.

[0454] A "natural language processing engine" is a software technology that analyzes text data and understands its meaning and emotions.

[0455] The "generative AI model" is an artificial intelligence model that generates new stamps based on user registration information and conversation analysis results.

[0456] The present invention is a system that automatically generates and suggests stamps that match the flow of conversation using images or characters that users like. The specific processing of this system is as follows.

[0457] First, the user performs initial setup to use the system. The user registers their favorite images and characters in the system. This information is sent from the user's device to the server. The server stores the received information in a database and holds data according to the preferences of each individual user. This data includes image data and keywords uploaded by the user.

[0458] Next, when a user starts chatting, the device periodically monitors the conversation content and sends the data to the server. The server then analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and determines whether a stamp appropriate for the conversation is needed. The natural language processing engine extracts emotions and intentions from the user's conversation content and performs analysis based on this.

[0459] If a stamp is deemed necessary, the server uses the analysis results and the user's favorite images and characters to generate a new stamp using an image generation AI (e.g., a standard image generation algorithm). The image generation AI is provided with a prompt that reflects the user's preferences and the context of the conversation.

[0460] As a concrete example, consider a case where a user has registered "cats" and "dogs" as their favorite characters in the system. When the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp that expresses "tiredness" using the "cat" registered by the user. The generated stamp is then suggested to the user, who then sends the stamp in the chat.

[0461] An example prompt is:

[0462] "The user's favorite character is a cat. The conversation is, 'I'm tired today.' Please generate an image of a cat that expresses this conversation."

[0463] This system allows users to use a variety of stamps featuring their favorite characters and send emotive messages to others. As described above, the purpose of this invention is to enrich users' communication experiences by automatically generating and suggesting stamps that match the user's preferences and the context of the conversation.

[0464] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0465] Step 1:

[0466] Users register their favorite images and characters.

[0467] Specific behavior:

[0468] The user opens the system's settings screen and registers their favorite image or character by entering the image file and keywords.

[0469] Input: Image files and keywords entered by the user

[0470] Output: Registration information is generated

[0471] Step 2:

[0472] The terminal sends the registration information to the server.

[0473] Specific behavior:

[0474] The terminal (for example, a smartphone or PC) sends the data entered by the user to the server using the HTTPS protocol.

[0475] Input: Image files and keywords registered by the user

[0476] Output: Registration information sent to the server

[0477] Step 3:

[0478] The server stores the information in a database.

[0479] Specific behavior:

[0480] The server analyzes the received data, organizes it by user, and stores it in a database (e.g., MySQL).

[0481] Input: Registration information received by the server

[0482] Output: User registration information saved in the database

[0483] Step 4:

[0484] The device monitors the conversation.

[0485] Specific behavior:

[0486] When a user launches a chat app and starts a conversation, the device monitors messages entered every few seconds.

[0487] Input: The chat message the user types

[0488] Output: Monitored conversation

[0489] Step 5:

[0490] The terminal transmits the conversation data to the server.

[0491] Specific behavior:

[0492] The device sends text data of the conversation to the server at regular intervals, and this communication also uses the HTTPS protocol.

[0493] Input: Monitored conversation

[0494] Output: Conversation data sent to the server

[0495] Step 6:

[0496] The server analyzes the conversation using a natural language processing engine.

[0497] Specific behavior:

[0498] The server analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and extracts emotions and intentions from the text.

[0499] Input: Conversation data sent to the server

[0500] Output: Analyzed conversation content (emotions, intentions, etc.)

[0501] Step 7:

[0502] The server determines whether a stamp is needed.

[0503] Specific behavior:

[0504] Based on the results of the natural language processing engine, the server uses logic to determine whether a stamp is necessary depending on the context and emotion of the conversation.

[0505] Input: Parsed conversation

[0506] Output: Result of stamp necessity determination

[0507] Step 8:

[0508] The server generates stamps using image generation AI.

[0509] Specific behavior:

[0510] If it is determined that a stamp is necessary, the server sends a prompt message to the image generation AI based on the analysis results and the image or character registered by the user. The image generation AI then generates a stamp based on the prompt message.

[0511] Input: The result of determining the necessity of the stamp, the image or character registered by the user, and the prompt text

[0512] Output: Generated stamp image

[0513] Step 9:

[0514] The server sends the generated stamp to the terminal.

[0515] Specific behavior:

[0516] The server sends the generated stamp image to the terminal. The image data is sent in binary format.

[0517] Input: Generated stamp image

[0518] Output: Stamp image sent to the device

[0519] Step 10:

[0520] The terminal displays the stamp on the user interface.

[0521] Specific behavior:

[0522] The terminal decodes the received stamp and displays it on the user interface.

[0523] Input: Stamp image sent to the device

[0524] Output: Stamp displayed in the user interface

[0525] Step 11:

[0526] A user sends a stamp in a chat.

[0527] Specific behavior:

[0528] The user checks the suggested stamp and presses the send button to send the stamp to the chat partner.

[0529] Input: Stamp displayed in the user interface

[0530] Output: Stamp sent to chat partner

[0531] (Application example 1)

[0532] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0533] Conventional electronic payment services have had the problem that communication between users is monotonous and does not allow for rich expression of emotions or intentions. The present invention aims to solve this problem and improve the user experience in electronic payment services.

[0534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0535] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring and analyzing conversation content, means for generating images using the user's registered pictures and characters based on the conversation content, means for suggesting the generated images to the user, means for including a chat function when the user transfers money or makes a payment, and means for automatically generating and suggesting stamps suitable for the chat function. This allows for richer communication during electronic payments and makes it possible to accurately convey the user's feelings and intentions.

[0536] - "Pictures and characters" refers to graphics, images, or specific characters that a user registers as their preference.

[0537] "Conversation content" refers to the text content exchanged via chat or messaging apps.

[0538] "Monitoring" means the process of watching and recording conversations in real time.

[0539] "Means of analysis" refers to the technical means of analyzing the content of a conversation and understanding its intentions and emotions.

[0540] "Means for generating images" refers to the process of creating new images using pictures and characters registered by the user based on the content of the conversation.

[0541] "Means for suggesting" refers to a process for suggesting generated images to a user.

[0542] "Chat function" refers to a means for users to exchange text messages with each other.

[0543] "Means for automatic generation and suggestion" means a process for automatically generating stamps based on the content of a user's chat and suggesting them to the user.

[0544] A "natural language processing engine" refers to software that analyzes text data and understands its meaning and sentiment.

[0545] "Electronic payment service" refers to a system for transferring money and making payments via the Internet.

[0546] The present invention provides a system that allows users to realize emotional communication through electronic payment services. Specific embodiments of the system are described below.

[0547] The system mainly consists of a user terminal, a server, a natural language processing engine, and an image generation AI.

[0548] First, the user performs the initial setup and registers their favorite designs and characters in the system. The information the user enters into the device is sent to the server and saved in a database. This is used to record the user's preferences.

[0549] Next, when a user transfers or makes a payment as part of an electronic payment service, they use the chat function. The chat content is periodically sent to a server. The server then analyzes the received conversation content using a natural language processing engine. This analysis allows the intent and sentiment of the conversation to be understood. The natural language processing engine uses software such as Hugging Face's Transformers.

[0550] Furthermore, based on the analysis results and the images and characters registered by the user, the server uses image generation AI to generate stamps. The generated stamps are based on the content of the user's chat and are intended to enrich the conversation. A specific generation AI model is used for the image generation AI.

[0551] The generated stamps are instantly sent to the user's device and displayed on the user interface. Users can easily send emotive messages by sending them in chats.

[0552] Specific examples

[0553] For example, if a user has registered "cats" and "dogs" as their favorite characters in the system, when they write "I'm tired today" in a chat, the chat content is sent to the server. The server analyzes the emotion "tired" using a natural language processing engine and generates a stamp of a cat that shows "tiredness." The generated stamp is then suggested to the user, who can then send it in the chat.

[0554] Prompt Sentence Examples

[0555] Based on the text below, please create a stamp image of a cat character that matches the emotion.

[0556] Text: "I'm tired today"

[0557] Character: "Cat"

[0558] Emotion: "Tired"

[0559] Thus, the present invention aims to improve the user's communication experience by generating stamps according to the user's preferences and conversation context.

[0560] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0561] Step 1:

[0562] The user registers their favorite designs and characters on the device. The device receives the registration information (designs and characters) entered by the user and sends it to the server. The server stores the received information in a database. This records the user's preferences and makes them available for later processing.

[0563] Input: Images or characters registered by the user

[0564] Data processing: The terminal converts the input information into a structured data format.

[0565] Output: Structured image and character data is sent to the server

[0566] Step 2:

[0567] A user uses the chat function within an electronic payment service to have a conversation. The device periodically sends the contents of the conversation to the server. The server receives the content and analyzes it using a natural language processing engine. The analysis results identify the intent and sentiment of the conversation.

[0568] Input: User chat content

[0569] Data processing: The device sends chat content in text format to the server

[0570] Output: The conversation is analyzed by a natural language processing engine to obtain sentiment and intent data.

[0571] Step 3:

[0572] The server uses the analyzed conversation content and the images and character information registered by the user to generate stamps using an image generation AI model. Specifically, it inputs prompts according to emotions and intentions into the generation AI model to generate stamp images.

[0573] Input: Analyzed emotions and intentions, registered images and characters

[0574] Data processing: The server generates prompts according to emotions and intentions, and inputs them into the image generation AI model.

[0575] Output: A new stamp image is generated based on the prompt.

[0576] Step 4:

[0577] The generated stamp image is sent from the server to the device. The device receives this information and displays it on the user interface. The user can then check the proposed stamp and send it to the chat if necessary.

[0578] Input: Generated stamp image

[0579] Data processing: The server sends the stamp image data to the terminal.

[0580] Output: The stamp image is displayed on the device's user interface.

[0581] Step 5:

[0582] Users can send suggested stamps in chat, which allows for richer emotional expression and enriches conversations.

[0583] Input: User confirmation and stamp sending operation

[0584] Data processing: The device receives the user's sending operation and inserts the stamp image into the chat.

[0585] Output: The stamp is sent in the chat and reflected in the conversation.

[0586] In this way, the system of the present invention generates and suggests emotive stamps in real time based on the user's registration information and conversational context, improving the user experience in electronic payment services.

[0587] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0588] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to suggest more appropriate stamps. The program processing is explained in detail below.

[0589] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[0590] Next, when a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server. The server then analyzes the received data using a natural language processing engine (NLP) and simultaneously analyzes the user's emotional state using an emotion engine. This analysis process identifies emotions such as positive, negative, and neutral from the conversation content.

[0591] Based on the analysis results of the emotion engine and NLP, the server generates new stamps based on the images and characters registered by the user. For example, if a user sends a message expressing the negative emotion of "tired," the server will generate a stamp expressing "tiredness" using a "cat" character.

[0592] The generated stamps are sent from the server to the device. The device displays the received stamps on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[0593] Specific examples

[0594] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. The server analyzes the context of "tired" using an NLP engine and simultaneously identifies "negative" emotions using an emotion engine. It then generates stamps that express the feeling of "tired" using the "cat" registered by the user. The generated stamps are then sent to the device and suggested to the user. The user can continue chatting using the suggested stamps.

[0595] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[0596] The processing flow will be explained below.

[0597] Step 1:

[0598] Users use the system to register their favorite designs and characters, and then input the desired character name or design name through the application interface.

[0599] Step 2:

[0600] The device receives the user's input information and sends it to the server, including the user ID and registered image and character information.

[0601] Step 3:

[0602] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0603] Step 4:

[0604] A user starts a chat. The user communicates with other users by sending messages as usual.

[0605] Step 5:

[0606] Your device will periodically monitor chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0607] Step 6:

[0608] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0609] Step 7:

[0610] The server uses a natural language processing engine (NLP) to analyze the chat content received and understand the user's intentions and emotions. At the same time, it uses an emotion engine to analyze the user's emotional state and identify whether it is positive, negative, or neutral.

[0611] Step 8:

[0612] If the server determines that a stamp needs to be generated based on the analysis results of the NLP and emotion engine, it obtains information about the design and character registered by the user.

[0613] Step 9:

[0614] The server activates the image generation AI and generates stamp images based on the user's preferences, conversation analysis results, and emotional state. For example, it generates a "cat" stamp with the theme of "tired."

[0615] Step 10:

[0616] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0617] Step 11:

[0618] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it via chat if necessary.

[0619] Example 2

[0620] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0621] When users chat, it takes time and effort to select and use appropriate stamps that match the content of the conversation and their own emotions. Furthermore, the stress and time required to search and select the appropriate stamps each time can be a problem. To solve this problem, a system is needed that can automatically generate and suggest stamps that match the user's emotions and conversation content.

[0622] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0623] In this invention, the server includes a means for registering characters and images that users like, a means for monitoring and analyzing conversation content, and a means for analyzing conversation content and emotions and generating images using the characters and images registered by the user. This allows users to be quickly and appropriately suggested stamps based on the chat content and emotions.

[0624] "User" refers to someone who uses the system to register their favorite characters or images and chat.

[0625] "Character" refers to visual elements such as characters, pictures, and images that users can register in the system.

[0626] "Picture" refers to a specific design or image that a user can register in the system.

[0627] "Conversation content" refers to text messages and statements entered into the chat by users.

[0628] "Monitoring" refers to the act of the terminal monitoring the user's chat content at regular intervals and obtaining the latest messages.

[0629] "Analyzing" refers to the act of analyzing the content and emotions of messages acquired by the server using a natural language processing engine or emotion engine.

[0630] "Emotion engine" refers to software or algorithms for identifying a user's emotion from the text of a message.

[0631] A "natural language processing engine" refers to software or algorithms that analyze the text of a conversation and understand its meaning and intent.

[0632] "Generative AI model" refers to an artificial intelligence model for generating new stamp images based on analyzed data.

[0633] "Generating an image" refers to the act of creating an image such as a stamp using a specified character or design based on the analysis results.

[0634] "Suggesting" refers to the act of displaying the generated image or stamp to the user and recommending its use.

[0635] "Terminal" refers to a device used by a user to chat, such as a smartphone or PC.

[0636] "Server" refers to a computer system that receives data sent by users and performs analysis and image generation.

[0637] "Database" refers to a data management system for storing information on characters and designs registered by users.

[0638] "Stamps" refer to images or emoticons that are generated based on the user's emotions or conversation content.

[0639] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using characters and images that users like. Furthermore, by combining it with an emotion engine, it is possible to suggest even more appropriate stamps.

[0640] First, the user opens the application interface and registers the name of their favorite character or design. The registered information is sent from the device to the server, which then stores this data in a database.

[0641] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. The retrieved messages are then sent to the server, which uses a natural language processing engine (NLP) to analyze the message content and an emotion engine to identify the user's emotions. This analysis process identifies emotions such as positive, negative, and neutral from the message content.

[0642] Based on the analysis results, the server generates new stamps using characters and images registered by the user. Using a generative AI model, stamps are created based on emotions and the content of the conversation. For example, if a user sends a message saying, "I'm tired today," the server analyzes the emotion "tired" using an NLP engine and identifies it as a negative emotion using an emotion engine. Based on this, a stamp expressing a "tired look" is generated using a "cat" registered by the user.

[0643] The generated stamps are sent from the server to the terminal, and the terminal displays them on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[0644] Specific examples

[0645] Let us take as an example the process when a user registers "cat" and "dog" as their favorite characters in the system.

[0646] If a user types "I'm tired today" in a chat, the device sends the chat content to the server. The server uses an NLP engine to analyze the context of "tired" and an emotion engine to identify "negative emotions." It then generates a stamp that expresses the feeling of "tired" using the "cat" registered by the user. The generated stamp is then sent to the device and suggested to the user. The user can continue chatting using the suggested stamp.

[0647] Prompt Sentence Examples

[0648] The following prompt will generate a stamp using a generative AI model:

[0649] "Brief description: A system that allows users to register their favorite characters and automatically generates and suggests stamps based on what they say in chat. It combines an emotion engine and natural language processing to provide stamps that reflect the user's emotions. For example, if a user types "I'm tired today," a stamp expressing "tiredness" will be generated using the "cat" character registered by the user."

[0650] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0651] Step 1: User character registration

[0652] Users open the application interface, enter the name of their favorite character or image, and register it.

[0653] Input: Character name or picture name entered by the user.

[0654] Operation: The user enters the character name "cat" and the picture name "dog" into the input form and presses the registration button.

[0655] Output: The terminal sends the entered information to the server.

[0656] Step 2: Save your registration information

[0657] The server receives the character names and design names sent from the terminal and stores them in a database.

[0658] Input: Character name or picture name sent from the device.

[0659] Operation: The server stores the received data in a database.

[0660] Output: Character and picture information stored in the database.

[0661] Step 3: Monitoring the chat

[0662] A user initiates a chat.

[0663] The device periodically monitors the chat content and retrieves the latest messages.

[0664] Input: The chat message typed by the user.

[0665] Operation: Periodically checks and retrieves messages entered on the chat screen, such as "I'm tired today."

[0666] Output: The most recent message data.

[0667] Step 4: Sending the message to the server

[0668] The terminal transmits the acquired chat message to the server.

[0669] Input: The most recent chat message.

[0670] Action: The device sends a message to the server.

[0671] Output: The message data received by the server.

[0672] Step 5: Parsing the message

[0673] The server analyzes the received message using a natural language processing engine (NLP).

[0674] Input: Received chat messages.

[0675] How it works: The NLP engine analyzes the context of "tired."

[0676] Output: Analysis results (e.g., context information such as "tired").

[0677] Step 6: Identify the emotion

[0678] The server uses an emotion engine to identify the emotion of the message.

[0679] Input: Analysis results of the NLP engine.

[0680] How it works: The emotion engine identifies negative emotions based on "tired."

[0681] Output: Identified emotion information (e.g., "negative").

[0682] Step 7: Generate stamps

[0683] The server is a generative AI model that generates stamps using characters and images registered by the user.

[0684] Input: Identified emotion information and user registration information.

[0685] How it works: A generative AI model creates a "Tired Cat" stamp.

[0686] Output: The generated stamp image.

[0687] Step 8: Send your stamp

[0688] The server transmits the generated stamp image to the terminal.

[0689] Input: The generated stamp image.

[0690] Operation: The server sends the stamp image to the device.

[0691] Output: Stamp image data for display on the device.

[0692] Step 9: Propose and use stamps

[0693] The terminal displays the received stamps on the user interface and suggests them to the user.

[0694] Input: The stamp image sent to the device.

[0695] What it does: A stamp appears on the chat screen.

[0696] Output: A user interface for the user to review and send the stamp.

[0697] (Application example 2)

[0698] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0699] Conventional chat applications lack a mechanism for automatically suggesting appropriate stickers based on the user's emotions or the flow of the conversation, forcing users to select stickers themselves. Furthermore, the emotions and nuances that users want to convey through chat messages may not be conveyed properly, making personalized communication difficult. Furthermore, technology for understanding users' emotions and providing appropriate content in real time based on those emotions remains immature, and there is a need for improved accuracy.

[0700] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0701] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring conversation content and analyzing the conversation content and the user's emotions, means for generating images using the user's registered pictures and characters based on the conversation content and the user's emotions, and means for suggesting the generated images to the user. This enables more intuitive communication by automatically suggesting personalized stamps according to the user's emotions and the flow of the conversation.

[0702] A "user" is a person who uses the system to chat.

[0703] "Pictures and characters" are images or illustrations registered by users and used as stamps.

[0704] "Means of registration" refers to a method by which a user inputs and saves their favorite designs or characters into the system.

[0705] "Means for monitoring conversation content and analyzing the conversation content and user emotions" refers to the process of monitoring chat text data and understanding its content and emotions using natural language processing and emotion analysis techniques.

[0706] "Means for generating images using user-registered pictures and characters based on the content of the conversation and the user's emotions" refers to a technology that generates stamps using pre-registered pictures and characters, taking into account the context of the conversation and the user's emotions.

[0707] The "means for suggesting the generated image to the user" is a method for suggesting the generated stamp to the user and displaying it on the chat interface.

[0708] The "means for periodically sending chat content and analysis results to the server" is a mechanism for sending users' chat text and the analysis results to the server at regular intervals.

[0709] A "natural language processing engine" is software that analyzes text data and understands its context and meaning.

[0710] An "emotion analysis engine" is an algorithm or software that identifies a user's emotions from text data.

[0711] "Stamps" are images or icons used to convey emotions or messages within chat.

[0712] This invention is a system that automatically generates and suggests stamps that match the flow of conversation and emotions using the user's favorite designs and characters. Specifically, it can be implemented as follows.

[0713] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[0714] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server, which then analyzes the received data using a natural language processing engine (NLP) and a sentiment analysis engine. This analysis process identifies sentiments, such as negative, positive, or neutral, from the conversation and stores the results in a database.

[0715] Based on the analysis results, the server generates new stamps using the images and characters registered by the user. A generative AI model is used to generate stamps, so for example, if a user sends a message expressing a negative emotion such as "I'm tired," the server will generate a stamp that expresses a "tired look" using a "cat" character.

[0716] The generated stamps are sent from the server to the terminal, and the terminal displays the received stamps on the user interface and suggests them to the user. The user can then check the suggested stamps and send them in chat.

[0717] The hardware used is a smartphone (e.g., iPhone, Android device), and the software can be Python, the Transformers library (Hugging Face), and a REST API (backend server).The natural language processing engine used is, for example, the Hugging Face NLP engine, and the Hugging Face model is used for emotion analysis.

[0718] As a specific example, if a user registers a "cat" and a "dog" character in the system and writes "I feel tired today" in a chat, the device will send the chat content to the server. The server will analyze the context of "tired" using an NLP engine and simultaneously identify "negative emotions" using an emotion engine. It will then use a generative AI model to generate a stamp that expresses a "tired appearance" according to the prompt "Generate a stamp showing cat feeling tired." The generated stamp will then be sent to the device and suggested to the user.

[0719] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[0720] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0721] Step 1:

[0722] The user uses a means for registering a favorite picture or character to input the desired character name or picture name through the interface of the application.

[0723] Input: Character name or picture name

[0724] Data processing: The terminal collects and formats input data

[0725] Output: Formatted user registration information

[0726] Step 2:

[0727] The terminal transmits the formatted user registration information to the server.

[0728] Input: Formatted user registration information

[0729] Data processing: Sending data from the device to the server

[0730] Output: User registration information stored on the server

[0731] Step 3:

[0732] Once a user starts chatting, the terminal periodically monitors the conversation and retrieves the latest messages.

[0733] Input: Chat message

[0734] Data processing: Collection and temporary storage of chat messages

[0735] Output: Latest chat messages

[0736] Step 4:

[0737] The latest chat message acquired by the device is sent to the server.

[0738] Input: Latest chat message

[0739] Data processing: Sending data from the device to the server

[0740] Output: Chat messages sent to the server

[0741] Step 5:

[0742] The server analyzes the chat messages it receives using a natural language processing engine (NLP) and sentiment analysis engine.

[0743] Input: Chat message

[0744] Data processing: Context analysis using NLP and sentiment analysis using a sentiment analysis engine

[0745] Output: Analysis results (emotions and contextual information)

[0746] Step 6:

[0747] Based on the analysis results, the server uses a generative AI model to generate new stamps based on the images and characters registered by the user.

[0748] Input: Analysis results (emotions and context information), user registration information

[0749] Data processing: Prompt sentence generation and stamp generation using generative AI models

[0750] Output: Generated stamp (image data)

[0751] As a concrete example, a stamp is generated according to the prompt sentence "Generate a stamp showing a cat feeling tired."

[0752] Step 7:

[0753] The server sends the generated stamp to the terminal.

[0754] Input: Generated stamp (image data)

[0755] Data processing: Sending generated stamp data

[0756] Output: Generated stamp sent to the device

[0757] Step 8:

[0758] The terminal displays the received stamp on the user interface and suggests it to the user.

[0759] Input: Generated stamp (image data)

[0760] Data processing: Stamp display in user interface

[0761] Output: Stamps suggested to the user

[0762] Step 9:

[0763] The user can check the suggested stamps and send them via chat if necessary.

[0764] Input: Suggested stamp

[0765] Data processing: Selecting and sending stamps

[0766] Output: Stickers sent in chat

[0767] In this way, a system is constructed that automatically generates appropriate stamps and suggests them to users based on the user's emotions and conversation content.

[0768] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0769] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0770] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0771] [Third embodiment]

[0772] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0773] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0774] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0775] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0776] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0777] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0778] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0779] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0780] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0781] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0782] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0783] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0784] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. The program processing is explained in detail below.

[0785] First, the user performs initial setup to use the system. The user registers their favorite designs and characters in the system, and this information is sent from the terminal to the server. The server stores the received information in a database and holds data according to the preferences of each individual user.

[0786] Once a user starts chatting, the device periodically monitors the conversation and sends the data to the server, which then analyzes the received conversation using a natural language processing engine to determine whether a stamp matching the conversation is needed.

[0787] If a stamp is deemed necessary, the server uses image generation AI to generate a new stamp based on the analysis results and the user's registered favorite designs and characters. The generated stamp is then sent from the server to the device.

[0788] The device displays the received stamps on the user interface and suggests them to the user. By sending the suggested stamps, the user can communicate more uniquely and expressively.

[0789] Specific examples

[0790] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp using the "cat" registered by the user to express a "tired look." The generated stamp is then suggested to the user, who then sends it in the chat.

[0791] This allows users to use a variety of stamps featuring their favorite characters and send emotionally rich messages to others.The purpose of the present invention is to improve the user's communication experience by generating stamps based on the user's preferences and the context of the conversation.

[0792] The processing flow will be explained below.

[0793] Step 1:

[0794] Users use the system to register their favorite designs and characters, and then input the names of each character and design through the application interface.

[0795] Step 2:

[0796] The device acquires the user's input information and sends it to the server, including the user ID and registered image and character information.

[0797] Step 3:

[0798] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0799] Step 4:

[0800] A user initiates a chat. The user chats by sending regular messages.

[0801] Step 5:

[0802] The device will periodically monitor the chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0803] Step 6:

[0804] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0805] Step 7:

[0806] The server uses a natural language processing engine (NLP) to analyze the chat content it receives, and determines whether the message needs an appropriate stamp based on its intent and sentiment.

[0807] Step 8:

[0808] If the server determines that a stamp needs to be generated based on the results of the NLP analysis, it obtains information about the design and character registered by the user.

[0809] Step 9:

[0810] The server activates the image generation AI and generates a stamp image based on the user's preferences and the results of NLP analysis. For example, it generates a "cat" stamp with the theme of "tired."

[0811] Step 10:

[0812] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0813] Step 11:

[0814] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it if necessary.

[0815] Example 1

[0816] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0817] In conventional chat systems, users had to manually select stamps, making it difficult to quickly use appropriate stamps that fit the flow of the conversation. Furthermore, because users had to choose from existing stamps, it was difficult to easily find stamps that matched individual users' preferences and the context of the conversation. This limited the user's communication experience and made it difficult to have expressive conversations.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0819] In this invention, the server includes means for users to register their favorite images and characters, means for monitoring and analyzing conversation content, means for generating stamps using the images and characters registered by the user based on the conversation content, and means for suggesting the generated stamps to the user. This enables users to quickly use unique and appropriate stamps in line with the flow of conversation, enabling more expressive communication.

[0820] A "user" is an entity that uses the system to register their favorite images and characters and chat.

[0821] "Images and characters" are visual representations such as favorite pictures and characters that users register in the system.

[0822] The "means for monitoring the content of conversations" is a function for continuously observing the content of chat conversations and extracting important information.

[0823] "Means for analyzing conversation content" refers to a function that uses natural language processing technology to analyze the conversation content of a user and understand their emotions and intentions.

[0824] A "stamp" is a type of image generated by the system to visually express the user's emotions and intentions.

[0825] "Means of generation" is a function that creates stamps suitable for conversation based on registered images and characters and analysis results.

[0826] The "means of suggesting" is a function of displaying the generated stamp to the user and encouraging them to use it.

[0827] A "server" is a central computer that processes and manages data for the entire system.

[0828] A "natural language processing engine" is a software technology that analyzes text data and understands its meaning and emotions.

[0829] The "generative AI model" is an artificial intelligence model that generates new stamps based on user registration information and conversation analysis results.

[0830] The present invention is a system that automatically generates and suggests stamps that match the flow of conversation using images or characters that users like. The specific processing of this system is as follows.

[0831] First, the user performs initial setup to use the system. The user registers their favorite images and characters in the system. This information is sent from the user's device to the server. The server stores the received information in a database and holds data according to the preferences of each individual user. This data includes image data and keywords uploaded by the user.

[0832] Next, when a user starts chatting, the device periodically monitors the conversation content and sends the data to the server. The server then analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and determines whether a stamp appropriate for the conversation is needed. The natural language processing engine extracts emotions and intentions from the user's conversation content and performs analysis based on this.

[0833] If a stamp is deemed necessary, the server uses the analysis results and the user's favorite images and characters to generate a new stamp using an image generation AI (e.g., a standard image generation algorithm). The image generation AI is provided with a prompt that reflects the user's preferences and the context of the conversation.

[0834] As a concrete example, consider a case where a user has registered "cats" and "dogs" as their favorite characters in the system. When the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp that expresses "tiredness" using the "cat" registered by the user. The generated stamp is then suggested to the user, who then sends the stamp in the chat.

[0835] An example prompt is:

[0836] "The user's favorite character is a cat. The conversation is, 'I'm tired today.' Please generate an image of a cat that expresses this conversation."

[0837] This system allows users to use a variety of stamps featuring their favorite characters and send emotive messages to others. As described above, the purpose of this invention is to enrich users' communication experiences by automatically generating and suggesting stamps that match the user's preferences and the context of the conversation.

[0838] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0839] Step 1:

[0840] Users register their favorite images and characters.

[0841] Specific behavior:

[0842] The user opens the system's settings screen and registers their favorite image or character by entering the image file and keywords.

[0843] Input: Image files and keywords entered by the user

[0844] Output: Registration information is generated

[0845] Step 2:

[0846] The terminal sends the registration information to the server.

[0847] Specific behavior:

[0848] The terminal (for example, a smartphone or PC) sends the data entered by the user to the server using the HTTPS protocol.

[0849] Input: Image files and keywords registered by the user

[0850] Output: Registration information sent to the server

[0851] Step 3:

[0852] The server stores the information in a database.

[0853] Specific behavior:

[0854] The server analyzes the received data, organizes it by user, and stores it in a database (e.g., MySQL).

[0855] Input: Registration information received by the server

[0856] Output: User registration information saved in the database

[0857] Step 4:

[0858] The device monitors the conversation.

[0859] Specific behavior:

[0860] When a user launches a chat app and starts a conversation, the device monitors messages entered every few seconds.

[0861] Input: The chat message the user types

[0862] Output: Monitored conversation

[0863] Step 5:

[0864] The terminal transmits the conversation data to the server.

[0865] Specific behavior:

[0866] The device sends text data of the conversation to the server at regular intervals, and this communication also uses the HTTPS protocol.

[0867] Input: Monitored conversation

[0868] Output: Conversation data sent to the server

[0869] Step 6:

[0870] The server analyzes the conversation using a natural language processing engine.

[0871] Specific behavior:

[0872] The server analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and extracts emotions and intentions from the text.

[0873] Input: Conversation data sent to the server

[0874] Output: Analyzed conversation content (emotions, intentions, etc.)

[0875] Step 7:

[0876] The server determines whether a stamp is needed.

[0877] Specific behavior:

[0878] Based on the results of the natural language processing engine, the server uses logic to determine whether a stamp is necessary depending on the context and emotion of the conversation.

[0879] Input: Parsed conversation

[0880] Output: Result of stamp necessity determination

[0881] Step 8:

[0882] The server generates stamps using image generation AI.

[0883] Specific behavior:

[0884] If it is determined that a stamp is necessary, the server sends a prompt message to the image generation AI based on the analysis results and the image or character registered by the user. The image generation AI then generates a stamp based on the prompt message.

[0885] Input: The result of determining the necessity of the stamp, the image or character registered by the user, and the prompt text

[0886] Output: Generated stamp image

[0887] Step 9:

[0888] The server sends the generated stamp to the terminal.

[0889] Specific behavior:

[0890] The server sends the generated stamp image to the terminal. The image data is sent in binary format.

[0891] Input: Generated stamp image

[0892] Output: Stamp image sent to the device

[0893] Step 10:

[0894] The terminal displays the stamp on the user interface.

[0895] Specific behavior:

[0896] The terminal decodes the received stamp and displays it on the user interface.

[0897] Input: Stamp image sent to the device

[0898] Output: Stamp displayed in the user interface

[0899] Step 11:

[0900] A user sends a stamp in a chat.

[0901] Specific behavior:

[0902] The user checks the suggested stamp and presses the send button to send the stamp to the chat partner.

[0903] Input: Stamp displayed in the user interface

[0904] Output: Stamp sent to chat partner

[0905] (Application example 1)

[0906] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0907] Conventional electronic payment services have had the problem that communication between users is monotonous and does not allow for rich expression of emotions or intentions. The present invention aims to solve this problem and improve the user experience in electronic payment services.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0909] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring and analyzing conversation content, means for generating images using the user's registered pictures and characters based on the conversation content, means for suggesting the generated images to the user, means for including a chat function when the user transfers money or makes a payment, and means for automatically generating and suggesting stamps suitable for the chat function. This allows for richer communication during electronic payments and makes it possible to accurately convey the user's feelings and intentions.

[0910] - "Pictures and characters" refers to graphics, images, or specific characters that a user registers as their preference.

[0911] "Conversation content" refers to the text content exchanged via chat or messaging apps.

[0912] "Monitoring" means the process of watching and recording conversations in real time.

[0913] "Means of analysis" refers to the technical means of analyzing the content of a conversation and understanding its intentions and emotions.

[0914] "Means for generating images" refers to the process of creating new images using pictures and characters registered by the user based on the content of the conversation.

[0915] "Means for suggesting" refers to a process for suggesting generated images to a user.

[0916] "Chat function" refers to a means for users to exchange text messages with each other.

[0917] "Means for automatic generation and suggestion" means a process for automatically generating stamps based on the content of a user's chat and suggesting them to the user.

[0918] A "natural language processing engine" refers to software that analyzes text data and understands its meaning and sentiment.

[0919] "Electronic payment service" refers to a system for transferring money and making payments via the Internet.

[0920] The present invention provides a system that allows users to realize emotional communication through electronic payment services. Specific embodiments of the system are described below.

[0921] The system mainly consists of a user terminal, a server, a natural language processing engine, and an image generation AI.

[0922] First, the user performs the initial setup and registers their favorite designs and characters in the system. The information the user enters into the device is sent to the server and saved in a database. This is used to record the user's preferences.

[0923] Next, when a user transfers or makes a payment as part of an electronic payment service, they use the chat function. The chat content is periodically sent to a server. The server then analyzes the received conversation content using a natural language processing engine. This analysis allows the intent and sentiment of the conversation to be understood. The natural language processing engine uses software such as Hugging Face's Transformers.

[0924] Furthermore, based on the analysis results and the images and characters registered by the user, the server uses image generation AI to generate stamps. The generated stamps are based on the content of the user's chat and are intended to enrich the conversation. A specific generation AI model is used for the image generation AI.

[0925] The generated stamps are instantly sent to the user's device and displayed on the user interface. Users can easily send emotive messages by sending them in chats.

[0926] Specific examples

[0927] For example, if a user has registered "cats" and "dogs" as their favorite characters in the system, when they write "I'm tired today" in a chat, the chat content is sent to the server. The server analyzes the emotion "tired" using a natural language processing engine and generates a stamp of a cat that shows "tiredness." The generated stamp is then suggested to the user, who can then send it in the chat.

[0928] Prompt Sentence Examples

[0929] Based on the text below, please create a stamp image of a cat character that matches the emotion.

[0930] Text: "I'm tired today"

[0931] Character: "Cat"

[0932] Emotion: "Tired"

[0933] Thus, the present invention aims to improve the user's communication experience by generating stamps according to the user's preferences and conversation context.

[0934] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0935] Step 1:

[0936] The user registers their favorite designs and characters on the device. The device receives the registration information (designs and characters) entered by the user and sends it to the server. The server stores the received information in a database. This records the user's preferences and makes them available for later processing.

[0937] Input: Images or characters registered by the user

[0938] Data processing: The terminal converts the input information into a structured data format.

[0939] Output: Structured image and character data is sent to the server

[0940] Step 2:

[0941] A user uses the chat function within an electronic payment service to have a conversation. The device periodically sends the contents of the conversation to the server. The server receives the content and analyzes it using a natural language processing engine. The analysis results identify the intent and sentiment of the conversation.

[0942] Input: User chat content

[0943] Data processing: The device sends chat content in text format to the server

[0944] Output: The conversation is analyzed by a natural language processing engine to obtain sentiment and intent data.

[0945] Step 3:

[0946] The server uses the analyzed conversation content and the images and character information registered by the user to generate stamps using an image generation AI model. Specifically, it inputs prompts according to emotions and intentions into the generation AI model to generate stamp images.

[0947] Input: Analyzed emotions and intentions, registered images and characters

[0948] Data processing: The server generates prompts according to emotions and intentions, and inputs them into the image generation AI model.

[0949] Output: A new stamp image is generated based on the prompt.

[0950] Step 4:

[0951] The generated stamp image is sent from the server to the device. The device receives this information and displays it on the user interface. The user can then check the proposed stamp and send it to the chat if necessary.

[0952] Input: Generated stamp image

[0953] Data processing: The server sends the stamp image data to the terminal.

[0954] Output: The stamp image is displayed on the device's user interface.

[0955] Step 5:

[0956] Users can send suggested stamps in chat, which allows for richer emotional expression and enriches conversations.

[0957] Input: User confirmation and stamp sending operation

[0958] Data processing: The device receives the user's sending operation and inserts the stamp image into the chat.

[0959] Output: The stamp is sent in the chat and reflected in the conversation.

[0960] In this way, the system of the present invention generates and suggests emotive stamps in real time based on the user's registration information and conversational context, improving the user experience in electronic payment services.

[0961] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0962] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to suggest more appropriate stamps. The program processing is explained in detail below.

[0963] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[0964] Next, when a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server. The server then analyzes the received data using a natural language processing engine (NLP) and simultaneously analyzes the user's emotional state using an emotion engine. This analysis process identifies emotions such as positive, negative, and neutral from the conversation content.

[0965] Based on the analysis results of the emotion engine and NLP, the server generates new stamps based on the images and characters registered by the user. For example, if a user sends a message expressing the negative emotion of "tired," the server will generate a stamp expressing "tiredness" using a "cat" character.

[0966] The generated stamps are sent from the server to the device. The device displays the received stamps on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[0967] Specific examples

[0968] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. The server analyzes the context of "tired" using an NLP engine and simultaneously identifies "negative" emotions using an emotion engine. It then generates stamps that express the feeling of "tired" using the "cat" registered by the user. The generated stamps are then sent to the device and suggested to the user. The user can continue chatting using the suggested stamps.

[0969] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[0970] The processing flow will be explained below.

[0971] Step 1:

[0972] Users use the system to register their favorite designs and characters, and then input the desired character name or design name through the application interface.

[0973] Step 2:

[0974] The device receives the user's input information and sends it to the server, including the user ID and registered image and character information.

[0975] Step 3:

[0976] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[0977] Step 4:

[0978] A user starts a chat. The user communicates with other users by sending messages as usual.

[0979] Step 5:

[0980] Your device will periodically monitor chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[0981] Step 6:

[0982] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[0983] Step 7:

[0984] The server uses a natural language processing engine (NLP) to analyze the chat content received and understand the user's intentions and emotions. At the same time, it uses an emotion engine to analyze the user's emotional state and identify whether it is positive, negative, or neutral.

[0985] Step 8:

[0986] If the server determines that a stamp needs to be generated based on the analysis results of the NLP and emotion engine, it obtains information about the design and character registered by the user.

[0987] Step 9:

[0988] The server activates the image generation AI and generates stamp images based on the user's preferences, conversation analysis results, and emotional state. For example, it generates a "cat" stamp based on the theme of "tired."

[0989] Step 10:

[0990] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[0991] Step 11:

[0992] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it via chat if necessary.

[0993] Example 2

[0994] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0995] When users chat, it takes time and effort to select and use appropriate stamps that match the content of the conversation and their own emotions. Furthermore, the stress and time required to search and select the appropriate stamps each time can be a problem. To solve this problem, a system is needed that can automatically generate and suggest stamps that match the user's emotions and conversation content.

[0996] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0997] In this invention, the server includes a means for registering characters and images that users like, a means for monitoring and analyzing conversation content, and a means for analyzing conversation content and emotions and generating images using the characters and images registered by the user. This allows users to be quickly and appropriately suggested stamps based on the chat content and emotions.

[0998] "User" refers to someone who uses the system to register their favorite characters or images and chat.

[0999] "Character" refers to visual elements such as characters, pictures, and images that users can register in the system.

[1000] "Picture" refers to a specific design or image that a user can register in the system.

[1001] "Conversation content" refers to text messages and statements entered into the chat by users.

[1002] "Monitoring" refers to the act of the terminal monitoring the user's chat content at regular intervals and obtaining the latest messages.

[1003] "Analyzing" refers to the act of analyzing the content and emotions of messages acquired by the server using a natural language processing engine or emotion engine.

[1004] "Emotion engine" refers to software or algorithms for identifying a user's emotion from the text of a message.

[1005] A "natural language processing engine" refers to software or algorithms that analyze the text of a conversation and understand its meaning and intent.

[1006] "Generative AI model" refers to an artificial intelligence model for generating new stamp images based on analyzed data.

[1007] "Generating an image" refers to the act of creating an image such as a stamp using a specified character or design based on the analysis results.

[1008] "Suggesting" refers to the act of displaying the generated image or stamp to the user and recommending its use.

[1009] "Terminal" refers to a device used by a user to chat, such as a smartphone or PC.

[1010] "Server" refers to a computer system that receives data sent by users and performs analysis and image generation.

[1011] "Database" refers to a data management system for storing information on characters and designs registered by users.

[1012] "Stamps" refer to images or emoticons that are generated based on the user's emotions or conversation content.

[1013] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using characters and images that users like. Furthermore, by combining it with an emotion engine, it is possible to suggest even more appropriate stamps.

[1014] First, the user opens the application interface and registers the name of their favorite character or design. The registered information is sent from the device to the server, which then stores this data in a database.

[1015] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. The retrieved messages are then sent to the server, which uses a natural language processing engine (NLP) to analyze the message content and an emotion engine to identify the user's emotions. This analysis process identifies emotions such as positive, negative, and neutral from the message content.

[1016] Based on the analysis results, the server generates new stamps using characters and images registered by the user. Using a generative AI model, stamps are created based on emotions and the content of the conversation. For example, if a user sends a message saying, "I'm tired today," the server analyzes the emotion "tired" using an NLP engine and identifies it as a negative emotion using an emotion engine. Based on this, a stamp expressing a "tired look" is generated using a "cat" registered by the user.

[1017] The generated stamps are sent from the server to the terminal, and the terminal displays them on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[1018] Specific examples

[1019] Let us take as an example the process when a user registers "cat" and "dog" as their favorite characters in the system.

[1020] If a user types "I'm tired today" in a chat, the device sends the chat content to the server. The server uses an NLP engine to analyze the context of "tired" and an emotion engine to identify "negative emotions." It then generates a stamp that expresses the feeling of "tired" using the "cat" registered by the user. The generated stamp is then sent to the device and suggested to the user. The user can continue chatting using the suggested stamp.

[1021] Prompt Sentence Examples

[1022] The following prompt will generate a stamp using a generative AI model:

[1023] "Brief description: A system that allows users to register their favorite characters and automatically generates and suggests stamps based on what they say in chat. It combines an emotion engine and natural language processing to provide stamps that reflect the user's emotions. For example, if a user types "I'm tired today," a stamp expressing "tiredness" will be generated using the "cat" character registered by the user."

[1024] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1025] Step 1: User character registration

[1026] Users open the application interface, enter the name of their favorite character or image, and register it.

[1027] Input: Character name or picture name entered by the user.

[1028] Operation: The user enters the character name "cat" and the picture name "dog" into the input form and presses the registration button.

[1029] Output: The terminal sends the entered information to the server.

[1030] Step 2: Save your registration information

[1031] The server receives the character names and design names sent from the terminal and stores them in a database.

[1032] Input: Character name or picture name sent from the device.

[1033] Operation: The server stores the received data in a database.

[1034] Output: Character and picture information stored in the database.

[1035] Step 3: Monitoring the chat

[1036] A user initiates a chat.

[1037] The device periodically monitors the chat content and retrieves the latest messages.

[1038] Input: The chat message typed by the user.

[1039] Operation: Periodically checks and retrieves messages entered on the chat screen, such as "I'm tired today."

[1040] Output: The most recent message data.

[1041] Step 4: Sending the message to the server

[1042] The terminal transmits the acquired chat message to the server.

[1043] Input: The most recent chat message.

[1044] Action: The device sends a message to the server.

[1045] Output: The message data received by the server.

[1046] Step 5: Parsing the message

[1047] The server analyzes the received message using a natural language processing engine (NLP).

[1048] Input: Received chat messages.

[1049] How it works: The NLP engine analyzes the context of "tired."

[1050] Output: Analysis results (e.g., context information such as "tired").

[1051] Step 6: Identify the emotion

[1052] The server uses an emotion engine to identify the emotion of the message.

[1053] Input: Analysis results of the NLP engine.

[1054] How it works: The emotion engine identifies negative emotions based on "tired."

[1055] Output: Identified emotion information (e.g., "negative").

[1056] Step 7: Generate stamps

[1057] The server is a generative AI model that generates stamps using characters and images registered by the user.

[1058] Input: Identified emotion information and user registration information.

[1059] How it works: A generative AI model creates a "Tired Cat" stamp.

[1060] Output: The generated stamp image.

[1061] Step 8: Send your stamp

[1062] The server transmits the generated stamp image to the terminal.

[1063] Input: The generated stamp image.

[1064] Operation: The server sends the stamp image to the device.

[1065] Output: Stamp image data for display on the device.

[1066] Step 9: Propose and use stamps

[1067] The terminal displays the received stamps on the user interface and suggests them to the user.

[1068] Input: The stamp image sent to the device.

[1069] What it does: A stamp appears on the chat screen.

[1070] Output: A user interface for the user to review and send the stamp.

[1071] (Application example 2)

[1072] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1073] Conventional chat applications lack a mechanism for automatically suggesting appropriate stickers based on the user's emotions or the flow of the conversation, forcing users to select stickers themselves. Furthermore, the emotions and nuances that users want to convey through chat messages may not be conveyed properly, making personalized communication difficult. Furthermore, technology for understanding users' emotions and providing appropriate content in real time based on those emotions remains immature, and there is a need for improved accuracy.

[1074] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1075] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring conversation content and analyzing the conversation content and the user's emotions, means for generating images using the user's registered pictures and characters based on the conversation content and the user's emotions, and means for suggesting the generated images to the user. This enables more intuitive communication by automatically suggesting personalized stamps according to the user's emotions and the flow of the conversation.

[1076] A "user" is a person who uses the system to chat.

[1077] "Pictures and characters" are images or illustrations registered by users and used as stamps.

[1078] "Means of registration" refers to a method by which a user inputs and saves their favorite designs or characters into the system.

[1079] "Means for monitoring conversation content and analyzing the conversation content and user emotions" refers to the process of monitoring chat text data and understanding its content and emotions using natural language processing and emotion analysis techniques.

[1080] "Means for generating images using user-registered pictures and characters based on the content of the conversation and the user's emotions" refers to a technology that generates stamps using pre-registered pictures and characters, taking into account the context of the conversation and the user's emotions.

[1081] The "means for suggesting the generated image to the user" is a method for suggesting the generated stamp to the user and displaying it on the chat interface.

[1082] The "means for periodically sending chat content and analysis results to the server" is a mechanism for sending users' chat text and the analysis results to the server at regular intervals.

[1083] A "natural language processing engine" is software that analyzes text data and understands its context and meaning.

[1084] An "emotion analysis engine" is an algorithm or software that identifies a user's emotions from text data.

[1085] "Stamps" are images or icons used to convey emotions or messages within chat.

[1086] This invention is a system that automatically generates and suggests stamps that match the flow of conversation and emotions using the user's favorite designs and characters. Specifically, it can be implemented as follows.

[1087] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[1088] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server, which then analyzes the received data using a natural language processing engine (NLP) and a sentiment analysis engine. This analysis process identifies sentiments, such as negative, positive, or neutral, from the conversation and stores the results in a database.

[1089] Based on the analysis results, the server generates new stamps using the images and characters registered by the user. A generative AI model is used to generate stamps, so for example, if a user sends a message expressing a negative emotion such as "I'm tired," the server will generate a stamp that expresses a "tired look" using a "cat" character.

[1090] The generated stamps are sent from the server to the terminal, and the terminal displays the received stamps on the user interface and suggests them to the user. The user can then check the suggested stamps and send them in chat.

[1091] The hardware used is a smartphone (e.g., iPhone, Android device), and the software can be Python, the Transformers library (Hugging Face), and a REST API (backend server).The natural language processing engine used is, for example, the Hugging Face NLP engine, and the Hugging Face model is used for emotion analysis.

[1092] As a specific example, if a user registers a "cat" and a "dog" character in the system and writes "I feel tired today" in a chat, the device will send the chat content to the server. The server will analyze the context of "tired" using an NLP engine and simultaneously identify "negative emotions" using an emotion engine. It will then use a generative AI model to generate a stamp that expresses a "tired appearance" according to the prompt "Generate a stamp showing cat feeling tired." The generated stamp will then be sent to the device and suggested to the user.

[1093] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[1094] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1095] Step 1:

[1096] The user uses a means for registering a favorite picture or character to input the desired character name or picture name through the interface of the application.

[1097] Input: Character name or picture name

[1098] Data processing: The terminal collects and formats input data

[1099] Output: Formatted user registration information

[1100] Step 2:

[1101] The terminal transmits the formatted user registration information to the server.

[1102] Input: Formatted user registration information

[1103] Data processing: Sending data from the device to the server

[1104] Output: User registration information stored on the server

[1105] Step 3:

[1106] Once a user starts chatting, the terminal periodically monitors the conversation and retrieves the latest messages.

[1107] Input: Chat message

[1108] Data processing: Collection and temporary storage of chat messages

[1109] Output: Latest chat messages

[1110] Step 4:

[1111] The latest chat message acquired by the device is sent to the server.

[1112] Input: Latest chat message

[1113] Data processing: Sending data from the device to the server

[1114] Output: Chat messages sent to the server

[1115] Step 5:

[1116] The server analyzes the chat messages it receives using a natural language processing engine (NLP) and sentiment analysis engine.

[1117] Input: Chat message

[1118] Data processing: Context analysis using NLP and sentiment analysis using a sentiment analysis engine

[1119] Output: Analysis results (emotions and contextual information)

[1120] Step 6:

[1121] Based on the analysis results, the server uses a generative AI model to generate new stamps based on the images and characters registered by the user.

[1122] Input: Analysis results (emotions and context information), user registration information

[1123] Data processing: Prompt sentence generation and stamp generation using generative AI models

[1124] Output: Generated stamp (image data)

[1125] As a concrete example, a stamp is generated according to the prompt sentence "Generate a stamp showing a cat feeling tired."

[1126] Step 7:

[1127] The server sends the generated stamp to the terminal.

[1128] Input: Generated stamp (image data)

[1129] Data processing: Sending generated stamp data

[1130] Output: Generated stamp sent to the device

[1131] Step 8:

[1132] The terminal displays the received stamp on the user interface and suggests it to the user.

[1133] Input: Generated stamp (image data)

[1134] Data processing: Stamp display in user interface

[1135] Output: Stamps suggested to the user

[1136] Step 9:

[1137] The user can check the suggested stamps and send them via chat if necessary.

[1138] Input: Suggested stamp

[1139] Data processing: Selecting and sending stamps

[1140] Output: Stickers sent in chat

[1141] In this way, a system is constructed that automatically generates appropriate stamps and suggests them to users based on the user's emotions and conversation content.

[1142] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1143] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1144] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1145] [Fourth embodiment]

[1146] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1147] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1148] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1149] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1150] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1151] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1152] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1153] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1154] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1155] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1156] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1157] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1158] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1159] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. The program processing is explained in detail below.

[1160] First, the user performs initial setup to use the system. The user registers their favorite designs and characters in the system, and this information is sent from the terminal to the server. The server stores the received information in a database and holds data according to the preferences of each individual user.

[1161] Once a user starts chatting, the device periodically monitors the conversation and sends the data to the server, which then analyzes the received conversation using a natural language processing engine to determine whether a stamp matching the conversation is needed.

[1162] If a stamp is deemed necessary, the server uses image generation AI to generate a new stamp based on the analysis results and the user's registered favorite designs and characters. The generated stamp is then sent from the server to the device.

[1163] The device displays the received stamps on the user interface and suggests them to the user. By sending the suggested stamps, the user can communicate more uniquely and expressively.

[1164] Specific examples

[1165] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp using the "cat" registered by the user to express a "tired look." The generated stamp is then suggested to the user, who then sends it in the chat.

[1166] This allows users to use a variety of stamps featuring their favorite characters and send emotionally rich messages to others.The purpose of the present invention is to improve the user's communication experience by generating stamps based on the user's preferences and the context of the conversation.

[1167] The processing flow will be explained below.

[1168] Step 1:

[1169] Users use the system to register their favorite designs and characters, and then input the names of each character and design through the application interface.

[1170] Step 2:

[1171] The device acquires the user's input information and sends it to the server, including the user ID and registered image and character information.

[1172] Step 3:

[1173] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[1174] Step 4:

[1175] A user initiates a chat. The user chats by sending regular messages.

[1176] Step 5:

[1177] The device will periodically monitor the chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[1178] Step 6:

[1179] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[1180] Step 7:

[1181] The server uses a natural language processing engine (NLP) to analyze the chat content it receives, and determines whether the message needs an appropriate stamp based on its intent and sentiment.

[1182] Step 8:

[1183] If the server determines that a stamp needs to be generated based on the results of the NLP analysis, it obtains information about the design and character registered by the user.

[1184] Step 9:

[1185] The server activates the image generation AI and generates a stamp image based on the user's preferences and the results of NLP analysis. For example, it generates a "cat" stamp with the theme of "tired."

[1186] Step 10:

[1187] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[1188] Step 11:

[1189] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it if necessary.

[1190] Example 1

[1191] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1192] In conventional chat systems, users had to manually select stamps, making it difficult to quickly use appropriate stamps that fit the flow of the conversation. Furthermore, because users had to choose from existing stamps, it was difficult to easily find stamps that matched individual users' preferences and the context of the conversation. This limited the user's communication experience and made it difficult to have expressive conversations.

[1193] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1194] In this invention, the server includes means for users to register their favorite images and characters, means for monitoring and analyzing conversation content, means for generating stamps using the images and characters registered by the user based on the conversation content, and means for suggesting the generated stamps to the user. This enables users to quickly use unique and appropriate stamps in line with the flow of conversation, enabling more expressive communication.

[1195] A "user" is an entity that uses the system to register their favorite images and characters and chat.

[1196] "Images and characters" are visual representations such as favorite pictures and characters that users register in the system.

[1197] The "means for monitoring the content of conversations" is a function for continuously observing the content of chat conversations and extracting important information.

[1198] "Means for analyzing conversation content" refers to a function that uses natural language processing technology to analyze the conversation content of a user and understand their emotions and intentions.

[1199] A "stamp" is a type of image generated by the system to visually express the user's emotions and intentions.

[1200] "Means of generation" is a function that creates stamps suitable for conversation based on registered images and characters and analysis results.

[1201] The "means of suggesting" is a function of displaying the generated stamp to the user and encouraging them to use it.

[1202] A "server" is a central computer that processes and manages data for the entire system.

[1203] A "natural language processing engine" is a software technology that analyzes text data and understands its meaning and emotions.

[1204] The "generative AI model" is an artificial intelligence model that generates new stamps based on user registration information and conversation analysis results.

[1205] The present invention is a system that automatically generates and suggests stamps that match the flow of conversation using images or characters that users like. The specific processing of this system is as follows.

[1206] First, the user performs initial setup to use the system. The user registers their favorite images and characters in the system. This information is sent from the user's device to the server. The server stores the received information in a database and holds data according to the preferences of each individual user. This data includes image data and keywords uploaded by the user.

[1207] Next, when a user starts chatting, the device periodically monitors the conversation content and sends the data to the server. The server then analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and determines whether a stamp appropriate for the conversation is needed. The natural language processing engine extracts emotions and intentions from the user's conversation content and performs analysis based on this.

[1208] If a stamp is deemed necessary, the server uses the analysis results and the user's favorite images and characters to generate a new stamp using an image generation AI (e.g., a standard image generation algorithm). The image generation AI is provided with a prompt that reflects the user's preferences and the context of the conversation.

[1209] As a concrete example, consider a case where a user has registered "cats" and "dogs" as their favorite characters in the system. When the user writes "I'm tired today" in a chat, the device sends the chat content to the server. Based on the conversation content "tired," the server generates a stamp that expresses "tiredness" using the "cat" registered by the user. The generated stamp is then suggested to the user, who then sends the stamp in the chat.

[1210] An example prompt is:

[1211] "The user's favorite character is a cat. The conversation is, 'I'm tired today.' Please generate an image of a cat that expresses this conversation."

[1212] This system allows users to use a variety of stamps featuring their favorite characters and send emotive messages to others. As described above, the purpose of this invention is to enrich users' communication experiences by automatically generating and suggesting stamps that match the user's preferences and the context of the conversation.

[1213] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1214] Step 1:

[1215] Users register their favorite images and characters.

[1216] Specific behavior:

[1217] The user opens the system's settings screen and registers their favorite image or character by entering the image file and keywords.

[1218] Input: Image files and keywords entered by the user

[1219] Output: Registration information is generated

[1220] Step 2:

[1221] The terminal sends the registration information to the server.

[1222] Specific behavior:

[1223] The terminal (for example, a smartphone or PC) sends the data entered by the user to the server using the HTTPS protocol.

[1224] Input: Image files and keywords registered by the user

[1225] Output: Registration information sent to the server

[1226] Step 3:

[1227] The server stores the information in a database.

[1228] Specific behavior:

[1229] The server analyzes the received data, organizes it by user, and stores it in a database (e.g., MySQL).

[1230] Input: Registration information received by the server

[1231] Output: User registration information saved in the database

[1232] Step 4:

[1233] The device monitors the conversation.

[1234] Specific behavior:

[1235] When a user launches a chat app and starts a conversation, the device monitors messages entered every few seconds.

[1236] Input: The chat message the user types

[1237] Output: Monitored conversation

[1238] Step 5:

[1239] The terminal transmits the conversation data to the server.

[1240] Specific behavior:

[1241] The device sends text data of the conversation to the server at regular intervals, and this communication also uses the HTTPS protocol.

[1242] Input: Monitored conversation

[1243] Output: Conversation data sent to the server

[1244] Step 6:

[1245] The server analyzes the conversation using a natural language processing engine.

[1246] Specific behavior:

[1247] The server analyzes the received conversation content using a natural language processing engine (e.g., a general text analysis engine) and extracts emotions and intentions from the text.

[1248] Input: Conversation data sent to the server

[1249] Output: Analyzed conversation content (emotions, intentions, etc.)

[1250] Step 7:

[1251] The server determines whether a stamp is needed.

[1252] Specific behavior:

[1253] Based on the results of the natural language processing engine, the server uses logic to determine whether a stamp is necessary depending on the context and emotion of the conversation.

[1254] Input: Parsed conversation

[1255] Output: Result of stamp necessity determination

[1256] Step 8:

[1257] The server generates stamps using image generation AI.

[1258] Specific behavior:

[1259] If it is determined that a stamp is necessary, the server sends a prompt message to the image generation AI based on the analysis results and the image or character registered by the user. The image generation AI then generates a stamp based on the prompt message.

[1260] Input: The result of determining the necessity of the stamp, the image or character registered by the user, and the prompt text

[1261] Output: Generated stamp image

[1262] Step 9:

[1263] The server sends the generated stamp to the terminal.

[1264] Specific behavior:

[1265] The server sends the generated stamp image to the terminal. The image data is sent in binary format.

[1266] Input: Generated stamp image

[1267] Output: Stamp image sent to the device

[1268] Step 10:

[1269] The terminal displays the stamp on the user interface.

[1270] Specific behavior:

[1271] The terminal decodes the received stamp and displays it on the user interface.

[1272] Input: Stamp image sent to the device

[1273] Output: Stamp displayed in the user interface

[1274] Step 11:

[1275] A user sends a stamp in a chat.

[1276] Specific behavior:

[1277] The user checks the suggested stamp and presses the send button to send the stamp to the chat partner.

[1278] Input: Stamp displayed in the user interface

[1279] Output: Stamp sent to chat partner

[1280] (Application example 1)

[1281] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1282] Conventional electronic payment services have had the problem that communication between users is monotonous and does not allow for rich expression of emotions or intentions. The present invention aims to solve this problem and improve the user experience in electronic payment services.

[1283] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1284] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring and analyzing conversation content, means for generating images using the user's registered pictures and characters based on the conversation content, means for suggesting the generated images to the user, means for including a chat function when the user transfers money or makes a payment, and means for automatically generating and suggesting stamps suitable for the chat function. This allows for richer communication during electronic payments and makes it possible to accurately convey the user's feelings and intentions.

[1285] - "Pictures and characters" refers to graphics, images, or specific characters that a user registers as their preference.

[1286] "Conversation content" refers to the text content exchanged via chat or messaging apps.

[1287] "Monitoring" means the process of watching and recording conversations in real time.

[1288] "Means of analysis" refers to the technical means of analyzing the content of a conversation and understanding its intentions and emotions.

[1289] "Means for generating images" refers to the process of creating new images using pictures and characters registered by the user based on the content of the conversation.

[1290] "Means for suggesting" refers to a process for suggesting generated images to a user.

[1291] "Chat function" refers to a means for users to exchange text messages with each other.

[1292] "Means for automatic generation and suggestion" means a process for automatically generating stamps based on the content of a user's chat and suggesting them to the user.

[1293] A "natural language processing engine" refers to software that analyzes text data and understands its meaning and sentiment.

[1294] "Electronic payment service" refers to a system for transferring money and making payments via the Internet.

[1295] The present invention provides a system that allows users to realize emotional communication through electronic payment services. Specific embodiments of the system are described below.

[1296] The system mainly consists of a user terminal, a server, a natural language processing engine, and an image generation AI.

[1297] First, the user performs the initial setup and registers their favorite designs and characters in the system. The information the user enters into the device is sent to the server and saved in a database. This is used to record the user's preferences.

[1298] Next, when a user transfers or makes a payment as part of an electronic payment service, they use the chat function. The chat content is periodically sent to a server. The server then analyzes the received conversation content using a natural language processing engine. This analysis allows the intent and sentiment of the conversation to be understood. The natural language processing engine uses software such as Hugging Face's Transformers.

[1299] Furthermore, based on the analysis results and the images and characters registered by the user, the server uses image generation AI to generate stamps. The generated stamps are based on the content of the user's chat and are intended to enrich the conversation. A specific generation AI model is used for the image generation AI.

[1300] The generated stamps are instantly sent to the user's device and displayed on the user interface. Users can easily send emotive messages by sending them in chats.

[1301] Specific examples

[1302] For example, if a user has registered "cats" and "dogs" as their favorite characters in the system, when they write "I'm tired today" in a chat, the chat content is sent to the server. The server analyzes the emotion "tired" using a natural language processing engine and generates a stamp of a cat that shows "tiredness." The generated stamp is then suggested to the user, who can then send it in the chat.

[1303] Prompt Sentence Examples

[1304] Based on the text below, please create a stamp image of a cat character that matches the emotion.

[1305] Text: "I'm tired today"

[1306] Character: "Cat"

[1307] Emotion: "Tired"

[1308] Thus, the present invention aims to improve the user's communication experience by generating stamps according to the user's preferences and conversation context.

[1309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1310] Step 1:

[1311] The user registers their favorite designs and characters on the device. The device receives the registration information (designs and characters) entered by the user and sends it to the server. The server stores the received information in a database. This records the user's preferences and makes them available for later processing.

[1312] Input: Images or characters registered by the user

[1313] Data processing: The terminal converts the input information into a structured data format.

[1314] Output: Structured image and character data is sent to the server

[1315] Step 2:

[1316] A user uses the chat function within an electronic payment service to have a conversation. The device periodically sends the contents of the conversation to the server. The server receives the content and analyzes it using a natural language processing engine. The analysis results identify the intent and sentiment of the conversation.

[1317] Input: User chat content

[1318] Data processing: The device sends chat content in text format to the server

[1319] Output: The conversation is analyzed by a natural language processing engine to obtain sentiment and intent data.

[1320] Step 3:

[1321] The server uses the analyzed conversation content and the images and character information registered by the user to generate stamps using an image generation AI model. Specifically, it inputs prompts according to emotions and intentions into the generation AI model to generate stamp images.

[1322] Input: Analyzed emotions and intentions, registered images and characters

[1323] Data processing: The server generates prompts according to emotions and intentions, and inputs them into the image generation AI model.

[1324] Output: A new stamp image is generated based on the prompt.

[1325] Step 4:

[1326] The generated stamp image is sent from the server to the device. The device receives this information and displays it on the user interface. The user can then check the proposed stamp and send it to the chat if necessary.

[1327] Input: Generated stamp image

[1328] Data processing: The server sends the stamp image data to the terminal.

[1329] Output: The stamp image is displayed on the device's user interface.

[1330] Step 5:

[1331] Users can send suggested stamps in chat, which allows for richer emotional expression and enriches conversations.

[1332] Input: User confirmation and stamp sending operation

[1333] Data processing: The device receives the user's sending operation and inserts the stamp image into the chat.

[1334] Output: The stamp is sent in the chat and reflected in the conversation.

[1335] In this way, the system of the present invention generates and suggests emotive stamps in real time based on the user's registration information and conversational context, improving the user experience in electronic payment services.

[1336] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1337] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using the user's favorite designs and characters. Furthermore, by combining this system with an emotion engine that recognizes the user's emotions, it is possible to suggest more appropriate stamps. The program processing is explained in detail below.

[1338] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[1339] Next, when a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server. The server then analyzes the received data using a natural language processing engine (NLP) and simultaneously analyzes the user's emotional state using an emotion engine. This analysis process identifies emotions such as positive, negative, and neutral from the conversation content.

[1340] Based on the analysis results of the emotion engine and NLP, the server generates new stamps based on the images and characters registered by the user. For example, if a user sends a message expressing the negative emotion of "tired," the server will generate a stamp expressing "tiredness" using a "cat" character.

[1341] The generated stamps are sent from the server to the device. The device displays the received stamps on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[1342] Specific examples

[1343] For example, suppose a user registers "cats" and "dogs" as their favorite characters in the system. If the user writes "I'm tired today" in a chat, the device sends the chat content to the server. The server analyzes the context of "tired" using an NLP engine and simultaneously identifies "negative" emotions using an emotion engine. It then generates stamps that express the feeling of "tired" using the "cat" registered by the user. The generated stamps are then sent to the device and suggested to the user. The user can continue chatting using the suggested stamps.

[1344] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[1345] The processing flow will be explained below.

[1346] Step 1:

[1347] Users use the system to register their favorite designs and characters, and then input the desired character name or design name through the application interface.

[1348] Step 2:

[1349] The device receives the user's input information and sends it to the server, including the user ID and registered image and character information.

[1350] Step 3:

[1351] The server processes the received data and stores it in the database. The server analyzes the request and inserts the user preference information into the corresponding database table.

[1352] Step 4:

[1353] A user starts a chat. The user communicates with other users by sending messages as usual.

[1354] Step 5:

[1355] Your device will periodically monitor chat content and retrieve the latest messages. For example, you can set it to check for new messages every second.

[1356] Step 6:

[1357] The device sends the latest chat content it has acquired to the server, including the user ID and the conversation text.

[1358] Step 7:

[1359] The server uses a natural language processing engine (NLP) to analyze the chat content received and understand the user's intentions and emotions. At the same time, it uses an emotion engine to analyze the user's emotional state and identify whether it is positive, negative, or neutral.

[1360] Step 8:

[1361] If the server determines that a stamp needs to be generated based on the analysis results of the NLP and emotion engine, it obtains information about the design and character registered by the user.

[1362] Step 9:

[1363] The server activates the image generation AI and generates stamp images based on the user's preferences, conversation analysis results, and emotional state. For example, it generates a "cat" stamp based on the theme of "tired."

[1364] Step 10:

[1365] The server sends the generated stamp image to the terminal, which includes the user ID and the generated stamp image.

[1366] Step 11:

[1367] The terminal displays the received stamp image on the user interface and suggests it to the user. The user can check the suggested stamp and send it via chat if necessary.

[1368] Example 2

[1369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1370] When users chat, it takes time and effort to select and use appropriate stamps that match the content of the conversation and their own emotions. Furthermore, the stress and time required to search and select the appropriate stamps each time can be a problem. To solve this problem, a system is needed that can automatically generate and suggest stamps that match the user's emotions and conversation content.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1372] In this invention, the server includes a means for registering characters and images that users like, a means for monitoring and analyzing conversation content, and a means for analyzing conversation content and emotions and generating images using the characters and images registered by the user. This allows users to be quickly and appropriately suggested stamps based on the chat content and emotions.

[1373] "User" refers to someone who uses the system to register their favorite characters or images and chat.

[1374] "Character" refers to visual elements such as characters, pictures, and images that users can register in the system.

[1375] "Picture" refers to a specific design or image that a user can register in the system.

[1376] "Conversation content" refers to text messages and statements entered into the chat by users.

[1377] "Monitoring" refers to the act of the terminal monitoring the user's chat content at regular intervals and obtaining the latest messages.

[1378] "Analyzing" refers to the act of analyzing the content and emotions of messages acquired by the server using a natural language processing engine or emotion engine.

[1379] "Emotion engine" refers to software or algorithms for identifying a user's emotion from the text of a message.

[1380] A "natural language processing engine" refers to software or algorithms that analyze the text of a conversation and understand its meaning and intent.

[1381] "Generative AI model" refers to an artificial intelligence model for generating new stamp images based on analyzed data.

[1382] "Generating an image" refers to the act of creating an image such as a stamp using a specified character or design based on the analysis results.

[1383] "Suggesting" refers to the act of displaying the generated image or stamp to the user and recommending its use.

[1384] "Terminal" refers to a device used by a user to chat, such as a smartphone or PC.

[1385] "Server" refers to a computer system that receives data sent by users and performs analysis and image generation.

[1386] "Database" refers to a data management system for storing information on characters and designs registered by users.

[1387] "Stamps" refer to images or emoticons that are generated based on the user's emotions or conversation content.

[1388] This invention is a system that automatically generates and suggests stamps that match the flow of conversation using characters and images that users like. Furthermore, by combining it with an emotion engine, it is possible to suggest even more appropriate stamps.

[1389] First, the user opens the application interface and registers the name of their favorite character or design. The registered information is sent from the device to the server, which then stores this data in a database.

[1390] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. The retrieved messages are then sent to the server, which uses a natural language processing engine (NLP) to analyze the message content and an emotion engine to identify the user's emotions. This analysis process identifies emotions such as positive, negative, and neutral from the message content.

[1391] Based on the analysis results, the server generates new stamps using characters and images registered by the user. Using a generative AI model, stamps are created based on emotions and the content of the conversation. For example, if a user sends a message saying, "I'm tired today," the server analyzes the emotion "tired" using an NLP engine and identifies it as a negative emotion using an emotion engine. Based on this, a stamp expressing a "tired look" is generated using a "cat" registered by the user.

[1392] The generated stamps are sent from the server to the terminal, and the terminal displays them on the user interface and suggests them to the user. The user can check the suggested stamps and send them in chats as needed.

[1393] Specific examples

[1394] Let us take as an example the process when a user registers "cat" and "dog" as their favorite characters in the system.

[1395] If a user types "I'm tired today" in a chat, the device sends the chat content to the server. The server uses an NLP engine to analyze the context of "tired" and an emotion engine to identify "negative emotions." It then generates a stamp that expresses the feeling of "tired" using the "cat" registered by the user. The generated stamp is then sent to the device and suggested to the user. The user can continue chatting using the suggested stamp.

[1396] Prompt Sentence Examples

[1397] The following prompt will generate a stamp using a generative AI model:

[1398] "Brief description: A system that allows users to register their favorite characters and automatically generates and suggests stamps based on what they say in chat. It combines an emotion engine and natural language processing to provide stamps that reflect the user's emotions. For example, if a user types "I'm tired today," a stamp expressing "tiredness" will be generated using the "cat" character registered by the user."

[1399] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1400] Step 1: User character registration

[1401] Users open the application interface, enter the name of their favorite character or image, and register it.

[1402] Input: Character name or picture name entered by the user.

[1403] Operation: The user enters the character name "cat" and the picture name "dog" into the input form and presses the registration button.

[1404] Output: The terminal sends the entered information to the server.

[1405] Step 2: Save your registration information

[1406] The server receives the character names and design names sent from the terminal and stores them in a database.

[1407] Input: Character name or picture name sent from the device.

[1408] Operation: The server stores the received data in a database.

[1409] Output: Character and picture information stored in the database.

[1410] Step 3: Monitoring the chat

[1411] A user initiates a chat.

[1412] The device periodically monitors the chat content and retrieves the latest messages.

[1413] Input: The chat message typed by the user.

[1414] Operation: Periodically checks and retrieves messages entered on the chat screen, such as "I'm tired today."

[1415] Output: The most recent message data.

[1416] Step 4: Sending the message to the server

[1417] The terminal transmits the acquired chat message to the server.

[1418] Input: The most recent chat message.

[1419] Action: The device sends a message to the server.

[1420] Output: The message data received by the server.

[1421] Step 5: Parsing the message

[1422] The server analyzes the received message using a natural language processing engine (NLP).

[1423] Input: Received chat messages.

[1424] How it works: The NLP engine analyzes the context of "tired."

[1425] Output: Analysis results (e.g., context information such as "tired").

[1426] Step 6: Identify the emotion

[1427] The server uses an emotion engine to identify the emotion of the message.

[1428] Input: Analysis results of the NLP engine.

[1429] How it works: The emotion engine identifies negative emotions based on "tired."

[1430] Output: Identified emotion information (e.g., "negative").

[1431] Step 7: Generate stamps

[1432] The server is a generative AI model that generates stamps using characters and images registered by the user.

[1433] Input: Identified emotion information and user registration information.

[1434] How it works: A generative AI model creates a "Tired Cat" stamp.

[1435] Output: The generated stamp image.

[1436] Step 8: Send your stamp

[1437] The server transmits the generated stamp image to the terminal.

[1438] Input: The generated stamp image.

[1439] Operation: The server sends the stamp image to the device.

[1440] Output: Stamp image data for display on the device.

[1441] Step 9: Propose and use stamps

[1442] The terminal displays the received stamps on the user interface and suggests them to the user.

[1443] Input: The stamp image sent to the device.

[1444] What it does: A stamp appears on the chat screen.

[1445] Output: A user interface for the user to review and send the stamp.

[1446] (Application example 2)

[1447] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1448] Conventional chat applications lack a mechanism for automatically suggesting appropriate stickers based on the user's emotions or the flow of the conversation, forcing users to select stickers themselves. Furthermore, the emotions and nuances that users want to convey through chat messages may not be conveyed properly, making personalized communication difficult. Furthermore, technology for understanding users' emotions and providing appropriate content in real time based on those emotions remains immature, and there is a need for improved accuracy.

[1449] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1450] In this invention, the server includes means for registering users' favorite pictures and characters, means for monitoring conversation content and analyzing the conversation content and the user's emotions, means for generating images using the user's registered pictures and characters based on the conversation content and the user's emotions, and means for suggesting the generated images to the user. This enables more intuitive communication by automatically suggesting personalized stamps according to the user's emotions and the flow of the conversation.

[1451] A "user" is a person who uses the system to chat.

[1452] "Pictures and characters" are images or illustrations registered by users and used as stamps.

[1453] "Means of registration" refers to a method by which a user inputs and saves their favorite designs or characters into the system.

[1454] "Means for monitoring conversation content and analyzing the conversation content and user emotions" refers to the process of monitoring chat text data and understanding its content and emotions using natural language processing and emotion analysis techniques.

[1455] "Means for generating images using user-registered pictures and characters based on the content of the conversation and the user's emotions" refers to a technology that generates stamps using pre-registered pictures and characters, taking into account the context of the conversation and the user's emotions.

[1456] The "means for suggesting the generated image to the user" is a method for suggesting the generated stamp to the user and displaying it on the chat interface.

[1457] The "means for periodically sending chat content and analysis results to the server" is a mechanism for sending users' chat text and the analysis results to the server at regular intervals.

[1458] A "natural language processing engine" is software that analyzes text data and understands its context and meaning.

[1459] An "emotion analysis engine" is an algorithm or software that identifies a user's emotions from text data.

[1460] "Stamps" are images or icons used to convey emotions or messages within chat.

[1461] This invention is a system that automatically generates and suggests stamps that match the flow of conversation and emotions using the user's favorite designs and characters. Specifically, it can be implemented as follows.

[1462] First, the user uses the system to register their favorite designs and characters. The user inputs the desired character name or design name through the application interface. This information is sent from the device to the server, which receives the data and stores it in a database.

[1463] Next, once a user starts chatting, the device periodically monitors the conversation and retrieves the latest messages. These messages are then sent to the server, which then analyzes the received data using a natural language processing engine (NLP) and a sentiment analysis engine. This analysis process identifies sentiments, such as negative, positive, or neutral, from the conversation and stores the results in a database.

[1464] Based on the analysis results, the server generates new stamps using the images and characters registered by the user. A generative AI model is used to generate stamps, so for example, if a user sends a message expressing a negative emotion such as "I'm tired," the server will generate a stamp that expresses a "tired look" using a "cat" character.

[1465] The generated stamps are sent from the server to the terminal, and the terminal displays the received stamps on the user interface and suggests them to the user. The user can then check the suggested stamps and send them in chat.

[1466] The hardware used is a smartphone (e.g., iPhone, Android device), and the software can be Python, the Transformers library (Hugging Face), and a REST API (backend server).The natural language processing engine used is, for example, the Hugging Face NLP engine, and the Hugging Face model is used for emotion analysis.

[1467] As a specific example, if a user registers a "cat" and a "dog" character in the system and writes "I feel tired today" in a chat, the device will send the chat content to the server. The server will analyze the context of "tired" using an NLP engine and simultaneously identify "negative emotions" using an emotion engine. It will then use a generative AI model to generate a stamp that expresses a "tired appearance" according to the prompt "Generate a stamp showing cat feeling tired." The generated stamp will then be sent to the device and suggested to the user.

[1468] In this way, the present invention can provide a more personalized chat experience based on the user's emotions and conversation content.

[1469] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1470] Step 1:

[1471] The user uses a means for registering a favorite picture or character to input the desired character name or picture name through the interface of the application.

[1472] Input: Character name or picture name

[1473] Data processing: The terminal collects and formats input data

[1474] Output: Formatted user registration information

[1475] Step 2:

[1476] The terminal transmits the formatted user registration information to the server.

[1477] Input: Formatted user registration information

[1478] Data processing: Sending data from the device to the server

[1479] Output: User registration information stored on the server

[1480] Step 3:

[1481] Once a user starts chatting, the terminal periodically monitors the conversation and retrieves the latest messages.

[1482] Input: Chat message

[1483] Data processing: Collection and temporary storage of chat messages

[1484] Output: Latest chat messages

[1485] Step 4:

[1486] The latest chat message acquired by the device is sent to the server.

[1487] Input: Latest chat message

[1488] Data processing: Sending data from the device to the server

[1489] Output: Chat messages sent to the server

[1490] Step 5:

[1491] The server analyzes the chat messages it receives using a natural language processing engine (NLP) and sentiment analysis engine.

[1492] Input: Chat message

[1493] Data processing: Context analysis using NLP and sentiment analysis using a sentiment analysis engine

[1494] Output: Analysis results (emotions and contextual information)

[1495] Step 6:

[1496] Based on the analysis results, the server uses a generative AI model to generate new stamps based on the images and characters registered by the user.

[1497] Input: Analysis results (emotions and context information), user registration information

[1498] Data processing: Prompt sentence generation and stamp generation using generative AI models

[1499] Output: Generated stamp (image data)

[1500] As a concrete example, a stamp is generated according to the prompt sentence "Generate a stamp showing a cat feeling tired."

[1501] Step 7:

[1502] The server sends the generated stamp to the terminal.

[1503] Input: Generated stamp (image data)

[1504] Data processing: Sending generated stamp data

[1505] Output: Generated stamp sent to the device

[1506] Step 8:

[1507] The terminal displays the received stamp on the user interface and suggests it to the user.

[1508] Input: Generated stamp (image data)

[1509] Data processing: Stamp display in user interface

[1510] Output: Stamps suggested to the user

[1511] Step 9:

[1512] The user can check the suggested stamps and send them via chat if necessary.

[1513] Input: Suggested stamp

[1514] Data processing: Selecting and sending stamps

[1515] Output: Stickers sent in chat

[1516] In this way, a system is constructed that automatically generates appropriate stamps and suggests them to users based on the user's emotions and conversation content.

[1517] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1518] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1519] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1520] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1521] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1522] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1523] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1524] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1525] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1526] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1527] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1528] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1529] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1530] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1531] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1532] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1533] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1534] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1535] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1536] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1537] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1538] The following is further disclosed regarding the above embodiment.

[1539] (Claim 1)

[1540] A means for users to register their favorite designs and characters,

[1541] a means for monitoring and analyzing the content of the conversation;

[1542] A means for generating an image using a picture or character registered by a user based on the content of the conversation;

[1543] means for suggesting the generated image to a user;

[1544] A system including:

[1545] (Claim 2)

[1546] 10. The system of claim 1, further comprising: means for periodically transmitting chat contents of the user to the server.

[1547] (Claim 3)

[1548] 10. The system of claim 1, further comprising means for analyzing the conversation content using a natural language processing engine and generating appropriate stamps.

[1549] "Example 1"

[1550] (Claim 1)

[1551] A means for users to register their favorite images and characters,

[1552] means for monitoring and analyzing the content of the conversation;

[1553] A means for generating stamps using images or characters registered by the user based on the content of the conversation;

[1554] means for suggesting the generated stamp to a user;

[1555] A system including:

[1556] (Claim 2)

[1557] 10. The system of claim 1, further comprising means for periodically transmitting user interaction content to the server.

[1558] (Claim 3)

[1559] 10. The system of claim 1, further comprising means for analyzing the conversation content using a natural language processing engine and generating appropriate stamps using a generative AI model.

[1560] "Application Example 1"

[1561] (Claim 1)

[1562] A means for users to register their favorite designs and characters,

[1563] a means for monitoring and analyzing the content of the conversation;

[1564] A means for generating an image using a picture or character registered by a user based on the content of the conversation;

[1565] means for suggesting the generated image to a user;

[1566] A means for users to send and receive money, including chat functionality;

[1567] A means to automatically generate and suggest stamps suitable for chat functions,

[1568] A system including:

[1569] (Claim 2)

[1570] 10. The system of claim 1, further comprising: means for periodically transmitting chat contents of the user to the server.

[1571] (Claim 3)

[1572] 10. The system of claim 1, further comprising means for analyzing the conversation content using a natural language processing engine and generating appropriate stamps.

[1573] "Example 2: Combining Emotion Engines"

[1574] (Claim 1)

[1575] A means for users to register their favorite characters and designs,

[1576] a means for monitoring and analyzing the content of the conversation;

[1577] A means for analyzing the content of conversations and emotions and generating images using characters and pictures registered by the user;

[1578] means for suggesting the generated image to a user;

[1579] A system including:

[1580] (Claim 2)

[1581] 10. The system of claim 1, further comprising: means for periodically transmitting chat contents of the user to the server.

[1582] (Claim 3)

[1583] 10. The system of claim 1, further comprising means for analyzing speech content and sentiment using a natural language processing engine and a sentiment analysis engine to generate appropriate stamps.

[1584] "Application example 2 when combining emotion engines"

[1585] (Claim 1)

[1586] A means for users to register their favorite designs and characters,

[1587] means for monitoring the content of the conversation and analyzing the content of the conversation and the emotions of the user;

[1588] means for generating an image using a picture or character registered by a user based on the content of the conversation and the user's emotions;

[1589] means for suggesting the generated image to a user;

[1590] A system including:

[1591] (Claim 2)

[1592] 10. The system according to claim 1, further comprising means for periodically transmitting the user's chat contents and analysis results to the server.

[1593] (Claim 3)

[1594] 10. The system of claim 1, further comprising means for analyzing conversation content and user sentiment using a natural language processing engine and a sentiment analysis engine to generate appropriate stamps. [Explanation of symbols]

[1595] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to register their favorite designs and characters, a means for monitoring and analyzing the content of the conversation; A means for generating an image using a picture or character registered by a user based on the content of the conversation; means for suggesting the generated image to a user; A system including:

2. 2. The system of claim 1, further comprising means for periodically transmitting chat contents of users to a server.

3. The system of claim 1 further comprising means for analyzing the conversation content using a natural language processing engine and generating appropriate stamps.

Images