System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to enter text in a message input field and perform a send action; means for a terminal to capture the entered text and send a request to a server; means for the server to parse the received text and extract keywords and emotions; means for the server to search for relevant emoticons and emojis from a database based on the extracted keywords and emotions; means for the server to return a search result to the terminal; means for the terminal to parse the received data and display the emoticons and emojis on a user screen; and means for the user to select from among the presented emoticons and emojis.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When using messaging apps, the sheer number of stickers and emojis makes it difficult for users to quickly find specific stickers or emojis. This problem prevents users from effectively using newly purchased stickers and also causes sticker creators to see limited sales. A system that solves this problem and allows users to efficiently select appropriate stickers and emojis is needed. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including: means for a user to enter text into a message input field and perform a send action; means for a terminal to capture the entered text and send a request to a server; means for the server to analyze the received text and extract keywords and emotions; means for the server to search a database for relevant stamps and emoticons based on the extracted keywords and emotions; means for the server to return the search results to the terminal; means for the terminal to analyze the received data and display stamps and emoticons on the user screen; and means for the user to select from the presented stamps and emoticons.

[0006] "User" refers to a person who uses a messaging app to input text, select stamps or emojis, and send them.

[0007] "Device" means a computer system or device used by a User to operate a messaging app, including, but not limited to, a smartphone, tablet, or PC.

[0008] A "server" refers to a computer system that receives requests from a terminal, performs text analysis and database searches, and returns the results to the terminal.

[0009] "Text" refers to the string of characters that a user enters into the message input field.

[0010] A "send action" refers to the act of sending text entered by a user as a message.

[0011] "Captcha" refers to the act of capturing text entered by a user as data.

[0012] A "request" refers to a request for data sent from a terminal to a server.

[0013] "Analysis" refers to the process by which the server understands the meaning of the text it receives and extracts keywords and sentiment.

[0014] "Keywords" refer to specific words extracted through text analysis.

[0015] "Emotion" refers to the user's state of mind or emotion extracted through text analysis.

[0016] "Database" refers to the data storage system that stores stamp and emoji information.

[0017] "Search" refers to the act of finding a specific stamp or emoji from the database.

[0018] "Search Results" refers to a list of relevant stickers or emojis retrieved from a database.

[0019] "Return" refers to the act of the server sending the search results to the terminal.

[0020] "Display" refers to the act of visually presenting data received by a terminal on a user screen.

[0021] "Selection" refers to the act of the user choosing the appropriate stamp or emoji from the ones presented. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] This invention relates to a system and method for quickly suggesting relevant stamps and emojis based on the text entered by a user when using a messaging application. The system analyzes the text entered by the user and displays appropriate stamps and emojis based on the content of the text, thereby facilitating communication between users.

[0044] Specific flow of program processing

[0045] User text input

[0046] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[0047] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[0048] Text Analysis

[0049] The server receives the text sent from the device and passes it to the text analysis module. The text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract the keyword "understanding." It also performs sentiment analysis if necessary.

[0050] Search for stamps and emojis

[0051] The server searches the database for relevant stickers and emojis based on the analysis results. In this case, it searches for stickers and emojis related to the keyword "gotcha" (for example, a thumbs-up emoji or a sticker that says "gotcha").

[0052] Returning and displaying search results

[0053] The server organizes the search results and sends them to the terminal in list form.

[0054] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[0055] User Selection and Submission

[0056] The user selects an appropriate stamp or emoji from the displayed ones.

[0057] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0058] Specific examples

[0059] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0060] 1. The user types "Thank you for your hard work" and presses the send button.

[0061] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0062] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0063] 4. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[0064] 5. The server sends these search results to the device.

[0065] 6. The device displays the received stamp and emoji candidates on the user's screen.

[0066] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0067] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to lead to increased sales.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] The user enters the text "OK" in the message input field and presses the send button.

[0071] Specifically, the user enters text using the keyboard and taps the send button.

[0072] Step 2:

[0073] The terminal captures the entered text "OK" and sends a request to the server.

[0074] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[0075] Step 3:

[0076] The server forwards the text received from the user to a text analysis module for analysis.

[0077] Specifically, the server extracts text data from the request body and passes it to the analysis engine.

[0078] Step 4:

[0079] The server's text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract keywords and sentiment.

[0080] Specifically, the NLP engine tokenizes the text and applies keyword extraction algorithms and sentiment analysis models.

[0081] Step 5:

[0082] Based on the analysis results, the server searches the database for related stamps and emojis.

[0083] Specifically, it generates a database query and searches a database of stamps and emojis.

[0084] Step 6:

[0085] The server organizes the search results in list form and prepares to return them to the user terminal.

[0086] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[0087] Step 7:

[0088] The server sends the generated response to the user's terminal.

[0089] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[0090] Step 8:

[0091] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[0092] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[0093] Step 9:

[0094] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[0095] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[0096] Step 10:

[0097] The user selects the appropriate stamp or emoji from the displayed ones.

[0098] Specifically, the user taps to select the stamp or emoji they need.

[0099] Step 11:

[0100] The device inserts the selected stamp or emoji into the message box and performs the send action.

[0101] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[0102] Step 12:

[0103] The terminal transmits a message including the selected stamps and pictograms to the server, and normal message processing is performed.

[0104] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[0105] The process including the above steps allows a user to quickly and efficiently select appropriate stamps and emojis based on text input.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] Conventional messaging applications lack the functionality to appropriately suggest relevant stamps and emojis when users enter text. High-precision sentiment analysis based on the content of the text and rapid search and display of relevant stamps and emojis are also challenges. As a result, users have to spend time selecting stamps and emojis, which disrupts smooth communication.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for visual information and symbols related to the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling prompt and appropriate suggestions of stamps and emojis related to the text entered by the user, thereby facilitating smooth communication.

[0111] "User" refers to the entity that operates the message application.

[0112] A "message input field" is an interface for a user to input text.

[0113] "Means for performing a submit action" refers to an action or button for submitting text entered by the user to a server.

[0114] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[0115] "Capture" refers to the operation of a terminal acquiring data input by a user.

[0116] A "server" is a central computer system that receives data sent from a user's terminal and performs analysis and response.

[0117] "Analysis" refers to the operations or processes performed by the server to extract useful information from the text received.

[0118] "Keywords" refer to important words extracted from the input text.

[0119] "Sentiment" refers to the emotional state of the user extracted from the text.

[0120] "Visual information and symbols" refers to visual elements such as stamps and emojis.

[0121] A "database" refers to a repository of information that stores visual information and symbols.

[0122] "Search" refers to the operation of finding appropriate visual information or symbols from a database.

[0123] "User interface (UI)" refers to the screens and functions that are displayed on a terminal and that allow users to interact with it.

[0124] "Pop-up" refers to a supplementary window or menu that appears at a specific location in the UI.

[0125] "Means for displaying visual information and symbol recommendations" refers to the system's ability to suggest appropriate stamps and emojis to users.

[0126] This invention relates to a system and method for quickly suggesting relevant visual information and symbols based on the text entered by a user when using a messaging application. This system analyzes the text entered by the user and displays appropriate visual information and symbols based on the content of the text, thereby facilitating communication between users.

[0127] When a user enters text in the message input field and performs the send action, the device captures the entered text and sends a request to the server. The server analyzes the received text and extracts keywords and emotions. A natural language processing engine (e.g., OpenAI GPT-3) is used for this analysis. The server searches a database (e.g., MySQL or MongoDB) for relevant visual information and symbols based on the extracted keywords and emotions. The search results are organized and sent to the device in list form. The device analyzes the received data and displays candidate visual information and symbols in the user interface (UI). The user selects the appropriate one from the presented candidates and sends it as a message.

[0128] As a specific example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0129] 1. The user types "Thank you for your hard work" and presses the send button.

[0130] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0131] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0132] 4. The server searches its database for visual information or symbols related to "good work" (for example, a tired face emoji or a stamp that says "good work!").

[0133] 5. The server sends these search results to the device.

[0134] 6. The terminal displays the received visual information and symbol candidates on the user screen.

[0135] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0136] This system allows users to quickly and intuitively select appropriate visual information and symbols when entering text, making communication smoother. Furthermore, because visual information and symbols can be used effectively, it is expected that they will be actively used, including newly acquired ones. This will also benefit creators, and is expected to lead to increased sales.

[0137] Below are some example prompts to input to a generative AI model:

[0138] Please explain in detail the steps of the system's process for suggesting related stickers and emojis when a user types "Thank you." Please clearly state the actions of the server, device, and user, including the specific actions at each step.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] The user enters text into the message input field and presses the send button.

[0142] Specific behavior:

[0143] The user types "Got it" in the message application and performs a send action.

[0144] Input: The text entered by the user: "Got it."

[0145] Output: A submit action is triggered to proceed to the next step.

[0146] Step 2:

[0147] The terminal captures the entered text and sends a request to the server.

[0148] Specific behavior:

[0149] Logic within the messaging application is activated and captures the text "Got it" from the input field.

[0150] The captured data is securely sent to the server using encrypted communication such as SSL / TLS.

[0151] Input: The captured text "Got it."

[0152] Output: The request to the server and the captured text "Got it."

[0153] Step 3:

[0154] The server analyzes the received text and extracts keywords and sentiment.

[0155] Specific behavior:

[0156] The text data received on the server side is managed in JSON format or similar and passed to the analysis module.

[0157] A natural language processing (NLP) engine (e.g., OpenAI GPT-3) runs, understands the meaning of "understood," and extracts it as a keyword.

[0158] In emotion analysis, this is classified as a normal emotion.

[0159] Input: Received text "Got it."

[0160] Output: Extracted keyword "understand" and sentiment data.

[0161] Step 4:

[0162] The server searches a database for relevant visual information and symbols based on the extracted keywords and emotions.

[0163] Specific behavior:

[0164] A database is queried to extract visuals or symbols associated with "got it," such as a thumbs-up emoji or a stamp that says "got it."

[0165] Input: The extracted keyword "understand" and emotion data.

[0166] Output: A list of relevant visual information and symbols.

[0167] Step 5:

[0168] The server returns the search results to the terminal.

[0169] Specific behavior:

[0170] The search results are organized in a list format such as JSON or XML and sent back to the device.

[0171] Input: A list of relevant visual information or symbols.

[0172] Output: Organized list data.

[0173] Step 6:

[0174] The terminal analyzes the received data and displays visual information and symbol candidates on the user interface.

[0175] Specific behavior:

[0176] The list data received by the device is parsed and visual information and symbols are popped up in the messaging application UI in an appropriate display format.

[0177] Input: Received list data.

[0178] Output: Suggested visual information and symbols displayed in the user interface.

[0179] Step 7:

[0180] The user selects from the presented visual information and symbols.

[0181] Specific behavior:

[0182] Users can view a list of suggested visual information and symbols and select by tapping or clicking.

[0183] The selected visual information or symbol is inserted into the message box.

[0184] Input: Presented visual information and symbol candidates.

[0185] Output: The selected visual information or symbol is inserted into the message box.

[0186] Step 8:

[0187] The terminal inserts the visual information or symbols selected by the user into the message box and sends it as the final message.

[0188] Specific behavior:

[0189] The final message is constructed and the send action is performed, sending the selected stickers and emojis to a friend.

[0190] Input: Selected visual information or symbol.

[0191] Output: The final message sent.

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] While there have been systems that quickly suggest related stamps and emojis when a user inputs text, they lack the functionality to automatically suggest related products and promotions based on product reviews and comments. As a result, no efficient means has been provided to increase a user's purchasing motivation. The objective of the present invention is to provide a system that appropriately suggests related products and promotions to a user simply by inputting text, thereby increasing the user's purchasing motivation.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes means for analyzing text sent from the terminal, means for extracting keywords and emotions from the text, means for searching a database for related products and promotions based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling related products and promotions to be quickly suggested when a user inputs a product review or comment.

[0197] "User" refers to a person who uses the system to enter messages or post product reviews.

[0198] A "message input field" refers to an interface for a user to input text.

[0199] A "send action" refers to the operation of confirming text entered by the user and sending it from the terminal to the server.

[0200] A "terminal" is a device that is directly operated by a user, and refers to an electronic device such as a smartphone or tablet.

[0201] "Server" refers to a computing system that receives data sent from a terminal, analyzes it, and performs the necessary processing.

[0202] "Text" refers to the string of characters that a user enters into an input field.

[0203] "Captcha" refers to the act of saving entered text and sending it to a server for processing.

[0204] A "request" refers to data or a processing request sent from a terminal to a server.

[0205] "Analysis" refers to the process of breaking down text and extracting meaning and keywords.

[0206] "Keywords" refer to important words or phrases extracted from the analyzed text.

[0207] "Emotion" refers to the user's emotional state as interpreted from the text.

[0208] A "database" refers to a system that stores related information in an organized manner and allows it to be searched and extracted.

[0209] A "stamp" is an image or animation that can be attached to a message or review.

[0210] "Emoji" refers to small images or icons used alongside text.

[0211] "Related products" refers to products suggested based on the content of a user's reviews and comments.

[0212] "Promotion" refers to information about special offers or campaigns relating to a particular product.

[0213] "User screen" refers to an interface for displaying information to a user on a terminal.

[0214] "User interface" refers to the visual elements and tools that allow a user to interact with a system.

[0215] "Selection" refers to the action of a user choosing a particular sticker, emoji, related product, or promotion from among the options presented.

[0216] The present invention is a system that suggests related stamps, emoticons, related products, and promotions when a user enters text into a message input field or a product review input field and performs a send action. The server analyzes the received text, extracts keywords and emotions, and searches a database for related information to suggest it. The detailed configuration and functions required to realize this invention are described below.

[0217] Hardware and Software

[0218] Hardware:

[0219] Device: A smartphone or tablet operated by a user

[0220] Server: A high-performance computing system for text analysis and information retrieval.

[0221] software:

[0222] Natural Language Processing libraries: Google NLP API, spaCy, etc.

[0223] Database management systems: MySQL, MongoDB, etc.

[0224] Front-end framework: React Native, etc.

[0225] Messaging applications, shopping site applications

[0226] Data processing and calculation

[0227] Device:

[0228] When a user enters text into the message input field or product review input field, the device captures the text and sends it to the server. For example, if a user enters "This camera is very good and I'm very satisfied," the text data is sent to the server.

[0229] server:

[0230] 1. Text analysis: The server passes the received text to the text analysis module, which analyzes it using a natural language processing engine. Through analysis, keywords and sentiment are extracted from the text. For example, keywords such as "camera," "performance," and "satisfaction" and the sentiment "positive" are extracted.

[0231] 2. Related information search: Based on the extracted keywords and sentiment, the database is searched for related stickers, emojis, products, or promotions. Search results may include, for example, high-end cameras, related accessories, and special offers.

[0232] 3. Returning the results: The server organizes the search results and sends them to the terminal in a list format.

[0233] Device:

[0234] The device analyzes the received data and displays relevant stickers, emojis, products, or promotions in a user interface. The user selects the appropriate option from the presented options and inserts it into a message or proceeds to a product purchase page.

[0235] Examples of concrete examples and prompts

[0236] Examples:

[0237] For example, if a user writes in a review, "These shoes are very comfortable to wear and I like the design," the system will extract keywords such as "comfortable shoes" and "design" and suggest related insoles and shoes with a high design.

[0238] Example prompt sentence:

[0239] If a user writes, "These shoes are very comfortable and I love the design," provide examples of relevant promotions and product suggestions.

[0240] The above is an embodiment of the present invention. The present invention allows users to receive appropriate stamps, emoticons, related products, and promotions simply by inputting text, thereby improving communication and purchasing experiences.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Step 1:

[0243] The user enters text into the message input field or product review input field and performs a send action. The input is text data such as "This camera has very good performance and I am very satisfied."

[0244] Step 2:

[0245] The terminal captures the entered text and sends that data to the server as a request. The input is the text entered by the user, and the output is the data sent to the server.

[0246] Step 3:

[0247] The server passes the received text to the text analysis module, which analyzes the text using a natural language processing engine (such as Google NLP API or spaCy). The input is the received text, and the output is extracted keywords and sentiment data (e.g., "camera," "performance," "satisfied," sentiment: "positive").

[0248] Step 4:

[0249] The server searches a database (e.g., MySQL or MongoDB) for related stickers, emojis, products, or promotions based on the extracted keywords and emotions. The input is the extracted keywords and emotions, and the output is a list of related stickers, emojis, products, or promotions.

[0250] Step 5:

[0251] The server organizes the search results and sends them to the terminal in list form. The input is a list of related information, and the output is the data to be sent to the terminal.

[0252] Step 6:

[0253] The device analyzes the data received and displays the relevant stickers, emojis, products, or promotions in a user interface (e.g., React Native). The input is the data received from the server, and the output is the stickers, emojis, products, or promotions displayed in the user interface.

[0254] Step 7:

[0255] The user selects a sticker or emoji from the list and sends it as a message, or selects a related product or promotion and proceeds to the purchase page. The input is the options displayed in the user interface, and the output reflects the selected sticker, emoji, product, or promotion.

[0256] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0257] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[0258] Specific flow of program processing

[0259] User text input

[0260] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[0261] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[0262] Text Analysis

[0263] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[0264] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[0265] Emotion extraction

[0266] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. For example, a positive emotion is extracted from the text "I understand."

[0267] Search for stamps and emojis

[0268] Based on the analysis results, the server searches the database for relevant stamps and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions).

[0269] Returning and displaying search results

[0270] The server organizes the search results and sends them to the device in list form.

[0271] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[0272] User Selection and Submission

[0273] The user selects an appropriate stamp or emoji from the displayed ones.

[0274] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0275] Specific examples

[0276] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0277] 1. The user types "Thank you for your hard work" and presses the send button.

[0278] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0279] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0280] 4. The server's emotion engine extracts emotions such as fatigue and gratitude from the text "Thank you for your hard work."

[0281] 5. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[0282] 6. The server sends these search results to the device.

[0283] 7. The device displays the received stamp and emoji candidates on the user's screen.

[0284] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0285] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it is possible to provide stamps and emojis that best suit the user's emotions, resulting in more effective communication. And because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to increase sales.

[0286] The processing flow will be explained below.

[0287] Step 1:

[0288] The user enters the text "Thank you for your hard work" in the message input field and presses the send button.

[0289] Specifically, the user enters text using the keyboard and taps the send button.

[0290] Step 2:

[0291] The terminal captures the input text "Thank you for your hard work" and sends a request to the server.

[0292] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[0293] Step 3:

[0294] The server forwards the text received from the user to the text analysis module.

[0295] Specifically, the server extracts text data from the request body and passes it to a natural language processing (NLP) engine.

[0296] Step 4:

[0297] The server's text analysis module uses a natural language processing (NLP) engine to analyze "otsukaresama" and extract the keyword "otsukaresama."

[0298] Specifically, the NLP engine tokenizes the text and applies a keyword extraction algorithm.

[0299] Step 5:

[0300] The server's emotion engine extracts emotions from the text "Thank you for your hard work" based on the keywords extracted by the text analysis module.

[0301] Specifically, the emotion engine uses an emotion analysis algorithm to analyze the input text for emotions such as fatigue and relief.

[0302] Step 6:

[0303] The server searches a database for related stamps and emojis based on the extracted keywords and emotions.

[0304] Specifically, the system generates a database query and searches a database of stamps and emojis corresponding to feelings of fatigue and comfort.

[0305] Step 7:

[0306] The server organizes the search results in list form and prepares to send them to the user's terminal.

[0307] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[0308] Step 8:

[0309] The server sends the generated response to the user's terminal.

[0310] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[0311] Step 9:

[0312] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[0313] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[0314] Step 10:

[0315] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[0316] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[0317] Step 11:

[0318] The user selects from the displayed stamps and emojis.

[0319] Specifically, the user taps to select the appropriate stamp or emoji from the displayed list of candidates.

[0320] Step 12:

[0321] The device inserts the selected stamp or emoji into the message box and performs the send action.

[0322] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[0323] Step 13:

[0324] The terminal again transmits the message including the selected stamps and pictograms to the server, and normal message processing is performed.

[0325] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[0326] The process including the above steps allows users to quickly and efficiently select appropriate stickers and emojis based on text input, and the emotion engine provides stickers and emojis that best suit the user's emotions, resulting in more effective communication.

[0327] Example 2

[0328] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0329] In modern communication, attaching appropriate stickers and emojis to text messages is required to more effectively convey the emotion and nuance of the message. However, manually selecting appropriate stickers and emojis is time-consuming and inefficient. Furthermore, there is a lack of systems that instantly suggest stickers and emojis that match the content of the text, which can disrupt the flow of communication. There is a need for a system that can solve these problems and improve users' communication experience.

[0330] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0331] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for relevant stamps and emojis based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling the server to quickly suggest appropriate stamps and emojis for the text entered by the user.

[0332] A "user" is an individual who operates the system to input text and send messages using stamps and emoticons.

[0333] A "terminal" is an electronic device that allows a user to input, send, or receive text, and includes smartphones, computers, and the like.

[0334] The "server" is a central processing unit that manages the processing of the entire system, analyzes text sent from the terminals, and searches for stamps and emoticons.

[0335] A "message input field" is a UI element that allows a user to input text on a terminal.

[0336] A "send action" is an operation for sending text entered by a user from a terminal to a server.

[0337] "Capture" is the process by which a terminal captures entered text and sends it to a server.

[0338] A "request" is a communication in which a terminal requests a server to analyze data or search for a stamp.

[0339] "Text analysis" is the process by which the server analyzes the text data it receives and extracts keywords and context.

[0340] A "keyword" is an important word that has a particular meaning or sentiment in text analysis.

[0341] "Emotion" is information extracted from text that indicates a psychological state, such as positive or negative.

[0342] A "natural language processing engine" is a system that includes algorithms and models for performing text analysis.

[0343] The "emotion analysis module" is a part of the system that has the function of analyzing emotions from text and extracting the type of emotion.

[0344] A "database" is a storage device within the system where stamp and emoji information is stored.

[0345] "Searching" is the process by which the server searches the database for relevant stamps or emojis based on specific criteria.

[0346] The "user interface" refers to the screen configuration and elements that allow the terminal to display stamp and pictogram candidates to the user.

[0347] A "pop-up display" is a display method that appears temporarily to show appropriate stamps or emojis for the text entered by the user.

[0348] A "message box" is a UI element that displays the final message, including stamps and emojis selected by the user.

[0349] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[0350] User text input

[0351] First, the user enters text into the message input field and presses the send button. For example, the user enters the text "Got it" and sends it. The device used by the user at this time is a device running a messaging application, such as a smartphone or computer.

[0352] Sending text data

[0353] The terminal captures the text "OK" entered when the send button is pressed and transmits this data to the server via the Internet.

[0354] Text analytics

[0355] When the server receives the text sent from the device, it passes it to a text analysis module and an emotion engine. The server's text analysis module analyzes the text "I understand" using a natural language processing (NLP) engine (e.g., BERT or GPT model). During the analysis, the text is tokenized, common stop words are removed, and the keyword "I understand" is extracted.

[0356] Emotion extraction

[0357] Next, the emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. The emotion engine uses an emotion identification algorithm to extract positive emotion from the text "Got it."

[0358] Search for stamps and emojis

[0359] Based on the analysis results, the server searches the database for related stickers and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions), and searches for related stickers and emojis using a specific algorithm.

[0360] Submit search results

[0361] The server organizes the search results and sends them to the device in list format, along with metadata such as the display order.

[0362] Display in the user interface

[0363] The device analyzes the received data and displays candidate stickers and emojis in the user interface (UI). Specifically, a list of stickers and emojis is displayed as a pop-up near the message input field. This pop-up display is designed to allow users to quickly select stickers and emojis that are highly relevant to the text they have entered.

[0364] User Selection and Submission

[0365] Finally, the user selects an appropriate stamp or emoji from the displayed list, and the terminal inserts the selected stamp or emoji into the message box and sends it as the final message.

[0366] Specific examples

[0367] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0368] 1. The user types "Thank you for your hard work" and presses the send button.

[0369] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0370] 3. The server analyzes the received text and extracts the keyword "Otsukaresama." This keyword is extracted using a natural language processing engine.

[0371] 4. The server's emotion engine extracts emotions such as fatigue and relief from the text "Thank you for your hard work." The emotion engine analyzes the emotions using an emotion identification algorithm.

[0372] 5. The server searches the database for stamps or emojis related to "good work" (for example, a tired face emoji or a stamp that says "good work!"). The database is a storage device that stores information about stamps and emojis.

[0373] 6. The server organizes these search results and adds metadata such as display order when sending them to the device.

[0374] 7. The device analyzes the received stamp and emoji candidates and displays them on the user's screen. The user interface module pops up the candidates to make it easier for the user to select.

[0375] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0376] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it can provide stamps and emojis that best match the user's emotions, resulting in more effective communication.

[0377] Prompt Sentence Examples

[0378] An example of an input prompt for a generative AI model is shown below.

[0379] plaintext

[0380] Explain how the system works for a user who types "good work" while chatting with a friend.

[0381] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0382] Program processing steps

[0383] Step 1:

[0384] The user enters the text "OK" in the message input field and presses the send button.

[0385] The input data is the text "OK." This input is captured on the terminal, preparing it for the next step.

[0386] Step 2:

[0387] The terminal detects that the send button has been pressed, captures the entered text "OK" and sends this data to the server.

[0388] Specifically, the terminal sends the user's input text to the server as an HTTP request, and the output data is request data including the text "OK."

[0389] Step 3:

[0390] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[0391] The input data is the text "OK". The server receives this and forwards it to the appropriate module for analysis. The output data is forwarded to the text analysis module and emotion engine.

[0392] Step 4:

[0393] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[0394] The input data is the text "OK". Specifically, the NLP engine tokenizes the text, removes stop words, and extracts important keywords. The output data is the keyword "OK".

[0395] Step 5:

[0396] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module.

[0397] The input data is the keyword "understanding." The emotion engine uses this to perform emotion analysis and extract, for example, positive emotions. The output data is positive emotion information.

[0398] Step 6:

[0399] Based on the analysis results, the server searches the database for related stamps and emojis.

[0400] The input data is the keyword "OK" and emotional information (positive emotion). The server uses this information to search for stamps and emojis in the database. Specifically, it retrieves relevant stamps and emojis using SQL queries. The output data is a list of appropriate stamps and emojis.

[0401] Step 7:

[0402] The server organizes the search results and sends them to the terminal in list form.

[0403] The input data is a list of stamps and emojis. The server organizes this data, adds metadata that determines the display order, and sends it to the device. The output data is a list of candidate stamps and emojis.

[0404] Step 8:

[0405] The device analyzes the data received and displays stamp and emoji candidates on the user interface (UI).

[0406] The input data is a list of candidate stamps and emojis received from the server. Specifically, the device analyzes this and displays it to the user in a popup format. The output data is a list of stamps and emojis to be displayed in the user interface.

[0407] Step 9:

[0408] The user selects the appropriate stamp or emoji from the displayed ones.

[0409] The input data is a list of stamps and emojis displayed on the device. The user selects from this list and chooses the stamps and emojis to use in the final message. The output data is the stamps and emojis selected by the user.

[0410] Step 10:

[0411] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0412] The input data are the stamps and emojis selected by the user. The device inserts them into the message box and sends them to friends etc. as the final message. The output data is the sent message.

[0413] (Application example 2)

[0414] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0415] Conventional messaging systems require users to manually select appropriate stamps and emojis based on the text they input, which is particularly inconvenient when many messages are exchanged in real time. This can disrupt communication between viewers and degrade the viewing experience, especially in live streaming services. To solve this problem, a system that automatically suggests appropriate stamps and emojis based on the user's text input is needed.

[0416] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0417] In this invention, the server includes: means for a user to input text into a message input field and perform a send action; means for a terminal to capture the input text and send a request to the server; means for the server to analyze the received text and extract keywords and emotions; means for the server to search a database for relevant stamps and emojis based on the extracted keywords and emotions; means for the server to return the search results to the terminal; means for the terminal to analyze the received data and display stamps and emojis on a user interface; means for the user to select from the presented stamps and emojis; means for the server to process real-time comments in the live streaming service; and means for the terminal to display recommended stamps and emojis when a user enters a comment during live streaming. This allows users to quickly and intuitively select appropriate stamps and emojis in real time, enabling smooth communication.

[0418] A "user" is a person who uses the system to enter and send messages.

[0419] A "message entry field" is an area on the screen where a user can enter text.

[0420] "Text" is a string of characters that the user inputs as a message.

[0421] A "send action" refers to an operation of sending text entered by a user from a terminal to a server.

[0422] A "terminal" is a device that a user uses to enter and send messages.

[0423] "Capture" refers to the terminal taking in and saving the text entered.

[0424] A "server" is a computer system that analyzes and processes received text.

[0425] A "request" refers to a data request sent by a terminal to a server.

[0426] "Analysis" is the process of linguistically breaking down received text and extracting meaning and emotion.

[0427] "Keywords" are words or phrases that have important meaning in text analysis.

[0428] "Emotion" refers to the user's psychological state or sensation extracted through text analysis.

[0429] "Related stamps and emojis" are image data that match based on the extracted keywords and emotions.

[0430] A "database" is a searchable collection of information in which stamps and emojis are stored.

[0431] "Search" is the act of locating information that meets specific criteria from a database.

[0432] "Return" refers to the server sending the search results back to the terminal.

[0433] A "user interface" is the display portion on the screen that can be directly operated by the user.

[0434] "Selection" refers to the act of the user choosing an appropriate stamp or emoji from the ones presented.

[0435] "Real-time comments" are messages that users enter instantly during live streaming.

[0436] A "live streaming service" refers to a service that distributes video and audio in real time over the Internet.

[0437] "Recommendation" refers to suggesting appropriate stamps or emojis to users.

[0438] In this invention, a system is constructed that automatically recommends appropriate stamps and emoticons to facilitate smooth communication between users in live distribution services. The system program and its processing are described below.

[0439] Explanation of program processing

[0440] First, a user enters text into the comment input field during a live broadcast and performs a send action. The device (smartphone or tablet) captures this input text and sends it directly to the server. The server analyzes the received text and extracts keywords and user sentiment.

[0441] The text analysis uses Python's TextBlob library, which determines the positive, negative, or neutral sentiment of the text and extracts related keywords based on that sentiment. The analysis results are then processed within the computer system, after which relevant stickers and emojis are searched for in a database that contains pre-stored stickers and emojis corresponding to each sentiment.

[0442] The recommended stickers and emojis are sent back from the server to the device and displayed on the user interface. The user can then select from the stickers and emojis presented and send them as the final message. This process allows users to quickly and intuitively select the appropriate stickers and emojis in real time, improving the viewing experience.

[0443] Hardware and software used

[0444] Hardware: smartphones, tablets, servers

[0445] Software: Python 3, TextBlob library, cloud database

[0446] Specific examples

[0447] For example, suppose a user enters a comment such as "This is the best live broadcast!" during a live broadcast and presses the send button. In this case, the system operates as follows:

[0448] 1. The user types "This is the best live show ever!" and presses the send button.

[0449] 2. The device captures this text and sends it to the server.

[0450] 3. The server analyzes the received text using a text analysis module to determine positive sentiment.

[0451] 4. The server searches its database for stickers or emojis that correspond to positive emotions (e.g., 😊, 👍, 🎉).

[0452] 5. The server sends these recommended emojis back to the device and displays them in the user interface.

[0453] 6. The user selects the appropriate emoji from the ones presented and sends it.

[0454] This system allows users to quickly select appropriate stamps and emojis for their comments in real time, making communication during live broadcasts smoother and improving the viewing experience.

[0455] Prompt Sentence Examples

[0456] The application is based on a generated AI model, and example prompts include:

[0457] "Create a system that analyzes comments entered by users and recommends appropriate emojis based on their sentiment."

[0458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0459] Step 1:

[0460] The user enters text into the message input field and performs a send action. This allows the user to provide text data to the terminal. The input is manual input by the user, and the output is text data. For example, the user might enter "Great live show!"

[0461] Step 2:

[0462] The terminal captures the input text and sends a request to the server. Specifically, the terminal takes the text entered by the user and converts the text data into request data to be sent to the server. The input is the user's input text, and the output is the request data to the server.

[0463] Step 3:

[0464] The server analyzes the received text. The text analysis module uses a natural language processing engine (e.g., TextBlob library) to perform semantic analysis and sentiment extraction of the text. Through this processing, keywords and sentiment data are extracted. The input is the text data received from the device, and the output is the analyzed keywords and sentiment data. For example, positive sentiment can be extracted from "This was a great live show!"

[0465] Step 4:

[0466] The server searches a database for relevant stickers and emojis based on the extracted keywords and emotions. Specifically, the server generates a database query to search for stickers and emojis that match the emotions and keywords. The input is the parsed keywords and emotion data, and the output is a list of related stickers and emojis. For example, "😊", "👍", "🎉", etc.

[0467] Step 5:

[0468] The server returns the search results to the device. The server organizes the search results and generates a response to the requesting device. The input is a list of related stamps or emojis, and the output is the response data to the device.

[0469] Step 6:

[0470] The terminal analyzes the received data and displays stamps and emojis on the user interface. Specifically, the terminal analyzes the returned data it receives and displays stamp and emoji options to the user. The input is the returned data from the server, and the output is the stamps and emojis displayed on the terminal's user interface.

[0471] Step 7:

[0472] The user selects from the presented stamps and emojis. The user selects the appropriate one from the presented stamps and emojis and performs a selection action to send it as a message. The input is a list of stamps and emojis displayed on the device, and the output is the selected stamp or emoji.

[0473] Through these steps, the system can recommend appropriate stamps and emojis in real time based on the user's input text, improving the viewing experience.

[0474] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0475] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0476] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0477] [Second embodiment]

[0478] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0479] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0480] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0481] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0482] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0483] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0484] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0485] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0486] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0487] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0488] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0489] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0490] This invention relates to a system and method for quickly suggesting relevant stamps and emojis based on the text entered by a user when using a messaging application. The system analyzes the text entered by the user and displays appropriate stamps and emojis based on the content of the text, thereby facilitating communication between users.

[0491] Specific flow of program processing

[0492] User text input

[0493] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[0494] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[0495] Text Analysis

[0496] The server receives the text sent from the device and passes it to the text analysis module. The text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract the keyword "understanding." It also performs sentiment analysis if necessary.

[0497] Search for stamps and emojis

[0498] The server searches the database for relevant stickers and emojis based on the analysis results. In this case, it searches for stickers and emojis related to the keyword "gotcha" (for example, a thumbs-up emoji or a sticker that says "gotcha").

[0499] Returning and displaying search results

[0500] The server organizes the search results and sends them to the terminal in list form.

[0501] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[0502] User Selection and Submission

[0503] The user selects an appropriate stamp or emoji from the displayed ones.

[0504] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0505] Specific examples

[0506] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0507] 1. The user types "Thank you for your hard work" and presses the send button.

[0508] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0509] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0510] 4. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[0511] 5. The server sends these search results to the device.

[0512] 6. The device displays the received stamp and emoji candidates on the user's screen.

[0513] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0514] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to lead to increased sales.

[0515] The processing flow will be explained below.

[0516] Step 1:

[0517] The user enters the text "OK" in the message input field and presses the send button.

[0518] Specifically, the user enters text using the keyboard and taps the send button.

[0519] Step 2:

[0520] The terminal captures the entered text "OK" and sends a request to the server.

[0521] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[0522] Step 3:

[0523] The server forwards the text received from the user to a text analysis module for analysis.

[0524] Specifically, the server extracts text data from the request body and passes it to the analysis engine.

[0525] Step 4:

[0526] The server's text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract keywords and sentiment.

[0527] Specifically, the NLP engine tokenizes the text and applies keyword extraction algorithms and sentiment analysis models.

[0528] Step 5:

[0529] Based on the analysis results, the server searches the database for related stamps and emojis.

[0530] Specifically, it generates a database query and searches a database of stamps and emojis.

[0531] Step 6:

[0532] The server organizes the search results in list form and prepares to return them to the user terminal.

[0533] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[0534] Step 7:

[0535] The server sends the generated response to the user's terminal.

[0536] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[0537] Step 8:

[0538] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[0539] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[0540] Step 9:

[0541] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[0542] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[0543] Step 10:

[0544] The user selects the appropriate stamp or emoji from the displayed ones.

[0545] Specifically, the user taps to select the stamp or emoji they need.

[0546] Step 11:

[0547] The device inserts the selected stamp or emoji into the message box and performs the send action.

[0548] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[0549] Step 12:

[0550] The terminal transmits a message including the selected stamps and pictograms to the server, and normal message processing is performed.

[0551] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[0552] The process including the above steps allows a user to quickly and efficiently select appropriate stamps and emojis based on text input.

[0553] Example 1

[0554] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0555] Conventional messaging applications lack the functionality to appropriately suggest relevant stamps and emojis when users enter text. High-precision sentiment analysis based on the content of the text and rapid search and display of relevant stamps and emojis are also challenges. As a result, users have to spend time selecting stamps and emojis, which disrupts smooth communication.

[0556] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0557] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for visual information and symbols related to the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling prompt and appropriate suggestions of stamps and emojis related to the text entered by the user, thereby facilitating smooth communication.

[0558] "User" refers to the entity that operates the message application.

[0559] A "message input field" is an interface for a user to input text.

[0560] "Means for performing a submit action" refers to an action or button for submitting text entered by the user to a server.

[0561] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[0562] "Capture" refers to the operation of a terminal acquiring data input by a user.

[0563] A "server" is a central computer system that receives data sent from a user's terminal and performs analysis and response.

[0564] "Analysis" refers to the operations or processes performed by the server to extract useful information from the text received.

[0565] "Keywords" refer to important words extracted from the input text.

[0566] "Sentiment" refers to the emotional state of the user extracted from the text.

[0567] "Visual information and symbols" refers to visual elements such as stamps and emojis.

[0568] A "database" refers to a repository of information that stores visual information and symbols.

[0569] "Search" refers to the operation of finding appropriate visual information or symbols from a database.

[0570] "User interface (UI)" refers to the screens and functions that are displayed on a terminal and that allow users to interact with it.

[0571] "Pop-up" refers to a supplementary window or menu that appears at a specific location in the UI.

[0572] "Means for displaying visual information and symbol recommendations" refers to the system's ability to suggest appropriate stamps and emojis to users.

[0573] This invention relates to a system and method for quickly suggesting relevant visual information and symbols based on the text entered by a user when using a messaging application. This system analyzes the text entered by the user and displays appropriate visual information and symbols based on the content of the text, thereby facilitating communication between users.

[0574] When a user enters text in the message input field and performs the send action, the device captures the entered text and sends a request to the server. The server analyzes the received text and extracts keywords and emotions. A natural language processing engine (e.g., OpenAI GPT-3) is used for this analysis. The server searches a database (e.g., MySQL or MongoDB) for relevant visual information and symbols based on the extracted keywords and emotions. The search results are organized and sent to the device in list form. The device analyzes the received data and displays candidate visual information and symbols in the user interface (UI). The user selects the appropriate one from the presented candidates and sends it as a message.

[0575] As a specific example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0576] 1. The user types "Thank you for your hard work" and presses the send button.

[0577] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0578] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0579] 4. The server searches its database for visual information or symbols related to "good work" (for example, a tired face emoji or a stamp that says "good work!").

[0580] 5. The server sends these search results to the device.

[0581] 6. The terminal displays the received visual information and symbol candidates on the user screen.

[0582] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0583] This system allows users to quickly and intuitively select appropriate visual information and symbols when entering text, making communication smoother. Furthermore, because visual information and symbols can be used effectively, it is expected that they will be actively used, including newly acquired ones. This will also benefit creators, and is expected to lead to increased sales.

[0584] Below are some example prompts to input to a generative AI model:

[0585] Please explain in detail the steps of the system's process for suggesting related stickers and emojis when a user types "Thank you." Please clearly state the actions of the server, device, and user, including the specific actions at each step.

[0586] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] The user enters text into the message input field and presses the send button.

[0589] Specific behavior:

[0590] The user types "Got it" in the message application and performs a send action.

[0591] Input: The text entered by the user: "Got it."

[0592] Output: A submit action is triggered to proceed to the next step.

[0593] Step 2:

[0594] The terminal captures the entered text and sends a request to the server.

[0595] Specific behavior:

[0596] Logic within the messaging application is activated and captures the text "Got it" from the input field.

[0597] The captured data is securely sent to the server using encrypted communication such as SSL / TLS.

[0598] Input: The captured text "Got it."

[0599] Output: The request to the server and the captured text "Got it."

[0600] Step 3:

[0601] The server analyzes the received text and extracts keywords and sentiment.

[0602] Specific behavior:

[0603] The text data received on the server side is managed in JSON format or similar and passed to the analysis module.

[0604] A natural language processing (NLP) engine (e.g., OpenAI GPT-3) runs, understands the meaning of "understood," and extracts it as a keyword.

[0605] In emotion analysis, this is classified as a normal emotion.

[0606] Input: Received text "Got it."

[0607] Output: Extracted keyword "understand" and sentiment data.

[0608] Step 4:

[0609] The server searches a database for relevant visual information and symbols based on the extracted keywords and emotions.

[0610] Specific behavior:

[0611] A database is queried to extract visuals or symbols associated with "got it," such as a thumbs-up emoji or a stamp that says "got it."

[0612] Input: The extracted keyword "understand" and emotion data.

[0613] Output: A list of relevant visual information and symbols.

[0614] Step 5:

[0615] The server returns the search results to the terminal.

[0616] Specific behavior:

[0617] The search results are organized in a list format such as JSON or XML and sent back to the device.

[0618] Input: A list of relevant visual information or symbols.

[0619] Output: Organized list data.

[0620] Step 6:

[0621] The terminal analyzes the received data and displays visual information and symbol candidates on the user interface.

[0622] Specific behavior:

[0623] The list data received by the device is parsed and visual information and symbols are popped up in the messaging application UI in an appropriate display format.

[0624] Input: Received list data.

[0625] Output: Suggested visual information and symbols displayed in the user interface.

[0626] Step 7:

[0627] The user selects from the presented visual information and symbols.

[0628] Specific behavior:

[0629] Users can view a list of suggested visual information and symbols and select by tapping or clicking.

[0630] The selected visual information or symbol is inserted into the message box.

[0631] Input: Presented visual information and symbol candidates.

[0632] Output: The selected visual information or symbol is inserted into the message box.

[0633] Step 8:

[0634] The terminal inserts the visual information or symbols selected by the user into the message box and sends it as the final message.

[0635] Specific behavior:

[0636] The final message is constructed and the send action is performed, sending the selected stickers and emojis to a friend.

[0637] Input: Selected visual information or symbol.

[0638] Output: The final message sent.

[0639] (Application example 1)

[0640] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0641] While there have been systems that quickly suggest related stamps and emojis when a user inputs text, they lack the functionality to automatically suggest related products and promotions based on product reviews and comments. As a result, no efficient means has been provided to increase a user's purchasing motivation. The objective of the present invention is to provide a system that appropriately suggests related products and promotions to a user simply by inputting text, thereby increasing the user's purchasing motivation.

[0642] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0643] In this invention, the server includes means for analyzing text sent from the terminal, means for extracting keywords and emotions from the text, means for searching a database for related products and promotions based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling related products and promotions to be quickly suggested when a user inputs a product review or comment.

[0644] "User" refers to a person who uses the system to enter messages or post product reviews.

[0645] A "message input field" refers to an interface for a user to input text.

[0646] A "send action" refers to the operation of confirming text entered by the user and sending it from the terminal to the server.

[0647] A "terminal" is a device that is directly operated by a user, and refers to an electronic device such as a smartphone or tablet.

[0648] "Server" refers to a computing system that receives data sent from a terminal, analyzes it, and performs the necessary processing.

[0649] "Text" refers to the string of characters that a user enters into an input field.

[0650] "Captcha" refers to the act of saving entered text and sending it to a server for processing.

[0651] A "request" refers to data or a processing request sent from a terminal to a server.

[0652] "Analysis" refers to the process of breaking down text and extracting meaning and keywords.

[0653] "Keywords" refer to important words or phrases extracted from the analyzed text.

[0654] "Emotion" refers to the user's emotional state as interpreted from the text.

[0655] A "database" refers to a system that stores related information in an organized manner and allows it to be searched and extracted.

[0656] A "stamp" is an image or animation that can be attached to a message or review.

[0657] "Emoji" refers to small images or icons used alongside text.

[0658] "Related products" refers to products suggested based on the content of a user's reviews and comments.

[0659] "Promotion" refers to information about special offers or campaigns relating to a particular product.

[0660] "User screen" refers to an interface for displaying information to a user on a terminal.

[0661] "User interface" refers to the visual elements and tools that allow a user to interact with a system.

[0662] "Selection" refers to the action of a user choosing a particular sticker, emoji, related product, or promotion from among the options presented.

[0663] The present invention is a system that suggests related stamps, emoticons, related products, and promotions when a user enters text into a message input field or a product review input field and performs a send action. The server analyzes the received text, extracts keywords and emotions, and searches a database for related information to suggest it. The detailed configuration and functions required to realize this invention are described below.

[0664] Hardware and Software

[0665] Hardware:

[0666] Device: A smartphone or tablet operated by a user

[0667] Server: A high-performance computing system for text analysis and information retrieval.

[0668] software:

[0669] Natural Language Processing libraries: Google NLP API, spaCy, etc.

[0670] Database management systems: MySQL, MongoDB, etc.

[0671] Front-end framework: React Native, etc.

[0672] Messaging applications, shopping site applications

[0673] Data processing and calculation

[0674] Device:

[0675] When a user enters text into the message input field or product review input field, the device captures the text and sends it to the server. For example, if a user enters "This camera is very good and I'm very satisfied," the text data is sent to the server.

[0676] server:

[0677] 1. Text analysis: The server passes the received text to the text analysis module, which analyzes it using a natural language processing engine. Through analysis, keywords and sentiment are extracted from the text. For example, keywords such as "camera," "performance," and "satisfaction" and the sentiment "positive" are extracted.

[0678] 2. Related information search: Based on the extracted keywords and sentiment, the database is searched for related stickers, emojis, products, or promotions. Search results may include, for example, high-end cameras, related accessories, and special offers.

[0679] 3. Returning the results: The server organizes the search results and sends them to the terminal in a list format.

[0680] Device:

[0681] The device analyzes the received data and displays relevant stickers, emojis, products, or promotions in a user interface. The user selects the appropriate option from the presented options and inserts it into a message or proceeds to a product purchase page.

[0682] Examples of concrete examples and prompts

[0683] Examples:

[0684] For example, if a user writes in a review, "These shoes are very comfortable to wear and I like the design," the system will extract keywords such as "comfortable shoes" and "design" and suggest related insoles and shoes with a high design.

[0685] Example prompt sentence:

[0686] If a user writes, "These shoes are very comfortable and I love the design," provide examples of relevant promotions and product suggestions.

[0687] The above is an embodiment of the present invention. The present invention allows users to receive appropriate stamps, emoticons, related products, and promotions simply by inputting text, thereby improving communication and purchasing experiences.

[0688] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0689] Step 1:

[0690] The user enters text into the message input field or product review input field and performs a send action. The input is text data such as "This camera has very good performance and I am very satisfied."

[0691] Step 2:

[0692] The terminal captures the entered text and sends that data to the server as a request. The input is the text entered by the user, and the output is the data sent to the server.

[0693] Step 3:

[0694] The server passes the received text to the text analysis module, which analyzes the text using a natural language processing engine (such as Google NLP API or spaCy). The input is the received text, and the output is extracted keywords and sentiment data (e.g., "camera," "performance," "satisfied," sentiment: "positive").

[0695] Step 4:

[0696] The server searches a database (e.g., MySQL or MongoDB) for related stickers, emojis, products, or promotions based on the extracted keywords and emotions. The input is the extracted keywords and emotions, and the output is a list of related stickers, emojis, products, or promotions.

[0697] Step 5:

[0698] The server organizes the search results and sends them to the terminal in list form. The input is a list of related information, and the output is the data to be sent to the terminal.

[0699] Step 6:

[0700] The device analyzes the data received and displays the relevant stickers, emojis, products, or promotions in a user interface (e.g., React Native). The input is the data received from the server, and the output is the stickers, emojis, products, or promotions displayed in the user interface.

[0701] Step 7:

[0702] The user selects a sticker or emoji from the list and sends it as a message, or selects a related product or promotion and proceeds to the purchase page. The input is the options displayed in the user interface, and the output reflects the selected sticker, emoji, product, or promotion.

[0703] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0704] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[0705] Specific flow of program processing

[0706] User text input

[0707] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[0708] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[0709] Text Analysis

[0710] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[0711] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[0712] Emotion extraction

[0713] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. For example, a positive emotion is extracted from the text "I understand."

[0714] Search for stamps and emojis

[0715] Based on the analysis results, the server searches the database for relevant stamps and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions).

[0716] Returning and displaying search results

[0717] The server organizes the search results and sends them to the device in list form.

[0718] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[0719] User Selection and Submission

[0720] The user selects an appropriate stamp or emoji from the displayed ones.

[0721] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0722] Specific examples

[0723] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0724] 1. The user types "Thank you for your hard work" and presses the send button.

[0725] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0726] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0727] 4. The server's emotion engine extracts emotions such as fatigue and gratitude from the text "Thank you for your hard work."

[0728] 5. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[0729] 6. The server sends these search results to the device.

[0730] 7. The device displays the received stamp and emoji candidates on the user's screen.

[0731] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0732] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it is possible to provide stamps and emojis that best suit the user's emotions, resulting in more effective communication. And because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to increase sales.

[0733] The processing flow will be explained below.

[0734] Step 1:

[0735] The user enters the text "Thank you for your hard work" in the message input field and presses the send button.

[0736] Specifically, the user enters text using the keyboard and taps the send button.

[0737] Step 2:

[0738] The terminal captures the input text "Thank you for your hard work" and sends a request to the server.

[0739] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[0740] Step 3:

[0741] The server forwards the text received from the user to the text analysis module.

[0742] Specifically, the server extracts text data from the request body and passes it to a natural language processing (NLP) engine.

[0743] Step 4:

[0744] The server's text analysis module uses a natural language processing (NLP) engine to analyze "otsukaresama" and extract the keyword "otsukaresama."

[0745] Specifically, the NLP engine tokenizes the text and applies a keyword extraction algorithm.

[0746] Step 5:

[0747] The server's emotion engine extracts emotions from the text "Thank you for your hard work" based on the keywords extracted by the text analysis module.

[0748] Specifically, the emotion engine uses an emotion analysis algorithm to analyze the input text for emotions such as fatigue and relief.

[0749] Step 6:

[0750] The server searches a database for related stamps and emojis based on the extracted keywords and emotions.

[0751] Specifically, the system generates a database query and searches a database of stamps and emojis corresponding to feelings of fatigue and comfort.

[0752] Step 7:

[0753] The server organizes the search results in list form and prepares to send them to the user's terminal.

[0754] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[0755] Step 8:

[0756] The server sends the generated response to the user's terminal.

[0757] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[0758] Step 9:

[0759] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[0760] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[0761] Step 10:

[0762] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[0763] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[0764] Step 11:

[0765] The user selects from the displayed stamps and emojis.

[0766] Specifically, the user taps to select the appropriate stamp or emoji from the displayed list of candidates.

[0767] Step 12:

[0768] The device inserts the selected stamp or emoji into the message box and performs the send action.

[0769] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[0770] Step 13:

[0771] The terminal again transmits the message including the selected stamps and pictograms to the server, and normal message processing is performed.

[0772] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[0773] The process including the above steps allows users to quickly and efficiently select appropriate stickers and emojis based on text input, and the emotion engine provides stickers and emojis that best suit the user's emotions, resulting in more effective communication.

[0774] Example 2

[0775] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0776] In modern communication, attaching appropriate stickers and emojis to text messages is required to more effectively convey the emotion and nuance of the message. However, manually selecting appropriate stickers and emojis is time-consuming and inefficient. Furthermore, there is a lack of systems that instantly suggest stickers and emojis that match the content of the text, which can disrupt the flow of communication. There is a need for a system that can solve these problems and improve users' communication experience.

[0777] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0778] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for relevant stamps and emojis based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling the server to quickly suggest appropriate stamps and emojis for the text entered by the user.

[0779] A "user" is an individual who operates the system to input text and send messages using stamps and emoticons.

[0780] A "terminal" is an electronic device that allows a user to input, send, or receive text, and includes smartphones, computers, and the like.

[0781] The "server" is a central processing unit that manages the processing of the entire system, analyzes text sent from the terminals, and searches for stamps and emoticons.

[0782] A "message input field" is a UI element that allows a user to input text on a terminal.

[0783] A "send action" is an operation for sending text entered by a user from a terminal to a server.

[0784] "Capture" is the process by which a terminal captures entered text and sends it to a server.

[0785] A "request" is a communication in which a terminal requests a server to analyze data or search for a stamp.

[0786] "Text analysis" is the process by which the server analyzes the text data it receives and extracts keywords and context.

[0787] A "keyword" is an important word that has a particular meaning or sentiment in text analysis.

[0788] "Emotion" is information extracted from text that indicates a psychological state, such as positive or negative.

[0789] A "natural language processing engine" is a system that includes algorithms and models for performing text analysis.

[0790] The "emotion analysis module" is a part of the system that has the function of analyzing emotions from text and extracting the type of emotion.

[0791] A "database" is a storage device within the system where stamp and emoji information is stored.

[0792] "Searching" is the process by which the server searches the database for relevant stamps or emojis based on specific criteria.

[0793] The "user interface" refers to the screen configuration and elements that allow the terminal to display stamp and pictogram candidates to the user.

[0794] A "pop-up display" is a display method that appears temporarily to show appropriate stamps or emojis for the text entered by the user.

[0795] A "message box" is a UI element that displays the final message, including stamps and emojis selected by the user.

[0796] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[0797] User text input

[0798] First, the user enters text into the message input field and presses the send button. For example, the user enters the text "Got it" and sends it. The device used by the user at this time is a device running a messaging application, such as a smartphone or computer.

[0799] Sending text data

[0800] The terminal captures the text "OK" entered when the send button is pressed and transmits this data to the server via the Internet.

[0801] Text analytics

[0802] When the server receives the text sent from the device, it passes it to a text analysis module and an emotion engine. The server's text analysis module analyzes the text "I understand" using a natural language processing (NLP) engine (e.g., BERT or GPT model). During the analysis, the text is tokenized, common stop words are removed, and the keyword "I understand" is extracted.

[0803] Emotion extraction

[0804] Next, the emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. The emotion engine uses an emotion identification algorithm to extract positive emotion from the text "Got it."

[0805] Search for stamps and emojis

[0806] Based on the analysis results, the server searches the database for related stickers and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions), and searches for related stickers and emojis using a specific algorithm.

[0807] Submit search results

[0808] The server organizes the search results and sends them to the device in list format, along with metadata such as the display order.

[0809] Display in the user interface

[0810] The device analyzes the received data and displays candidate stickers and emojis in the user interface (UI). Specifically, a list of stickers and emojis is displayed as a pop-up near the message input field. This pop-up display is designed to allow users to quickly select stickers and emojis that are highly relevant to the text they have entered.

[0811] User Selection and Submission

[0812] Finally, the user selects an appropriate stamp or emoji from the displayed list, and the terminal inserts the selected stamp or emoji into the message box and sends it as the final message.

[0813] Specific examples

[0814] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0815] 1. The user types "Thank you for your hard work" and presses the send button.

[0816] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0817] 3. The server analyzes the received text and extracts the keyword "Otsukaresama." This keyword is extracted using a natural language processing engine.

[0818] 4. The server's emotion engine extracts emotions such as fatigue and relief from the text "Thank you for your hard work." The emotion engine analyzes the emotions using an emotion identification algorithm.

[0819] 5. The server searches the database for stamps or emojis related to "good work" (for example, a tired face emoji or a stamp that says "good work!"). The database is a storage device that stores information about stamps and emojis.

[0820] 6. The server organizes these search results and adds metadata such as display order when sending them to the device.

[0821] 7. The device analyzes the received stamp and emoji candidates and displays them on the user's screen. The user interface module pops up the candidates to make it easier for the user to select.

[0822] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0823] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it can provide stamps and emojis that best match the user's emotions, resulting in more effective communication.

[0824] Prompt Sentence Examples

[0825] An example of an input prompt for a generative AI model is shown below.

[0826] plaintext

[0827] Explain how the system works for a user who types "good work" while chatting with a friend.

[0828] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0829] Program processing steps

[0830] Step 1:

[0831] The user enters the text "OK" in the message input field and presses the send button.

[0832] The input data is the text "OK." This input is captured on the terminal, preparing it for the next step.

[0833] Step 2:

[0834] The terminal detects that the send button has been pressed, captures the entered text "OK" and sends this data to the server.

[0835] Specifically, the terminal sends the user's input text to the server as an HTTP request, and the output data is request data including the text "OK."

[0836] Step 3:

[0837] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[0838] The input data is the text "OK". The server receives this and forwards it to the appropriate module for analysis. The output data is forwarded to the text analysis module and emotion engine.

[0839] Step 4:

[0840] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[0841] The input data is the text "OK". Specifically, the NLP engine tokenizes the text, removes stop words, and extracts important keywords. The output data is the keyword "OK".

[0842] Step 5:

[0843] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module.

[0844] The input data is the keyword "understanding." The emotion engine uses this to perform emotion analysis and extract, for example, positive emotions. The output data is positive emotion information.

[0845] Step 6:

[0846] Based on the analysis results, the server searches the database for related stamps and emojis.

[0847] The input data is the keyword "OK" and emotional information (positive emotion). The server uses this information to search for stamps and emojis in the database. Specifically, it retrieves relevant stamps and emojis using SQL queries. The output data is a list of appropriate stamps and emojis.

[0848] Step 7:

[0849] The server organizes the search results and sends them to the terminal in list form.

[0850] The input data is a list of stamps and emojis. The server organizes this data, adds metadata that determines the display order, and sends it to the device. The output data is a list of candidate stamps and emojis.

[0851] Step 8:

[0852] The device analyzes the data received and displays stamp and emoji candidates on the user interface (UI).

[0853] The input data is a list of candidate stamps and emojis received from the server. Specifically, the device analyzes this and displays it to the user in a popup format. The output data is a list of stamps and emojis to be displayed in the user interface.

[0854] Step 9:

[0855] The user selects the appropriate stamp or emoji from the displayed ones.

[0856] The input data is a list of stamps and emojis displayed on the device. The user selects from this list and chooses the stamps and emojis to use in the final message. The output data is the stamps and emojis selected by the user.

[0857] Step 10:

[0858] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0859] The input data are the stamps and emojis selected by the user. The device inserts them into the message box and sends them to friends etc. as the final message. The output data is the sent message.

[0860] (Application example 2)

[0861] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0862] Conventional messaging systems require users to manually select appropriate stamps and emojis based on the text they input, which is particularly inconvenient when many messages are exchanged in real time. This can disrupt communication between viewers and degrade the viewing experience, especially in live streaming services. To solve this problem, a system that automatically suggests appropriate stamps and emojis based on the user's text input is needed.

[0863] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0864] In this invention, the server includes: means for a user to input text into a message input field and perform a send action; means for a terminal to capture the input text and send a request to the server; means for the server to analyze the received text and extract keywords and emotions; means for the server to search a database for relevant stamps and emojis based on the extracted keywords and emotions; means for the server to return the search results to the terminal; means for the terminal to analyze the received data and display stamps and emojis on a user interface; means for the user to select from the presented stamps and emojis; means for the server to process real-time comments in the live streaming service; and means for the terminal to display recommended stamps and emojis when a user enters a comment during live streaming. This allows users to quickly and intuitively select appropriate stamps and emojis in real time, enabling smooth communication.

[0865] A "user" is a person who uses the system to enter and send messages.

[0866] A "message entry field" is an area on the screen where a user can enter text.

[0867] "Text" is a string of characters that the user inputs as a message.

[0868] A "send action" refers to an operation of sending text entered by a user from a terminal to a server.

[0869] A "terminal" is a device that a user uses to enter and send messages.

[0870] "Capture" refers to the terminal taking in and saving the text entered.

[0871] A "server" is a computer system that analyzes and processes received text.

[0872] A "request" refers to a data request sent by a terminal to a server.

[0873] "Analysis" is the process of linguistically breaking down received text and extracting meaning and emotion.

[0874] "Keywords" are words or phrases that have important meaning in text analysis.

[0875] "Emotion" refers to the user's psychological state or sensation extracted through text analysis.

[0876] "Related stamps and emojis" are image data that match based on the extracted keywords and emotions.

[0877] A "database" is a searchable collection of information in which stamps and emojis are stored.

[0878] "Search" is the act of locating information that meets specific criteria from a database.

[0879] "Return" refers to the server sending the search results back to the terminal.

[0880] A "user interface" is the display portion on the screen that can be directly operated by the user.

[0881] "Selection" refers to the act of the user choosing an appropriate stamp or emoji from the ones presented.

[0882] "Real-time comments" are messages that users enter instantly during live streaming.

[0883] A "live streaming service" refers to a service that distributes video and audio in real time over the Internet.

[0884] "Recommendation" refers to suggesting appropriate stamps or emojis to users.

[0885] In this invention, a system is constructed that automatically recommends appropriate stamps and emoticons to facilitate smooth communication between users in live distribution services. The system program and its processing are described below.

[0886] Explanation of program processing

[0887] First, a user enters text into the comment input field during a live broadcast and performs a send action. The device (smartphone or tablet) captures this input text and sends it directly to the server. The server analyzes the received text and extracts keywords and user sentiment.

[0888] The text analysis uses Python's TextBlob library, which determines the positive, negative, or neutral sentiment of the text and extracts related keywords based on that sentiment. The analysis results are then processed within the computer system, after which relevant stickers and emojis are searched for in a database that contains pre-stored stickers and emojis corresponding to each sentiment.

[0889] The recommended stickers and emojis are sent back from the server to the device and displayed on the user interface. The user can then select from the stickers and emojis presented and send them as the final message. This process allows users to quickly and intuitively select the appropriate stickers and emojis in real time, improving the viewing experience.

[0890] Hardware and software used

[0891] Hardware: smartphones, tablets, servers

[0892] Software: Python 3, TextBlob library, cloud database

[0893] Specific examples

[0894] For example, suppose a user enters a comment such as "This is the best live broadcast!" during a live broadcast and presses the send button. In this case, the system operates as follows:

[0895] 1. The user types "This is the best live show ever!" and presses the send button.

[0896] 2. The device captures this text and sends it to the server.

[0897] 3. The server analyzes the received text using a text analysis module to determine positive sentiment.

[0898] 4. The server searches its database for stickers or emojis that correspond to positive emotions (e.g., 😊, 👍, 🎉).

[0899] 5. The server sends these recommended emojis back to the device and displays them in the user interface.

[0900] 6. The user selects the appropriate emoji from the ones presented and sends it.

[0901] This system allows users to quickly select appropriate stamps and emojis for their comments in real time, making communication during live broadcasts smoother and improving the viewing experience.

[0902] Prompt Sentence Examples

[0903] The application is based on a generated AI model, and example prompts include:

[0904] "Create a system that analyzes comments entered by users and recommends appropriate emojis based on their sentiment."

[0905] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0906] Step 1:

[0907] The user enters text into the message input field and performs a send action. This allows the user to provide text data to the terminal. The input is manual input by the user, and the output is text data. For example, the user might enter "Great live show!"

[0908] Step 2:

[0909] The terminal captures the input text and sends a request to the server. Specifically, the terminal takes the text entered by the user and converts the text data into request data to be sent to the server. The input is the user's input text, and the output is the request data to the server.

[0910] Step 3:

[0911] The server analyzes the received text. The text analysis module uses a natural language processing engine (e.g., TextBlob library) to perform semantic analysis and sentiment extraction of the text. Through this processing, keywords and sentiment data are extracted. The input is the text data received from the device, and the output is the analyzed keywords and sentiment data. For example, positive sentiment can be extracted from "This was a great live show!"

[0912] Step 4:

[0913] The server searches a database for relevant stickers and emojis based on the extracted keywords and emotions. Specifically, the server generates a database query to search for stickers and emojis that match the emotions and keywords. The input is the parsed keywords and emotion data, and the output is a list of related stickers and emojis. For example, "😊", "👍", "🎉", etc.

[0914] Step 5:

[0915] The server returns the search results to the device. The server organizes the search results and generates a response to the requesting device. The input is a list of related stamps or emojis, and the output is the response data to the device.

[0916] Step 6:

[0917] The terminal analyzes the received data and displays stamps and emojis on the user interface. Specifically, the terminal analyzes the returned data it receives and displays stamp and emoji options to the user. The input is the returned data from the server, and the output is the stamps and emojis displayed on the terminal's user interface.

[0918] Step 7:

[0919] The user selects from the presented stamps and emojis. The user selects the appropriate one from the presented stamps and emojis and performs a selection action to send it as a message. The input is a list of stamps and emojis displayed on the device, and the output is the selected stamp or emoji.

[0920] Through these steps, the system can recommend appropriate stamps and emojis in real time based on the user's input text, improving the viewing experience.

[0921] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0922] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0923] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0924] [Third embodiment]

[0925] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0926] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0927] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0928] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0929] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0930] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0931] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0932] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0933] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0934] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0935] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0936] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0937] This invention relates to a system and method for quickly suggesting relevant stamps and emojis based on the text entered by a user when using a messaging application. The system analyzes the text entered by the user and displays appropriate stamps and emojis based on the content of the text, thereby facilitating communication between users.

[0938] Specific flow of program processing

[0939] User text input

[0940] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[0941] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[0942] Text Analysis

[0943] The server receives the text sent from the device and passes it to the text analysis module. The text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract the keyword "understanding." It also performs sentiment analysis if necessary.

[0944] Search for stamps and emojis

[0945] The server searches the database for relevant stickers and emojis based on the analysis results. In this case, it searches for stickers and emojis related to the keyword "gotcha" (for example, a thumbs-up emoji or a sticker that says "gotcha").

[0946] Returning and displaying search results

[0947] The server organizes the search results and sends them to the terminal in list form.

[0948] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[0949] User Selection and Submission

[0950] The user selects an appropriate stamp or emoji from the displayed ones.

[0951] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[0952] Specific examples

[0953] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[0954] 1. The user types "Thank you for your hard work" and presses the send button.

[0955] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[0956] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[0957] 4. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[0958] 5. The server sends these search results to the device.

[0959] 6. The device displays the received stamp and emoji candidates on the user's screen.

[0960] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[0961] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to lead to increased sales.

[0962] The processing flow will be explained below.

[0963] Step 1:

[0964] The user enters the text "OK" in the message input field and presses the send button.

[0965] Specifically, the user enters text using the keyboard and taps the send button.

[0966] Step 2:

[0967] The terminal captures the entered text "OK" and sends a request to the server.

[0968] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[0969] Step 3:

[0970] The server forwards the text received from the user to a text analysis module for analysis.

[0971] Specifically, the server extracts text data from the request body and passes it to the analysis engine.

[0972] Step 4:

[0973] The server's text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract keywords and sentiment.

[0974] Specifically, the NLP engine tokenizes the text and applies keyword extraction algorithms and sentiment analysis models.

[0975] Step 5:

[0976] Based on the analysis results, the server searches the database for related stamps and emojis.

[0977] Specifically, it generates a database query and searches a database of stamps and emojis.

[0978] Step 6:

[0979] The server organizes the search results in list form and prepares to return them to the user terminal.

[0980] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[0981] Step 7:

[0982] The server sends the generated response to the user's terminal.

[0983] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[0984] Step 8:

[0985] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[0986] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[0987] Step 9:

[0988] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[0989] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[0990] Step 10:

[0991] The user selects the appropriate stamp or emoji from the displayed ones.

[0992] Specifically, the user taps to select the stamp or emoji they need.

[0993] Step 11:

[0994] The device inserts the selected stamp or emoji into the message box and performs the send action.

[0995] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[0996] Step 12:

[0997] The terminal transmits a message including the selected stamps and pictograms to the server, and normal message processing is performed.

[0998] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[0999] The process including the above steps allows a user to quickly and efficiently select appropriate stamps and emojis based on text input.

[1000] Example 1

[1001] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1002] Conventional messaging applications lack the functionality to appropriately suggest relevant stamps and emojis when users enter text. High-precision sentiment analysis based on the content of the text and rapid search and display of relevant stamps and emojis are also challenges. As a result, users have to spend time selecting stamps and emojis, which disrupts smooth communication.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1004] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for visual information and symbols related to the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling prompt and appropriate suggestions of stamps and emojis related to the text entered by the user, thereby facilitating smooth communication.

[1005] "User" refers to the entity that operates the message application.

[1006] A "message input field" is an interface for a user to input text.

[1007] "Means for performing a submit action" refers to an action or button for submitting text entered by the user to a server.

[1008] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[1009] "Capture" refers to the operation of a terminal acquiring data input by a user.

[1010] A "server" is a central computer system that receives data sent from a user's terminal and performs analysis and response.

[1011] "Analysis" refers to the operations or processes performed by the server to extract useful information from the text received.

[1012] "Keywords" refer to important words extracted from the input text.

[1013] "Sentiment" refers to the emotional state of the user extracted from the text.

[1014] "Visual information and symbols" refers to visual elements such as stamps and emojis.

[1015] A "database" refers to a repository of information that stores visual information and symbols.

[1016] "Search" refers to the operation of finding appropriate visual information or symbols from a database.

[1017] "User interface (UI)" refers to the screens and functions that are displayed on a terminal and that allow users to interact with it.

[1018] "Pop-up" refers to a supplementary window or menu that appears at a specific location in the UI.

[1019] "Means for displaying visual information and symbol recommendations" refers to the system's ability to suggest appropriate stamps and emojis to users.

[1020] This invention relates to a system and method for quickly suggesting relevant visual information and symbols based on the text entered by a user when using a messaging application. This system analyzes the text entered by the user and displays appropriate visual information and symbols based on the content of the text, thereby facilitating communication between users.

[1021] When a user enters text in the message input field and performs the send action, the device captures the entered text and sends a request to the server. The server analyzes the received text and extracts keywords and emotions. A natural language processing engine (e.g., OpenAI GPT-3) is used for this analysis. The server searches a database (e.g., MySQL or MongoDB) for relevant visual information and symbols based on the extracted keywords and emotions. The search results are organized and sent to the device in list form. The device analyzes the received data and displays candidate visual information and symbols in the user interface (UI). The user selects the appropriate one from the presented candidates and sends it as a message.

[1022] As a specific example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1023] 1. The user types "Thank you for your hard work" and presses the send button.

[1024] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1025] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[1026] 4. The server searches its database for visual information or symbols related to "good work" (for example, a tired face emoji or a stamp that says "good work!").

[1027] 5. The server sends these search results to the device.

[1028] 6. The terminal displays the received visual information and symbol candidates on the user screen.

[1029] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1030] This system allows users to quickly and intuitively select appropriate visual information and symbols when entering text, making communication smoother. Furthermore, because visual information and symbols can be used effectively, it is expected that they will be actively used, including newly acquired ones. This will also benefit creators, and is expected to lead to increased sales.

[1031] Below are some example prompts to input to a generative AI model:

[1032] Please explain in detail the steps of the system's process for suggesting related stickers and emojis when a user types "Thank you." Please clearly state the actions of the server, device, and user, including the specific actions at each step.

[1033] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1034] Step 1:

[1035] The user enters text into the message input field and presses the send button.

[1036] Specific behavior:

[1037] The user types "Got it" in the message application and performs a send action.

[1038] Input: The text entered by the user: "Got it."

[1039] Output: A submit action is triggered to proceed to the next step.

[1040] Step 2:

[1041] The terminal captures the entered text and sends a request to the server.

[1042] Specific behavior:

[1043] Logic within the messaging application is activated and captures the text "Got it" from the input field.

[1044] The captured data is securely sent to the server using encrypted communication such as SSL / TLS.

[1045] Input: The captured text "Got it."

[1046] Output: The request to the server and the captured text "Got it."

[1047] Step 3:

[1048] The server analyzes the received text and extracts keywords and sentiment.

[1049] Specific behavior:

[1050] The text data received on the server side is managed in JSON format or similar and passed to the analysis module.

[1051] A natural language processing (NLP) engine (e.g., OpenAI GPT-3) runs, understands the meaning of "understood," and extracts it as a keyword.

[1052] In emotion analysis, this is classified as a normal emotion.

[1053] Input: Received text "Got it."

[1054] Output: Extracted keyword "understand" and sentiment data.

[1055] Step 4:

[1056] The server searches a database for relevant visual information and symbols based on the extracted keywords and emotions.

[1057] Specific behavior:

[1058] A database is queried to extract visuals or symbols associated with "got it," such as a thumbs-up emoji or a stamp that says "got it."

[1059] Input: The extracted keyword "understand" and emotion data.

[1060] Output: A list of relevant visual information and symbols.

[1061] Step 5:

[1062] The server returns the search results to the terminal.

[1063] Specific behavior:

[1064] The search results are organized in a list format such as JSON or XML and sent back to the device.

[1065] Input: A list of relevant visual information or symbols.

[1066] Output: Organized list data.

[1067] Step 6:

[1068] The terminal analyzes the received data and displays visual information and symbol candidates on the user interface.

[1069] Specific behavior:

[1070] The list data received by the device is parsed and visual information and symbols are popped up in the messaging application UI in an appropriate display format.

[1071] Input: Received list data.

[1072] Output: Suggested visual information and symbols displayed in the user interface.

[1073] Step 7:

[1074] The user selects from the presented visual information and symbols.

[1075] Specific behavior:

[1076] Users can view a list of suggested visual information and symbols and select by tapping or clicking.

[1077] The selected visual information or symbol is inserted into the message box.

[1078] Input: Presented visual information and symbol candidates.

[1079] Output: The selected visual information or symbol is inserted into the message box.

[1080] Step 8:

[1081] The terminal inserts the visual information or symbols selected by the user into the message box and sends it as the final message.

[1082] Specific behavior:

[1083] The final message is constructed and the send action is performed, sending the selected stickers and emojis to a friend.

[1084] Input: Selected visual information or symbol.

[1085] Output: The final message sent.

[1086] (Application example 1)

[1087] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1088] While there have been systems that quickly suggest related stamps and emojis when a user inputs text, they lack the functionality to automatically suggest related products and promotions based on product reviews and comments. As a result, no efficient means has been provided to increase a user's purchasing motivation. The objective of the present invention is to provide a system that appropriately suggests related products and promotions to a user simply by inputting text, thereby increasing the user's purchasing motivation.

[1089] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1090] In this invention, the server includes means for analyzing text sent from the terminal, means for extracting keywords and emotions from the text, means for searching a database for related products and promotions based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling related products and promotions to be quickly suggested when a user inputs a product review or comment.

[1091] "User" refers to a person who uses the system to enter messages or post product reviews.

[1092] A "message input field" refers to an interface for a user to input text.

[1093] A "send action" refers to the operation of confirming text entered by the user and sending it from the terminal to the server.

[1094] A "terminal" is a device that is directly operated by a user, and refers to an electronic device such as a smartphone or tablet.

[1095] "Server" refers to a computing system that receives data sent from a terminal, analyzes it, and performs the necessary processing.

[1096] "Text" refers to the string of characters that a user enters into an input field.

[1097] "Captcha" refers to the act of saving entered text and sending it to a server for processing.

[1098] A "request" refers to data or a processing request sent from a terminal to a server.

[1099] "Analysis" refers to the process of breaking down text and extracting meaning and keywords.

[1100] "Keywords" refer to important words or phrases extracted from the analyzed text.

[1101] "Emotion" refers to the user's emotional state as interpreted from the text.

[1102] A "database" refers to a system that stores related information in an organized manner and allows it to be searched and extracted.

[1103] A "stamp" is an image or animation that can be attached to a message or review.

[1104] "Emoji" refers to small images or icons used alongside text.

[1105] "Related products" refers to products suggested based on the content of a user's reviews and comments.

[1106] "Promotion" refers to information about special offers or campaigns relating to a particular product.

[1107] "User screen" refers to an interface for displaying information to a user on a terminal.

[1108] "User interface" refers to the visual elements and tools that allow a user to interact with a system.

[1109] "Selection" refers to the action of a user choosing a particular sticker, emoji, related product, or promotion from among the options presented.

[1110] The present invention is a system that suggests related stamps, emoticons, related products, and promotions when a user enters text into a message input field or a product review input field and performs a send action. The server analyzes the received text, extracts keywords and emotions, and searches a database for related information to suggest it. The detailed configuration and functions required to realize this invention are described below.

[1111] Hardware and Software

[1112] Hardware:

[1113] Device: A smartphone or tablet operated by a user

[1114] Server: A high-performance computing system for text analysis and information retrieval.

[1115] software:

[1116] Natural Language Processing libraries: Google NLP API, spaCy, etc.

[1117] Database management systems: MySQL, MongoDB, etc.

[1118] Front-end framework: React Native, etc.

[1119] Messaging applications, shopping site applications

[1120] Data processing and calculation

[1121] Device:

[1122] When a user enters text into the message input field or product review input field, the device captures the text and sends it to the server. For example, if a user enters "This camera is very good and I'm very satisfied," the text data is sent to the server.

[1123] server:

[1124] 1. Text analysis: The server passes the received text to the text analysis module, which analyzes it using a natural language processing engine. Through analysis, keywords and sentiment are extracted from the text. For example, keywords such as "camera," "performance," and "satisfaction" and the sentiment "positive" are extracted.

[1125] 2. Related information search: Based on the extracted keywords and sentiment, the database is searched for related stickers, emojis, products, or promotions. Search results may include, for example, high-end cameras, related accessories, and special offers.

[1126] 3. Returning the results: The server organizes the search results and sends them to the terminal in a list format.

[1127] Device:

[1128] The device analyzes the received data and displays relevant stickers, emojis, products, or promotions in a user interface. The user selects the appropriate option from the presented options and inserts it into a message or proceeds to a product purchase page.

[1129] Examples of concrete examples and prompts

[1130] Examples:

[1131] For example, if a user writes in a review, "These shoes are very comfortable to wear and I like the design," the system will extract keywords such as "comfortable shoes" and "design" and suggest related insoles and shoes with a high design.

[1132] Example prompt sentence:

[1133] If a user writes, "These shoes are very comfortable and I love the design," provide examples of relevant promotions and product suggestions.

[1134] The above is an embodiment of the present invention. The present invention allows users to receive appropriate stamps, emoticons, related products, and promotions simply by inputting text, thereby improving communication and purchasing experiences.

[1135] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1136] Step 1:

[1137] The user enters text into the message input field or product review input field and performs a send action. The input is text data such as "This camera has very good performance and I am very satisfied."

[1138] Step 2:

[1139] The terminal captures the entered text and sends that data to the server as a request. The input is the text entered by the user, and the output is the data sent to the server.

[1140] Step 3:

[1141] The server passes the received text to the text analysis module, which analyzes the text using a natural language processing engine (such as Google NLP API or spaCy). The input is the received text, and the output is extracted keywords and sentiment data (e.g., "camera," "performance," "satisfied," sentiment: "positive").

[1142] Step 4:

[1143] The server searches a database (e.g., MySQL or MongoDB) for related stickers, emojis, products, or promotions based on the extracted keywords and emotions. The input is the extracted keywords and emotions, and the output is a list of related stickers, emojis, products, or promotions.

[1144] Step 5:

[1145] The server organizes the search results and sends them to the terminal in list form. The input is a list of related information, and the output is the data to be sent to the terminal.

[1146] Step 6:

[1147] The device analyzes the data received and displays the relevant stickers, emojis, products, or promotions in a user interface (e.g., React Native). The input is the data received from the server, and the output is the stickers, emojis, products, or promotions displayed in the user interface.

[1148] Step 7:

[1149] The user selects a sticker or emoji from the list and sends it as a message, or selects a related product or promotion and proceeds to the purchase page. The input is the options displayed in the user interface, and the output reflects the selected sticker, emoji, product, or promotion.

[1150] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1151] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[1152] Specific flow of program processing

[1153] User text input

[1154] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[1155] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[1156] Text Analysis

[1157] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[1158] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[1159] Emotion extraction

[1160] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. For example, a positive emotion is extracted from the text "I understand."

[1161] Search for stamps and emojis

[1162] Based on the analysis results, the server searches the database for relevant stamps and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions).

[1163] Returning and displaying search results

[1164] The server organizes the search results and sends them to the device in list form.

[1165] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[1166] User Selection and Submission

[1167] The user selects an appropriate stamp or emoji from the displayed ones.

[1168] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[1169] Specific examples

[1170] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1171] 1. The user types "Thank you for your hard work" and presses the send button.

[1172] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1173] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[1174] 4. The server's emotion engine extracts emotions such as fatigue and gratitude from the text "Thank you for your hard work."

[1175] 5. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[1176] 6. The server sends these search results to the device.

[1177] 7. The device displays the received stamp and emoji candidates on the user's screen.

[1178] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1179] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it is possible to provide stamps and emojis that best suit the user's emotions, resulting in more effective communication. And because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to increase sales.

[1180] The processing flow will be explained below.

[1181] Step 1:

[1182] The user enters the text "Thank you for your hard work" in the message input field and presses the send button.

[1183] Specifically, the user enters text using the keyboard and taps the send button.

[1184] Step 2:

[1185] The terminal captures the input text "Thank you for your hard work" and sends a request to the server.

[1186] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[1187] Step 3:

[1188] The server forwards the text received from the user to the text analysis module.

[1189] Specifically, the server extracts text data from the request body and passes it to a natural language processing (NLP) engine.

[1190] Step 4:

[1191] The server's text analysis module uses a natural language processing (NLP) engine to analyze "otsukaresama" and extract the keyword "otsukaresama."

[1192] Specifically, the NLP engine tokenizes the text and applies a keyword extraction algorithm.

[1193] Step 5:

[1194] The server's emotion engine extracts emotions from the text "Thank you for your hard work" based on the keywords extracted by the text analysis module.

[1195] Specifically, the emotion engine uses an emotion analysis algorithm to analyze the input text for emotions such as fatigue and relief.

[1196] Step 6:

[1197] The server searches a database for related stamps and emojis based on the extracted keywords and emotions.

[1198] Specifically, the system generates a database query and searches a database of stamps and emojis corresponding to feelings of fatigue and comfort.

[1199] Step 7:

[1200] The server organizes the search results in list form and prepares to send them to the user's terminal.

[1201] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[1202] Step 8:

[1203] The server sends the generated response to the user's terminal.

[1204] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[1205] Step 9:

[1206] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[1207] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[1208] Step 10:

[1209] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[1210] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[1211] Step 11:

[1212] The user selects from the displayed stamps and emojis.

[1213] Specifically, the user taps to select the appropriate stamp or emoji from the displayed list of candidates.

[1214] Step 12:

[1215] The device inserts the selected stamp or emoji into the message box and performs the send action.

[1216] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[1217] Step 13:

[1218] The terminal again transmits the message including the selected stamps and pictograms to the server, and normal message processing is performed.

[1219] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[1220] The process including the above steps allows users to quickly and efficiently select appropriate stickers and emojis based on text input, and the emotion engine provides stickers and emojis that best suit the user's emotions, resulting in more effective communication.

[1221] Example 2

[1222] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1223] In modern communication, attaching appropriate stickers and emojis to text messages is required to more effectively convey the emotion and nuance of the message. However, manually selecting appropriate stickers and emojis is time-consuming and inefficient. Furthermore, there is a lack of systems that instantly suggest stickers and emojis that match the content of the text, which can disrupt the flow of communication. There is a need for a system that can solve these problems and improve users' communication experience.

[1224] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1225] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for relevant stamps and emojis based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling the server to quickly suggest appropriate stamps and emojis for the text entered by the user.

[1226] A "user" is an individual who operates the system to input text and send messages using stamps and emoticons.

[1227] A "terminal" is an electronic device that allows a user to input, send, or receive text, and includes smartphones, computers, and the like.

[1228] The "server" is a central processing unit that manages the processing of the entire system, analyzes text sent from the terminals, and searches for stamps and emoticons.

[1229] A "message input field" is a UI element that allows a user to input text on a terminal.

[1230] A "send action" is an operation for sending text entered by a user from a terminal to a server.

[1231] "Capture" is the process by which a terminal captures entered text and sends it to a server.

[1232] A "request" is a communication in which a terminal requests a server to analyze data or search for a stamp.

[1233] "Text analysis" is the process by which the server analyzes the text data it receives and extracts keywords and context.

[1234] A "keyword" is an important word that has a particular meaning or sentiment in text analysis.

[1235] "Emotion" is information extracted from text that indicates a psychological state, such as positive or negative.

[1236] A "natural language processing engine" is a system that includes algorithms and models for performing text analysis.

[1237] The "emotion analysis module" is a part of the system that has the function of analyzing emotions from text and extracting the type of emotion.

[1238] A "database" is a storage device within the system where stamp and emoji information is stored.

[1239] "Searching" is the process by which the server searches the database for relevant stamps or emojis based on specific criteria.

[1240] The "user interface" refers to the screen configuration and elements that allow the terminal to display stamp and pictogram candidates to the user.

[1241] A "pop-up display" is a display method that appears temporarily to show appropriate stamps or emojis for the text entered by the user.

[1242] A "message box" is a UI element that displays the final message, including stamps and emojis selected by the user.

[1243] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[1244] User text input

[1245] First, the user enters text into the message input field and presses the send button. For example, the user enters the text "Got it" and sends it. The device used by the user at this time is a device running a messaging application, such as a smartphone or computer.

[1246] Sending text data

[1247] The terminal captures the text "OK" entered when the send button is pressed and transmits this data to the server via the Internet.

[1248] Text analytics

[1249] When the server receives the text sent from the device, it passes it to a text analysis module and an emotion engine. The server's text analysis module analyzes the text "I understand" using a natural language processing (NLP) engine (e.g., BERT or GPT model). During the analysis, the text is tokenized, common stop words are removed, and the keyword "I understand" is extracted.

[1250] Emotion extraction

[1251] Next, the emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. The emotion engine uses an emotion identification algorithm to extract positive emotion from the text "Got it."

[1252] Search for stamps and emojis

[1253] Based on the analysis results, the server searches the database for related stickers and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions), and searches for related stickers and emojis using a specific algorithm.

[1254] Submit search results

[1255] The server organizes the search results and sends them to the device in list format, along with metadata such as the display order.

[1256] Display in the user interface

[1257] The device analyzes the received data and displays candidate stickers and emojis in the user interface (UI). Specifically, a list of stickers and emojis is displayed as a pop-up near the message input field. This pop-up display is designed to allow users to quickly select stickers and emojis that are highly relevant to the text they have entered.

[1258] User Selection and Submission

[1259] Finally, the user selects an appropriate stamp or emoji from the displayed list, and the terminal inserts the selected stamp or emoji into the message box and sends it as the final message.

[1260] Specific examples

[1261] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1262] 1. The user types "Thank you for your hard work" and presses the send button.

[1263] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1264] 3. The server analyzes the received text and extracts the keyword "Otsukaresama." This keyword is extracted using a natural language processing engine.

[1265] 4. The server's emotion engine extracts emotions such as fatigue and relief from the text "Thank you for your hard work." The emotion engine analyzes the emotions using an emotion identification algorithm.

[1266] 5. The server searches the database for stamps or emojis related to "good work" (for example, a tired face emoji or a stamp that says "good work!"). The database is a storage device that stores information about stamps and emojis.

[1267] 6. The server organizes these search results and adds metadata such as display order when sending them to the device.

[1268] 7. The device analyzes the received stamp and emoji candidates and displays them on the user's screen. The user interface module pops up the candidates to make it easier for the user to select.

[1269] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1270] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it can provide stamps and emojis that best match the user's emotions, resulting in more effective communication.

[1271] Prompt Sentence Examples

[1272] An example of an input prompt for a generative AI model is shown below.

[1273] plaintext

[1274] Explain how the system works for a user who types "good work" while chatting with a friend.

[1275] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1276] Program processing steps

[1277] Step 1:

[1278] The user enters the text "OK" in the message input field and presses the send button.

[1279] The input data is the text "OK." This input is captured on the terminal, preparing it for the next step.

[1280] Step 2:

[1281] The terminal detects that the send button has been pressed, captures the entered text "OK" and sends this data to the server.

[1282] Specifically, the terminal sends the user's input text to the server as an HTTP request, and the output data is request data including the text "OK."

[1283] Step 3:

[1284] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[1285] The input data is the text "OK". The server receives this and forwards it to the appropriate module for analysis. The output data is forwarded to the text analysis module and emotion engine.

[1286] Step 4:

[1287] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[1288] The input data is the text "OK". Specifically, the NLP engine tokenizes the text, removes stop words, and extracts important keywords. The output data is the keyword "OK".

[1289] Step 5:

[1290] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module.

[1291] The input data is the keyword "understanding." The emotion engine uses this to perform emotion analysis and extract, for example, positive emotions. The output data is positive emotion information.

[1292] Step 6:

[1293] Based on the analysis results, the server searches the database for related stamps and emojis.

[1294] The input data is the keyword "OK" and emotional information (positive emotion). The server uses this information to search for stamps and emojis in the database. Specifically, it retrieves relevant stamps and emojis using SQL queries. The output data is a list of appropriate stamps and emojis.

[1295] Step 7:

[1296] The server organizes the search results and sends them to the terminal in list form.

[1297] The input data is a list of stamps and emojis. The server organizes this data, adds metadata that determines the display order, and sends it to the device. The output data is a list of candidate stamps and emojis.

[1298] Step 8:

[1299] The device analyzes the data received and displays stamp and emoji candidates on the user interface (UI).

[1300] The input data is a list of candidate stamps and emojis received from the server. Specifically, the device analyzes this and displays it to the user in a popup format. The output data is a list of stamps and emojis to be displayed in the user interface.

[1301] Step 9:

[1302] The user selects the appropriate stamp or emoji from the displayed ones.

[1303] The input data is a list of stamps and emojis displayed on the device. The user selects from this list and chooses the stamps and emojis to use in the final message. The output data is the stamps and emojis selected by the user.

[1304] Step 10:

[1305] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[1306] The input data are the stamps and emojis selected by the user. The device inserts them into the message box and sends them to friends etc. as the final message. The output data is the sent message.

[1307] (Application example 2)

[1308] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1309] Conventional messaging systems require users to manually select appropriate stamps and emojis based on the text they input, which is particularly inconvenient when many messages are exchanged in real time. This can disrupt communication between viewers and degrade the viewing experience, especially in live streaming services. To solve this problem, a system that automatically suggests appropriate stamps and emojis based on the user's text input is needed.

[1310] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1311] In this invention, the server includes: means for a user to input text into a message input field and perform a send action; means for a terminal to capture the input text and send a request to the server; means for the server to analyze the received text and extract keywords and emotions; means for the server to search a database for relevant stamps and emojis based on the extracted keywords and emotions; means for the server to return the search results to the terminal; means for the terminal to analyze the received data and display stamps and emojis on a user interface; means for the user to select from the presented stamps and emojis; means for the server to process real-time comments in the live streaming service; and means for the terminal to display recommended stamps and emojis when a user enters a comment during live streaming. This allows users to quickly and intuitively select appropriate stamps and emojis in real time, enabling smooth communication.

[1312] A "user" is a person who uses the system to enter and send messages.

[1313] A "message entry field" is an area on the screen where a user can enter text.

[1314] "Text" is a string of characters that the user inputs as a message.

[1315] A "send action" refers to an operation of sending text entered by a user from a terminal to a server.

[1316] A "terminal" is a device that a user uses to enter and send messages.

[1317] "Capture" refers to the terminal taking in and saving the text entered.

[1318] A "server" is a computer system that analyzes and processes received text.

[1319] A "request" refers to a data request sent by a terminal to a server.

[1320] "Analysis" is the process of linguistically breaking down received text and extracting meaning and emotion.

[1321] "Keywords" are words or phrases that have important meaning in text analysis.

[1322] "Emotion" refers to the user's psychological state or sensation extracted through text analysis.

[1323] "Related stamps and emojis" are image data that match based on the extracted keywords and emotions.

[1324] A "database" is a searchable collection of information in which stamps and emojis are stored.

[1325] "Search" is the act of locating information that meets specific criteria from a database.

[1326] "Return" refers to the server sending the search results back to the terminal.

[1327] A "user interface" is the display portion on the screen that can be directly operated by the user.

[1328] "Selection" refers to the act of the user choosing an appropriate stamp or emoji from the ones presented.

[1329] "Real-time comments" are messages that users enter instantly during live streaming.

[1330] A "live streaming service" refers to a service that distributes video and audio in real time over the Internet.

[1331] "Recommendation" refers to suggesting appropriate stamps or emojis to users.

[1332] In this invention, a system is constructed that automatically recommends appropriate stamps and emoticons to facilitate smooth communication between users in live distribution services. The system program and its processing are described below.

[1333] Explanation of program processing

[1334] First, a user enters text into the comment input field during a live broadcast and performs a send action. The device (smartphone or tablet) captures this input text and sends it directly to the server. The server analyzes the received text and extracts keywords and user sentiment.

[1335] The text analysis uses Python's TextBlob library, which determines the positive, negative, or neutral sentiment of the text and extracts related keywords based on that sentiment. The analysis results are then processed within the computer system, after which relevant stickers and emojis are searched for in a database that contains pre-stored stickers and emojis corresponding to each sentiment.

[1336] The recommended stickers and emojis are sent back from the server to the device and displayed on the user interface. The user can then select from the stickers and emojis presented and send them as the final message. This process allows users to quickly and intuitively select the appropriate stickers and emojis in real time, improving the viewing experience.

[1337] Hardware and software used

[1338] Hardware: smartphones, tablets, servers

[1339] Software: Python 3, TextBlob library, cloud database

[1340] Specific examples

[1341] For example, suppose a user enters a comment such as "This is the best live broadcast!" during a live broadcast and presses the send button. In this case, the system operates as follows:

[1342] 1. The user types "This is the best live show ever!" and presses the send button.

[1343] 2. The device captures this text and sends it to the server.

[1344] 3. The server analyzes the received text using a text analysis module to determine positive sentiment.

[1345] 4. The server searches its database for stickers or emojis that correspond to positive emotions (e.g., 😊, 👍, 🎉).

[1346] 5. The server sends these recommended emojis back to the device and displays them in the user interface.

[1347] 6. The user selects the appropriate emoji from the ones presented and sends it.

[1348] This system allows users to quickly select appropriate stamps and emojis for their comments in real time, making communication during live broadcasts smoother and improving the viewing experience.

[1349] Prompt Sentence Examples

[1350] The application is based on a generated AI model, and example prompts include:

[1351] "Create a system that analyzes comments entered by users and recommends appropriate emojis based on their sentiment."

[1352] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1353] Step 1:

[1354] The user enters text into the message input field and performs a send action. This allows the user to provide text data to the terminal. The input is manual input by the user, and the output is text data. For example, the user might enter "Great live show!"

[1355] Step 2:

[1356] The terminal captures the input text and sends a request to the server. Specifically, the terminal takes the text entered by the user and converts the text data into request data to be sent to the server. The input is the user's input text, and the output is the request data to the server.

[1357] Step 3:

[1358] The server analyzes the received text. The text analysis module uses a natural language processing engine (e.g., TextBlob library) to perform semantic analysis and sentiment extraction of the text. Through this processing, keywords and sentiment data are extracted. The input is the text data received from the device, and the output is the analyzed keywords and sentiment data. For example, positive sentiment can be extracted from "This was a great live show!"

[1359] Step 4:

[1360] The server searches a database for relevant stickers and emojis based on the extracted keywords and emotions. Specifically, the server generates a database query to search for stickers and emojis that match the emotions and keywords. The input is the parsed keywords and emotion data, and the output is a list of related stickers and emojis. For example, "😊", "👍", "🎉", etc.

[1361] Step 5:

[1362] The server returns the search results to the device. The server organizes the search results and generates a response to the requesting device. The input is a list of related stamps or emojis, and the output is the response data to the device.

[1363] Step 6:

[1364] The terminal analyzes the received data and displays stamps and emojis on the user interface. Specifically, the terminal analyzes the returned data it receives and displays stamp and emoji options to the user. The input is the returned data from the server, and the output is the stamps and emojis displayed on the terminal's user interface.

[1365] Step 7:

[1366] The user selects from the presented stamps and emojis. The user selects the appropriate one from the presented stamps and emojis and performs a selection action to send it as a message. The input is a list of stamps and emojis displayed on the device, and the output is the selected stamp or emoji.

[1367] Through these steps, the system can recommend appropriate stamps and emojis in real time based on the user's input text, improving the viewing experience.

[1368] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1369] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1370] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1371] [Fourth embodiment]

[1372] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1373] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1374] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1375] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1376] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1377] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1378] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1379] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1380] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1381] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1382] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1383] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1384] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1385] This invention relates to a system and method for quickly suggesting relevant stamps and emojis based on the text entered by a user when using a messaging application. The system analyzes the text entered by the user and displays appropriate stamps and emojis based on the content of the text, thereby facilitating communication between users.

[1386] Specific flow of program processing

[1387] User text input

[1388] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[1389] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[1390] Text Analysis

[1391] The server receives the text sent from the device and passes it to the text analysis module. The text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract the keyword "understanding." It also performs sentiment analysis if necessary.

[1392] Search for stamps and emojis

[1393] The server searches the database for relevant stickers and emojis based on the analysis results. In this case, it searches for stickers and emojis related to the keyword "gotcha" (for example, a thumbs-up emoji or a sticker that says "gotcha").

[1394] Returning and displaying search results

[1395] The server organizes the search results and sends them to the terminal in list form.

[1396] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[1397] User Selection and Submission

[1398] The user selects an appropriate stamp or emoji from the displayed ones.

[1399] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[1400] Specific examples

[1401] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1402] 1. The user types "Thank you for your hard work" and presses the send button.

[1403] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1404] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[1405] 4. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[1406] 5. The server sends these search results to the device.

[1407] 6. The device displays the received stamp and emoji candidates on the user's screen.

[1408] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1409] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to lead to increased sales.

[1410] The processing flow will be explained below.

[1411] Step 1:

[1412] The user enters the text "OK" in the message input field and presses the send button.

[1413] Specifically, the user enters text using the keyboard and taps the send button.

[1414] Step 2:

[1415] The terminal captures the entered text "OK" and sends a request to the server.

[1416] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[1417] Step 3:

[1418] The server forwards the text received from the user to a text analysis module for analysis.

[1419] Specifically, the server extracts text data from the request body and passes it to the analysis engine.

[1420] Step 4:

[1421] The server's text analysis module uses a natural language processing (NLP) engine to analyze the text "understanding" and extract keywords and sentiment.

[1422] Specifically, the NLP engine tokenizes the text and applies keyword extraction algorithms and sentiment analysis models.

[1423] Step 5:

[1424] Based on the analysis results, the server searches the database for related stamps and emojis.

[1425] Specifically, it generates a database query and searches a database of stamps and emojis.

[1426] Step 6:

[1427] The server organizes the search results in list form and prepares to return them to the user terminal.

[1428] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[1429] Step 7:

[1430] The server sends the generated response to the user's terminal.

[1431] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[1432] Step 8:

[1433] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[1434] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[1435] Step 9:

[1436] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[1437] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[1438] Step 10:

[1439] The user selects the appropriate stamp or emoji from the displayed ones.

[1440] Specifically, the user taps to select the stamp or emoji they need.

[1441] Step 11:

[1442] The device inserts the selected stamp or emoji into the message box and performs the send action.

[1443] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[1444] Step 12:

[1445] The terminal transmits a message including the selected stamps and pictograms to the server, and normal message processing is performed.

[1446] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[1447] The process including the above steps allows a user to quickly and efficiently select appropriate stamps and emojis based on text input.

[1448] Example 1

[1449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1450] Conventional messaging applications lack the functionality to appropriately suggest relevant stamps and emojis when users enter text. High-precision sentiment analysis based on the content of the text and rapid search and display of relevant stamps and emojis are also challenges. As a result, users have to spend time selecting stamps and emojis, which disrupts smooth communication.

[1451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1452] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for visual information and symbols related to the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling prompt and appropriate suggestions of stamps and emojis related to the text entered by the user, thereby facilitating smooth communication.

[1453] "User" refers to the entity that operates the message application.

[1454] A "message input field" is an interface for a user to input text.

[1455] "Means for performing a submit action" refers to an action or button for submitting text entered by the user to a server.

[1456] A "terminal" is an electronic device used by a user, such as a smartphone or computer.

[1457] "Capture" refers to the operation of a terminal acquiring data input by a user.

[1458] A "server" is a central computer system that receives data sent from a user's terminal and performs analysis and response.

[1459] "Analysis" refers to the operations or processes performed by the server to extract useful information from the text received.

[1460] "Keywords" refer to important words extracted from the input text.

[1461] "Sentiment" refers to the emotional state of the user extracted from the text.

[1462] "Visual information and symbols" refers to visual elements such as stamps and emojis.

[1463] A "database" refers to a repository of information that stores visual information and symbols.

[1464] "Search" refers to the operation of finding appropriate visual information or symbols from a database.

[1465] "User interface (UI)" refers to the screens and functions that are displayed on a terminal and that allow users to interact with it.

[1466] "Pop-up" refers to a supplementary window or menu that appears at a specific location in the UI.

[1467] "Means for displaying visual information and symbol recommendations" refers to the system's ability to suggest appropriate stamps and emojis to users.

[1468] This invention relates to a system and method for quickly suggesting relevant visual information and symbols based on the text entered by a user when using a messaging application. This system analyzes the text entered by the user and displays appropriate visual information and symbols based on the content of the text, thereby facilitating communication between users.

[1469] When a user enters text in the message input field and performs the send action, the device captures the entered text and sends a request to the server. The server analyzes the received text and extracts keywords and emotions. A natural language processing engine (e.g., OpenAI GPT-3) is used for this analysis. The server searches a database (e.g., MySQL or MongoDB) for relevant visual information and symbols based on the extracted keywords and emotions. The search results are organized and sent to the device in list form. The device analyzes the received data and displays candidate visual information and symbols in the user interface (UI). The user selects the appropriate one from the presented candidates and sends it as a message.

[1470] As a specific example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1471] 1. The user types "Thank you for your hard work" and presses the send button.

[1472] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1473] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[1474] 4. The server searches its database for visual information or symbols related to "good work" (for example, a tired face emoji or a stamp that says "good work!").

[1475] 5. The server sends these search results to the device.

[1476] 6. The terminal displays the received visual information and symbol candidates on the user screen.

[1477] 7. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1478] This system allows users to quickly and intuitively select appropriate visual information and symbols when entering text, making communication smoother. Furthermore, because visual information and symbols can be used effectively, it is expected that they will be actively used, including newly acquired ones. This will also benefit creators, and is expected to lead to increased sales.

[1479] Below are some example prompts to input to a generative AI model:

[1480] Please explain in detail the steps of the system's process for suggesting related stickers and emojis when a user types "Thank you." Please clearly state the actions of the server, device, and user, including the specific actions at each step.

[1481] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1482] Step 1:

[1483] The user enters text into the message input field and presses the send button.

[1484] Specific behavior:

[1485] The user types "Got it" in the message application and performs a send action.

[1486] Input: The text entered by the user: "Got it."

[1487] Output: A submit action is triggered to proceed to the next step.

[1488] Step 2:

[1489] The terminal captures the entered text and sends a request to the server.

[1490] Specific behavior:

[1491] Logic within the messaging application is activated and captures the text "Got it" from the input field.

[1492] The captured data is securely sent to the server using encrypted communication such as SSL / TLS.

[1493] Input: The captured text "Got it."

[1494] Output: The request to the server and the captured text "Got it."

[1495] Step 3:

[1496] The server analyzes the received text and extracts keywords and sentiment.

[1497] Specific behavior:

[1498] The text data received on the server side is managed in JSON format or similar and passed to the analysis module.

[1499] A natural language processing (NLP) engine (e.g., OpenAI GPT-3) runs, understands the meaning of "understood," and extracts it as a keyword.

[1500] In emotion analysis, this is classified as a normal emotion.

[1501] Input: Received text "Got it."

[1502] Output: Extracted keyword "understand" and sentiment data.

[1503] Step 4:

[1504] The server searches a database for relevant visual information and symbols based on the extracted keywords and emotions.

[1505] Specific behavior:

[1506] A database is queried to extract visuals or symbols associated with "got it," such as a thumbs-up emoji or a stamp that says "got it."

[1507] Input: The extracted keyword "understand" and emotion data.

[1508] Output: A list of relevant visual information and symbols.

[1509] Step 5:

[1510] The server returns the search results to the terminal.

[1511] Specific behavior:

[1512] The search results are organized in a list format such as JSON or XML and sent back to the device.

[1513] Input: A list of relevant visual information or symbols.

[1514] Output: Organized list data.

[1515] Step 6:

[1516] The terminal analyzes the received data and displays visual information and symbol candidates on the user interface.

[1517] Specific behavior:

[1518] The list data received by the device is parsed and visual information and symbols are popped up in the messaging application UI in an appropriate display format.

[1519] Input: Received list data.

[1520] Output: Suggested visual information and symbols displayed in the user interface.

[1521] Step 7:

[1522] The user selects from the presented visual information and symbols.

[1523] Specific behavior:

[1524] Users can view a list of suggested visual information and symbols and select by tapping or clicking.

[1525] The selected visual information or symbol is inserted into the message box.

[1526] Input: Presented visual information and symbol candidates.

[1527] Output: The selected visual information or symbol is inserted into the message box.

[1528] Step 8:

[1529] The terminal inserts the visual information or symbols selected by the user into the message box and sends it as the final message.

[1530] Specific behavior:

[1531] The final message is constructed and the send action is performed, sending the selected stickers and emojis to a friend.

[1532] Input: Selected visual information or symbol.

[1533] Output: The final message sent.

[1534] (Application example 1)

[1535] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1536] While there have been systems that quickly suggest related stamps and emojis when a user inputs text, they lack the functionality to automatically suggest related products and promotions based on product reviews and comments. As a result, no efficient means has been provided to increase a user's purchasing motivation. The objective of the present invention is to provide a system that appropriately suggests related products and promotions to a user simply by inputting text, thereby increasing the user's purchasing motivation.

[1537] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1538] In this invention, the server includes means for analyzing text sent from the terminal, means for extracting keywords and emotions from the text, means for searching a database for related products and promotions based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling related products and promotions to be quickly suggested when a user inputs a product review or comment.

[1539] "User" refers to a person who uses the system to enter messages or post product reviews.

[1540] A "message input field" refers to an interface for a user to input text.

[1541] A "send action" refers to the operation of confirming text entered by the user and sending it from the terminal to the server.

[1542] A "terminal" is a device that is directly operated by a user, and refers to an electronic device such as a smartphone or tablet.

[1543] "Server" refers to a computing system that receives data sent from a terminal, analyzes it, and performs the necessary processing.

[1544] "Text" refers to the string of characters that a user enters into an input field.

[1545] "Captcha" refers to the act of saving entered text and sending it to a server for processing.

[1546] A "request" refers to data or a processing request sent from a terminal to a server.

[1547] "Analysis" refers to the process of breaking down text and extracting meaning and keywords.

[1548] "Keywords" refer to important words or phrases extracted from the analyzed text.

[1549] "Emotion" refers to the user's emotional state as interpreted from the text.

[1550] A "database" refers to a system that stores related information in an organized manner and allows it to be searched and extracted.

[1551] A "stamp" is an image or animation that can be attached to a message or review.

[1552] "Emoji" refers to small images or icons used alongside text.

[1553] "Related products" refers to products suggested based on the content of a user's reviews and comments.

[1554] "Promotion" refers to information about special offers or campaigns relating to a particular product.

[1555] "User screen" refers to an interface for displaying information to a user on a terminal.

[1556] "User interface" refers to the visual elements and tools that allow a user to interact with a system.

[1557] "Selection" refers to the action of a user choosing a particular sticker, emoji, related product, or promotion from among the options presented.

[1558] The present invention is a system that suggests related stamps, emoticons, related products, and promotions when a user enters text into a message input field or a product review input field and performs a send action. The server analyzes the received text, extracts keywords and emotions, and searches a database for related information to suggest it. The detailed configuration and functions required to realize this invention are described below.

[1559] Hardware and Software

[1560] Hardware:

[1561] Device: A smartphone or tablet operated by a user

[1562] Server: A high-performance computing system for text analysis and information retrieval.

[1563] software:

[1564] Natural Language Processing libraries: Google NLP API, spaCy, etc.

[1565] Database management systems: MySQL, MongoDB, etc.

[1566] Front-end framework: React Native, etc.

[1567] Messaging applications, shopping site applications

[1568] Data processing and calculation

[1569] Device:

[1570] When a user enters text into the message input field or product review input field, the device captures the text and sends it to the server. For example, if a user enters "This camera is very good and I'm very satisfied," the text data is sent to the server.

[1571] server:

[1572] 1. Text analysis: The server passes the received text to the text analysis module, which analyzes it using a natural language processing engine. Through analysis, keywords and sentiment are extracted from the text. For example, keywords such as "camera," "performance," and "satisfaction" and the sentiment "positive" are extracted.

[1573] 2. Related information search: Based on the extracted keywords and sentiment, the database is searched for related stickers, emojis, products, or promotions. Search results may include, for example, high-end cameras, related accessories, and special offers.

[1574] 3. Returning the results: The server organizes the search results and sends them to the terminal in a list format.

[1575] Device:

[1576] The device analyzes the received data and displays relevant stickers, emojis, products, or promotions in a user interface. The user selects the appropriate option from the presented options and inserts it into a message or proceeds to a product purchase page.

[1577] Examples of concrete examples and prompts

[1578] Examples:

[1579] For example, if a user writes in a review, "These shoes are very comfortable to wear and I like the design," the system will extract keywords such as "comfortable shoes" and "design" and suggest related insoles and shoes with a high design.

[1580] Example prompt sentence:

[1581] If a user writes, "These shoes are very comfortable and I love the design," provide examples of relevant promotions and product suggestions.

[1582] The above is an embodiment of the present invention. The present invention allows users to receive appropriate stamps, emoticons, related products, and promotions simply by inputting text, thereby improving communication and purchasing experiences.

[1583] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1584] Step 1:

[1585] The user enters text into the message input field or product review input field and performs a send action. The input is text data such as "This camera has very good performance and I am very satisfied."

[1586] Step 2:

[1587] The terminal captures the entered text and sends that data to the server as a request. The input is the text entered by the user, and the output is the data sent to the server.

[1588] Step 3:

[1589] The server passes the received text to the text analysis module, which analyzes the text using a natural language processing engine (such as Google NLP API or spaCy). The input is the received text, and the output is extracted keywords and sentiment data (e.g., "camera," "performance," "satisfied," sentiment: "positive").

[1590] Step 4:

[1591] The server searches a database (e.g., MySQL or MongoDB) for related stickers, emojis, products, or promotions based on the extracted keywords and emotions. The input is the extracted keywords and emotions, and the output is a list of related stickers, emojis, products, or promotions.

[1592] Step 5:

[1593] The server organizes the search results and sends them to the terminal in list form. The input is a list of related information, and the output is the data to be sent to the terminal.

[1594] Step 6:

[1595] The device analyzes the data received and displays the relevant stickers, emojis, products, or promotions in a user interface (e.g., React Native). The input is the data received from the server, and the output is the stickers, emojis, products, or promotions displayed in the user interface.

[1596] Step 7:

[1597] The user selects a sticker or emoji from the list and sends it as a message, or selects a related product or promotion and proceeds to the purchase page. The input is the options displayed in the user interface, and the output reflects the selected sticker, emoji, product, or promotion.

[1598] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1599] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[1600] Specific flow of program processing

[1601] User text input

[1602] Suppose a user enters the text "Got it" in the message input field and presses the send button.

[1603] The terminal captures the text "OK" entered when the send button is pressed and sends this data as is to the server.

[1604] Text Analysis

[1605] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[1606] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[1607] Emotion extraction

[1608] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. For example, a positive emotion is extracted from the text "I understand."

[1609] Search for stamps and emojis

[1610] Based on the analysis results, the server searches the database for relevant stamps and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions).

[1611] Returning and displaying search results

[1612] The server organizes the search results and sends them to the device in list form.

[1613] The device analyzes the received data and displays stamp and emoji candidates in the user interface (UI). Specifically, a list of stamps and emojis is displayed as a pop-up near the message input field.

[1614] User Selection and Submission

[1615] The user selects an appropriate stamp or emoji from the displayed ones.

[1616] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[1617] Specific examples

[1618] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1619] 1. The user types "Thank you for your hard work" and presses the send button.

[1620] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1621] 3. The server analyzes the received text and extracts the keyword "Thank you for your hard work."

[1622] 4. The server's emotion engine extracts emotions such as fatigue and gratitude from the text "Thank you for your hard work."

[1623] 5. The server searches its database for stamps and emojis related to "good work" (such as a tired face emoji or a stamp that says "good work!").

[1624] 6. The server sends these search results to the device.

[1625] 7. The device displays the received stamp and emoji candidates on the user's screen.

[1626] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1627] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it is possible to provide stamps and emojis that best suit the user's emotions, resulting in more effective communication. And because stamps and emojis can be used effectively, it is expected that they will be used actively, including newly purchased ones. This will also benefit stamp creators, and is expected to increase sales.

[1628] The processing flow will be explained below.

[1629] Step 1:

[1630] The user enters the text "Thank you for your hard work" in the message input field and presses the send button.

[1631] Specifically, the user enters text using the keyboard and taps the send button.

[1632] Step 2:

[1633] The terminal captures the input text "Thank you for your hard work" and sends a request to the server.

[1634] Specifically, when the submit button is pressed, the terminal obtains the text in the input field and prepares to send it to the server as an HTTP POST request.

[1635] Step 3:

[1636] The server forwards the text received from the user to the text analysis module.

[1637] Specifically, the server extracts text data from the request body and passes it to a natural language processing (NLP) engine.

[1638] Step 4:

[1639] The server's text analysis module uses a natural language processing (NLP) engine to analyze "otsukaresama" and extract the keyword "otsukaresama."

[1640] Specifically, the NLP engine tokenizes the text and applies a keyword extraction algorithm.

[1641] Step 5:

[1642] The server's emotion engine extracts emotions from the text "Thank you for your hard work" based on the keywords extracted by the text analysis module.

[1643] Specifically, the emotion engine uses an emotion analysis algorithm to analyze the input text for emotions such as fatigue and relief.

[1644] Step 6:

[1645] The server searches a database for related stamps and emojis based on the extracted keywords and emotions.

[1646] Specifically, the system generates a database query and searches a database of stamps and emojis corresponding to feelings of fatigue and comfort.

[1647] Step 7:

[1648] The server organizes the search results in list form and prepares to send them to the user's terminal.

[1649] Specifically, it receives the search results, packages them in JSON format, and prepares them as an HTTP response.

[1650] Step 8:

[1651] The server sends the generated response to the user's terminal.

[1652] Specifically, the operation involves sending an HTTP response and delivering data to the user terminal.

[1653] Step 9:

[1654] The terminal analyzes the received data and prepares to display stamps and emojis on the user screen.

[1655] Specifically, it parses the received JSON data and passes stamp and emoji information to the user interface component.

[1656] Step 10:

[1657] The terminal displays candidate stamps and pictograms on the user screen to be displayed to the user.

[1658] Specifically, the UI component displays a list of stamps and emojis on the screen in a pop-up format.

[1659] Step 11:

[1660] The user selects from the displayed stamps and emojis.

[1661] Specifically, the user taps to select the appropriate stamp or emoji from the displayed list of candidates.

[1662] Step 12:

[1663] The device inserts the selected stamp or emoji into the message box and performs the send action.

[1664] Specifically, the selected stamp or emoji is added to the message input field, and the message is sent by tapping the send button again.

[1665] Step 13:

[1666] The terminal again transmits the message including the selected stamps and pictograms to the server, and normal message processing is performed.

[1667] Specifically, the message data is sent to the server, and the server delivers the message to the recipient.

[1668] The process including the above steps allows users to quickly and efficiently select appropriate stickers and emojis based on text input, and the emotion engine provides stickers and emojis that best suit the user's emotions, resulting in more effective communication.

[1669] Example 2

[1670] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1671] In modern communication, attaching appropriate stickers and emojis to text messages is required to more effectively convey the emotion and nuance of the message. However, manually selecting appropriate stickers and emojis is time-consuming and inefficient. Furthermore, there is a lack of systems that instantly suggest stickers and emojis that match the content of the text, which can disrupt the flow of communication. There is a need for a system that can solve these problems and improve users' communication experience.

[1672] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1673] In this invention, the server includes means for analyzing received text and extracting keywords and emotions, means for searching a database for relevant stamps and emojis based on the extracted keywords and emotions, and means for returning the search results to the terminal, thereby enabling the server to quickly suggest appropriate stamps and emojis for the text entered by the user.

[1674] A "user" is an individual who operates the system to input text and send messages using stamps and emoticons.

[1675] A "terminal" is an electronic device that allows a user to input, send, or receive text, and includes smartphones, computers, and the like.

[1676] The "server" is a central processing unit that manages the processing of the entire system, analyzes text sent from the terminals, and searches for stamps and emoticons.

[1677] A "message input field" is a UI element that allows a user to input text on a terminal.

[1678] A "send action" is an operation for sending text entered by a user from a terminal to a server.

[1679] "Capture" is the process by which a terminal captures entered text and sends it to a server.

[1680] A "request" is a communication in which a terminal requests a server to analyze data or search for a stamp.

[1681] "Text analysis" is the process by which the server analyzes the text data it receives and extracts keywords and context.

[1682] A "keyword" is an important word that has a particular meaning or sentiment in text analysis.

[1683] "Emotion" is information extracted from text that indicates a psychological state, such as positive or negative.

[1684] A "natural language processing engine" is a system that includes algorithms and models for performing text analysis.

[1685] The "emotion analysis module" is a part of the system that has the function of analyzing emotions from text and extracting the type of emotion.

[1686] A "database" is a storage device within the system where stamp and emoji information is stored.

[1687] "Searching" is the process by which the server searches the database for relevant stamps or emojis based on specific criteria.

[1688] The "user interface" refers to the screen configuration and elements that allow the terminal to display stamp and pictogram candidates to the user.

[1689] A "pop-up display" is a display method that appears temporarily to show appropriate stamps or emojis for the text entered by the user.

[1690] A "message box" is a UI element that displays the final message, including stamps and emojis selected by the user.

[1691] This invention relates to a system and method for analyzing a user's input text and suggesting appropriate stamps and emojis based on the analysis results and the user's emotion recognition. This system automates the entire process from recognizing the user's input to recommending and displaying stamps and emojis, improving the user experience.

[1692] User text input

[1693] First, the user enters text into the message input field and presses the send button. For example, the user enters the text "Got it" and sends it. The device used by the user at this time is a device running a messaging application, such as a smartphone or computer.

[1694] Sending text data

[1695] The terminal captures the text "OK" entered when the send button is pressed and transmits this data to the server via the Internet.

[1696] Text analytics

[1697] When the server receives the text sent from the device, it passes it to a text analysis module and an emotion engine. The server's text analysis module analyzes the text "I understand" using a natural language processing (NLP) engine (e.g., BERT or GPT model). During the analysis, the text is tokenized, common stop words are removed, and the keyword "I understand" is extracted.

[1698] Emotion extraction

[1699] Next, the emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module. The emotion engine uses an emotion identification algorithm to extract positive emotion from the text "Got it."

[1700] Search for stamps and emojis

[1701] Based on the analysis results, the server searches the database for related stickers and emojis, taking into account the results extracted by the emotion engine (e.g., positive emotions), and searches for related stickers and emojis using a specific algorithm.

[1702] Submit search results

[1703] The server organizes the search results and sends them to the device in list format, along with metadata such as the display order.

[1704] Display in the user interface

[1705] The device analyzes the received data and displays candidate stickers and emojis in the user interface (UI). Specifically, a list of stickers and emojis is displayed as a pop-up near the message input field. This pop-up display is designed to allow users to quickly select stickers and emojis that are highly relevant to the text they have entered.

[1706] User Selection and Submission

[1707] Finally, the user selects an appropriate stamp or emoji from the displayed list, and the terminal inserts the selected stamp or emoji into the message box and sends it as the final message.

[1708] Specific examples

[1709] For example, consider the case where a user types "Thank you for your hard work" while chatting with a friend and presses the send button. This system operates as follows.

[1710] 1. The user types "Thank you for your hard work" and presses the send button.

[1711] 2. The device captures the text "Thank you for your hard work" and sends it to the server.

[1712] 3. The server analyzes the received text and extracts the keyword "Otsukaresama." This keyword is extracted using a natural language processing engine.

[1713] 4. The server's emotion engine extracts emotions such as fatigue and relief from the text "Thank you for your hard work." The emotion engine analyzes the emotions using an emotion identification algorithm.

[1714] 5. The server searches the database for stamps or emojis related to "good work" (for example, a tired face emoji or a stamp that says "good work!"). The database is a storage device that stores information about stamps and emojis.

[1715] 6. The server organizes these search results and adds metadata such as display order when sending them to the device.

[1716] 7. The device analyzes the received stamp and emoji candidates and displays them on the user's screen. The user interface module pops up the candidates to make it easier for the user to select.

[1717] 8. The user selects the appropriate stamp or emoji from the displayed suggestions, inserts it into a message, and sends it to a friend.

[1718] This system allows users to quickly and intuitively select appropriate stamps and emojis when entering text, making communication smoother. Furthermore, by using an emotion engine, it can provide stamps and emojis that best match the user's emotions, resulting in more effective communication.

[1719] Prompt Sentence Examples

[1720] An example of an input prompt for a generative AI model is shown below.

[1721] plaintext

[1722] Explain how the system works for a user who types "good work" while chatting with a friend.

[1723] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1724] Program processing steps

[1725] Step 1:

[1726] The user enters the text "OK" in the message input field and presses the send button.

[1727] The input data is the text "OK." This input is captured on the terminal, preparing it for the next step.

[1728] Step 2:

[1729] The terminal detects that the send button has been pressed, captures the entered text "OK" and sends this data to the server.

[1730] Specifically, the terminal sends the user's input text to the server as an HTTP request, and the output data is request data including the text "OK."

[1731] Step 3:

[1732] The server receives the text sent from the terminal and passes it to the text analysis module and emotion engine.

[1733] The input data is the text "OK". The server receives this and forwards it to the appropriate module for analysis. The output data is forwarded to the text analysis module and emotion engine.

[1734] Step 4:

[1735] The server's text analysis module analyzes the text "understanding" using a natural language processing (NLP) engine and extracts the keyword "understanding."

[1736] The input data is the text "OK". Specifically, the NLP engine tokenizes the text, removes stop words, and extracts important keywords. The output data is the keyword "OK".

[1737] Step 5:

[1738] The emotion engine of the server performs emotion analysis using the keywords extracted by the text analysis module.

[1739] The input data is the keyword "understanding." The emotion engine uses this to perform emotion analysis and extract, for example, positive emotions. The output data is positive emotion information.

[1740] Step 6:

[1741] Based on the analysis results, the server searches the database for related stamps and emojis.

[1742] The input data is the keyword "OK" and emotional information (positive emotion). The server uses this information to search for stamps and emojis in the database. Specifically, it retrieves relevant stamps and emojis using SQL queries. The output data is a list of appropriate stamps and emojis.

[1743] Step 7:

[1744] The server organizes the search results and sends them to the terminal in list form.

[1745] The input data is a list of stamps and emojis. The server organizes this data, adds metadata that determines the display order, and sends it to the device. The output data is a list of candidate stamps and emojis.

[1746] Step 8:

[1747] The device analyzes the data received and displays stamp and emoji candidates on the user interface (UI).

[1748] The input data is a list of candidate stamps and emojis received from the server. Specifically, the device analyzes this and displays it to the user in a popup format. The output data is a list of stamps and emojis to be displayed in the user interface.

[1749] Step 9:

[1750] The user selects the appropriate stamp or emoji from the displayed ones.

[1751] The input data is a list of stamps and emojis displayed on the device. The user selects from this list and chooses the stamps and emojis to use in the final message. The output data is the stamps and emojis selected by the user.

[1752] Step 10:

[1753] The device inserts the stamps and emojis selected by the user into the message box and sends it as the final message.

[1754] The input data are the stamps and emojis selected by the user. The device inserts them into the message box and sends them to friends etc. as the final message. The output data is the sent message.

[1755] (Application example 2)

[1756] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1757] Conventional messaging systems require users to manually select appropriate stamps and emojis based on the text they input, which is particularly inconvenient when many messages are exchanged in real time. This can disrupt communication between viewers and degrade the viewing experience, especially in live streaming services. To solve this problem, a system that automatically suggests appropriate stamps and emojis based on the user's text input is needed.

[1758] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1759] In this invention, the server includes: means for a user to input text into a message input field and perform a send action; means for a terminal to capture the input text and send a request to the server; means for the server to analyze the received text and extract keywords and emotions; means for the server to search a database for relevant stamps and emojis based on the extracted keywords and emotions; means for the server to return the search results to the terminal; means for the terminal to analyze the received data and display stamps and emojis on a user interface; means for the user to select from the presented stamps and emojis; means for the server to process real-time comments in the live streaming service; and means for the terminal to display recommended stamps and emojis when a user enters a comment during live streaming. This allows users to quickly and intuitively select appropriate stamps and emojis in real time, enabling smooth communication.

[1760] A "user" is a person who uses the system to enter and send messages.

[1761] A "message entry field" is an area on the screen where a user can enter text.

[1762] "Text" is a string of characters that the user inputs as a message.

[1763] A "send action" refers to an operation of sending text entered by a user from a terminal to a server.

[1764] A "terminal" is a device that a user uses to enter and send messages.

[1765] "Capture" refers to the terminal taking in and saving the text entered.

[1766] A "server" is a computer system that analyzes and processes received text.

[1767] A "request" refers to a data request sent by a terminal to a server.

[1768] "Analysis" is the process of linguistically breaking down received text and extracting meaning and emotion.

[1769] "Keywords" are words or phrases that have important meaning in text analysis.

[1770] "Emotion" refers to the user's psychological state or sensation extracted through text analysis.

[1771] "Related stamps and emojis" are image data that match based on the extracted keywords and emotions.

[1772] A "database" is a searchable collection of information in which stamps and emojis are stored.

[1773] "Search" is the act of locating information that meets specific criteria from a database.

[1774] "Return" refers to the server sending the search results back to the terminal.

[1775] A "user interface" is the display portion on the screen that can be directly operated by the user.

[1776] "Selection" refers to the act of the user choosing an appropriate stamp or emoji from the ones presented.

[1777] "Real-time comments" are messages that users enter instantly during live streaming.

[1778] A "live streaming service" refers to a service that distributes video and audio in real time over the Internet.

[1779] "Recommendation" refers to suggesting appropriate stamps or emojis to users.

[1780] In this invention, a system is constructed that automatically recommends appropriate stamps and emoticons to facilitate smooth communication between users in live distribution services. The system program and its processing are described below.

[1781] Explanation of program processing

[1782] First, a user enters text into the comment input field during a live broadcast and performs a send action. The device (smartphone or tablet) captures this input text and sends it directly to the server. The server analyzes the received text and extracts keywords and user sentiment.

[1783] The text analysis uses Python's TextBlob library, which determines the positive, negative, or neutral sentiment of the text and extracts related keywords based on that sentiment. The analysis results are then processed within the computer system, after which relevant stickers and emojis are searched for in a database that contains pre-stored stickers and emojis corresponding to each sentiment.

[1784] The recommended stickers and emojis are sent back from the server to the device and displayed on the user interface. The user can then select from the stickers and emojis presented and send them as the final message. This process allows users to quickly and intuitively select the appropriate stickers and emojis in real time, improving the viewing experience.

[1785] Hardware and software used

[1786] Hardware: smartphones, tablets, servers

[1787] Software: Python 3, TextBlob library, cloud database

[1788] Specific examples

[1789] For example, suppose a user enters a comment such as "This is the best live broadcast!" during a live broadcast and presses the send button. In this case, the system operates as follows:

[1790] 1. The user types "This is the best live show ever!" and presses the send button.

[1791] 2. The device captures this text and sends it to the server.

[1792] 3. The server analyzes the received text using a text analysis module to determine positive sentiment.

[1793] 4. The server searches its database for stickers or emojis that correspond to positive emotions (e.g., 😊, 👍, 🎉).

[1794] 5. The server sends these recommended emojis back to the device and displays them in the user interface.

[1795] 6. The user selects the appropriate emoji from the ones presented and sends it.

[1796] This system allows users to quickly select appropriate stamps and emojis for their comments in real time, making communication during live broadcasts smoother and improving the viewing experience.

[1797] Prompt Sentence Examples

[1798] The application is based on a generated AI model, and example prompts include:

[1799] "Create a system that analyzes comments entered by users and recommends appropriate emojis based on their sentiment."

[1800] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1801] Step 1:

[1802] The user enters text into the message input field and performs a send action. This allows the user to provide text data to the terminal. The input is manual input by the user, and the output is text data. For example, the user might enter "Great live show!"

[1803] Step 2:

[1804] The terminal captures the input text and sends a request to the server. Specifically, the terminal takes the text entered by the user and converts the text data into request data to be sent to the server. The input is the user's input text, and the output is the request data to the server.

[1805] Step 3:

[1806] The server analyzes the received text. The text analysis module uses a natural language processing engine (e.g., TextBlob library) to perform semantic analysis and sentiment extraction of the text. Through this processing, keywords and sentiment data are extracted. The input is the text data received from the device, and the output is the analyzed keywords and sentiment data. For example, positive sentiment can be extracted from "This was a great live show!"

[1807] Step 4:

[1808] The server searches a database for relevant stickers and emojis based on the extracted keywords and emotions. Specifically, the server generates a database query to search for stickers and emojis that match the emotions and keywords. The input is the parsed keywords and emotion data, and the output is a list of related stickers and emojis. For example, "😊", "👍", "🎉", etc.

[1809] Step 5:

[1810] The server returns the search results to the device. The server organizes the search results and generates a response to the requesting device. The input is a list of related stamps or emojis, and the output is the response data to the device.

[1811] Step 6:

[1812] The terminal analyzes the received data and displays stamps and emojis on the user interface. Specifically, the terminal analyzes the returned data it receives and displays stamp and emoji options to the user. The input is the returned data from the server, and the output is the stamps and emojis displayed on the terminal's user interface.

[1813] Step 7:

[1814] The user selects from the presented stamps and emojis. The user selects the appropriate one from the presented stamps and emojis and performs a selection action to send it as a message. The input is a list of stamps and emojis displayed on the device, and the output is the selected stamp or emoji.

[1815] Through these steps, the system can recommend appropriate stamps and emojis in real time based on the user's input text, improving the viewing experience.

[1816] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1817] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1818] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1819] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1820] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1821] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1822] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1823] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1824] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1825] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1826] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1827] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1828] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1829] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1830] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1831] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1832] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1833] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1834] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1835] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1836] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1837] The following is further disclosed regarding the above embodiment.

[1838] (Claim 1)

[1839] A means for a user to enter text into a message input field and perform a send action;

[1840] means for the terminal to capture the input text and send a request to the server;

[1841] A means for the server to analyze the received text and extract keywords and sentiments;

[1842] A means for searching a database for relevant stamps and emojis based on the keywords and emotions extracted by the server;

[1843] A means for the server to return search results to the terminal;

[1844] A means for analyzing the data received by the terminal and displaying stamps and emojis on the user's screen;

[1845] The system includes a means for a user to select from presented stamps or emojis.

[1846] (Claim 2)

[1847] 10. The system of claim 1, wherein the server comprises means for using a natural language processing engine for text analysis.

[1848] (Claim 3)

[1849] 2. The system according to claim 1, wherein the terminal comprises means for displaying recommended stamps and emojis through a user interface, allowing the user to select one and send it as a message.

[1850] "Example 1"

[1851] (Claim 1)

[1852] A means for a user to enter text into a message input field and perform a send action;

[1853] means for the terminal to capture the input text and send a request to the server;

[1854] A means for the server to analyze the received text and extract keywords and sentiments;

[1855] A means for searching a database for relevant visual information or symbols based on the keywords or emotions extracted by the server;

[1856] A means for the server to return search results to the terminal;

[1857] means for analyzing the data received by the terminal and displaying visual information or symbols on a user screen;

[1858] A system that includes a means for a user to select from presented visual information or symbols.

[1859] (Claim 2)

[1860] 10. The system of claim 1, wherein the server comprises means for using a natural language processing engine for text analysis.

[1861] (Claim 3)

[1862] 2. The system according to claim 1, wherein the terminal comprises means for displaying visual information or symbol recommendations through a user interface, which the user can select and send as a message.

[1863] "Application Example 1"

[1864] (Claim 1)

[1865] A means for a user to enter text into a message input field and perform a send action;

[1866] means for the terminal to capture the input text and send a request to the server;

[1867] A means for the server to analyze the received text and extract keywords and sentiments;

[1868] A means for searching a database for relevant stamps and emojis based on the keywords and emotions extracted by the server;

[1869] A means for the server to return search results to the terminal;

[1870] A means for analyzing the data received by the terminal and displaying stamps and emojis on the user's screen;

[1871] A means for the user to select from among the stamps and emojis presented;

[1872] A means for a user to enter text into the product review input field and perform a submit action;

[1873] means for the terminal to capture the entered text and transmit it to a server for suggesting related products and promotions;

[1874] A means for the server to analyze the received text and extract keywords and sentiments;

[1875] A means for searching a database for related products and promotions based on the extracted keywords and sentiments by the server;

[1876] A means for the server to return search results to the terminal;

[1877] means for analyzing the data received by the terminal and displaying related products and promotions on the user's screen;

[1878] The system includes a means for a user to select from presented related products and promotions.

[1879] (Claim 2)

[1880] 10. The system of claim 1, wherein the server comprises means for using a natural language processing engine for text analysis.

[1881] (Claim 3)

[1882] A means for the terminal to display recommended stamps and emojis through a user interface, and for the user to select one and send it as a message;

[1883] The system according to claim 1, further comprising means for displaying recommendations of related products and promotions through a user interface, allowing the user to select one and proceed to a purchase page.

[1884] "Example 2: Combining Emotion Engines"

[1885] (Claim 1)

[1886] A means for a user to enter text into a message input field and perform a send action;

[1887] means for the terminal to capture the input text and send a request to the server;

[1888] A means for the server to analyze the received text and extract keywords and sentiments;

[1889] A means for searching a database for relevant stamps and emojis based on the keywords and emotions extracted by the server;

[1890] A means for the server to return search results to the terminal;

[1891] A means for analyzing the data received by the terminal and displaying stamps and emojis on a user interface;

[1892] A means for the user to select from among the stamps and emojis presented;

[1893] A means for the device to insert the selected stamp or emoji into the message box and send it as the final message;

[1894] A system including:

[1895] (Claim 2)

[1896] 10. The system of claim 1, wherein the server comprises means for using a natural language processing engine and a sentiment analysis module for text analysis.

[1897] (Claim 3)

[1898] 2. The system according to claim 1, further comprising means for the terminal to pop up recommended stamps or emojis through a user interface, allowing the user to select one and send it as a message.

[1899] "Application example 2 when combining emotion engines"

[1900] (Claim 1)

[1901] A means for a user to enter text into a message input field and perform a send action;

[1902] means for the terminal to capture the input text and send a request to the server;

[1903] A means for the server to analyze the received text and extract keywords and sentiments;

[1904] A means for searching a database for relevant stamps and emojis based on the keywords and emotions extracted by the server;

[1905] A means for the server to return search results to the terminal;

[1906] A means for analyzing the data received by the terminal and displaying stamps and emojis on a user interface;

[1907] A means for the user to select from among the stamps and emojis presented;

[1908] A means for the server to process real-time comments in the live streaming service;

[1909] The system includes a means for displaying recommended stamps and emojis when a user enters a comment while the terminal is live streaming.

[1910] (Claim 2)

[1911] 10. The system of claim 1, wherein the server comprises means for using a natural language processing engine for text analysis.

[1912] (Claim 3)

[1913] The system according to claim 1, further comprising means for displaying recommended stamps and emojis during live streaming through a user interface, allowing the user to select one and send it as a message. [Explanation of symbols]

[1914] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to enter text into a message input field and perform a send action; means for the terminal to capture the input text and send a request to the server; A means for the server to analyze the received text and extract keywords and sentiments; A means for searching a database for relevant stamps and emojis based on the keywords and emotions extracted by the server; A means for the server to return search results to the terminal; A means for analyzing the data received by the terminal and displaying stamps and emojis on the user's screen; The system includes a means for a user to select from presented stamps or emojis.

2. The system of claim 1 , wherein the server comprises means for using a natural language processing engine for text analysis.

3. 2. The system according to claim 1, wherein the terminal comprises means for displaying recommended stamps and emoticons through a user interface, allowing the user to select one and send it as a message.

Images