system

Abstract

We provide the system. [Solution] A system including means for looking up the meaning of a word selected by the user on the chat screen of a messenger app, means for displaying the meaning of that word on the chat screen, means for automatically creating a question for a generative AI, and means for setting a character limit to prevent the display result from becoming redundant.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When a word with an unclear meaning appears on the talk screen of a messenger app, it is usually necessary to switch windows to look up the meaning of the word, resulting in the problem that the flow of the conversation is interrupted.

Means for Solving the Problems

[0005] [[ID=4"]] As a means for solving this problem, a system is provided that looks up the meaning of a word selected by a user and directly displays the result on the talk screen. In this system, a question to a generative AI is automatically created based on the selected word, and the result is displayed on the talk screen. Also, a character limit is set for the question to the generative AI to prevent the display result from being redundant.

Brief Description of the Drawings

[0006] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 1 of Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2 of Embodiment 2. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2 of Form Example 2. [Figure 15] This is a sequence diagram showing the processing flow of the data processing system in Embodiment 3 of Example 3. [Figure 16]This is a sequence diagram showing the processing flow of the data processing system in Application Example 3 of Form Example 3. [Figure 17] This is a sequence diagram showing the processing flow of the data processing system in Example 1 of the Form 1 when an emotion engine is combined. [Figure 18] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] This is a sequence diagram showing the processing flow of a data processing system in another embodiment. [Modes for carrying out the invention]

[0007] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0008] First, let's explain the terminology used in the following explanation.

[0009] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (TENSOR PROCESSING UNIT®).

[0010] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0011] In the following embodiments, the tagged storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0012] In the following embodiments, the tagged communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0014] [First Embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0019] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0022] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0026] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0027] "Example of form 1"

[0028] One embodiment of the present invention provides a function in the chat screen of a messenger app that allows the user to look up the meaning of a word they have selected. Specifically, when a user selects a particular word on the chat screen, a question about that word is automatically sent to a generative AI. The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen. At this time, a character limit is imposed on the question sent to the generative AI to prevent the answer from becoming redundant.

[0029] "Example of form 2"

[0030] As a concrete example, consider the case where a user selects the word "blockchain." When this word is selected, the generative AI automatically receives the question, "What is blockchain?" The generative AI generates an answer to this question, and that answer is displayed directly on the chat screen. At this time, the answer is kept concise due to a character limit.

[0031] The following describes the processing flow for each example of the form.

[0032] "Example of form 1"

[0033] Step 1: The user selects a specific word on the chat screen of the messenger app. Step 2: Based on the selected word, a question is automatically created for the generative AI.

[0034] Step 3: The generated question is sent to the generation AI.

[0035] Step 4: The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen.

[0036] "Example of form 2"

[0037] Step 1: The user selects the word "blockchain".

[0038] Step 2: The question "What is blockchain?" is automatically sent to the generative AI.

[0039] Step 3: The generative AI generates an answer to this question.

[0040] Step 4: The generated response will be displayed directly on the chat screen. At this time, the response will be kept concise due to a character limit.

[0041] (Example 1)

[0042] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0043] Modern information processing devices require users to quickly and easily look up the meaning of specific words and phrases. However, conventional methods require users to open other applications or websites, resulting in cumbersome operation. Furthermore, responses from generative AI models can be redundant, making it difficult for users to quickly obtain the information they need.

[0044] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0045] In this invention, the server includes means for looking up the meaning of a word or phrase selected by a user on the display screen of the information processing device, means for automatically creating a query to a generating AI model, and means for receiving a response from the generating AI model and displaying it on the display screen. This allows the user to quickly and directly confirm the meaning of the selected word or phrase on the display screen.

[0046] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and includes devices such as computers and smartphones.

[0047] "Display screen" refers to a screen or display used to visually display information in an information processing device.

[0048] "User" refers to an individual or group that operates an information processing device and utilizes specific functions or services.

[0049] "Words" refer to words or phrases that have meaning within a text or conversation.

[0050] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate natural language or provide responses based on input data.

[0051] An "inquiry" refers to a question or request made to obtain specific information.

[0052] "Answer" refers to the information or explanation provided in response to an inquiry.

[0053] "Character limit" refers to the maximum number of characters set to control the length of the information displayed.

[0054] This invention relates to a system for information processing that enables users to quickly look up the meaning of specific words or phrases. Specifically, it uses the chat screen of a messenger app to acquire information about words or phrases selected by the user through a generating AI model and displays it directly on the display screen.

[0055] The server uses, for example, OpenAI's GPT-3® as a generative AI model. This model has the ability to generate appropriate responses based on input prompt sentences using natural language processing techniques. The terminal detects words selected by the user on the chat screen and automatically generates prompt sentences related to those words. For example, if the user selects the word "algorithm," the terminal will generate the prompt sentence "Tell me what algorithm means."

[0056] The device sends this prompt message to the server, which queries the generative AI model. The generative AI model generates a response based on the received prompt message and sends it back to the server. The server sends this response back to the device, which displays the response on the chat screen. This allows the user to directly confirm the meaning of the selected words on the chat screen.

[0057] This system offers the advantage of allowing users to quickly obtain information without opening other applications or websites. Furthermore, the AI ​​model's responses have a character limit, eliminating redundant information and providing concise and essential information.

[0058] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0059] Step 1:

[0060] The user selects the word or phrase they want to look up in the chat screen of the messenger app. The selected word or phrase is detected by the device. As a result of this action, the device receives the selected word or phrase as input.

[0061] Step 2:

[0062] The device generates a prompt based on the selected word or phrase. Specifically, it creates a prompt in the format of "Tell me the meaning of the selected word or phrase." This prompt serves as input data for querying the generating AI model.

[0063] Step 3:

[0064] The terminal sends the generated prompt message to the server. The server receives this prompt message and prepares to query the generative AI model. Here, the input is the prompt message, and the output is the query to the generative AI model.

[0065] Step 4:

[0066] The server sends a prompt to the generation AI model. The generation AI model generates an answer based on the received prompt. In this process, the prompt is taken as input and an appropriate answer is output using natural language processing techniques.

[0067] Step 5:

[0068] The server receives a response from the AI ​​model. This response is generated based on the prompt and is concise due to character limits. The server sends this response to the terminal.

[0069] Step 6:

[0070] The device displays the response received from the server on the chat screen. The user can directly check the meaning of the selected word or phrase on the chat screen. In this step, the response from the server is received as input, and the display on the chat screen is considered output.

[0071] (Application Example 1)

[0072] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0073] Modern information display systems present a challenge in that users often struggle to quickly and accurately obtain detailed information about specific terms or products. In particular, in retail settings, store employees are required to provide timely and appropriate information in response to customer inquiries, but current systems sometimes fall short in this regard.

[0074] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0075] In this invention, the server includes means for looking up the meaning of a term selected by the user, means for displaying the meaning of that term on an information display device, and means for automatically creating questions for a generative AI. This enables the provision of detailed information about the term selected by the user in real time, and allows for the rapid and accurate provision of information through the information display device.

[0076] An "information display device" is a device that allows users to visually confirm information, and includes smart glasses and displays.

[0077] "User" refers to an individual or group that operates an information display device and obtains information.

[0078] A "term" is a word or phrase selected on an information display device that has a specific meaning or information associated with it.

[0079] A "generative AI" is an artificial intelligence system that generates answers in natural language to input questions.

[0080] "Method for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on terms selected by the user.

[0081] "Character limit" is a setting that restricts the number of characters displayed in the response from the generation AI to prevent it from becoming redundant.

[0082] "Detailed information" refers to additional information or explanations related to the selected term, intended to help users gain a deeper understanding.

[0083] "Providing information in real time" means providing information immediately in response to user requests, and means that information is displayed without delay.

[0084] To implement this invention, smart glasses are used as an information display device. Smart glasses are devices that visually confirm terms selected by the user and display their meaning and related information. When a user selects a specific term through the interface of the smart glasses, a question about that term is automatically generated and sent to a generative AI.

[0085] The server uses a generative AI, such as OpenAI's GPT-4 (registered trademark). This AI has the capability to generate natural language answers to input questions. The server automatically creates questions based on terms selected by the user and sends them to the generative AI. The AI's answers are adjusted to account for character limits and displayed on the smart glasses' screen.

[0086] As a concrete example, consider a scenario where a user asks a question about a product in a physical store. The user asks, "What material is this product made of?" and selects the term "material" using smart glasses. The server generates a prompt, "Tell me the details of the materials used in this product," and sends it to a generative AI. The AI ​​generates an answer such as, "This product is made from organic cotton," and displays it on the smart glasses.

[0087] Examples of prompt phrases include "Tell me the meaning of this word" or "Provide me information related to this product." This allows users to obtain detailed information in real time, enabling quick and accurate information provision.

[0088] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0089] Step 1:

[0090] The user selects a specific term through smart glasses. The input is the user's gaze or touch input, and the output is the text data of the selected term. This text data is used for subsequent processing.

[0091] Step 2:

[0092] The device receives a selected term and automatically generates a prompt message to send to the generative AI based on that term. The input is the text data of the term obtained in step 1, and the output is the generated prompt message. Specifically, it analyzes the term and generates a prompt message such as "Tell me the meaning of this word."

[0093] Step 3:

[0094] The server sends the generated prompt to the generative AI. The input is the prompt generated in step 2, and the output is the response from the generative AI. The server sends the prompt to the AI, and the AI ​​generates a response in natural language.

[0095] Step 4:

[0096] The server receives the response from the generative AI and adjusts it, taking into account character limits. The input is the response from the generative AI, and the output is the adjusted response text. Specifically, it limits the character count to prevent the response from becoming redundant and summarizes it as needed.

[0097] Step 5:

[0098] The device displays the adjusted response on the smart glasses' display. The input is the adjusted response text from step 4, and the output is information that the user can visually confirm. Specifically, the response is displayed on the smart glasses' display, allowing the user to confirm the information.

[0099] (Example 2)

[0100] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0101] Conventional information processing devices face the challenge of quickly and concisely acquiring and displaying information related to user-selected terms. Furthermore, in response generation using generative AI models, the displayed information tends to be redundant, highlighting the need to provide information in a user-friendly format.

[0102] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0103] In this invention, the server includes means for acquiring information about a word or phrase selected by a user on the display screen of an information processing device, means for automatically generating input sentences for a generating AI model based on that word or phrase, and means for generating a response to the input sentences using the generating AI model. This makes it possible to quickly and concisely acquire and display information about the word or phrase selected by the user.

[0104] An "information processing device" is a device used for inputting, processing, and outputting data, and includes computers, smartphones, and other similar devices.

[0105] A "display screen" is a screen in an information processing device that visually presents information to the user.

[0106] "User" refers to a person who operates an information processing device and acquires or processes information.

[0107] "Words" refer to words or phrases that have meaning within a text or conversation.

[0108] A "generative AI model" is a model that uses artificial intelligence technology to generate responses and information based on input data.

[0109] An "input sentence" is a sentence that is input to a generative AI model in order to generate information.

[0110] "Response" refers to the information or answer that a generative AI model generates based on the input sentence.

[0111] A "character limit" is a constraint that restricts the number of characters displayed in the information, in order to prevent the information from becoming redundant.

[0112] A description of embodiments for carrying out this invention will be given.

[0113] When a user selects a specific word or phrase on the device, the device detects this selection. Based on the selected word or phrase, the device generates a prompt. This prompt is used as input to a generative AI model. For example, a model using natural language processing techniques can be used as the generative AI model. Specifically, OpenAI's GPT-3 is a suitable example.

[0114] The terminal sends the generated prompt message to the server. The server receives this prompt message and generates a response using a generation AI model. The generated response is sent from the server to the terminal. The terminal displays the received response on its screen. At this time, the displayed response is kept concise due to a character limit.

[0115] As a concrete example, if the user selects the term "blockchain," the device generates a prompt message such as "What is blockchain?". The server receives this prompt message and uses a generation AI model to generate a response such as "Blockchain is a distributed ledger technology used to securely manage records of transactions." The device displays this response on its screen, which the user can then review.

[0116] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0117] Step 1:

[0118] The user selects a specific word or phrase on the device. The device detects this selection and receives the selected word or phrase as input. Based on this input, the device prepares to generate a prompt. Specifically, it monitors click and touch events on the user interface and retrieves the selected word or phrase.

[0119] Step 2:

[0120] The terminal generates a prompt based on the selected phrase. Using the phrase received as input, it creates a prompt in the format "What is this phrase?". This prompt is used as input to the generation AI model. Specifically, it performs string manipulation and incorporates the selected phrase into the prompt.

[0121] Step 3:

[0122] The terminal sends the generated prompt message to the server. The server receives this prompt message as input and prepares to pass it to the generating AI model. Specifically, it sends the prompt message to the server via network communication.

[0123] Step 4:

[0124] The server uses a generative AI model to generate a response to the received prompt. The generative AI model uses natural language processing with the prompt as input and outputs an appropriate response. Specifically, the process involves calling the generative AI model, providing the prompt as input, and generating the response.

[0125] Step 5:

[0126] The server sends the generated response to the terminal. The terminal receives this response as input and prepares to display it on the display screen. Specifically, it performs the action of sending the response to the terminal via network communication.

[0127] Step 6:

[0128] The terminal displays the received response on the display screen. At this time, a character limit is applied to ensure the response is concise and avoids redundancy. Specifically, the response is placed within the display area, and adjustments are made to ensure it does not exceed the character limit.

[0129] (Application Example 2)

[0130] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0131] In electronic transactions, a challenge exists in that users often find it difficult to quickly and concisely understand unfamiliar terminology or functions. This can hinder smooth transactions and potentially reduce transaction efficiency.

[0132] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0133] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for displaying the meaning of that term on the display screen, and means for automatically creating questions for a generative artificial intelligence. This enables the user to quickly understand terms related to electronic transactions and to proceed with transactions smoothly.

[0134] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data.

[0135] A "display screen" is a screen used to visually display information.

[0136] A "user" is a person who operates an information processing device.

[0137] A "term" is a word or phrase that has a specific meaning.

[0138] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate information based on given input.

[0139] "Methods for automatically generating questions" refers to a function that automatically generates relevant questions based on selected terms.

[0140] "Character limit" is a setting that restricts the number of characters in the displayed information to a certain range.

[0141] "Electronic transactions" refer to commercial transactions conducted via the internet.

[0142] A "concise explanation" is a short explanation that gets straight to the point.

[0143] The system for carrying out this invention includes a terminal as an information processing device and a server that utilizes generative artificial intelligence. The terminal has a display screen and provides an interface for inputting terms selected by the user. When the user selects a specific term, the terminal sends that term to the server.

[0144] Based on the received terms, the server automatically creates a relevant question for the generative artificial intelligence and sends it as a prompt. This prompt might be in the format of, for example, "What is cryptocurrency?". The generative AI then generates a concise explanation based on this prompt.

[0145] The generated explanation is sent from the server to the terminal and displayed on the terminal's screen. The displayed explanation is kept concise due to character limits, allowing users to quickly understand terminology related to electronic transactions.

[0146] For example, if a user selects the term "QR code payment," the server sends a prompt to the generative AI asking, "What is QR code payment?" The generative AI then generates a concise explanation such as, "QR code payment is a method of payment that involves scanning a QR code using a smartphone," and displays it on the terminal.

[0147] This system allows users to instantly understand unfamiliar terminology and conduct electronic transactions smoothly.

[0148] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0149] Step 1:

[0150] The user selects a specific term on the terminal's display screen. The input is the term selected by the user, and the output is that the term is sent to the terminal's system. The terminal then prepares this term for the next processing step.

[0151] Step 2:

[0152] The terminal sends the selected term to the server. The input is the term received from the terminal, and the output is the term data sent to the server. The server receives this data and proceeds to the next step.

[0153] Step 3:

[0154] The server automatically generates prompts to send to the AI ​​model based on the terms it receives. The input is the terms received by the server, and the output is the generated prompt. Specifically, the server generates a prompt in the format "What is a term?".

[0155] Step 4:

[0156] The server sends the generated prompt to the AI ​​model. The input is the prompt, and the output is the request sent to the AI ​​model. The AI ​​model receives this prompt and generates a response.

[0157] Step 5:

[0158] The generative AI model generates a concise explanation based on a prompt. The input is the prompt, and the output is the generated explanation. The generative AI model uses its internal database and algorithms to extract relevant information and create a concise explanation.

[0159] Step 6:

[0160] The server sends the descriptive text received from the generated AI model to the terminal. The input is the descriptive text from the generated AI model, and the output is the descriptive text sent to the terminal. The server transfers this data to the terminal.

[0161] Step 7:

[0162] The terminal displays the received description on its screen. The input is the description received from the server, and the output is the information visually displayed to the user. The terminal displays the description concisely, taking into account character limits.

[0163] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0164] "Example of form 1"

[0165] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine recognizes the user's emotions from the content of their messages on the messenger app's chat screen. Specifically, if a user says "I am very sad" on the messenger app's chat screen, the emotion engine recognizes from this statement that the user is feeling sad. This recognized emotion is then used to adjust the questions posed to the generative AI. For example, if it is recognized that the user is feeling sad, the questions posed to the generative AI are adjusted to take the user's emotions into account. As a result, more appropriate answers that reflect the user's emotions are displayed on the chat screen.

[0166] "Example of form 2"

[0167] In another embodiment of the present invention, after the emotion engine recognizes the user's emotion, the questions to the generative AI are adjusted based on that emotion. Specifically, if the user says, "I'm very angry," the emotion engine recognizes from this statement that the user is feeling angry. This recognized emotion is then used to adjust the questions to the generative AI. For example, if it is recognized that the user is feeling angry, the questions to the generative AI are adjusted to take the user's emotion into account, and a question such as "Why am I angry?" is generated. As a result, a more appropriate answer that corresponds to the user's emotion is displayed on the chat screen.

[0168] The following describes the processing flow for each example of the form.

[0169] "Example of form 1"

[0170] Step 1: The user makes a message on the chat screen of the messenger app.

[0171] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes sadness from the statement "I am very sad."

[0172] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is feeling sad, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0173] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0174] "Example of form 2"

[0175] Step 1: The user makes a message on the chat screen of the messenger app.

[0176] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes anger from the statement, "I am very angry."

[0177] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is angry, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0178] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0179] (Example 1)

[0180] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0181] Traditional messaging applications were inconvenient because users had to leave the application and use a separate dictionary application or website to look up the meaning of a specific word. Furthermore, they failed to provide information that took user emotions into consideration, resulting in inappropriate information tailored to the user's situation.

[0182] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0183] In this invention, the server includes means for looking up the meaning of a word selected by the user on the messaging application's communication screen, means for displaying the meaning of the word on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for setting a character limit to prevent the display result from becoming redundant, means for recognizing emotions from the user's statements, and means for adjusting the query to the generative artificial intelligence based on the recognized emotions. As a result, the user can quickly check the meaning of a word selected within the application and obtain appropriate information according to their emotions at that time.

[0184] A "messaging application" is software that allows users to send and receive text messages.

[0185] A "communication screen" is an interface within a messaging application that allows users to view and input messages.

[0186] "User" refers to an individual or group using a messaging application.

[0187] A "word or phrase" refers to a word or phrase that a user selects within a messaging application.

[0188] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate responses in natural language based on input information.

[0189] An "inquiry" refers to a question or request sent to a generative artificial intelligence to obtain information.

[0190] "Means of recognizing emotions" refers to technologies or algorithms that analyze the content of a user's statements and identify their emotional state.

[0191] "Character limit" refers to a constraint on the maximum number of characters that can be used to prevent queries to and responses to generative artificial intelligence from becoming redundant.

[0192] This invention is a system for looking up the meaning of a word or phrase selected by a user on the communication screen of a messaging application. The system provides a function that, when a user selects a specific word or phrase on the communication screen, automatically sends an inquiry about that word or phrase to a generative artificial intelligence. The generative artificial intelligence generates an answer to the inquiry and displays that answer directly on the communication screen.

[0193] The server generates a query to send to the generative artificial intelligence based on the words selected by the user. A character limit is imposed to prevent the query from becoming redundant. Furthermore, the server analyzes the user's utterance and uses an emotion engine to recognize the user's emotions. Based on the recognized emotions, the server adjusts the query to the generative AI to provide an appropriate response that reflects the user's feelings.

[0194] As a concrete example, if a user selects the word "empathy" on the communication screen, the server generates the inquiry "Please tell me the meaning of empathy." If the user says "I'm very tired today," the server recognizes that the user is tired and adjusts the inquiry to "Please briefly explain the meaning of empathy." Generative artificial intelligence generates an answer based on this inquiry, and the server displays that answer on the communication screen.

[0195] This system allows users to quickly confirm the meaning of selected words within messaging applications and obtain appropriate information tailored to their current emotions.

[0196] The flow of the specific processing in Example 1 will be explained using Figure 15.

[0197] Step 1:

[0198] The user selects a word or phrase on the communication screen.

[0199] Input: The word or phrase selected by the user on the communication screen.

[0200] Operation: The user selects the word or phrase they want to look up by long-pressing it on the messaging application's communication screen.

[0201] Output: The selected phrase is recognized by the system.

[0202] Step 2:

[0203] The terminal generates the query.

[0204] Input: Selected word or phrase.

[0205] Operation: The terminal generates a query in the format "Please tell me the meaning of the selected phrase." based on the selected phrase.

[0206] Output: The generated query.

[0207] Step 3:

[0208] The terminal sends a query to the server.

[0209] Input: Generated query.

[0210] Operation: The terminal sends the generated query to the server.

[0211] Output: The server receives the query.

[0212] Step 4:

[0213] The server recognizes the user's emotions.

[0214] Input: User's statement.

[0215] Operation: The server analyzes the user's statements on the communication screen and uses an emotion engine to recognize the user's emotions.

[0216] Output: Recognized user emotions.

[0217] Step 5:

[0218] The server will coordinate the queries.

[0219] Input: Recognized user sentiment, generated query.

[0220] Operation: The server adjusts queries to the generative artificial intelligence based on the perceived emotions. For example, if the server detects that the user is tired, the query will be adjusted to "Please briefly explain the meaning of the selected phrase."

[0221] Output: Adjusted query.

[0222] Step 6:

[0223] The server sends a query to the generative artificial intelligence.

[0224] Input: Adjusted query.

[0225] Operation: The server sends a pre-configured query to the generative artificial intelligence.

[0226] Output: Generative artificial intelligence receives the inquiry.

[0227] Step 7:

[0228] Generative artificial intelligence generates the answer.

[0229] Input: Adjusted query.

[0230] Operation: Generative artificial intelligence generates answers based on the received inquiry.

[0231] Output: Generated answer.

[0232] Step 8:

[0233] The server receives the response and sends it to the terminal.

[0234] Input: Generated response.

[0235] Operation: The server receives a response from the generative artificial intelligence and sends it to the terminal.

[0236] Output: The terminal receives the response.

[0237] Step 9:

[0238] The device displays the answer on the communication screen.

[0239] Input: Response received from the server.

[0240] Operation: The terminal displays the received response on the communication screen.

[0241] Output: The user confirms the answer on the communication screen.

[0242] (Application Example 1)

[0243] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0244] In today's information-saturated environment, it is difficult for users to quickly and accurately obtain information related to the content they are watching. Furthermore, there is a lack of information that resonates with users' emotions, resulting in a non-personalized viewing experience. Moreover, there is a need to provide appropriate information that responds to users' emotions while preventing information from becoming redundant.

[0245] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0246] In this invention, the server includes means for looking up the meaning of a word selected by the user on the chat screen of a messenger app, means for displaying the meaning of that word on the chat screen, and means for automatically creating a question for a generative AI. This allows the user to obtain information related to the content they are watching in real time and receive information that is tailored to their emotions.

[0247] A "messenger app" is software that allows users to send and receive text messages.

[0248] The "chat screen" is the interface within a messenger app that allows users to view and input messages.

[0249] "Methods for looking up the meaning of a word" refers to functions that allow users to obtain the definition and related information of a word they have selected.

[0250] "Generative AI" refers to artificial intelligence that generates answers in natural language to input questions.

[0251] "Methods for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on words and emotions selected by the user.

[0252] "Methods for setting character limits" refer to functions that restrict the number of characters in questions and answers to prevent the responses from generative AI from becoming redundant.

[0253] "Means of recognizing emotions" refers to technologies that analyze and recognize emotions from a user's statements and actions.

[0254] "Content being watched" refers to the media, such as movies or TV shows, that the user is currently viewing.

[0255] "Means of providing information in real time" refers to a function that provides information immediately in response to user requests.

[0256] "Means of providing information that resonates with emotions" refers to a function that provides appropriate and empathetic information according to the user's emotional state.

[0257] The system for carrying out this invention operates based on a messenger application installed on the user's device. The device utilizes a generative AI to look up the meaning of a word selected by the user on the chat screen. For example, OpenAI's GPT-3 can be used as the generative AI.

[0258] The device sends the word selected by the user to a generative AI, and the response is displayed on the chat screen. To prevent the response from being redundant, a character limit is imposed to maintain the conciseness of the information.

[0259] Furthermore, the device analyzes the user's emotions using an emotion recognition engine. For this emotion recognition, for example, Microsoft® Azure®'s Emotion API can be used. Based on the user's emotions, the questions to the generative AI are adjusted to provide information that is sensitive to those emotions.

[0260] For example, if a user selects the word "endemic" while watching a movie, the device sends this word to a generative AI to retrieve its meaning. If the device recognizes that the user is emotionally moved, it provides an emotionally resonant explanation such as, "This scene depicts the feelings of people affected by an endemic."

[0261] An example of a prompt message might be something like, "What is endemic? Please explain it to the emotionally moved user."

[0262] In this way, users can obtain information related to the content they are watching in real time and receive information that resonates with their emotions.

[0263] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[0264] Step 1:

[0265] The user selects a word on the chat screen of the messenger app.

[0266] The input is a word selected by the user, and the output is information about the selected word. The terminal retrieves this word and prepares for the next process.

[0267] Step 2:

[0268] The terminal generates a prompt sentence to send to the AI ​​model for generating the selected word.

[0269] The input is the selected word, and the output is the prompt sentence sent to the generating AI model. The terminal generates a prompt sentence in the format "What is the word?" based on the selected word.

[0270] Step 3:

[0271] The device sends a prompt message to the generating AI model and obtains a response.

[0272] The input is a prompt sentence, and the output is the answer from the generative AI model. The terminal sends the prompt sentence to the generative AI model (e.g., GPT-3) and receives an answer regarding the meaning of the word.

[0273] Step 4:

[0274] The terminal analyzes the user's emotion using an emotion recognition engine.

[0275] The input is the user's recent speech and actions, and the output is the recognized emotion. The terminal uses an emotion recognition engine (e.g., the Emotion API of Microsoft Azure) to analyze the user's emotion.

[0276] Step 5:

[0277] The terminal adjusts the question to the generative AI model based on the user's emotion.

[0278] The input is the recognized emotion and the answer from the generative AI model, and the output is information that conforms to the emotion. When the user is touched, the terminal adjusts the answer in a form that conforms to the emotion.

[0279] Step 6:

[0280] <​​​​​​​​​​​​​​​​Conventional information processing devices faced challenges in efficiently providing information using generative artificial intelligence when users looked up the meaning of terms they selected. Furthermore, they failed to consider the user's emotions, making it difficult to obtain appropriate answers that met the user's needs.

[0285] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0286] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for automatically creating a query to a generative artificial intelligence, and means for sentiment analysis to recognize the user's emotions. This enables efficient information provision based on the term selected by the user and the provision of appropriate answers according to the user's emotions.

[0287] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and can interact with users through a user interface.

[0288] A "display screen" is a screen or display used in an information processing device to visually display information such as text and images.

[0289] "User" refers to an individual or group that operates an information processing device and acquires or inputs information.

[0290] A "term" is a word or phrase with a specific meaning, and is the object selected on an information processing device.

[0291] "Generative artificial intelligence" refers to an artificial intelligence system that uses natural language processing technology to automatically generate answers and information based on input data.

[0292] An "inquiry" is a question or request sent to a generative artificial intelligence to seek information or answers.

[0293] "Emotion analysis means" refers to a technology or device for recognizing emotions from a user's statements and actions, and for processing information based on those emotions.

[0294] In an embodiment of this invention, the information processing device comprises a system that efficiently provides information by utilizing generative artificial intelligence based on terms selected by the user. Specifically, the terminal detects terms selected by the user through a display screen and automatically generates prompt sentences related to those terms. These prompt sentences are transmitted to a generative artificial intelligence model, which then generates the corresponding information or answers.

[0295] The server uses a model employing natural language processing technology as a generative artificial intelligence model. For example, by utilizing an advanced natural language processing model such as GPT-3, it is possible to generate appropriate answers to user questions. The generated answers are sent to the terminal and displayed concisely on the screen.

[0296] Furthermore, the device can recognize the user's emotions using emotion analysis tools. If the user says, "I'm very angry," the emotion analysis tools will recognize this emotion and adjust the prompt text for the generative artificial intelligence. For example, it might generate a prompt text such as, "Why are you angry?" and provide a response that corresponds to the user's emotions.

[0297] For example, if the user selects the term "blockchain," the prompt will be "What is blockchain?". This prompt is sent to a generative artificial intelligence, which then generates an answer. An example of a prompt that responds to the user's emotions would be "Why am I angry?". This allows for the provision of information tailored to the user's needs.

[0298] The flow of the specific processing in Example 2 will be explained using Figure 17.

[0299] Step 1:

[0300] The user selects a specific term on the display screen of the terminal. The terminal detects this selection and obtains the selected term as input data. Based on this input data, preparations are made to generate a prompt sentence. As a specific operation, when the user clicks on the term "blockchain", the terminal recognizes the term.

[0301] Step 2:

[0302] The terminal generates a prompt sentence based on the selected term. The input data is the selected term, and the output is the generated prompt sentence. As data processing, the term is converted into a question in the form of "What is XX?". As a specific operation, when the term "blockchain" is selected, the prompt sentence "What is blockchain?" is generated.

[0303] Step 3:

[0304] The terminal sends the generated prompt sentence to the server. The input is the generated prompt sentence, and the output is the completion of sending to the server. As a specific operation, the terminal sends the prompt sentence to the server via the network.

[0305] Step 4:

[0306] The server inputs the received prompt sentence into the generation AI model. The input is the prompt sentence, and the output is the answer by the generation AI model. As data calculation, the generation AI model uses natural language processing technology to generate an answer to the question. As a specific operation, the server inputs the prompt sentence into the generation AI model and obtains the answer.

[0307] Step 5:

[0308] The server sends the generated answer to the terminal. The input is the answer by the generation AI model, and the output is the completion of sending to the terminal. As a specific operation, the server sends the answer to the terminal via the network.

[0309] Step 6:

[0310] The device displays the received response on its screen. The input is the response received from the server, and the output is the provision of visual information to the user. Specifically, the device displays the response concisely on the chat screen.

[0311] Step 7:

[0312] (Another embodiment) When a user makes a statement expressing an emotion, the terminal recognizes the user's emotion using emotion analysis means. The input is the user's statement, and the output is the recognized emotion. Specifically, if the user says, "I am very angry," the terminal analyzes that emotion.

[0313] Step 8:

[0314] (Another embodiment) The terminal adjusts the prompt message based on the recognized emotion and sends it to the server. The input is the recognized emotion, and the output is the adjusted prompt message. Specifically, if the emotion of anger is recognized, the prompt message "Why am I angry?" is generated and sent to the server.

[0315] (Application Example 2)

[0316] Next, we will describe Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server," and the smart device 14 will be referred to as a "terminal."

[0317] In today's information and communication environment, users are increasingly exposed to vast amounts of information, but they face the challenge of efficiently acquiring information that aligns with their interests and emotions. Furthermore, systems capable of personalizing information based on user emotions are limited, highlighting the need for technologies to improve the user experience.

[0318] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0319] In this invention, the server includes means for looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen, means for displaying the meaning of that word or phrase on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for recognizing the user's emotions and adjusting the query to the generative artificial intelligence based on those emotions, and means for providing additional information related to the information the user is viewing in real time. This makes it possible to efficiently acquire information that matches the user's interests and emotions and improve the user experience.

[0320] A "messaging application" is software that allows users to send and receive information such as text, voice, and images in real time.

[0321] A "communication screen" is an interface within a messaging application that allows users to exchange information.

[0322] A "user" is an individual or organization that uses a messaging application to send and receive information.

[0323] A "word or phrase" is a word or phrase selected by the user, and is a unit of language that has a specific meaning.

[0324] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate natural language text based on given input.

[0325] An "inquiry" is a question or request sent to a generative artificial intelligence system in order to obtain information.

[0326] "Recognizing emotions" means analyzing and judging a user's emotional state based on their words and actions.

[0327] "Additional information" refers to supplementary data or explanations provided in relation to the information the user is viewing.

[0328] "Real-time" refers to information being processed immediately and provided without delay.

[0329] The system for implementing this invention consists of a server and a user terminal. The server has the function of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen and displaying that meaning on the communication screen. It also has the function of automatically creating queries to a generative artificial intelligence, recognizing the user's emotions, and adjusting the queries based on those emotions. Furthermore, it can provide additional information related to the information the user is viewing in real time.

[0330] In this system, the terminal sends the user's selected words to the server, which then analyzes their meaning. Natural language processing techniques are used for the analysis, specifically utilizing generative artificial intelligence (e.g., OpenAI GPT-4). The server also analyzes the user's emotions using an emotion recognition engine (e.g., Microsoft Azure Emotion API) and adjusts the prompts to the generative AI based on the results.

[0331] As a concrete example, when a user is watching a video about "blockchain," their device sends that phrase to the server. The server then sends the prompt "What is blockchain?" to a generative artificial intelligence system and displays the resulting answer on the communication screen. Furthermore, if the system recognizes that the user is in an excited state, it generates the prompt "What are some of the latest applications of blockchain?" and provides related information.

[0332] Examples of prompts include, "What are some of the latest applications of blockchain?" and "What blockchain-related news would you recommend for someone excited about blockchain?" This allows users to efficiently obtain information tailored to their interests and emotions.

[0333] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[0334] Step 1:

[0335] The terminal sends the words selected by the user on the messaging application's communication screen to the server. The input is the words selected by the user, and the output is the transmission of those words to the server. This action allows the server to receive the words to be analyzed.

[0336] Step 2:

[0337] The server analyzes the meaning of the received words. The input is words sent from the terminal, and the output is semantic information of those words. The server uses natural language processing techniques to analyze the meaning of the words and creates prompts to send to the generative artificial intelligence.

[0338] Step 3:

[0339] The server analyzes the user's emotions using an emotion recognition engine. The input is the user's statements and behavioral data, and the output is the user's emotional state. Based on this emotional information, the server adjusts prompts to the generative artificial intelligence.

[0340] Step 4:

[0341] The server sends a prompt to a generative artificial intelligence and receives a response. The input is a prepared prompt, and the output is the response from the generative artificial intelligence. The server sends a prompt and receives the response obtained from the generative artificial intelligence.

[0342] Step 5:

[0343] The server sends the acquired response to the terminal and displays it on the communication screen. The input is the response from the generative artificial intelligence, and the output is the transmission of the response to the terminal. The terminal displays the received response on the communication screen and provides it to the user.

[0344] Step 6:

[0345] The server provides additional information in real time related to the information the user is viewing. The input is the user's viewing information and emotional state, and the output is the related additional information. Based on the viewing information and emotional state, the server generates the related additional information and sends it to the terminal.

[0346] (Other examples)

[0347] Next, other embodiments will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0348] In today's information society, it is crucial for users to quickly and accurately obtain detailed meanings and related information for specific terms. However, traditional methods require users to manually search dictionaries or the internet, which is time-consuming and laborious. Furthermore, the information obtained is often redundant or does not meet the user's needs. An efficient system is needed to solve these problems.

[0349] The identification process performed by the identification processing unit 290 of the data processing device 12 in other embodiments is realized by the following means.

[0350] In this invention, the server includes means for querying a dictionary database for a word or phrase selected by the user to obtain semantic information; means for generating a prompt sentence to instruct a generative AI model to generate a specific response based on the selected word or phrase; and means for receiving the response from the generative AI model and adjusting the displayed content by applying a character limit. This enables the user to quickly and accurately obtain detailed information about the word or phrase.

[0351] An "information processing device" is a system that includes hardware and software for inputting, processing, and outputting data, and has the function of looking up the meaning of a word or phrase selected by the user.

[0352] A "display screen" is an interface in an information processing device that visually presents information to the user and is used to display acquired semantic information and responses from generative AI models.

[0353] A "generative AI model" is a model that uses artificial intelligence technology to generate responses based on given inputs, and is capable of natural language processing.

[0354] A "prompt sentence" is a sentence used to instruct a generative AI model to generate a specific response, and it is automatically generated based on selected words or phrases.

[0355] A "dictionary database" is a database containing words and their meanings, which a server queries to look up the meaning of a word selected by the user.

[0356] "Character limit" refers to a standard used to restrict the number of characters displayed to prevent the information from becoming redundant, and is applied to provide users with an appropriate amount of information.

[0357] The following describes "modes for carrying out the invention."

[0358] ---

[0359] This invention provides a system for quickly and accurately obtaining detailed meanings and related information of words selected by the user. The system includes an information processing device, a display screen, a generating AI model, and a dictionary database.

[0360] The server receives the word or phrase selected by the user on the terminal's display screen. The server queries a dictionary database for the selected word or phrase and retrieves its semantic information. This dictionary database can utilize online dictionary services such as the "Oxford Dictionaries API".

[0361] The acquired semantic information is sent to the user's device and displayed visually through the device's user interface. The user can then verify the displayed semantic information.

[0362] Next, the server creates a query for the generative AI model based on the selected phrase. To do this, the server generates a "prompt statement" that instructs the generative AI model to generate a specific response. An example of a prompt statement is, "Generate a detailed explanation of this phrase."

[0363] The generated prompt message is sent from the server to the generative AI model. Examples of generative AI models used include "OpenAI GPT-3". The generative AI model generates a response based on the received prompt message.

[0364] The generated response is returned to the server, which then sends this response to the user's terminal. The terminal displays the received response on its screen. To prevent the display from becoming redundant, the server applies a character limit to adjust the displayed content. For example, if the response is too long, it will either be summarized or the user will be able to expand the details.

[0365] In this way, a system is realized in which the server, terminal, and user work together to look up the meaning of selected words and phrases, and to provide detailed information using a generative AI model.

[0366] The flow of a specific process in another embodiment will be explained using Figure 19.

[0367] Step 1:

[0368] The user selects the word or phrase they want to look up on the terminal's display screen. The terminal sends the selected word or phrase to the server. The input is the word or phrase selected by the user, and the output is the transmission of the word or phrase to the server.

[0369] Step 2:

[0370] The server queries a dictionary database for the received word or phrase and retrieves its semantic information. Specifically, the server uses online dictionary services such as the "Oxford Dictionaries API" to search for semantic information corresponding to the word or phrase. The input is the selected word or phrase, and the output is the retrieved semantic information.

[0371] Step 3:

[0372] The server sends the acquired semantic information to the user's terminal. The terminal visually displays the semantic information through its user interface. The input is the semantic information sent from the server, and the output is the display of the semantic information on the terminal.

[0373] Step 4:

[0374] The server generates prompt statements to create queries to the generative AI model based on the selected phrase. Specifically, the server creates "prompt statements that instruct the generative AI model to generate a specific response." An example of a prompt statement is, "Generate a detailed explanation of this phrase." The input is the selected phrase, and the output is the generated prompt statement.

[0375] Step 5:

[0376] The server sends the generated prompt to the generative AI model. The generative AI model used is typically "OpenAI GPT-3". The generative AI model generates a response based on the received prompt. The input is the prompt, and the output is the response from the generative AI model.

[0377] Step 6:

[0378] The server receives responses from the generative AI model and adjusts the displayed content by applying character limits. Specifically, if the response is too long, it is summarized, or the user is allowed to expand on the details. The input is the response from the generative AI model, and the output is the adjusted response.

[0379] Step 7:

[0380] The server sends a pre-arranged response to the user's terminal. The terminal displays the received response on its screen. The user has the option to expand the details. The input is the pre-arranged response sent from the server, and the output is the display of the response on the terminal.

[0381] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0382] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0383] Other examples of generative AI include Gemini® (registered trademark) (Internet search). <url: https: gemini.google.com ?hl="ja">) are some examples.

[0384] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0385] [Second Embodiment]

[0386] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0387] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0388] The data processing device 12 includes a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a “computer” related to the technology of this disclosure.

[0389] Computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. A database 24 and a communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0390] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0391] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0392] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0393] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0394] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0395] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0396] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0397] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0398] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0399] "Example of form 1"

[0400] One embodiment of the present invention provides a function in the chat screen of a messenger app that allows the user to look up the meaning of a word they have selected. Specifically, when a user selects a particular word on the chat screen, a question about that word is automatically sent to a generative AI. The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen. At this time, a character limit is imposed on the question sent to the generative AI to prevent the answer from becoming redundant.

[0401] "Example of form 2"

[0402] As a concrete example, consider the case where a user selects the word "blockchain." When this word is selected, the generative AI automatically receives the question, "What is blockchain?" The generative AI generates an answer to this question, and that answer is displayed directly on the chat screen. At this time, the answer is kept concise due to a character limit.

[0403] The following describes the processing flow for each example of the form.

[0404] "Example of form 1"

[0405] Step 1: The user selects a specific word on the chat screen of the messenger app. Step 2: Based on the selected word, a question is automatically created for the generative AI.

[0406] Step 3: The generated question is sent to the generation AI.

[0407] Step 4: The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen.

[0408] "Example of form 2"

[0409] Step 1: The user selects the word "blockchain".

[0410] Step 2: The question "What is blockchain?" is automatically sent to the generative AI.

[0411] Step 3: The generative AI generates an answer to this question.

[0412] Step 4: The generated response will be displayed directly on the chat screen. At this time, the response will be kept concise due to a character limit.

[0413] (Example 1)

[0414] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0415] Modern information processing devices require users to quickly and easily look up the meaning of specific words and phrases. However, conventional methods require users to open other applications or websites, resulting in cumbersome operation. Furthermore, responses from generative AI models can be redundant, making it difficult for users to quickly obtain the information they need.

[0416] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0417] In this invention, the server includes means for looking up the meaning of a word or phrase selected by a user on the display screen of the information processing device, means for automatically creating a query to a generating AI model, and means for receiving a response from the generating AI model and displaying it on the display screen. This allows the user to quickly and directly confirm the meaning of the selected word or phrase on the display screen.

[0418] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and includes devices such as computers and smartphones.

[0419] "Display screen" refers to a screen or display used to visually display information in an information processing device.

[0420] "User" refers to an individual or group that operates an information processing device and utilizes specific functions or services.

[0421] "Words" refer to words or phrases that have meaning within a text or conversation.

[0422] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate natural language or provide responses based on input data.

[0423] An "inquiry" refers to a question or request made to obtain specific information.

[0424] "Answer" refers to the information or explanation provided in response to an inquiry.

[0425] "Character limit" refers to the maximum number of characters set to control the length of the information displayed.

[0426] This invention relates to a system for information processing that enables users to quickly look up the meaning of specific words or phrases. Specifically, it uses the chat screen of a messenger app to acquire information about words or phrases selected by the user through a generating AI model and displays it directly on the display screen.

[0427] The server uses, for example, OpenAI's GPT-3 as a generative AI model. This model has the ability to generate appropriate responses based on input prompt sentences using natural language processing techniques. The terminal detects words selected by the user on the chat screen and automatically generates prompt sentences related to those words. For example, if the user selects the word "algorithm," the terminal will generate the prompt sentence "Tell me the meaning of algorithm."

[0428] The device sends this prompt message to the server, which queries the generative AI model. The generative AI model generates a response based on the received prompt message and sends it back to the server. The server sends this response back to the device, which displays the response on the chat screen. This allows the user to directly confirm the meaning of the selected words on the chat screen.

[0429] This system offers the advantage of allowing users to quickly obtain information without opening other applications or websites. Furthermore, the AI ​​model's responses have a character limit, eliminating redundant information and providing concise and essential information.

[0430] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0431] Step 1:

[0432] The user selects the word or phrase they want to look up in the chat screen of the messenger app. The selected word or phrase is detected by the device. As a result of this action, the device receives the selected word or phrase as input.

[0433] Step 2:

[0434] The device generates a prompt based on the selected word or phrase. Specifically, it creates a prompt in the format of "Tell me the meaning of the selected word or phrase." This prompt serves as input data for querying the generating AI model.

[0435] Step 3:

[0436] The terminal sends the generated prompt message to the server. The server receives this prompt message and prepares to query the generative AI model. Here, the input is the prompt message, and the output is the query to the generative AI model.

[0437] Step 4:

[0438] The server sends a prompt to the generation AI model. The generation AI model generates an answer based on the received prompt. In this process, the prompt is taken as input and an appropriate answer is output using natural language processing techniques.

[0439] Step 5:

[0440] The server receives a response from the AI ​​model. This response is generated based on the prompt and is concise due to character limits. The server sends this response to the terminal.

[0441] Step 6:

[0442] The device displays the response received from the server on the chat screen. The user can directly check the meaning of the selected word or phrase on the chat screen. In this step, the response from the server is received as input, and the display on the chat screen is considered output.

[0443] (Application Example 1)

[0444] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0445] Modern information display systems present a challenge in that users often struggle to quickly and accurately obtain detailed information about specific terms or products. In particular, in retail settings, store employees are required to provide timely and appropriate information in response to customer inquiries, but current systems sometimes fall short in this regard.

[0446] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0447] In this invention, the server includes means for looking up the meaning of a term selected by the user, means for displaying the meaning of that term on an information display device, and means for automatically creating questions for a generative AI. This enables the provision of detailed information about the term selected by the user in real time, and allows for the rapid and accurate provision of information through the information display device.

[0448] An "information display device" is a device that allows users to visually confirm information, and includes smart glasses and displays.

[0449] "User" refers to an individual or group that operates an information display device and obtains information.

[0450] A "term" is a word or phrase selected on an information display device that has a specific meaning or information associated with it.

[0451] A "generative AI" is an artificial intelligence system that generates answers in natural language to input questions.

[0452] "Method for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on terms selected by the user.

[0453] "Character limit" is a setting that restricts the number of characters displayed in the response from the generation AI to prevent it from becoming redundant.

[0454] "Detailed information" refers to additional information or explanations related to the selected term, intended to help users gain a deeper understanding.

[0455] "Providing information in real time" means providing information immediately in response to user requests, and means that information is displayed without delay.

[0456] To implement this invention, smart glasses are used as an information display device. Smart glasses are devices that visually confirm terms selected by the user and display their meaning and related information. When a user selects a specific term through the interface of the smart glasses, a question about that term is automatically generated and sent to a generative AI.

[0457] The server uses a generative AI, such as OpenAI's GPT-4. This AI has the capability to generate natural language answers to input questions. The server automatically creates questions based on terms selected by the user and sends them to the generative AI. The AI's answers are adjusted to account for character limits and displayed on the smart glasses' screen.

[0458] As a concrete example, consider a scenario where a user asks a question about a product in a physical store. The user asks, "What material is this product made of?" and selects the term "material" using smart glasses. The server generates a prompt, "Tell me the details of the materials used in this product," and sends it to a generative AI. The AI ​​generates an answer such as, "This product is made from organic cotton," and displays it on the smart glasses.

[0459] Examples of prompt phrases include "Tell me the meaning of this word" or "Provide me information related to this product." This allows users to obtain detailed information in real time, enabling quick and accurate information provision.

[0460] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0461] Step 1:

[0462] The user selects a specific term through smart glasses. The input is the user's gaze or touch input, and the output is the text data of the selected term. This text data is used for subsequent processing.

[0463] Step 2:

[0464] The device receives a selected term and automatically generates a prompt message to send to the generative AI based on that term. The input is the text data of the term obtained in step 1, and the output is the generated prompt message. Specifically, it analyzes the term and generates a prompt message such as "Tell me the meaning of this word."

[0465] Step 3:

[0466] The server sends the generated prompt to the generative AI. The input is the prompt generated in step 2, and the output is the response from the generative AI. The server sends the prompt to the AI, and the AI ​​generates a response in natural language.

[0467] Step 4:

[0468] The server receives the response from the generative AI and adjusts it, taking into account character limits. The input is the response from the generative AI, and the output is the adjusted response text. Specifically, it limits the character count to prevent the response from becoming redundant and summarizes it as needed.

[0469] Step 5:

[0470] The device displays the adjusted response on the smart glasses' display. The input is the adjusted response text from step 4, and the output is information that the user can visually confirm. Specifically, the response is displayed on the smart glasses' display, allowing the user to confirm the information.

[0471] (Example 2)

[0472] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0473] Conventional information processing devices face the challenge of quickly and concisely acquiring and displaying information related to user-selected terms. Furthermore, in response generation using generative AI models, the displayed information tends to be redundant, highlighting the need to provide information in a user-friendly format.

[0474] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0475] In this invention, the server includes means for acquiring information about a word or phrase selected by a user on the display screen of an information processing device, means for automatically generating input sentences for a generating AI model based on that word or phrase, and means for generating a response to the input sentences using the generating AI model. This makes it possible to quickly and concisely acquire and display information about the word or phrase selected by the user.

[0476] An "information processing device" is a device used for inputting, processing, and outputting data, and includes computers, smartphones, and other similar devices.

[0477] A "display screen" is a screen in an information processing device that visually presents information to the user.

[0478] "User" refers to a person who operates an information processing device and acquires or processes information.

[0479] "Words" refer to words or phrases that have meaning within a text or conversation.

[0480] A "generative AI model" is a model that uses artificial intelligence technology to generate responses and information based on input data.

[0481] An "input sentence" is a sentence that is input to a generative AI model in order to generate information.

[0482] "Response" refers to the information or answer that a generative AI model generates based on the input sentence.

[0483] A "character limit" is a constraint that restricts the number of characters displayed in the information, in order to prevent the information from becoming redundant.

[0484] A description of embodiments for carrying out this invention will be given.

[0485] When a user selects a specific word or phrase on the device, the device detects this selection. Based on the selected word or phrase, the device generates a prompt. This prompt is used as input to a generative AI model. For example, a model using natural language processing techniques can be used as the generative AI model. Specifically, OpenAI's GPT-3 is a suitable example.

[0486] The terminal sends the generated prompt message to the server. The server receives this prompt message and generates a response using a generation AI model. The generated response is sent from the server to the terminal. The terminal displays the received response on its screen. At this time, the displayed response is kept concise due to a character limit.

[0487] As a concrete example, if the user selects the term "blockchain," the device generates a prompt message such as "What is blockchain?". The server receives this prompt message and uses a generation AI model to generate a response such as "Blockchain is a distributed ledger technology used to securely manage records of transactions." The device displays this response on its screen, which the user can then review.

[0488] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0489] Step 1:

[0490] The user selects a specific word or phrase on the device. The device detects this selection and receives the selected word or phrase as input. Based on this input, the device prepares to generate a prompt. Specifically, it monitors click and touch events on the user interface and retrieves the selected word or phrase.

[0491] Step 2:

[0492] The terminal generates a prompt based on the selected phrase. Using the phrase received as input, it creates a prompt in the format "What is this phrase?". This prompt is used as input to the generation AI model. Specifically, it performs string manipulation and incorporates the selected phrase into the prompt.

[0493] Step 3:

[0494] The terminal sends the generated prompt message to the server. The server receives this prompt message as input and prepares to pass it to the generating AI model. Specifically, it sends the prompt message to the server via network communication.

[0495] Step 4:

[0496] The server uses a generative AI model to generate a response to the received prompt. The generative AI model uses natural language processing with the prompt as input and outputs an appropriate response. Specifically, the process involves calling the generative AI model, providing the prompt as input, and generating the response.

[0497] Step 5:

[0498] The server sends the generated response to the terminal. The terminal receives this response as input and prepares to display it on the display screen. Specifically, it performs the action of sending the response to the terminal via network communication.

[0499] Step 6:

[0500] The terminal displays the received response on the display screen. At this time, a character limit is applied to ensure the response is concise and avoids redundancy. Specifically, the response is placed within the display area, and adjustments are made to ensure it does not exceed the character limit.

[0501] (Application Example 2)

[0502] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0503] In electronic transactions, a challenge exists in that users often find it difficult to quickly and concisely understand unfamiliar terminology or functions. This can hinder smooth transactions and potentially reduce transaction efficiency.

[0504] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0505] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for displaying the meaning of that term on the display screen, and means for automatically creating questions for a generative artificial intelligence. This enables the user to quickly understand terms related to electronic transactions and to proceed with transactions smoothly.

[0506] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data.

[0507] A "display screen" is a screen used to visually display information.

[0508] A "user" is a person who operates an information processing device.

[0509] A "term" is a word or phrase that has a specific meaning.

[0510] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate information based on given input.

[0511] "Methods for automatically generating questions" refers to a function that automatically generates relevant questions based on selected terms.

[0512] "Character limit" is a setting that restricts the number of characters in the displayed information to a certain range.

[0513] "Electronic transactions" refer to commercial transactions conducted via the internet.

[0514] A "concise explanation" is a short explanation that gets straight to the point.

[0515] The system for carrying out this invention includes a terminal as an information processing device and a server that utilizes generative artificial intelligence. The terminal has a display screen and provides an interface for inputting terms selected by the user. When the user selects a specific term, the terminal sends that term to the server.

[0516] Based on the received terms, the server automatically creates a relevant question for the generative artificial intelligence and sends it as a prompt. This prompt might be in the format of, for example, "What is cryptocurrency?". The generative AI then generates a concise explanation based on this prompt.

[0517] The generated explanation is sent from the server to the terminal and displayed on the terminal's screen. The displayed explanation is kept concise due to character limits, allowing users to quickly understand terminology related to electronic transactions.

[0518] For example, if a user selects the term "QR code payment," the server sends a prompt to the generative AI asking, "What is QR code payment?" The generative AI then generates a concise explanation such as, "QR code payment is a method of payment that involves scanning a QR code using a smartphone," and displays it on the device.

[0519] This system allows users to instantly understand unfamiliar terminology and conduct electronic transactions smoothly.

[0520] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0521] Step 1:

[0522] The user selects a specific term on the terminal's display screen. The input is the term selected by the user, and the output is that the term is sent to the terminal's system. The terminal then prepares this term for the next processing step.

[0523] Step 2:

[0524] The terminal sends the selected term to the server. The input is the term received from the terminal, and the output is the term data sent to the server. The server receives this data and proceeds to the next step.

[0525] Step 3:

[0526] The server automatically generates prompts to send to the AI ​​model based on the terms it receives. The input is the terms received by the server, and the output is the generated prompt. Specifically, the server generates a prompt in the format "What is a term?".

[0527] Step 4:

[0528] The server sends the generated prompt to the AI ​​model. The input is the prompt, and the output is the request sent to the AI ​​model. The AI ​​model receives this prompt and generates a response.

[0529] Step 5:

[0530] The generative AI model generates a concise explanation based on a prompt. The input is the prompt, and the output is the generated explanation. The generative AI model uses its internal database and algorithms to extract relevant information and create a concise explanation.

[0531] Step 6:

[0532] The server sends the descriptive text received from the generated AI model to the terminal. The input is the descriptive text from the generated AI model, and the output is the descriptive text sent to the terminal. The server transfers this data to the terminal.

[0533] Step 7:

[0534] The terminal displays the received description on its screen. The input is the description received from the server, and the output is the information visually displayed to the user. The terminal displays the description concisely, taking into account character limits.

[0535] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0536] "Example of form 1"

[0537] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine recognizes the user's emotions from the content of their messages on the messenger app's chat screen. Specifically, if a user says "I am very sad" on the messenger app's chat screen, the emotion engine recognizes from this statement that the user is feeling sad. This recognized emotion is then used to adjust the questions posed to the generative AI. For example, if it is recognized that the user is feeling sad, the questions posed to the generative AI are adjusted to take the user's emotions into account. As a result, more appropriate answers that reflect the user's emotions are displayed on the chat screen.

[0538] "Example of form 2"

[0539] In another embodiment of the present invention, after the emotion engine recognizes the user's emotion, the questions to the generative AI are adjusted based on that emotion. Specifically, if the user says, "I'm very angry," the emotion engine recognizes from this statement that the user is feeling angry. This recognized emotion is then used to adjust the questions to the generative AI. For example, if it is recognized that the user is feeling angry, the questions to the generative AI are adjusted to take the user's emotion into account, and a question such as "Why am I angry?" is generated. As a result, a more appropriate answer that corresponds to the user's emotion is displayed on the chat screen.

[0540] The following describes the processing flow for each example of the form.

[0541] "Example of form 1"

[0542] Step 1: The user makes a message on the chat screen of the messenger app.

[0543] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes sadness from the statement "I am very sad."

[0544] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is feeling sad, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0545] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0546] "Example of form 2"

[0547] Step 1: The user makes a message on the chat screen of the messenger app.

[0548] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes anger from the statement, "I am very angry."

[0549] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is angry, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0550] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0551] (Example 1)

[0552] Next, we will describe Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0553] Traditional messaging applications were inconvenient because users had to leave the application and use a separate dictionary application or website to look up the meaning of a specific word. Furthermore, they failed to provide information that took user emotions into consideration, resulting in inappropriate information tailored to the user's situation.

[0554] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0555] In this invention, the server includes means for looking up the meaning of a word selected by the user on the messaging application's communication screen, means for displaying the meaning of the word on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for setting a character limit to prevent the display result from becoming redundant, means for recognizing emotions from the user's statements, and means for adjusting the query to the generative artificial intelligence based on the recognized emotions. As a result, the user can quickly check the meaning of a word selected within the application and obtain appropriate information according to their emotions at that time.

[0556] A "messaging application" is software that allows users to send and receive text messages.

[0557] A "communication screen" is an interface within a messaging application that allows users to view and input messages.

[0558] "User" refers to an individual or group using a messaging application.

[0559] A "word or phrase" refers to a word or phrase that a user selects within a messaging application.

[0560] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate responses in natural language based on input information.

[0561] An "inquiry" refers to a question or request sent to a generative artificial intelligence to obtain information.

[0562] "Means of recognizing emotions" refers to technologies or algorithms that analyze the content of a user's statements and identify their emotional state.

[0563] "Character limit" refers to a constraint on the maximum number of characters that can be used to prevent queries to and responses to generative artificial intelligence from becoming redundant.

[0564] This invention is a system for looking up the meaning of a word or phrase selected by a user on the communication screen of a messaging application. The system provides a function that, when a user selects a specific word or phrase on the communication screen, automatically sends an inquiry about that word or phrase to a generative artificial intelligence. The generative artificial intelligence generates an answer to the inquiry and displays that answer directly on the communication screen.

[0565] The server generates a query to send to the generative artificial intelligence based on the words selected by the user. A character limit is imposed to prevent the query from becoming redundant. Furthermore, the server analyzes the user's utterance and uses an emotion engine to recognize the user's emotions. Based on the recognized emotions, the server adjusts the query to the generative AI to provide an appropriate response that reflects the user's feelings.

[0566] As a concrete example, if a user selects the word "empathy" on the communication screen, the server generates the inquiry "Please tell me the meaning of empathy." If the user says "I'm very tired today," the server recognizes that the user is tired and adjusts the inquiry to "Please briefly explain the meaning of empathy." Generative artificial intelligence generates an answer based on this inquiry, and the server displays that answer on the communication screen.

[0567] This system allows users to quickly confirm the meaning of selected words within messaging applications and obtain appropriate information tailored to their current emotions.

[0568] The flow of the specific processing in Example 1 will be explained using Figure 15.

[0569] Step 1:

[0570] The user selects a word or phrase on the communication screen.

[0571] Input: The word or phrase selected by the user on the communication screen.

[0572] Operation: The user selects the word or phrase they want to look up by long-pressing it on the messaging application's communication screen.

[0573] Output: The selected phrase is recognized by the system.

[0574] Step 2:

[0575] The terminal generates the query.

[0576] Input: Selected word or phrase.

[0577] Operation: The terminal generates a query in the format "Please tell me the meaning of the selected phrase." based on the selected phrase.

[0578] Output: The generated query.

[0579] Step 3:

[0580] The terminal sends a query to the server.

[0581] Input: Generated query.

[0582] Operation: The terminal sends the generated query to the server.

[0583] Output: The server receives the query.

[0584] Step 4:

[0585] The server recognizes the user's emotions.

[0586] Input: User's statement.

[0587] Operation: The server analyzes the user's statements on the communication screen and uses an emotion engine to recognize the user's emotions.

[0588] Output: Recognized user emotions.

[0589] Step 5:

[0590] The server will coordinate the queries.

[0591] Input: Recognized user sentiment, generated query.

[0592] Operation: The server adjusts queries to the generative artificial intelligence based on the perceived emotions. For example, if the server detects that the user is tired, the query will be adjusted to "Please briefly explain the meaning of the selected phrase."

[0593] Output: Adjusted query.

[0594] Step 6:

[0595] The server sends a query to the generative artificial intelligence.

[0596] Input: Adjusted query.

[0597] Operation: The server sends a pre-configured query to the generative artificial intelligence.

[0598] Output: Generative artificial intelligence receives the inquiry.

[0599] Step 7:

[0600] Generative artificial intelligence generates the answer.

[0601] Input: Adjusted query.

[0602] Operation: Generative artificial intelligence generates answers based on the received inquiry.

[0603] Output: Generated answer.

[0604] Step 8:

[0605] The server receives the response and sends it to the terminal.

[0606] Input: Generated response.

[0607] Operation: The server receives a response from the generative artificial intelligence and sends it to the terminal.

[0608] Output: The terminal receives the response.

[0609] Step 9:

[0610] The device displays the answer on the communication screen.

[0611] Input: Response received from the server.

[0612] Operation: The terminal displays the received response on the communication screen.

[0613] Output: The user confirms the answer on the communication screen.

[0614] (Application Example 1)

[0615] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0616] In today's information-saturated environment, it is difficult for users to quickly and accurately obtain information related to the content they are watching. Furthermore, there is a lack of information that resonates with users' emotions, resulting in a non-personalized viewing experience. Moreover, there is a need to provide appropriate information that responds to users' emotions while preventing information from becoming redundant.

[0617] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0618] In this invention, the server includes means for looking up the meaning of a word selected by the user on the chat screen of a messenger app, means for displaying the meaning of that word on the chat screen, and means for automatically creating a question for a generative AI. This allows the user to obtain information related to the content they are watching in real time and receive information that is tailored to their emotions.

[0619] A "messenger app" is software that allows users to send and receive text messages.

[0620] The "chat screen" is the interface within a messenger app that allows users to view and input messages.

[0621] "Methods for looking up the meaning of a word" refers to functions that allow users to obtain the definition and related information of a word they have selected.

[0622] "Generative AI" refers to artificial intelligence that generates answers in natural language to input questions.

[0623] "Methods for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on words and emotions selected by the user.

[0624] "Methods for setting character limits" refer to functions that restrict the number of characters in questions and answers to prevent the responses from generative AI from becoming redundant.

[0625] "Means of recognizing emotions" refers to technologies that analyze and recognize emotions from a user's statements and actions.

[0626] "Content being watched" refers to the media, such as movies or TV shows, that the user is currently viewing.

[0627] "Means of providing information in real time" refers to a function that provides information immediately in response to user requests.

[0628] "Means of providing information that resonates with emotions" refers to a function that provides appropriate and empathetic information according to the user's emotional state.

[0629] The system for carrying out this invention operates based on a messenger application installed on the user's device. The device utilizes a generative AI to look up the meaning of a word selected by the user on the chat screen. For example, OpenAI's GPT-3 can be used as the generative AI.

[0630] The device sends the word selected by the user to a generative AI, and the response is displayed on the chat screen. To prevent the response from being redundant, a character limit is imposed to maintain the conciseness of the information.

[0631] Furthermore, the device uses an emotion recognition engine to analyze the user's emotions. For this emotion recognition, for example, Microsoft Azure's Emotion API can be used. Based on the user's emotions, the questions to the generative AI are adjusted to provide information that is sensitive to those emotions.

[0632] For example, if a user selects the word "endemic" while watching a movie, the device sends this word to a generative AI to retrieve its meaning. If the device recognizes that the user is emotionally moved, it provides an emotionally resonant explanation such as, "This scene depicts the feelings of people affected by an endemic."

[0633] An example of a prompt message might be something like, "What is endemic? Please explain it to the emotionally moved user."

[0634] In this way, users can obtain information related to the content they are watching in real time and receive information that resonates with their emotions.

[0635] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[0636] Step 1:

[0637] The user selects a word on the chat screen of the messenger app.

[0638] The input is a word selected by the user, and the output is information about the selected word. The terminal retrieves this word and prepares for the next process.

[0639] Step 2:

[0640] The terminal generates a prompt sentence to send to the AI ​​model for generating the selected word.

[0641] The input is the selected word, and the output is the prompt sentence sent to the generating AI model. The terminal generates a prompt sentence in the format "What is the word?" based on the selected word.

[0642] Step 3:

[0643] The device sends a prompt message to the generating AI model and obtains a response.

[0644] The input is a prompt sentence, and the output is the response from the generative AI model. The terminal sends a prompt sentence to the generative AI model (e.g., GPT-3) and receives a response regarding the meaning of the words.

[0645] Step 4:

[0646] The device uses an emotion recognition engine to analyze the user's emotions.

[0647] The input is the user's recent statements and actions, and the output is the recognized emotion. The device uses an emotion recognition engine (e.g., Microsoft Azure's Emotion API) to analyze the user's emotion.

[0648] Step 5:

[0649] The device adjusts the questions it asks the generated AI model based on the user's emotions.

[0650] The input consists of recognized emotions and responses from a generative AI model, while the output is information that aligns with those emotions. If the user is emotional, the device adjusts the response to reflect those emotions.

[0651] Step 6:

[0652] The device displays the adjusted information on the chat screen.

[0653] The input is information that resonates with emotions, and the output is information displayed on the chat screen. The device displays the meaning of words to the user in a way that resonates with their emotions.

[0654] (Example 2)

[0655] Next, we will describe Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0656] Conventional information processing devices faced challenges in efficiently providing information using generative artificial intelligence when users looked up the meaning of terms they selected. Furthermore, they failed to consider the user's emotions, making it difficult to obtain appropriate answers that met the user's needs.

[0657] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0658] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for automatically creating a query to a generative artificial intelligence, and means for sentiment analysis to recognize the user's emotions. This enables efficient information provision based on the term selected by the user and the provision of appropriate answers according to the user's emotions.

[0659] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and can interact with users through a user interface.

[0660] A "display screen" is a screen or display used in an information processing device to visually display information such as text and images.

[0661] "User" refers to an individual or group that operates an information processing device and acquires or inputs information.

[0662] A "term" is a word or phrase with a specific meaning, and is the object selected on an information processing device.

[0663] "Generative artificial intelligence" refers to an artificial intelligence system that uses natural language processing technology to automatically generate answers and information based on input data.

[0664] An "inquiry" is a question or request sent to a generative artificial intelligence to seek information or answers.

[0665] "Emotion analysis means" refers to a technology or device for recognizing emotions from a user's statements and actions, and for processing information based on those emotions.

[0666] In an embodiment of this invention, the information processing device constitutes a system that efficiently provides information by utilizing generative artificial intelligence based on terms selected by the user. Specifically, the terminal detects terms selected by the user through the display screen and automatically generates prompt sentences related to those terms. These prompt sentences are transmitted to a generative artificial intelligence model, which generates the corresponding information or answers.

[0667] The server uses a model employing natural language processing technology as a generative artificial intelligence model. For example, by utilizing an advanced natural language processing model such as GPT-3, it is possible to generate appropriate answers to user questions. The generated answers are sent to the terminal and displayed concisely on the screen.

[0668] Furthermore, the device can recognize the user's emotions using emotion analysis tools. If the user says, "I'm very angry," the emotion analysis tools will recognize this emotion and adjust the prompt text for the generative artificial intelligence. For example, it might generate a prompt text such as, "Why are you angry?" and provide a response that corresponds to the user's emotions.

[0669] For example, if the user selects the term "blockchain," the prompt will be "What is blockchain?". This prompt is sent to a generative artificial intelligence, which then generates an answer. An example of a prompt that responds to the user's emotions would be "Why am I angry?". This allows for the provision of information tailored to the user's needs.

[0670] The flow of the specific processing in Example 2 will be explained using Figure 17.

[0671] Step 1:

[0672] The user selects a specific term on the device's display screen. The device detects this selection and retrieves the selected term as input data. Based on this input data, it prepares to generate a prompt message. Specifically, when the user clicks the term "blockchain," the device recognizes that term.

[0673] Step 2:

[0674] The terminal generates a prompt based on the selected term. The input data is the selected term, and the output is the generated prompt. As a data processing step, the term is converted into a question in the format "What is XX?". Specifically, if the term "blockchain" is selected, the prompt "What is blockchain?" is generated.

[0675] Step 3:

[0676] The terminal sends the generated prompt message to the server. The input is the generated prompt message, and the output is the completion of the transmission to the server. Specifically, the terminal sends the prompt message to the server via the network.

[0677] Step 4:

[0678] The server inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is the answer generated by the generative AI model. As a data calculation, the generative AI model uses natural language processing techniques to generate an answer to the question. Specifically, the server inputs the prompt sentence into the generative AI model and obtains the answer.

[0679] Step 5:

[0680] The server sends the generated response to the terminal. The input is the response from the generating AI model, and the output is the completion of the transmission to the terminal. Specifically, the server sends the response to the terminal via the network.

[0681] Step 6:

[0682] The device displays the received response on its screen. The input is the response received from the server, and the output is the provision of visual information to the user. Specifically, the device displays the response concisely on the chat screen.

[0683] Step 7:

[0684] (Another embodiment) When a user makes a statement expressing an emotion, the terminal recognizes the user's emotion using emotion analysis means. The input is the user's statement, and the output is the recognized emotion. Specifically, if the user says, "I am very angry," the terminal analyzes that emotion.

[0685] Step 8:

[0686] (Another embodiment) The terminal adjusts the prompt message based on the recognized emotion and sends it to the server. The input is the recognized emotion, and the output is the adjusted prompt message. Specifically, if the emotion of anger is recognized, the prompt message "Why am I angry?" is generated and sent to the server.

[0687] (Application Example 2)

[0688] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0689] In today's information and communication environment, users are increasingly exposed to vast amounts of information, but they face the challenge of efficiently acquiring information that aligns with their interests and emotions. Furthermore, systems capable of personalizing information based on user emotions are limited, highlighting the need for technologies to improve the user experience.

[0690] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0691] In this invention, the server includes means for looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen, means for displaying the meaning of that word or phrase on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for recognizing the user's emotions and adjusting the query to the generative artificial intelligence based on those emotions, and means for providing additional information related to the information the user is viewing in real time. This makes it possible to efficiently acquire information that matches the user's interests and emotions and improve the user experience.

[0692] A "messaging application" is software that allows users to send and receive information such as text, voice, and images in real time.

[0693] A "communication screen" is an interface within a messaging application that allows users to exchange information.

[0694] A "user" is an individual or organization that uses a messaging application to send and receive information.

[0695] A "word or phrase" is a word or phrase selected by the user, and is a unit of language that has a specific meaning.

[0696] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate natural language text based on given input.

[0697] An "inquiry" is a question or request sent to a generative artificial intelligence system in order to obtain information.

[0698] "Recognizing emotions" means analyzing and judging a user's emotional state based on their words and actions.

[0699] "Additional information" refers to supplementary data or explanations provided in relation to the information the user is viewing.

[0700] "Real-time" refers to information being processed immediately and provided without delay.

[0701] The system for implementing this invention consists of a server and a user terminal. The server has the function of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen and displaying that meaning on the communication screen. It also has the function of automatically creating queries to a generative artificial intelligence, recognizing the user's emotions, and adjusting the queries based on those emotions. Furthermore, it can provide additional information related to the information the user is viewing in real time.

[0702] In this system, the terminal sends the user's selected words to the server, which then analyzes their meaning. Natural language processing techniques are used for the analysis, specifically utilizing generative artificial intelligence (e.g., OpenAI GPT-4). The server also analyzes the user's emotions using an emotion recognition engine (e.g., Microsoft Azure Emotion API) and adjusts the prompts to the generative AI based on the results.

[0703] As a concrete example, when a user is watching a video about "blockchain," their device sends that phrase to the server. The server then sends the prompt "What is blockchain?" to a generative artificial intelligence system and displays the resulting answer on the communication screen. Furthermore, if the system recognizes that the user is in an excited state, it generates the prompt "What are some of the latest applications of blockchain?" and provides related information.

[0704] Examples of prompts include, "What are some of the latest applications of blockchain?" and "What blockchain-related news would you recommend for someone excited about blockchain?" This allows users to efficiently obtain information tailored to their interests and emotions.

[0705] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[0706] Step 1:

[0707] The terminal sends the words selected by the user on the messaging application's communication screen to the server. The input is the words selected by the user, and the output is the transmission of those words to the server. This action allows the server to receive the words to be analyzed.

[0708] Step 2:

[0709] The server analyzes the meaning of the received words. The input is words sent from the terminal, and the output is semantic information of those words. The server uses natural language processing techniques to analyze the meaning of the words and creates prompts to send to the generative artificial intelligence.

[0710] Step 3:

[0711] The server analyzes the user's emotions using an emotion recognition engine. The input is the user's statements and behavioral data, and the output is the user's emotional state. Based on this emotional information, the server adjusts prompts to the generative artificial intelligence.

[0712] Step 4:

[0713] The server sends a prompt to a generative artificial intelligence and obtains a response. The input is a prepared prompt, and the output is the response from the generative artificial intelligence. The server sends a prompt and receives the response obtained from the generative artificial intelligence.

[0714] Step 5:

[0715] The server sends the acquired response to the terminal and displays it on the communication screen. The input is the response from the generative artificial intelligence, and the output is the transmission of the response to the terminal. The terminal displays the received response on the communication screen and provides it to the user.

[0716] Step 6:

[0717] The server provides additional information in real time related to the information the user is viewing. The input is the user's viewing information and emotional state, and the output is the related additional information. Based on the viewing information and emotional state, the server generates the related additional information and sends it to the terminal.

[0718] (Other examples)

[0719] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[0720] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0721] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0722] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[0723] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0724] [Third Embodiment]

[0725] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0726] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0727] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0728] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0729] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0730] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0731] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0732] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0733] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0734] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0735] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0736] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[0737] "Example of form 1"

[0738] One embodiment of the present invention provides a function in the chat screen of a messenger app that allows the user to look up the meaning of a word they have selected. Specifically, when a user selects a particular word on the chat screen, a question about that word is automatically sent to a generative AI. The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen. At this time, a character limit is imposed on the question sent to the generative AI to prevent the answer from becoming redundant.

[0739] "Example of form 2"

[0740] As a concrete example, consider the case where a user selects the word "blockchain." When this word is selected, the generative AI automatically receives the question, "What is blockchain?" The generative AI generates an answer to this question, and that answer is displayed directly on the chat screen. At this time, the answer is kept concise due to a character limit.

[0741] The following describes the processing flow for each example of the form.

[0742] "Example of form 1"

[0743] Step 1: The user selects a specific word on the chat screen of the messenger app. Step 2: Based on the selected word, a question is automatically created for the generative AI.

[0744] Step 3: The generated question is sent to the generation AI.

[0745] Step 4: The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen.

[0746] "Example of form 2"

[0747] Step 1: The user selects the word "blockchain".

[0748] Step 2: The question "What is blockchain?" is automatically sent to the generative AI.

[0749] Step 3: The generative AI generates an answer to this question.

[0750] Step 4: The generated response will be displayed directly on the chat screen. At this time, the response will be kept concise due to a character limit.

[0751] (Example 1)

[0752] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0753] Modern information processing devices require users to quickly and easily look up the meaning of specific words and phrases. However, conventional methods require users to open other applications or websites, resulting in cumbersome operation. Furthermore, responses from generative AI models can be redundant, making it difficult for users to quickly obtain the information they need.

[0754] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0755] In this invention, the server includes means for looking up the meaning of a word or phrase selected by a user on the display screen of the information processing device, means for automatically creating a query to a generating AI model, and means for receiving a response from the generating AI model and displaying it on the display screen. This allows the user to quickly and directly confirm the meaning of the selected word or phrase on the display screen.

[0756] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and includes devices such as computers and smartphones.

[0757] "Display screen" refers to a screen or display used to visually display information in an information processing device.

[0758] "User" refers to an individual or group that operates an information processing device and utilizes specific functions or services.

[0759] "Words" refer to words or phrases that have meaning within a text or conversation.

[0760] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate natural language or provide responses based on input data.

[0761] An "inquiry" refers to a question or request made to obtain specific information.

[0762] "Answer" refers to the information or explanation provided in response to an inquiry.

[0763] "Character limit" refers to the maximum number of characters set to control the length of the information displayed.

[0764] This invention relates to a system for information processing that enables users to quickly look up the meaning of specific words or phrases. Specifically, it uses the chat screen of a messenger app to acquire information about words or phrases selected by the user through a generating AI model and displays it directly on the display screen.

[0765] The server uses, for example, OpenAI's GPT-3 as a generative AI model. This model has the ability to generate appropriate responses based on input prompt sentences using natural language processing techniques. The terminal detects words selected by the user on the chat screen and automatically generates prompt sentences related to those words. For example, if the user selects the word "algorithm," the terminal will generate the prompt sentence "Tell me the meaning of algorithm."

[0766] The device sends this prompt message to the server, which queries the generative AI model. The generative AI model generates a response based on the received prompt message and sends it back to the server. The server sends this response back to the device, which displays the response on the chat screen. This allows the user to directly confirm the meaning of the selected words on the chat screen.

[0767] This system offers the advantage of allowing users to quickly obtain information without opening other applications or websites. Furthermore, the AI ​​model's responses have a character limit, eliminating redundant information and providing concise and essential information.

[0768] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0769] Step 1:

[0770] The user selects the word or phrase they want to look up in the chat screen of the messenger app. The selected word or phrase is detected by the device. As a result of this action, the device receives the selected word or phrase as input.

[0771] Step 2:

[0772] The device generates a prompt based on the selected word or phrase. Specifically, it creates a prompt in the format of "Tell me the meaning of the selected word or phrase." This prompt serves as input data for querying the generating AI model.

[0773] Step 3:

[0774] The terminal sends the generated prompt message to the server. The server receives this prompt message and prepares to query the generative AI model. Here, the input is the prompt message, and the output is the query to the generative AI model.

[0775] Step 4:

[0776] The server sends a prompt to the generation AI model. The generation AI model generates an answer based on the received prompt. In this process, the prompt is taken as input and an appropriate answer is output using natural language processing techniques.

[0777] Step 5:

[0778] The server receives a response from the AI ​​model. This response is generated based on the prompt and is concise due to character limits. The server sends this response to the terminal.

[0779] Step 6:

[0780] The device displays the response received from the server on the chat screen. The user can directly check the meaning of the selected word or phrase on the chat screen. In this step, the response from the server is received as input, and the display on the chat screen is considered output.

[0781] (Application Example 1)

[0782] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0783] Modern information display systems present a challenge in that users often struggle to quickly and accurately obtain detailed information about specific terms or products. In particular, in retail settings, store employees are required to provide timely and appropriate information in response to customer inquiries, but current systems sometimes fall short in this regard.

[0784] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0785] In this invention, the server includes means for looking up the meaning of a term selected by the user, means for displaying the meaning of that term on an information display device, and means for automatically creating questions for a generative AI. This enables the provision of detailed information about the term selected by the user in real time, and allows for the rapid and accurate provision of information through the information display device.

[0786] An "information display device" is a device that allows users to visually confirm information, and includes smart glasses and displays.

[0787] "User" refers to an individual or group that operates an information display device and obtains information.

[0788] A "term" is a word or phrase selected on an information display device that has a specific meaning or information associated with it.

[0789] A "generative AI" is an artificial intelligence system that generates answers in natural language to input questions.

[0790] "Method for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on terms selected by the user.

[0791] "Character limit" is a setting that restricts the number of characters displayed in the response from the generation AI to prevent it from becoming redundant.

[0792] "Detailed information" refers to additional information or explanations related to the selected term, intended to help users gain a deeper understanding.

[0793] "Providing information in real time" means providing information immediately in response to user requests, and means that information is displayed without delay.

[0794] To implement this invention, smart glasses are used as an information display device. Smart glasses are devices that visually confirm terms selected by the user and display their meaning and related information. When a user selects a specific term through the interface of the smart glasses, a question about that term is automatically generated and sent to a generative AI.

[0795] The server uses a generative AI, such as OpenAI's GPT-4. This AI has the capability to generate natural language answers to input questions. The server automatically creates questions based on terms selected by the user and sends them to the generative AI. The AI's answers are adjusted to account for character limits and displayed on the smart glasses' screen.

[0796] As a concrete example, consider a scenario where a user asks a question about a product in a physical store. The user asks, "What material is this product made of?" and selects the term "material" using smart glasses. The server generates a prompt, "Tell me the details of the materials used in this product," and sends it to a generative AI. The AI ​​generates an answer such as, "This product is made from organic cotton," and displays it on the smart glasses.

[0797] Examples of prompt phrases include "Tell me the meaning of this word" or "Provide me information related to this product." This allows users to obtain detailed information in real time, enabling quick and accurate information provision.

[0798] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0799] Step 1:

[0800] The user selects a specific term through smart glasses. The input is the user's gaze or touch input, and the output is the text data of the selected term. This text data is used for subsequent processing.

[0801] Step 2:

[0802] The device receives a selected term and automatically generates a prompt message to send to the generative AI based on that term. The input is the text data of the term obtained in step 1, and the output is the generated prompt message. Specifically, it analyzes the term and generates a prompt message such as "Tell me the meaning of this word."

[0803] Step 3:

[0804] The server sends the generated prompt to the generative AI. The input is the prompt generated in step 2, and the output is the response from the generative AI. The server sends the prompt to the AI, and the AI ​​generates a response in natural language.

[0805] Step 4:

[0806] The server receives the response from the generative AI and adjusts it, taking into account character limits. The input is the response from the generative AI, and the output is the adjusted response text. Specifically, it limits the character count to prevent the response from becoming redundant and summarizes it as needed.

[0807] Step 5:

[0808] The device displays the adjusted response on the smart glasses' display. The input is the adjusted response text from step 4, and the output is information that the user can visually confirm. Specifically, the response is displayed on the smart glasses' display, allowing the user to confirm the information.

[0809] (Example 2)

[0810] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0811] Conventional information processing devices face the challenge of quickly and concisely acquiring and displaying information related to user-selected terms. Furthermore, in response generation using generative AI models, the displayed information tends to be redundant, highlighting the need to provide information in a user-friendly format.

[0812] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0813] In this invention, the server includes means for acquiring information about a word or phrase selected by a user on the display screen of an information processing device, means for automatically generating input sentences for a generating AI model based on that word or phrase, and means for generating a response to the input sentences using the generating AI model. This makes it possible to quickly and concisely acquire and display information about the word or phrase selected by the user.

[0814] An "information processing device" is a device used for inputting, processing, and outputting data, and includes computers, smartphones, and other similar devices.

[0815] A "display screen" is a screen in an information processing device that visually presents information to the user.

[0816] "User" refers to a person who operates an information processing device and acquires or processes information.

[0817] "Words" refer to words or phrases that have meaning within a text or conversation.

[0818] A "generative AI model" is a model that uses artificial intelligence technology to generate responses and information based on input data.

[0819] An "input sentence" is a sentence that is input to a generative AI model in order to generate information.

[0820] "Response" refers to the information or answer that a generative AI model generates based on the input sentence.

[0821] A "character limit" is a constraint that restricts the number of characters displayed in the information, in order to prevent the information from becoming redundant.

[0822] A description of embodiments for carrying out this invention will be given.

[0823] When a user selects a specific word or phrase on the device, the device detects this selection. Based on the selected word or phrase, the device generates a prompt. This prompt is used as input to a generative AI model. For example, a model using natural language processing techniques can be used as the generative AI model. Specifically, OpenAI's GPT-3 is a suitable example.

[0824] The terminal sends the generated prompt message to the server. The server receives this prompt message and generates a response using a generation AI model. The generated response is sent from the server to the terminal. The terminal displays the received response on its screen. At this time, the displayed response is kept concise due to a character limit.

[0825] As a concrete example, if the user selects the term "blockchain," the device generates a prompt message such as "What is blockchain?". The server receives this prompt message and uses a generation AI model to generate a response such as "Blockchain is a distributed ledger technology used to securely manage records of transactions." The device displays this response on its screen, which the user can then review.

[0826] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0827] Step 1:

[0828] The user selects a specific word or phrase on the device. The device detects this selection and receives the selected word or phrase as input. Based on this input, the device prepares to generate a prompt. Specifically, it monitors click and touch events on the user interface and retrieves the selected word or phrase.

[0829] Step 2:

[0830] The terminal generates a prompt based on the selected phrase. Using the phrase received as input, it creates a prompt in the format "What is this phrase?". This prompt is used as input to the generation AI model. Specifically, it performs string manipulation and incorporates the selected phrase into the prompt.

[0831] Step 3:

[0832] The terminal sends the generated prompt message to the server. The server receives this prompt message as input and prepares to pass it to the generating AI model. Specifically, it sends the prompt message to the server via network communication.

[0833] Step 4:

[0834] The server uses a generative AI model to generate a response to the received prompt. The generative AI model uses natural language processing with the prompt as input and outputs an appropriate response. Specifically, the process involves calling the generative AI model, providing the prompt as input, and generating the response.

[0835] Step 5:

[0836] The server sends the generated response to the terminal. The terminal receives this response as input and prepares to display it on the display screen. Specifically, it performs the action of sending the response to the terminal via network communication.

[0837] Step 6:

[0838] The terminal displays the received response on the display screen. At this time, a character limit is applied to ensure the response is concise and avoids redundancy. Specifically, the response is placed within the display area, and adjustments are made to ensure it does not exceed the character limit.

[0839] (Application Example 2)

[0840] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

[0841] In electronic transactions, a challenge exists in that users often find it difficult to quickly and concisely understand unfamiliar terminology or functions. This can hinder smooth transactions and potentially reduce transaction efficiency.

[0842] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0843] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for displaying the meaning of that term on the display screen, and means for automatically creating questions for a generative artificial intelligence. This enables the user to quickly understand terms related to electronic transactions and to proceed with transactions smoothly.

[0844] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data.

[0845] A "display screen" is a screen used to visually display information.

[0846] A "user" is a person who operates an information processing device.

[0847] A "term" is a word or phrase that has a specific meaning.

[0848] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate information based on given input.

[0849] "Methods for automatically generating questions" refers to a function that automatically generates relevant questions based on selected terms.

[0850] "Character limit" is a setting that restricts the number of characters in the displayed information to a certain range.

[0851] "Electronic transactions" refer to commercial transactions conducted via the internet.

[0852] A "concise explanation" is a short explanation that gets straight to the point.

[0853] The system for carrying out this invention includes a terminal as an information processing device and a server that utilizes generative artificial intelligence. The terminal has a display screen and provides an interface for inputting terms selected by the user. When the user selects a specific term, the terminal sends that term to the server.

[0854] Based on the received terms, the server automatically creates a relevant question for the generative artificial intelligence and sends it as a prompt. This prompt might be in the format of, for example, "What is cryptocurrency?". The generative AI then generates a concise explanation based on this prompt.

[0855] The generated explanation is sent from the server to the terminal and displayed on the terminal's screen. The displayed explanation is kept concise due to character limits, allowing users to quickly understand terminology related to electronic transactions.

[0856] For example, if a user selects the term "QR code payment," the server sends a prompt to the generative AI asking, "What is QR code payment?" The generative AI then generates a concise explanation such as, "QR code payment is a method of payment that involves scanning a QR code using a smartphone," and displays it on the device.

[0857] This system allows users to instantly understand unfamiliar terminology and conduct electronic transactions smoothly.

[0858] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0859] Step 1:

[0860] The user selects a specific term on the terminal's display screen. The input is the term selected by the user, and the output is that the term is sent to the terminal's system. The terminal then prepares this term for the next processing step.

[0861] Step 2:

[0862] The terminal sends the selected term to the server. The input is the term received from the terminal, and the output is the term data sent to the server. The server receives this data and proceeds to the next step.

[0863] Step 3:

[0864] The server automatically generates prompts to send to the AI ​​model based on the terms it receives. The input is the terms received by the server, and the output is the generated prompt. Specifically, the server generates a prompt in the format "What is a term?".

[0865] Step 4:

[0866] The server sends the generated prompt to the AI ​​model. The input is the prompt, and the output is the request sent to the AI ​​model. The AI ​​model receives this prompt and generates a response.

[0867] Step 5:

[0868] The generative AI model generates a concise explanation based on a prompt. The input is the prompt, and the output is the generated explanation. The generative AI model uses its internal database and algorithms to extract relevant information and create a concise explanation.

[0869] Step 6:

[0870] The server sends the descriptive text received from the generated AI model to the terminal. The input is the descriptive text from the generated AI model, and the output is the descriptive text sent to the terminal. The server transfers this data to the terminal.

[0871] Step 7:

[0872] The terminal displays the received description on its screen. The input is the description received from the server, and the output is the information visually displayed to the user. The terminal displays the description concisely, taking into account character limits.

[0873] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0874] "Example of form 1"

[0875] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine recognizes the user's emotions from the content of their messages on the messenger app's chat screen. Specifically, if a user says "I am very sad" on the messenger app's chat screen, the emotion engine recognizes from this statement that the user is feeling sad. This recognized emotion is then used to adjust the questions posed to the generative AI. For example, if it is recognized that the user is feeling sad, the questions posed to the generative AI are adjusted to take the user's emotions into account. As a result, more appropriate answers that reflect the user's emotions are displayed on the chat screen.

[0876] "Example of form 2"

[0877] In another embodiment of the present invention, after the emotion engine recognizes the user's emotion, the questions to the generative AI are adjusted based on that emotion. Specifically, if the user says, "I'm very angry," the emotion engine recognizes from this statement that the user is feeling angry. This recognized emotion is then used to adjust the questions to the generative AI. For example, if it is recognized that the user is feeling angry, the questions to the generative AI are adjusted to take the user's emotion into account, and a question such as "Why am I angry?" is generated. As a result, a more appropriate answer that corresponds to the user's emotion is displayed on the chat screen.

[0878] The following describes the processing flow for each example of the form.

[0879] "Example of form 1"

[0880] Step 1: The user makes a message on the chat screen of the messenger app.

[0881] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes sadness from the statement "I am very sad."

[0882] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is feeling sad, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0883] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0884] "Example of form 2"

[0885] Step 1: The user makes a message on the chat screen of the messenger app.

[0886] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes anger from the statement, "I am very angry."

[0887] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is angry, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[0888] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[0889] (Example 1)

[0890] Next, we will describe Embodiment 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0891] Traditional messaging applications were inconvenient because users had to leave the application and use a separate dictionary application or website to look up the meaning of a specific word. Furthermore, they failed to provide information that took user emotions into consideration, resulting in inappropriate information tailored to the user's situation.

[0892] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0893] In this invention, the server includes means for looking up the meaning of a word selected by the user on the messaging application's communication screen, means for displaying the meaning of the word on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for setting a character limit to prevent the display result from becoming redundant, means for recognizing emotions from the user's statements, and means for adjusting the query to the generative artificial intelligence based on the recognized emotions. As a result, the user can quickly check the meaning of a word selected within the application and obtain appropriate information according to their emotions at that time.

[0894] A "messaging application" is software that allows users to send and receive text messages.

[0895] A "communication screen" is an interface within a messaging application that allows users to view and input messages.

[0896] "User" refers to an individual or group using a messaging application.

[0897] A "word or phrase" refers to a word or phrase that a user selects within a messaging application.

[0898] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate responses in natural language based on input information.

[0899] An "inquiry" refers to a question or request sent to a generative artificial intelligence to obtain information.

[0900] "Means of recognizing emotions" refers to technologies or algorithms that analyze the content of a user's statements and identify their emotional state.

[0901] "Character limit" refers to a constraint on the maximum number of characters that can be used to prevent queries to and responses to generative artificial intelligence from becoming redundant.

[0902] This invention is a system for looking up the meaning of a word or phrase selected by a user on the communication screen of a messaging application. The system provides a function that, when a user selects a specific word or phrase on the communication screen, automatically sends an inquiry about that word or phrase to a generative artificial intelligence. The generative artificial intelligence generates an answer to the inquiry and displays that answer directly on the communication screen.

[0903] The server generates a query to send to the generative artificial intelligence based on the words selected by the user. A character limit is imposed to prevent the query from becoming redundant. Furthermore, the server analyzes the user's utterance and uses an emotion engine to recognize the user's emotions. Based on the recognized emotions, the server adjusts the query to the generative AI to provide an appropriate response that reflects the user's feelings.

[0904] As a concrete example, if a user selects the word "empathy" on the communication screen, the server generates the inquiry "Please tell me the meaning of empathy." If the user says "I'm very tired today," the server recognizes that the user is tired and adjusts the inquiry to "Please briefly explain the meaning of empathy." Generative artificial intelligence generates an answer based on this inquiry, and the server displays that answer on the communication screen.

[0905] This system allows users to quickly confirm the meaning of selected words within messaging applications and obtain appropriate information tailored to their current emotions.

[0906] The flow of the specific processing in Example 1 will be explained using Figure 15.

[0907] Step 1:

[0908] The user selects a word or phrase on the communication screen.

[0909] Input: The word or phrase selected by the user on the communication screen.

[0910] Operation: The user selects the word or phrase they want to look up by long-pressing it on the messaging application's communication screen.

[0911] Output: The selected phrase is recognized by the system.

[0912] Step 2:

[0913] The terminal generates the query.

[0914] Input: Selected word or phrase.

[0915] Operation: The terminal generates a query in the format "Please tell me the meaning of the selected phrase." based on the selected phrase.

[0916] Output: The generated query.

[0917] Step 3:

[0918] The terminal sends a query to the server.

[0919] Input: Generated query.

[0920] Operation: The terminal sends the generated query to the server.

[0921] Output: The server receives the query.

[0922] Step 4:

[0923] The server recognizes the user's emotions.

[0924] Input: User's statement.

[0925] Operation: The server analyzes the user's statements on the communication screen and uses an emotion engine to recognize the user's emotions.

[0926] Output: Recognized user emotions.

[0927] Step 5:

[0928] The server will coordinate the queries.

[0929] Input: Recognized user sentiment, generated query.

[0930] Operation: The server adjusts queries to the generative artificial intelligence based on the perceived emotions. For example, if the server detects that the user is tired, the query will be adjusted to "Please briefly explain the meaning of the selected phrase."

[0931] Output: Adjusted query.

[0932] Step 6:

[0933] The server sends a query to the generative artificial intelligence.

[0934] Input: Adjusted query.

[0935] Operation: The server sends a pre-configured query to the generative artificial intelligence.

[0936] Output: Generative artificial intelligence receives the inquiry.

[0937] Step 7:

[0938] Generative artificial intelligence generates the answer.

[0939] Input: Adjusted query.

[0940] Operation: Generative artificial intelligence generates answers based on the received inquiry.

[0941] Output: Generated answer.

[0942] Step 8:

[0943] The server receives the response and sends it to the terminal.

[0944] Input: Generated response.

[0945] Operation: The server receives a response from the generative artificial intelligence and sends it to the terminal.

[0946] Output: The terminal receives the response.

[0947] Step 9:

[0948] The device displays the answer on the communication screen.

[0949] Input: Response received from the server.

[0950] Operation: The terminal displays the received response on the communication screen.

[0951] Output: The user confirms the answer on the communication screen.

[0952] (Application Example 1)

[0953] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0954] In today's information-saturated environment, it is difficult for users to quickly and accurately obtain information related to the content they are watching. Furthermore, there is a lack of information that resonates with users' emotions, resulting in a non-personalized viewing experience. Moreover, there is a need to provide appropriate information that responds to users' emotions while preventing information from becoming redundant.

[0955] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0956] In this invention, the server includes means for looking up the meaning of a word selected by the user on the chat screen of a messenger app, means for displaying the meaning of that word on the chat screen, and means for automatically creating a question for a generative AI. This allows the user to obtain information related to the content they are watching in real time and receive information that is tailored to their emotions.

[0957] A "messenger app" is software that allows users to send and receive text messages.

[0958] The "chat screen" is the interface within a messenger app that allows users to view and input messages.

[0959] "Methods for looking up the meaning of a word" refers to functions that allow users to obtain the definition and related information of a word they have selected.

[0960] "Generative AI" refers to artificial intelligence that generates answers in natural language to input questions.

[0961] "Methods for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on words and emotions selected by the user.

[0962] "Methods for setting character limits" refer to functions that restrict the number of characters in questions and answers to prevent the responses from generative AI from becoming redundant.

[0963] "Means of recognizing emotions" refers to technologies that analyze and recognize emotions from a user's statements and actions.

[0964] "Content being watched" refers to the media, such as movies or TV shows, that the user is currently viewing.

[0965] "Means of providing information in real time" refers to a function that provides information immediately in response to user requests.

[0966] "Means of providing information that resonates with emotions" refers to a function that provides appropriate and empathetic information according to the user's emotional state.

[0967] The system for carrying out this invention operates based on a messenger application installed on the user's device. The device utilizes a generative AI to look up the meaning of a word selected by the user on the chat screen. For example, OpenAI's GPT-3 can be used as the generative AI.

[0968] The device sends the word selected by the user to a generative AI, and the response is displayed on the chat screen. To prevent the response from being redundant, a character limit is imposed to maintain the conciseness of the information.

[0969] Furthermore, the device uses an emotion recognition engine to analyze the user's emotions. For this emotion recognition, for example, Microsoft Azure's Emotion API can be used. Based on the user's emotions, the questions to the generative AI are adjusted to provide information that is sensitive to those emotions.

[0970] For example, if a user selects the word "endemic" while watching a movie, the device sends this word to a generative AI to retrieve its meaning. If the device recognizes that the user is emotionally moved, it provides an emotionally resonant explanation such as, "This scene depicts the feelings of people affected by an endemic."

[0971] An example of a prompt message might be something like, "What is endemic? Please explain it to the emotionally moved user."

[0972] In this way, users can obtain information related to the content they are watching in real time and receive information that resonates with their emotions.

[0973] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[0974] Step 1:

[0975] The user selects a word on the chat screen of the messenger app.

[0976] The input is a word selected by the user, and the output is information about the selected word. The terminal retrieves this word and prepares for the next process.

[0977] Step 2:

[0978] The terminal generates a prompt sentence to send to the AI ​​model for generating the selected word.

[0979] The input is the selected word, and the output is the prompt sentence sent to the generating AI model. The terminal generates a prompt sentence in the format "What is the word?" based on the selected word.

[0980] Step 3:

[0981] The device sends a prompt message to the generating AI model and obtains a response.

[0982] The input is a prompt sentence, and the output is the response from the generative AI model. The terminal sends a prompt sentence to the generative AI model (e.g., GPT-3) and receives a response regarding the meaning of the words.

[0983] Step 4:

[0984] The device uses an emotion recognition engine to analyze the user's emotions.

[0985] The input is the user's recent statements and actions, and the output is the recognized emotion. The device uses an emotion recognition engine (e.g., Microsoft Azure's Emotion API) to analyze the user's emotion.

[0986] Step 5:

[0987] The device adjusts the questions it asks the generated AI model based on the user's emotions.

[0988] The input consists of recognized emotions and responses from a generative AI model, while the output is information that aligns with those emotions. If the user is emotional, the device adjusts the response to reflect those emotions.

[0989] Step 6:

[0990] The device displays the adjusted information on the chat screen.

[0991] The input is information that resonates with emotions, and the output is information displayed on the chat screen. The device displays the meaning of words to the user in a way that resonates with their emotions.

[0992] (Example 2)

[0993] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0994] Conventional information processing devices faced challenges in efficiently providing information using generative artificial intelligence when users looked up the meaning of terms they selected. Furthermore, they failed to consider the user's emotions, making it difficult to obtain appropriate answers that met the user's needs.

[0995] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0996] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for automatically creating a query to a generative artificial intelligence, and means for sentiment analysis to recognize the user's emotions. This enables efficient information provision based on the term selected by the user and the provision of appropriate answers according to the user's emotions.

[0997] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and can interact with users through a user interface.

[0998] A "display screen" is a screen or display used in an information processing device to visually display information such as text and images.

[0999] "User" refers to an individual or group that operates an information processing device and acquires or inputs information.

[1000] A "term" is a word or phrase with a specific meaning, and is the object selected on an information processing device.

[1001] "Generative artificial intelligence" refers to an artificial intelligence system that uses natural language processing technology to automatically generate answers and information based on input data.

[1002] An "inquiry" is a question or request sent to a generative artificial intelligence to seek information or answers.

[1003] "Emotion analysis means" refers to a technology or device for recognizing emotions from a user's statements and actions, and for processing information based on those emotions.

[1004] In an embodiment of this invention, the information processing device constitutes a system that efficiently provides information by utilizing generative artificial intelligence based on terms selected by the user. Specifically, the terminal detects terms selected by the user through the display screen and automatically generates prompt sentences related to those terms. These prompt sentences are transmitted to a generative artificial intelligence model, which generates the corresponding information or answers.

[1005] The server uses a model employing natural language processing technology as a generative artificial intelligence model. For example, by utilizing an advanced natural language processing model such as GPT-3, it is possible to generate appropriate answers to user questions. The generated answers are sent to the terminal and displayed concisely on the screen.

[1006] Furthermore, the device can recognize the user's emotions using emotion analysis tools. If the user says, "I'm very angry," the emotion analysis tools will recognize this emotion and adjust the prompt text for the generative artificial intelligence. For example, it might generate a prompt text such as, "Why are you angry?" and provide a response that corresponds to the user's emotions.

[1007] For example, if the user selects the term "blockchain," the prompt will be "What is blockchain?". This prompt is sent to a generative artificial intelligence, which then generates an answer. An example of a prompt that responds to the user's emotions would be "Why am I angry?". This allows for the provision of information tailored to the user's needs.

[1008] The flow of the specific processing in Example 2 will be explained using Figure 17.

[1009] Step 1:

[1010] The user selects a specific term on the device's display screen. The device detects this selection and retrieves the selected term as input data. Based on this input data, it prepares to generate a prompt message. Specifically, when the user clicks the term "blockchain," the device recognizes that term.

[1011] Step 2:

[1012] The terminal generates a prompt based on the selected term. The input data is the selected term, and the output is the generated prompt. As a data processing step, the term is converted into a question in the format "What is XX?". Specifically, if the term "blockchain" is selected, the prompt "What is blockchain?" is generated.

[1013] Step 3:

[1014] The terminal sends the generated prompt message to the server. The input is the generated prompt message, and the output is the completion of the transmission to the server. Specifically, the terminal sends the prompt message to the server via the network.

[1015] Step 4:

[1016] The server inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is the answer generated by the generative AI model. As a data calculation, the generative AI model uses natural language processing techniques to generate an answer to the question. Specifically, the server inputs the prompt sentence into the generative AI model and obtains the answer.

[1017] Step 5:

[1018] The server sends the generated response to the terminal. The input is the response from the generating AI model, and the output is the completion of the transmission to the terminal. Specifically, the server sends the response to the terminal via the network.

[1019] Step 6:

[1020] The device displays the received response on its screen. The input is the response received from the server, and the output is the provision of visual information to the user. Specifically, the device displays the response concisely on the chat screen.

[1021] Step 7:

[1022] (Another embodiment) When a user makes a statement expressing an emotion, the terminal recognizes the user's emotion using emotion analysis means. The input is the user's statement, and the output is the recognized emotion. Specifically, if the user says, "I am very angry," the terminal analyzes that emotion.

[1023] Step 8:

[1024] (Another embodiment) The terminal adjusts the prompt message based on the recognized emotion and sends it to the server. The input is the recognized emotion, and the output is the adjusted prompt message. Specifically, if the emotion of anger is recognized, the prompt message "Why am I angry?" is generated and sent to the server.

[1025] (Application Example 2)

[1026] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as a "server," and the headset-type terminal 314 will be referred to as a "terminal."

[1027] In today's information and communication environment, users are increasingly exposed to vast amounts of information, but they face the challenge of efficiently acquiring information that aligns with their interests and emotions. Furthermore, systems capable of personalizing information based on user emotions are limited, highlighting the need for technologies to improve the user experience.

[1028] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1029] In this invention, the server includes means for looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen, means for displaying the meaning of that word or phrase on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for recognizing the user's emotions and adjusting the query to the generative artificial intelligence based on those emotions, and means for providing additional information related to the information the user is viewing in real time. This makes it possible to efficiently acquire information that matches the user's interests and emotions and improve the user experience.

[1030] A "messaging application" is software that allows users to send and receive information such as text, voice, and images in real time.

[1031] A "communication screen" is an interface within a messaging application that allows users to exchange information.

[1032] A "user" is an individual or organization that uses a messaging application to send and receive information.

[1033] A "word or phrase" is a word or phrase selected by the user, and is a unit of language that has a specific meaning.

[1034] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate natural language text based on given input.

[1035] An "inquiry" is a question or request sent to a generative artificial intelligence system in order to obtain information.

[1036] "Recognizing emotions" means analyzing and judging a user's emotional state based on their words and actions.

[1037] "Additional information" refers to supplementary data or explanations provided in relation to the information the user is viewing.

[1038] "Real-time" refers to information being processed immediately and provided without delay.

[1039] The system for implementing this invention consists of a server and a user terminal. The server has the function of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen and displaying that meaning on the communication screen. It also has the function of automatically creating queries to a generative artificial intelligence, recognizing the user's emotions, and adjusting the queries based on those emotions. Furthermore, it can provide additional information related to the information the user is viewing in real time.

[1040] In this system, the terminal sends the user's selected words to the server, which then analyzes their meaning. Natural language processing techniques are used for the analysis, specifically utilizing generative artificial intelligence (e.g., OpenAI GPT-4). The server also analyzes the user's emotions using an emotion recognition engine (e.g., Microsoft Azure Emotion API) and adjusts the prompts to the generative AI based on the results.

[1041] As a concrete example, when a user is watching a video about "blockchain," their device sends that phrase to the server. The server then sends the prompt "What is blockchain?" to a generative artificial intelligence system and displays the resulting answer on the communication screen. Furthermore, if the system recognizes that the user is in an excited state, it generates the prompt "What are some of the latest applications of blockchain?" and provides related information.

[1042] Examples of prompts include, "What are some of the latest applications of blockchain?" and "What blockchain-related news would you recommend for someone excited about blockchain?" This allows users to efficiently obtain information tailored to their interests and emotions.

[1043] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[1044] Step 1:

[1045] The terminal sends the words selected by the user on the messaging application's communication screen to the server. The input is the words selected by the user, and the output is the transmission of those words to the server. This action allows the server to receive the words to be analyzed.

[1046] Step 2:

[1047] The server analyzes the meaning of the received words. The input is words sent from the terminal, and the output is semantic information of those words. The server uses natural language processing techniques to analyze the meaning of the words and creates prompts to send to the generative artificial intelligence.

[1048] Step 3:

[1049] The server analyzes the user's emotions using an emotion recognition engine. The input is the user's statements and behavioral data, and the output is the user's emotional state. Based on this emotional information, the server adjusts prompts to the generative artificial intelligence.

[1050] Step 4:

[1051] The server sends a prompt to a generative artificial intelligence and obtains a response. The input is a prepared prompt, and the output is the response from the generative artificial intelligence. The server sends a prompt and receives the response obtained from the generative artificial intelligence.

[1052] Step 5:

[1053] The server sends the acquired response to the terminal and displays it on the communication screen. The input is the response from the generative artificial intelligence, and the output is the transmission of the response to the terminal. The terminal displays the received response on the communication screen and provides it to the user.

[1054] Step 6:

[1055] The server provides additional information in real time related to the information the user is viewing. The input is the user's viewing information and emotional state, and the output is the related additional information. Based on the viewing information and emotional state, the server generates the related additional information and sends it to the terminal.

[1056] (Other examples)

[1057] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[1058] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1059] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1060] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1061] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1062] [Fourth Embodiment]

[1063] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1064] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1065] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1066] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1067] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1068] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1069] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1070] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1071] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1072] The specific processing program 56 is an example of a “program” relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and reads the specific processing program 56.

[1073] The processing program 56 is executed on RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on RAM 30.

[1074] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1075] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1076] Next, the identification process performed by the identification processing unit 290 of the data processing device 12 will be described.

[1077] "Example of form 1"

[1078] One embodiment of the present invention provides a function in the chat screen of a messenger app that allows the user to look up the meaning of a word they have selected. Specifically, when a user selects a particular word on the chat screen, a question about that word is automatically sent to a generative AI. The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen. At this time, a character limit is imposed on the question sent to the generative AI to prevent the answer from becoming redundant.

[1079] "Example of form 2"

[1080] As a concrete example, consider the case where a user selects the word "blockchain." When this word is selected, the generative AI automatically receives the question, "What is blockchain?" The generative AI generates an answer to this question, and that answer is displayed directly on the chat screen. At this time, the answer is kept concise due to a character limit.

[1081] The following describes the processing flow for each example of the form.

[1082] "Example of form 1"

[1083] Step 1: The user selects a specific word on the chat screen of the messenger app. Step 2: Based on the selected word, a question is automatically created for the generative AI.

[1084] Step 3: The generated question is sent to the generation AI.

[1085] Step 4: The generative AI generates an answer to the question, and that answer is displayed directly on the chat screen.

[1086] "Example of form 2"

[1087] Step 1: The user selects the word "blockchain".

[1088] Step 2: The question "What is blockchain?" is automatically sent to the generative AI.

[1089] Step 3: The generative AI generates an answer to this question.

[1090] Step 4: The generated response will be displayed directly on the chat screen. At this time, the response will be kept concise due to a character limit.

[1091] (Example 1)

[1092] Next, we will describe Embodiment 1 of Example Form 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1093] Modern information processing devices require users to quickly and easily look up the meaning of specific words and phrases. However, conventional methods require users to open other applications or websites, resulting in cumbersome operation. Furthermore, responses from generative AI models can be redundant, making it difficult for users to quickly obtain the information they need.

[1094] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1095] In this invention, the server includes means for looking up the meaning of a word or phrase selected by a user on the display screen of the information processing device, means for automatically creating a query to a generating AI model, and means for receiving a response from the generating AI model and displaying it on the display screen. This allows the user to quickly and directly confirm the meaning of the selected word or phrase on the display screen.

[1096] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and includes devices such as computers and smartphones.

[1097] "Display screen" refers to a screen or display used to visually display information in an information processing device.

[1098] "User" refers to an individual or group that operates an information processing device and utilizes specific functions or services.

[1099] "Words" refer to words or phrases that have meaning within a text or conversation.

[1100] A "generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate natural language or provide responses based on input data.

[1101] An "inquiry" refers to a question or request made to obtain specific information.

[1102] "Answer" refers to the information or explanation provided in response to an inquiry.

[1103] "Character limit" refers to the maximum number of characters set to control the length of the information displayed.

[1104] This invention relates to a system for information processing that enables users to quickly look up the meaning of specific words or phrases. Specifically, it uses the chat screen of a messenger app to acquire information about words or phrases selected by the user through a generating AI model and displays it directly on the display screen.

[1105] The server uses, for example, OpenAI's GPT-3 as a generative AI model. This model has the ability to generate appropriate responses based on input prompt sentences using natural language processing techniques. The terminal detects words selected by the user on the chat screen and automatically generates prompt sentences related to those words. For example, if the user selects the word "algorithm," the terminal will generate the prompt sentence "Tell me the meaning of algorithm."

[1106] The device sends this prompt message to the server, which queries the generative AI model. The generative AI model generates a response based on the received prompt message and sends it back to the server. The server sends this response back to the device, which displays the response on the chat screen. This allows the user to directly confirm the meaning of the selected words on the chat screen.

[1107] This system offers the advantage of allowing users to quickly obtain information without opening other applications or websites. Furthermore, the AI ​​model's responses have a character limit, eliminating redundant information and providing concise and essential information.

[1108] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1109] Step 1:

[1110] The user selects the word or phrase they want to look up in the chat screen of the messenger app. The selected word or phrase is detected by the device. As a result of this action, the device receives the selected word or phrase as input.

[1111] Step 2:

[1112] The device generates a prompt based on the selected word or phrase. Specifically, it creates a prompt in the format of "Tell me the meaning of the selected word or phrase." This prompt serves as input data for querying the generating AI model.

[1113] Step 3:

[1114] The terminal sends the generated prompt message to the server. The server receives this prompt message and prepares to query the generative AI model. Here, the input is the prompt message, and the output is the query to the generative AI model.

[1115] Step 4:

[1116] The server sends a prompt to the generation AI model. The generation AI model generates an answer based on the received prompt. In this process, the prompt is taken as input and an appropriate answer is output using natural language processing techniques.

[1117] Step 5:

[1118] The server receives a response from the AI ​​model. This response is generated based on the prompt and is concise due to character limits. The server sends this response to the terminal.

[1119] Step 6:

[1120] The device displays the response received from the server on the chat screen. The user can directly check the meaning of the selected word or phrase on the chat screen. In this step, the response from the server is received as input, and the display on the chat screen is considered output.

[1121] (Application Example 1)

[1122] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1123] Modern information display systems present a challenge in that users often struggle to quickly and accurately obtain detailed information about specific terms or products. In particular, in retail settings, store employees are required to provide timely and appropriate information in response to customer inquiries, but current systems sometimes fall short in this regard.

[1124] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1125] In this invention, the server includes means for looking up the meaning of a term selected by the user, means for displaying the meaning of that term on an information display device, and means for automatically creating questions for a generative AI. This enables the provision of detailed information about the term selected by the user in real time, and allows for the rapid and accurate provision of information through the information display device.

[1126] An "information display device" is a device that allows users to visually confirm information, and includes smart glasses and displays.

[1127] "User" refers to an individual or group that operates an information display device and obtains information.

[1128] A "term" is a word or phrase selected on an information display device that has a specific meaning or information associated with it.

[1129] A "generative AI" is an artificial intelligence system that generates answers in natural language to input questions.

[1130] "Method for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on terms selected by the user.

[1131] "Character limit" is a setting that restricts the number of characters displayed in the response from the generation AI to prevent it from becoming redundant.

[1132] "Detailed information" refers to additional information or explanations related to the selected term, intended to help users gain a deeper understanding.

[1133] "Providing information in real time" means providing information immediately in response to user requests, and means that information is displayed without delay.

[1134] To implement this invention, smart glasses are used as an information display device. Smart glasses are devices that visually confirm terms selected by the user and display their meaning and related information. When a user selects a specific term through the interface of the smart glasses, a question about that term is automatically generated and sent to a generative AI.

[1135] The server uses a generative AI, such as OpenAI's GPT-4. This AI has the capability to generate natural language answers to input questions. The server automatically creates questions based on terms selected by the user and sends them to the generative AI. The AI's answers are adjusted to account for character limits and displayed on the smart glasses' screen.

[1136] As a concrete example, consider a scenario where a user asks a question about a product in a physical store. The user asks, "What material is this product made of?" and selects the term "material" using smart glasses. The server generates a prompt, "Tell me the details of the materials used in this product," and sends it to a generative AI. The AI ​​generates an answer such as, "This product is made from organic cotton," and displays it on the smart glasses.

[1137] Examples of prompt phrases include "Tell me the meaning of this word" or "Provide me information related to this product." This allows users to obtain detailed information in real time, enabling quick and accurate information provision.

[1138] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1139] Step 1:

[1140] The user selects a specific term through smart glasses. The input is the user's gaze or touch input, and the output is the text data of the selected term. This text data is used for subsequent processing.

[1141] Step 2:

[1142] The device receives a selected term and automatically generates a prompt message to send to the generative AI based on that term. The input is the text data of the term obtained in step 1, and the output is the generated prompt message. Specifically, it analyzes the term and generates a prompt message such as "Tell me the meaning of this word."

[1143] Step 3:

[1144] The server sends the generated prompt to the generative AI. The input is the prompt generated in step 2, and the output is the response from the generative AI. The server sends the prompt to the AI, and the AI ​​generates a response in natural language.

[1145] Step 4:

[1146] The server receives the response from the generative AI and adjusts it, taking into account character limits. The input is the response from the generative AI, and the output is the adjusted response text. Specifically, it limits the character count to prevent the response from becoming redundant and summarizes it as needed.

[1147] Step 5:

[1148] The device displays the adjusted response on the smart glasses' display. The input is the adjusted response text from step 4, and the output is information that the user can visually confirm. Specifically, the response is displayed on the smart glasses' display, allowing the user to confirm the information.

[1149] (Example 2)

[1150] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1151] Conventional information processing devices face the challenge of quickly and concisely acquiring and displaying information related to user-selected terms. Furthermore, in response generation using generative AI models, the displayed information tends to be redundant, highlighting the need to provide information in a user-friendly format.

[1152] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1153] In this invention, the server includes means for acquiring information about a word or phrase selected by a user on the display screen of an information processing device, means for automatically generating input sentences for a generating AI model based on that word or phrase, and means for generating a response to the input sentences using the generating AI model. This makes it possible to quickly and concisely acquire and display information about the word or phrase selected by the user.

[1154] An "information processing device" is a device used for inputting, processing, and outputting data, and includes computers, smartphones, and other similar devices.

[1155] A "display screen" is a screen in an information processing device that visually presents information to the user.

[1156] "User" refers to a person who operates an information processing device and acquires or processes information.

[1157] "Words" refer to words or phrases that have meaning within a text or conversation.

[1158] A "generative AI model" is a model that uses artificial intelligence technology to generate responses and information based on input data.

[1159] An "input sentence" is a sentence that is input to a generative AI model in order to generate information.

[1160] "Response" refers to the information or answer that a generative AI model generates based on the input sentence.

[1161] A "character limit" is a constraint that restricts the number of characters displayed in the information, in order to prevent the information from becoming redundant.

[1162] A description of embodiments for carrying out this invention will be given.

[1163] When a user selects a specific word or phrase on the device, the device detects this selection. Based on the selected word or phrase, the device generates a prompt. This prompt is used as input to a generative AI model. For example, a model using natural language processing techniques can be used as the generative AI model. Specifically, OpenAI's GPT-3 is a suitable example.

[1164] The terminal sends the generated prompt message to the server. The server receives this prompt message and generates a response using a generation AI model. The generated response is sent from the server to the terminal. The terminal displays the received response on its screen. At this time, the displayed response is kept concise due to a character limit.

[1165] As a concrete example, if the user selects the term "blockchain," the device generates a prompt message such as "What is blockchain?". The server receives this prompt message and uses a generation AI model to generate a response such as "Blockchain is a distributed ledger technology used to securely manage records of transactions." The device displays this response on its screen, which the user can then review.

[1166] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1167] Step 1:

[1168] The user selects a specific word or phrase on the device. The device detects this selection and receives the selected word or phrase as input. Based on this input, the device prepares to generate a prompt. Specifically, it monitors click and touch events on the user interface and retrieves the selected word or phrase.

[1169] Step 2:

[1170] The terminal generates a prompt based on the selected phrase. Using the phrase received as input, it creates a prompt in the format "What is this phrase?". This prompt is used as input to the generation AI model. Specifically, it performs string manipulation and incorporates the selected phrase into the prompt.

[1171] Step 3:

[1172] The terminal sends the generated prompt message to the server. The server receives this prompt message as input and prepares to pass it to the generating AI model. Specifically, it sends the prompt message to the server via network communication.

[1173] Step 4:

[1174] The server uses a generative AI model to generate a response to the received prompt. The generative AI model uses natural language processing with the prompt as input and outputs an appropriate response. Specifically, the process involves calling the generative AI model, providing the prompt as input, and generating the response.

[1175] Step 5:

[1176] The server sends the generated response to the terminal. The terminal receives this response as input and prepares to display it on the display screen. Specifically, it performs the action of sending the response to the terminal via network communication.

[1177] Step 6:

[1178] The terminal displays the received response on the display screen. At this time, a character limit is applied to ensure the response is concise and avoids redundancy. Specifically, the response is placed within the display area, and adjustments are made to ensure it does not exceed the character limit.

[1179] (Application Example 2)

[1180] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1181] In electronic transactions, a challenge exists in that users often find it difficult to quickly and concisely understand unfamiliar terminology or functions. This can hinder smooth transactions and potentially reduce transaction efficiency.

[1182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1183] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for displaying the meaning of that term on the display screen, and means for automatically creating questions for a generative artificial intelligence. This enables the user to quickly understand terms related to electronic transactions and to proceed with transactions smoothly.

[1184] An "information processing device" is an electronic device that has the function of inputting, processing, and outputting data.

[1185] A "display screen" is a screen used to visually display information.

[1186] A "user" is a person who operates an information processing device.

[1187] A "term" is a word or phrase that has a specific meaning.

[1188] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate information based on given input.

[1189] "Methods for automatically generating questions" refers to a function that automatically generates relevant questions based on selected terms.

[1190] "Character limit" is a setting that restricts the number of characters in the displayed information to a certain range.

[1191] "Electronic transactions" refer to commercial transactions conducted via the internet.

[1192] A "concise explanation" is a short explanation that gets straight to the point.

[1193] The system for carrying out this invention includes a terminal as an information processing device and a server that utilizes generative artificial intelligence. The terminal has a display screen and provides an interface for inputting terms selected by the user. When the user selects a specific term, the terminal sends that term to the server.

[1194] Based on the received terms, the server automatically creates a relevant question for the generative artificial intelligence and sends it as a prompt. This prompt might be in the format of, for example, "What is cryptocurrency?". The generative AI then generates a concise explanation based on this prompt.

[1195] The generated explanation is sent from the server to the terminal and displayed on the terminal's screen. The displayed explanation is kept concise due to character limits, allowing users to quickly understand terminology related to electronic transactions.

[1196] For example, if a user selects the term "QR code payment," the server sends a prompt to the generative AI asking, "What is QR code payment?" The generative AI then generates a concise explanation such as, "QR code payment is a method of payment that involves scanning a QR code using a smartphone," and displays it on the device.

[1197] This system allows users to instantly understand unfamiliar terminology and conduct electronic transactions smoothly.

[1198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1199] Step 1:

[1200] The user selects a specific term on the terminal's display screen. The input is the term selected by the user, and the output is that the term is sent to the terminal's system. The terminal then prepares this term for the next processing step.

[1201] Step 2:

[1202] The terminal sends the selected term to the server. The input is the term received from the terminal, and the output is the term data sent to the server. The server receives this data and proceeds to the next step.

[1203] Step 3:

[1204] The server automatically generates prompts to send to the AI ​​model based on the terms it receives. The input is the terms received by the server, and the output is the generated prompt. Specifically, the server generates a prompt in the format "What is a term?".

[1205] Step 4:

[1206] The server sends the generated prompt to the AI ​​model. The input is the prompt, and the output is the request sent to the AI ​​model. The AI ​​model receives this prompt and generates a response.

[1207] Step 5:

[1208] The generative AI model generates a concise explanation based on a prompt. The input is the prompt, and the output is the generated explanation. The generative AI model uses its internal database and algorithms to extract relevant information and create a concise explanation.

[1209] Step 6:

[1210] The server sends the descriptive text received from the generated AI model to the terminal. The input is the descriptive text from the generated AI model, and the output is the descriptive text sent to the terminal. The server transfers this data to the terminal.

[1211] Step 7:

[1212] The terminal displays the received description on its screen. The input is the description received from the server, and the output is the information visually displayed to the user. The terminal displays the description concisely, taking into account character limits.

[1213] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1214] "Example of form 1"

[1215] In one embodiment of the present invention, an emotion engine that recognizes the user's emotions is incorporated into the system. This emotion engine recognizes the user's emotions from the content of their messages on the messenger app's chat screen. Specifically, if a user says "I am very sad" on the messenger app's chat screen, the emotion engine recognizes from this statement that the user is feeling sad. This recognized emotion is then used to adjust the questions posed to the generative AI. For example, if it is recognized that the user is feeling sad, the questions posed to the generative AI are adjusted to take the user's emotions into account. As a result, more appropriate answers that reflect the user's emotions are displayed on the chat screen.

[1216] "Example of form 2"

[1217] In another embodiment of the present invention, after the emotion engine recognizes the user's emotion, the questions to the generative AI are adjusted based on that emotion. Specifically, if the user says, "I'm very angry," the emotion engine recognizes from this statement that the user is feeling angry. This recognized emotion is then used to adjust the questions to the generative AI. For example, if it is recognized that the user is feeling angry, the questions to the generative AI are adjusted to take the user's emotion into account, and a question such as "Why am I angry?" is generated. As a result, a more appropriate answer that corresponds to the user's emotion is displayed on the chat screen.

[1218] The following describes the processing flow for each example of the form.

[1219] "Example of form 1"

[1220] Step 1: The user makes a message on the chat screen of the messenger app.

[1221] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes sadness from the statement "I am very sad."

[1222] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is feeling sad, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[1223] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[1224] "Example of form 2"

[1225] Step 1: The user makes a message on the chat screen of the messenger app.

[1226] Step 2: The emotion engine recognizes emotions from the user's statements. For example, it recognizes anger from the statement, "I am very angry."

[1227] Step 3: The recognized emotions are used to adjust the questions posed to the generative AI. For example, if the system recognizes that the user is angry, the questions posed to the generative AI will be adjusted to take the user's emotions into account.

[1228] Step 4: The generative AI generates an answer to the question, and that answer is displayed on the chat screen.

[1229] (Example 1)

[1230] Next, we will describe Embodiment 1 of Example Form 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1231] Traditional messaging applications were inconvenient because users had to leave the application and use a separate dictionary application or website to look up the meaning of a specific word. Furthermore, they failed to provide information that took user emotions into consideration, resulting in inappropriate information tailored to the user's situation.

[1232] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1233] In this invention, the server includes means for looking up the meaning of a word selected by the user on the messaging application's communication screen, means for displaying the meaning of the word on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for setting a character limit to prevent the display result from becoming redundant, means for recognizing emotions from the user's statements, and means for adjusting the query to the generative artificial intelligence based on the recognized emotions. As a result, the user can quickly check the meaning of a word selected within the application and obtain appropriate information according to their emotions at that time.

[1234] A "messaging application" is software that allows users to send and receive text messages.

[1235] A "communication screen" is an interface within a messaging application that allows users to view and input messages.

[1236] "User" refers to an individual or group using a messaging application.

[1237] A "word or phrase" refers to a word or phrase that a user selects within a messaging application.

[1238] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate responses in natural language based on input information.

[1239] An "inquiry" refers to a question or request sent to a generative artificial intelligence to obtain information.

[1240] "Means of recognizing emotions" refers to technologies or algorithms that analyze the content of a user's statements and identify their emotional state.

[1241] "Character limit" refers to a constraint on the maximum number of characters that can be used to prevent queries to and responses to generative artificial intelligence from becoming redundant.

[1242] This invention is a system for looking up the meaning of a word or phrase selected by a user on the communication screen of a messaging application. The system provides a function that, when a user selects a specific word or phrase on the communication screen, automatically sends an inquiry about that word or phrase to a generative artificial intelligence. The generative artificial intelligence generates an answer to the inquiry and displays that answer directly on the communication screen.

[1243] The server generates a query to send to the generative artificial intelligence based on the words selected by the user. A character limit is imposed to prevent the query from becoming redundant. Furthermore, the server analyzes the user's utterance and uses an emotion engine to recognize the user's emotions. Based on the recognized emotions, the server adjusts the query to the generative AI to provide an appropriate response that reflects the user's feelings.

[1244] As a concrete example, if a user selects the word "empathy" on the communication screen, the server generates the inquiry "Please tell me the meaning of empathy." If the user says "I'm very tired today," the server recognizes that the user is tired and adjusts the inquiry to "Please briefly explain the meaning of empathy." Generative artificial intelligence generates an answer based on this inquiry, and the server displays that answer on the communication screen.

[1245] This system allows users to quickly confirm the meaning of selected words within messaging applications and obtain appropriate information tailored to their current emotions.

[1246] The flow of the specific processing in Example 1 will be explained using Figure 15.

[1247] Step 1:

[1248] The user selects a word or phrase on the communication screen.

[1249] Input: The word or phrase selected by the user on the communication screen.

[1250] Operation: The user selects the word or phrase they want to look up by long-pressing it on the messaging application's communication screen.

[1251] Output: The selected phrase is recognized by the system.

[1252] Step 2:

[1253] The terminal generates the query.

[1254] Input: Selected word or phrase.

[1255] Operation: The terminal generates a query in the format "Please tell me the meaning of the selected phrase." based on the selected phrase.

[1256] Output: The generated query.

[1257] Step 3:

[1258] The terminal sends a query to the server.

[1259] Input: Generated query.

[1260] Operation: The terminal sends the generated query to the server.

[1261] Output: The server receives the query.

[1262] Step 4:

[1263] The server recognizes the user's emotions.

[1264] Input: User's statement.

[1265] Operation: The server analyzes the user's statements on the communication screen and uses an emotion engine to recognize the user's emotions.

[1266] Output: Recognized user emotions.

[1267] Step 5:

[1268] The server will coordinate the queries.

[1269] Input: Recognized user sentiment, generated query.

[1270] Operation: The server adjusts queries to the generative artificial intelligence based on the perceived emotions. For example, if the server detects that the user is tired, the query will be adjusted to "Please briefly explain the meaning of the selected phrase."

[1271] Output: Adjusted query.

[1272] Step 6:

[1273] The server sends a query to the generative artificial intelligence.

[1274] Input: Adjusted query.

[1275] Operation: The server sends a pre-configured query to the generative artificial intelligence.

[1276] Output: Generative artificial intelligence receives the inquiry.

[1277] Step 7:

[1278] Generative artificial intelligence generates the answer.

[1279] Input: Adjusted query.

[1280] Operation: Generative artificial intelligence generates answers based on the received inquiry.

[1281] Output: Generated answer.

[1282] Step 8:

[1283] The server receives the response and sends it to the terminal.

[1284] Input: Generated response.

[1285] Operation: The server receives a response from the generative artificial intelligence and sends it to the terminal.

[1286] Output: The terminal receives the response.

[1287] Step 9:

[1288] The device displays the answer on the communication screen.

[1289] Input: Response received from the server.

[1290] Operation: The terminal displays the received response on the communication screen.

[1291] Output: The user confirms the answer on the communication screen.

[1292] (Application Example 1)

[1293] Next, we will describe Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1294] In today's information-saturated environment, it is difficult for users to quickly and accurately obtain information related to the content they are watching. Furthermore, there is a lack of information that resonates with users' emotions, resulting in a non-personalized viewing experience. Moreover, there is a need to provide appropriate information that responds to users' emotions while preventing information from becoming redundant.

[1295] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1296] In this invention, the server includes means for looking up the meaning of a word selected by the user on the chat screen of a messenger app, means for displaying the meaning of that word on the chat screen, and means for automatically creating a question for a generative AI. This allows the user to obtain information related to the content they are watching in real time and receive information that is tailored to their emotions.

[1297] A "messenger app" is software that allows users to send and receive text messages.

[1298] The "chat screen" is the interface within a messenger app that allows users to view and input messages.

[1299] "Methods for looking up the meaning of a word" refers to functions that allow users to obtain the definition and related information of a word they have selected.

[1300] "Generative AI" refers to artificial intelligence that generates answers in natural language to input questions.

[1301] "Methods for automatically generating questions" refers to a function that automatically generates questions to send to a generative AI based on words and emotions selected by the user.

[1302] "Methods for setting character limits" refer to functions that restrict the number of characters in questions and answers to prevent the responses from generative AI from becoming redundant.

[1303] "Means of recognizing emotions" refers to technologies that analyze and recognize emotions from a user's statements and actions.

[1304] "Content being watched" refers to the media, such as movies or TV shows, that the user is currently viewing.

[1305] "Means of providing information in real time" refers to a function that provides information immediately in response to user requests.

[1306] "Means of providing information that resonates with emotions" refers to a function that provides appropriate and empathetic information according to the user's emotional state.

[1307] The system for carrying out this invention operates based on a messenger application installed on the user's device. The device utilizes a generative AI to look up the meaning of a word selected by the user on the chat screen. For example, OpenAI's GPT-3 can be used as the generative AI.

[1308] The device sends the word selected by the user to a generative AI, and the response is displayed on the chat screen. To prevent the response from being redundant, a character limit is imposed to maintain the conciseness of the information.

[1309] Furthermore, the device uses an emotion recognition engine to analyze the user's emotions. For this emotion recognition, for example, Microsoft Azure's Emotion API can be used. Based on the user's emotions, the questions to the generative AI are adjusted to provide information that is sensitive to those emotions.

[1310] For example, if a user selects the word "endemic" while watching a movie, the device sends this word to a generative AI to retrieve its meaning. If the device recognizes that the user is emotionally moved, it provides an emotionally resonant explanation such as, "This scene depicts the feelings of people affected by an endemic."

[1311] An example of a prompt message might be something like, "What is endemic? Please explain it to the emotionally moved user."

[1312] In this way, users can obtain information related to the content they are watching in real time and receive information that resonates with their emotions.

[1313] The flow of a specific process in Application Example 1 will be explained using Figure 16.

[1314] Step 1:

[1315] The user selects a word on the chat screen of the messenger app.

[1316] The input is a word selected by the user, and the output is information about the selected word. The terminal retrieves this word and prepares for the next process.

[1317] Step 2:

[1318] The terminal generates a prompt sentence to send to the AI ​​model for generating the selected word.

[1319] The input is the selected word, and the output is the prompt sentence sent to the generating AI model. The terminal generates a prompt sentence in the format "What is the word?" based on the selected word.

[1320] Step 3:

[1321] The device sends a prompt message to the generating AI model and obtains a response.

[1322] The input is a prompt sentence, and the output is the response from the generative AI model. The terminal sends a prompt sentence to the generative AI model (e.g., GPT-3) and receives a response regarding the meaning of the words.

[1323] Step 4:

[1324] The device uses an emotion recognition engine to analyze the user's emotions.

[1325] The input is the user's recent statements and actions, and the output is the recognized emotion. The device uses an emotion recognition engine (e.g., Microsoft Azure's Emotion API) to analyze the user's emotion.

[1326] Step 5:

[1327] The device adjusts the questions it asks the generated AI model based on the user's emotions.

[1328] The input consists of recognized emotions and responses from a generative AI model, while the output is information that aligns with those emotions. If the user is emotional, the device adjusts the response to reflect those emotions.

[1329] Step 6:

[1330] The device displays the adjusted information on the chat screen.

[1331] The input is information that resonates with emotions, and the output is information displayed on the chat screen. The device displays the meaning of words to the user in a way that resonates with their emotions.

[1332] (Example 2)

[1333] Next, we will describe Example 2 of the morphological example. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1334] Conventional information processing devices faced challenges in efficiently providing information using generative artificial intelligence when users looked up the meaning of terms they selected. Furthermore, they failed to consider the user's emotions, making it difficult to obtain appropriate answers that met the user's needs.

[1335] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1336] In this invention, the server includes means for looking up the meaning of a term selected by the user on the display screen of the information processing device, means for automatically creating a query to a generative artificial intelligence, and means for sentiment analysis to recognize the user's emotions. This enables efficient information provision based on the term selected by the user and the provision of appropriate answers according to the user's emotions.

[1337] An "information processing device" is an electronic device used for inputting, processing, and outputting data, and can interact with users through a user interface.

[1338] A "display screen" is a screen or display used in an information processing device to visually display information such as text and images.

[1339] "User" refers to an individual or group that operates an information processing device and acquires or inputs information.

[1340] A "term" is a word or phrase with a specific meaning, and is the object selected on an information processing device.

[1341] "Generative artificial intelligence" refers to an artificial intelligence system that uses natural language processing technology to automatically generate answers and information based on input data.

[1342] An "inquiry" is a question or request sent to a generative artificial intelligence to seek information or answers.

[1343] "Emotion analysis means" refers to a technology or device for recognizing emotions from a user's statements and actions, and for processing information based on those emotions.

[1344] In an embodiment of this invention, the information processing device constitutes a system that efficiently provides information by utilizing generative artificial intelligence based on terms selected by the user. Specifically, the terminal detects terms selected by the user through the display screen and automatically generates prompt sentences related to those terms. These prompt sentences are transmitted to a generative artificial intelligence model, which generates the corresponding information or answers.

[1345] The server uses a model employing natural language processing technology as a generative artificial intelligence model. For example, by utilizing an advanced natural language processing model such as GPT-3, it is possible to generate appropriate answers to user questions. The generated answers are sent to the terminal and displayed concisely on the screen.

[1346] Furthermore, the device can recognize the user's emotions using emotion analysis tools. If the user says, "I'm very angry," the emotion analysis tools will recognize this emotion and adjust the prompt text for the generative artificial intelligence. For example, it might generate a prompt text such as, "Why are you angry?" and provide a response that corresponds to the user's emotions.

[1347] For example, if the user selects the term "blockchain," the prompt will be "What is blockchain?". This prompt is sent to a generative artificial intelligence, which then generates an answer. An example of a prompt that responds to the user's emotions would be "Why am I angry?". This allows for the provision of information tailored to the user's needs.

[1348] The flow of the specific processing in Example 2 will be explained using Figure 17.

[1349] Step 1:

[1350] The user selects a specific term on the device's display screen. The device detects this selection and retrieves the selected term as input data. Based on this input data, it prepares to generate a prompt message. Specifically, when the user clicks the term "blockchain," the device recognizes that term.

[1351] Step 2:

[1352] The terminal generates a prompt based on the selected term. The input data is the selected term, and the output is the generated prompt. As a data processing step, the term is converted into a question in the format "What is XX?". Specifically, if the term "blockchain" is selected, the prompt "What is blockchain?" is generated.

[1353] Step 3:

[1354] The terminal sends the generated prompt message to the server. The input is the generated prompt message, and the output is the completion of the transmission to the server. Specifically, the terminal sends the prompt message to the server via the network.

[1355] Step 4:

[1356] The server inputs the received prompt sentence into the generative AI model. The input is the prompt sentence, and the output is the answer generated by the generative AI model. As a data calculation, the generative AI model uses natural language processing techniques to generate an answer to the question. Specifically, the server inputs the prompt sentence into the generative AI model and obtains the answer.

[1357] Step 5:

[1358] The server sends the generated response to the terminal. The input is the response from the generating AI model, and the output is the completion of the transmission to the terminal. Specifically, the server sends the response to the terminal via the network.

[1359] Step 6:

[1360] The device displays the received response on its screen. The input is the response received from the server, and the output is the provision of visual information to the user. Specifically, the device displays the response concisely on the chat screen.

[1361] Step 7:

[1362] (Another embodiment) When a user makes a statement expressing an emotion, the terminal recognizes the user's emotion using emotion analysis means. The input is the user's statement, and the output is the recognized emotion. Specifically, if the user says, "I am very angry," the terminal analyzes that emotion.

[1363] Step 8:

[1364] (Another embodiment) The terminal adjusts the prompt message based on the recognized emotion and sends it to the server. The input is the recognized emotion, and the output is the adjusted prompt message. Specifically, if the emotion of anger is recognized, the prompt message "Why am I angry?" is generated and sent to the server.

[1365] (Application Example 2)

[1366] Next, we will describe application example 2 of form example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1367] In today's information and communication environment, users are increasingly exposed to vast amounts of information, but they face the challenge of efficiently acquiring information that aligns with their interests and emotions. Furthermore, systems capable of personalizing information based on user emotions are limited, highlighting the need for technologies to improve the user experience.

[1368] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1369] In this invention, the server includes means for looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen, means for displaying the meaning of that word or phrase on the communication screen, means for automatically creating a query to a generative artificial intelligence, means for recognizing the user's emotions and adjusting the query to the generative artificial intelligence based on those emotions, and means for providing additional information related to the information the user is viewing in real time. This makes it possible to efficiently acquire information that matches the user's interests and emotions and improve the user experience.

[1370] A "messaging application" is software that allows users to send and receive information such as text, voice, and images in real time.

[1371] A "communication screen" is an interface within a messaging application that allows users to exchange information.

[1372] A "user" is an individual or organization that uses a messaging application to send and receive information.

[1373] A "word or phrase" is a word or phrase selected by the user, and is a unit of language that has a specific meaning.

[1374] "Generative artificial intelligence" refers to an artificial intelligence system that has the ability to generate natural language text based on given input.

[1375] An "inquiry" is a question or request sent to a generative artificial intelligence system in order to obtain information.

[1376] "Recognizing emotions" means analyzing and judging a user's emotional state based on their words and actions.

[1377] "Additional information" refers to supplementary data or explanations provided in relation to the information the user is viewing.

[1378] "Real-time" refers to information being processed immediately and provided without delay.

[1379] The system for implementing this invention consists of a server and a user terminal. The server has the function of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen and displaying that meaning on the communication screen. It also has the function of automatically creating queries to a generative artificial intelligence, recognizing the user's emotions, and adjusting the queries based on those emotions. Furthermore, it can provide additional information related to the information the user is viewing in real time.

[1380] In this system, the terminal sends the user's selected words to the server, which then analyzes their meaning. Natural language processing techniques are used for the analysis, specifically utilizing generative artificial intelligence (e.g., OpenAI GPT-4). The server also analyzes the user's emotions using an emotion recognition engine (e.g., Microsoft Azure Emotion API) and adjusts the prompts to the generative AI based on the results.

[1381] As a concrete example, when a user is watching a video about "blockchain," their device sends that phrase to the server. The server then sends the prompt "What is blockchain?" to a generative artificial intelligence system and displays the resulting answer on the communication screen. Furthermore, if the system recognizes that the user is in an excited state, it generates the prompt "What are some of the latest applications of blockchain?" and provides related information.

[1382] Examples of prompts include, "What are some of the latest applications of blockchain?" and "What blockchain-related news would you recommend for someone excited about blockchain?" This allows users to efficiently obtain information tailored to their interests and emotions.

[1383] The flow of a specific process in Application Example 2 will be explained using Figure 18.

[1384] Step 1:

[1385] The terminal sends the words selected by the user on the messaging application's communication screen to the server. The input is the words selected by the user, and the output is the transmission of those words to the server. This action allows the server to receive the words to be analyzed.

[1386] Step 2:

[1387] The server analyzes the meaning of the received words. The input is words sent from the terminal, and the output is semantic information of those words. The server uses natural language processing techniques to analyze the meaning of the words and creates prompts to send to the generative artificial intelligence.

[1388] Step 3:

[1389] The server analyzes the user's emotions using an emotion recognition engine. The input is the user's statements and behavioral data, and the output is the user's emotional state. Based on this emotional information, the server adjusts prompts to the generative artificial intelligence.

[1390] Step 4:

[1391] The server sends a prompt to a generative artificial intelligence and obtains a response. The input is a prepared prompt, and the output is the response from the generative artificial intelligence. The server sends a prompt and receives the response obtained from the generative artificial intelligence.

[1392] Step 5:

[1393] The server sends the acquired response to the terminal and displays it on the communication screen. The input is the response from the generative artificial intelligence, and the output is the transmission of the response to the terminal. The terminal displays the received response on the communication screen and provides it to the user.

[1394] Step 6:

[1395] The server provides additional information in real time related to the information the user is viewing. The input is the user's viewing information and emotional state, and the output is the related additional information. Based on the viewing information and emotional state, the server generates the related additional information and sends it to the terminal.

[1396] (Other examples)

[1397] Since this is the same as the specific processing described in the other embodiments of the first embodiment above, the explanation will be omitted.

[1398] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1399] The data generation model 58 is a form of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1400] Other examples of generative AI include Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are some examples.

[1401] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1402] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1403] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1404] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1405] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1406] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1407] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1408] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1409] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[1410] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1411] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1412] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1413] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1414] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1415] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1416] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1417] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1418] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1419] The following is further disclosed regarding the embodiments described above.

[1420] (Claim 1)

[1421] A system comprising: means for looking up the meaning of a word selected by the user on the chat screen of a messenger app; means for displaying the meaning of that word on the chat screen; means for automatically creating a question for a generative AI; and means for setting a character limit to prevent the display result from becoming redundant. (Claim 2)

[1422] The system according to claim 1, wherein a question for the generative AI is automatically generated based on selected words.

[1423] (Claim 3)

[1424] The system according to claim 1, wherein the display result is displayed directly on the chat screen.

[1425] (Claim 4)

[1426] The system according to claim 1, further comprising an emotion engine that recognizes the user's emotions.

[1427] (Claim 5)

[1428] The system according to claim 4, wherein the emotion engine recognizes emotions from the content of the user's messages on the chat screen of the messenger app.

[1429] (Claim 6)

[1430] The system according to claim 4, wherein the questions to the generative AI are adjusted based on the emotions recognized by the emotion engine.

[1431] "Example 1"

[1432] (Claim 1)

[1433] A means of looking up the meaning of a word or phrase selected by the user on the display screen of an information processing device,

[1434] A means of displaying the meaning of that word or phrase on the display screen,

[1435] A method for automatically creating queries for a generative AI model,

[1436] A means of setting a character limit to prevent the display results from becoming redundant,

[1437] A means of receiving responses from a generative AI model and displaying them on a screen,

[1438] A system that includes this.

[1439] (Claim 2)

[1440] The system according to claim 1, wherein queries to the generating AI model are automatically created based on selected words or phrases.

[1441] (Claim 3)

[1442] The system according to claim 1, wherein the display result is displayed directly on the display screen.

[1443] "Application Example 1"

[1444] (Claim 1)

[1445] In an information display device, a means for looking up the meaning of a term selected by the user,

[1446] A means for displaying the meaning of that term on an information display device,

[1447] A method for automatically generating questions for generative AI,

[1448] A means of setting a character limit to prevent the display results from becoming redundant,

[1449] A means of providing detailed information about selected terms in real time through an information display device,

[1450] A means for displaying the answer from a generative AI based on the user's question on an information display device,

[1451] A system that includes this.

[1452] (Claim 2)

[1453] The system according to claim 1, wherein questions for the generative AI are automatically generated based on selected terms.

[1454] (Claim 3)

[1455] The system according to claim 1, wherein the display result is displayed directly on an information display device.

[1456] Example 2

[1457] (Claim 1)

[1458] A means of obtaining information about a word or phrase selected by the user on the display screen of an information processing device,

[1459] A means of automatically generating input sentences for a generative AI model based on those words,

[1460] A means for generating a response to the input sentence using a generative AI model,

[1461] A means for displaying the generated response on a screen,

[1462] A means of setting a character limit to prevent the displayed response from becoming redundant,

[1463] A system that includes this.

[1464] (Claim 2)

[1465] The system according to claim 1, wherein the input sentence to the generation AI model is automatically generated based on selected words or phrases.

[1466] (Claim 3)

[1467] The system according to claim 1, wherein the generated response is displayed directly on a display screen.

[1468] "Application Example 2"

[1469] (Claim 1)

[1470] A means of looking up the meaning of a term selected by the user on the display screen of an information processing device,

[1471] A means of displaying the meaning of that term on the screen,

[1472] A method for automatically generating questions for generative artificial intelligence,

[1473] A means of setting a character limit to prevent the display results from becoming redundant,

[1474] When selecting terms related to electronic transactions, a means of providing a concise explanation of those terms is provided.

[1475] A system that includes this.

[1476] (Claim 2)

[1477] The system according to claim 1, wherein a question for the generative artificial intelligence is automatically generated based on selected terms.

[1478] (Claim 3)

[1479] The system according to claim 1, wherein the display result is displayed directly on the display screen.

[1480] "Example 1 of combining an emotion engine"

[1481] (Claim 1)

[1482] A means of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen,

[1483] A means of displaying the meaning of that phrase on the communication screen,

[1484] A method for automatically creating queries to generative artificial intelligence,

[1485] A means of setting a character limit to prevent the display results from becoming redundant,

[1486] A means of recognizing emotions from the content of user statements,

[1487] A system that includes means for adjusting queries to generative artificial intelligence based on recognized emotions.

[1488] (Claim 2)

[1489] The system according to claim 1, wherein a query to the generative artificial intelligence is automatically generated based on selected words or phrases.

[1490] (Claim 3)

[1491] The system according to claim 1, wherein the display result is directly displayed on the communication screen.

[1492] "Application example 1 of combining emotional engines"

[1493] (Claim 1)

[1494] A way to look up the meaning of a word selected by the user in the chat screen of a messenger app,

[1495] A means of displaying the meaning of that word on the chat screen,

[1496] A method for automatically generating questions for generative AI,

[1497] A means of setting a character limit to prevent the display results from becoming redundant,

[1498] Means of recognizing user emotions,

[1499] A means of tailoring questions to a generative AI based on the user's emotions,

[1500] A means of providing information related to the content being viewed in real time,

[1501] A means of providing information that resonates with the user's emotions.

[1502] A system that includes this.

[1503] (Claim 2)

[1504] The system according to claim 1, wherein questions for the generative AI are automatically generated based on selected words and the user's emotions.

[1505] (Claim 3)

[1506] The system according to claim 1, wherein the display result is displayed directly on the chat screen, and information corresponding to the user's emotions is provided.

[1507] "Example 2 of combining an emotion engine"

[1508] (Claim 1)

[1509] A means of looking up the meaning of a term selected by the user on the display screen of an information processing device,

[1510] A means of displaying the meaning of that term on the screen,

[1511] A method for automatically creating queries to generative artificial intelligence,

[1512] A means of setting a character limit to prevent the display results from becoming redundant,

[1513] A means of analyzing user emotions,

[1514] A means of adjusting queries to generative artificial intelligence based on recognized emotions,

[1515] A system that includes this.

[1516] (Claim 2)

[1517] The system according to claim 1, wherein a query to the generative artificial intelligence is automatically generated based on selected terms.

[1518] (Claim 3)

[1519] The system according to claim 1, wherein the display result is displayed directly on the display screen.

[1520] "Application example 2 when combining with an emotional engine"

[1521] (Claim 1)

[1522] A means of looking up the meaning of a word or phrase selected by the user on the messaging application's communication screen,

[1523] A means of displaying the meaning of that phrase on the communication screen,

[1524] A method for automatically creating queries to generative artificial intelligence,

[1525] A means of setting a character limit to prevent the display results from becoming redundant,

[1526] A means of recognizing the user's emotions and adjusting the query to the generative artificial intelligence based on those emotions,

[1527] A means of providing additional information related to the information the user is viewing in real time,

[1528] A system that includes this.

[1529] (Claim 2)

[1530] The system according to claim 1, wherein a query to the generative artificial intelligence is automatically generated based on selected words and the user's emotions.

[1531] (Claim 3)

[1532] The system according to claim 1, wherein the display result is displayed directly on the communication screen and adjusted to attract the user's interest. [Explanation of symbols]

[1533] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

[Claim 1] Equipped with a processor, The aforementioned processor, The system retrieves the words selected by the user from the conversation text displayed on the chat screen of the messenger application. The selected phrase is queried from a dictionary database to obtain semantic information for that phrase. The aforementioned semantic information is transmitted to the user's terminal and displayed on the chat screen, A prompt statement is generated to instruct the generating AI model to generate detailed information about the selected phrase, Based on the content of the user's statements included in the aforementioned conversation, the emotions of the user are recognized. Based on the recognized emotion, adjust the prompt message. The adjusted prompt statement is sent to the generating AI model to obtain a response from the generating AI model that includes the detailed information. The above response is adjusted by applying a character limit. The adjusted response is sent to the user's terminal and displayed on the chat screen. system.

Images