System

Abstract

A system is provided.SOLUTION: A system in which a user makes an inquiry in an advertisement and provides a real-time answer to the inquiry, the system including: means for transmitting advertisement data to a user terminal; means for receiving an inquiry content with respect to the advertisement displayed on the user terminal; means including a generation AI model for analyzing the received inquiry content and generating a corresponding answer; means for transmitting the generated answer to the user terminal; and means for displaying the answer on the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With traditional advertising, users only view ads and are unable to directly communicate or inquire, making it difficult to provide solutions that address users' concerns and needs. Furthermore, advertisers are unable to obtain direct feedback through dialogue with users, making it difficult to find improvements that will maximize advertising effectiveness. For these reasons, new methods are needed to increase advertising effectiveness. [Means for solving the problem]

[0005] The present invention provides a system that allows users to make inquiries within advertisements and provides real-time responses to those inquiries. This system includes the following means: a means for transmitting advertisement data to a user terminal, a means for receiving inquiries regarding advertisements displayed on the user terminal, a means for providing a generative AI model that analyzes the received inquiries and generates corresponding responses, a means for transmitting the generated responses to the user terminal, and a means for displaying the responses on the user terminal. This allows users to make inquiries directly within advertisements and receive responses on the spot, enabling advertisers to understand user needs in real time and maximize the effectiveness of their advertisements. Furthermore, by providing a means for receiving user feedback and providing it to advertisers, it becomes possible to further consider ways to improve the advertisements.

[0006] "Advertising data" refers to advertising content such as images, text, videos, etc. that are sent to and displayed on a user terminal.

[0007] "User terminal" refers to a device such as a computer, smartphone, or tablet that a user uses to view advertisements and make inquiries.

[0008] "Inquiry content" refers to questions or doubts that users directly input regarding advertisements.

[0009] "Means for receiving" refers to the communication protocol or interface that allows the server to receive the inquiry content sent from the user terminal.

[0010] A "generative AI model" refers to an artificial intelligence system that analyzes the content of a user's inquiry and automatically generates the optimal answer that corresponds to that content.

[0011] "Means for sending" refers to a communication means by which the server sends the generated answer to the user terminal.

[0012] The "means for displaying" refers to an interface for visually presenting the submitted answer to the user on the user terminal.

[0013] "Feedback" refers to the opinions and thoughts that users who receive answers provide to the system.

[0014] "Advertiser" refers to an entity that provides advertisements and receives feedback from users to improve the advertisements. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries. The system includes functions for transmitting advertisement data to a user terminal, receiving inquiries about advertisements displayed on the user terminal, providing a generative AI model that analyzes the received inquiries and generates corresponding answers, transmitting the generated answers to the user terminal, and displaying the answers on the user terminal.

[0037] Explanation of program processing

[0038] Sending advertising data

[0039] Server: Sends advertising data over the Internet to advertising spaces on web pages or applications. Advertising data can be in the form of images, text, videos, etc.

[0040] User views ad and initiates inquiry

[0041] Terminal: Displays the received advertising data to the user. Displays an inquiry button while the advertisement is being displayed and sets a click event.

[0042] User: Watch the ad and if there is anything they would like to inquire about, click the inquiry button.

[0043] Enter and submit your inquiry

[0044] Terminal: Detects the click event of the inquiry button, displays an inquiry interface (chat window, etc.), and sends the inquiry content entered by the user to the server.

[0045] User: Type your question or concern into the chat window and click the send button.

[0046] Processing inquiries and generating responses

[0047] Server: Passes the inquiry received from the user to the generative AI model and requests it to analyze and generate an answer.

[0048] Generative AI model: Analyzes the content of the received inquiry and generates the optimal answer. If the question is about smartphone performance, this answer might be, "The battery of this smartphone lasts for 24 hours on average."

[0049] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0050] Show Answers

[0051] Terminal: The received reply is displayed on the user's terminal. The user can immediately check the reply displayed in the chat window.

[0052] User Feedback (Optional)

[0053] User: After receiving the answer, enter feedback on the ad they watched and the answer.

[0054] Terminal: displays the interface for feedback and sends user input to the server.

[0055] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0056] Advertisers: Based on the feedback, they can consider ways to improve their advertising and decide on special offers and discounts for users.

[0057] Specific examples

[0058] Scenario 1: Product Question

[0059] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0060] Scenario 2: Providing feedback

[0061] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0062] This system allows users to make direct inquiries about advertisements and receive answers on the spot in real time. It also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0063] The processing flow will be explained below.

[0064] Step 1:

[0065] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[0066] Step 2:

[0067] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[0068] Step 3:

[0069] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[0070] Step 4:

[0071] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[0072] Step 5:

[0073] User: Enter their question or concern into the inquiry interface and click the submit button.

[0074] Step 6:

[0075] Terminal: Sends the query entered by the user to the server.

[0076] Step 7:

[0077] Server: Receives the user's inquiry and passes it to the generative AI model to analyze and generate an answer.

[0078] Step 8:

[0079] Generative AI model: Analyzes the content of the inquiry and generates the optimal answer, such as "The battery of this smartphone lasts for an average of 24 hours."

[0080] Step 9:

[0081] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0082] Step 10:

[0083] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[0084] Step 11 (Optional):

[0085] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[0086] Step 12 (Optional):

[0087] Terminal: Sends the feedback entered by the user to the server.

[0088] Step 13 (Optional):

[0089] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0090] Step 14 (Optional):

[0091] Advertisers: Based on feedback, they consider ways to improve their advertising and decide on special offers and discounts for users.

[0092] Step 15 (Optional):

[0093] Server: Sends special offers and discount information from advertisers to user terminals.

[0094] Step 16 (Optional):

[0095] Terminal: Displays special offers and discount information to the user.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] In today's advertising systems, there is a lack of a way for users to ask questions directly within the ad and receive answers in real time. There is also no mechanism for advertisers to quickly collect user feedback to maximize advertising effectiveness. This makes it difficult to resolve user questions and improve advertising effectiveness.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes a means for transmitting advertisement data to a user terminal, a means for receiving inquiries about an advertisement displayed on the user terminal, a means for analyzing the received inquiries and providing a generative AI model for generating corresponding answers, and a means for receiving feedback from users and providing the feedback to advertisers. This allows users to receive answers to inquiries in advertisements in real time, and enables advertisers to quickly collect user feedback and maximize the effectiveness of their advertisements.

[0101] "Advertising data" refers to digital content, including the content of advertisements, that is sent to and displayed on a user's device. The format may include images, text, video, etc.

[0102] "User terminal" refers to the device on which a user displays and interacts with advertisements. Examples include computers, smartphones, and tablets.

[0103] A "generative AI model" is an artificial intelligence system that analyzes the content of incoming inquiries and generates optimal answers, for example, using natural language processing technology.

[0104] "Feedback" refers to opinions and ratings provided by users regarding advertisements and responses, which allows advertisers to evaluate the effectiveness of their advertisements and obtain information to improve them.

[0105] A "server" is a computer system that manages communication with user terminals, delivers advertising data, receives inquiries, and provides data to generative AI models.

[0106] An "inquiry interface" is a screen or window that allows users to enter inquiries while an ad is being displayed. For example, it could be a chat window.

[0107] A "prompt" is a sentence that helps the generative AI model interpret the query, allowing it to generate a more accurate answer.

[0108] "Advertiser" means a company or individual that distributes advertisements and uses user feedback to evaluate and improve their effectiveness.

[0109] The present invention is a system that allows users to ask questions within advertisements and provides real-time answers to those questions. The system consists of the following main components:

[0110] Overall system configuration

[0111] The system includes a server that transmits advertising data to user terminals, a terminal that displays advertisements and receives inquiries from users, and a generative AI model that processes the inquiries and generates answers.

[0112] Sending advertising data

[0113] The server transmits the advertising data to the user terminal via the Internet. This advertising data is in the form of images, text, video, etc. For example, server software for delivering advertisements (AdServer) is used.

[0114] User views ad and initiates inquiry

[0115] The terminal displays the received advertising data to the user and provides a query interface. The query interface sets a click event and is displayed when the user clicks the query button. The terminal generates this interface using, for example, JavaScript (registered trademark) or native application code.

[0116] Enter and submit your inquiry

[0117] When the device detects a click event from the user, it displays an inquiry interface (e.g., a chat window). The user enters a question in the inquiry interface and clicks the send button. The entered inquiry content is sent to the server in JSON format. The device uses a UI library such as React or Vue.js.

[0118] Processing inquiries and generating responses

[0119] The server passes the query received from the user to the generative AI model. The generative AI model analyzes the query, generates a prompt sentence, and generates the optimal answer. An example of a prompt sentence is "Please tell me about the battery life." The generative AI model can use, for example, OpenAI's GPT-4 (registered trademark).

[0120] Submitting and viewing answers

[0121] The server receives the answers returned by the generative AI model and sends them to the user's device. The device displays the received answers in a chat window, allowing the user to get answers to their questions in real time. For front-end processing, a JavaScript library is used to dynamically update the DOM.

[0122] Collect user feedback (optional)

[0123] After receiving the answer, the user can enter feedback on the advertisement or the answer. A feedback interface is also displayed on the terminal, and the user's input is sent to the server. The server stores the collected feedback in a database and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve the advertisement.

[0124] Specific examples

[0125] Scenario 1: Product Question

[0126] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0127] Scenario 2: Providing feedback

[0128] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0129] This allows users to make direct inquiries about advertisements and receive answers on the spot in real time, and also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0130] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0131] Step 1:

[0132] The server transmits advertising data to the user terminal. Specifically, the server retrieves advertising data from a database and transmits it to the user terminal via the Internet. This advertising data includes images, text, video, etc. The input is the advertising data in the database, and the output is the transmission of the advertising data to the user terminal.

[0133] Step 2:

[0134] The terminal displays the received advertising data to the user. Furthermore, while the advertisement is being displayed, an inquiry button is displayed on the screen and a click event is set for this button. If the user is interested in the advertisement and clicks the inquiry button, the process proceeds to the next step. The input is advertising data from the server, and the output is the display of the advertisement to the user and the setting of an inquiry button.

[0135] Step 3:

[0136] When a user watches an advertisement and has an inquiry, they click the inquiry button. Clicking this button displays the inquiry interface. The input is the advertisement data on the device and the inquiry button, and the output is the interface displayed on the device.

[0137] Step 4:

[0138] The device detects the user's click event and displays an interface for inquiries (for example, a chat window). The inquiry entered by the user is converted into JSON format and sent to the server. The specific operation here is to capture the click event using JavaScript or native app code, dynamically generate the interface, and send the data to the server. The input is the user's inquiry, and the output is the JSON data sent to the server.

[0139] Step 5:

[0140] The server analyzes the query received from the user. It passes this to the generative AI model and processes the data to generate a prompt and obtain the optimal answer. Specific operations include organizing the data through a normalization process and sending the query to the generative AI model. The input is the user's query, and the output is the prompt to the generative AI model.

[0141] Step 6:

[0142] The generative AI model analyzes the received prompt and generates the optimal answer. For example, if the prompt is "Please tell me about the battery life," the model generates the answer "The battery of this smartphone lasts for 24 hours on average." The input is the prompt from the server, and the output is the generated answer text.

[0143] Step 7:

[0144] The server receives the answer returned by the generative AI model and sends it to the user's device. Specific operations include receiving the answer and transferring the data. The input is the answer from the generative AI model, and the output is sending the answer to the device.

[0145] Step 8:

[0146] The device displays the response received from the server in the chat window. The user can immediately check the response. Specifically, it updates the DOM using a JavaScript library and displays the response. The input is the response data from the server, and the output is the response displayed in the chat window.

[0147] Step 9:

[0148] The user receives the answer and can input feedback on the advertisement or answer they have viewed. The input is the user's feedback comment, and the output is the generation of feedback data.

[0149] Step 10:

[0150] The terminal displays a feedback interface and sends user input to the server. Specific operations include generating a screen for feedback input and sending data. The input is user feedback, and the output is sending feedback to the server.

[0151] Step 11:

[0152] The server stores the feedback from the user in a database and provides it to the advertiser. The input is the user feedback data, and the output is storing it in the database and providing the feedback to the advertiser.

[0153] Step 12:

[0154] Advertisers can use the collected feedback to improve their ads and offer special offers or discounts. Specific operations include analyzing feedback data and formulating improvement plans. The input is feedback data, and the output is improved ads and special offer information.

[0155] (Application example 1)

[0156] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0157] Systems that enable real-time communication between users and advertisers through advertising are limited, and traditional advertising systems lack a means to instantly answer user questions. They also lack a mechanism for efficiently collecting user feedback and providing it to advertisers to help improve their advertising. Since there is insufficient effort to improve engagement through the provision of user rewards, maximizing advertising effectiveness while improving user experience is a challenge.

[0158] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0159] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for transmitting the generated answer to the user terminal, means for displaying the answer on the user terminal, means for collecting user feedback and storing the feedback in a database, means for providing the stored feedback to the advertiser, and means for providing a benefit to the user based on the feedback. This allows users to make inquiries in real time while viewing an advertisement and receive an answer that is generated immediately. Furthermore, advertisers can improve the content of their advertisements based on the feedback, thereby increasing user engagement.

[0160] "Advertising data" refers to advertising information transmitted over the Internet and displayed on a user terminal, and may be in the form of text, images, video, or the like.

[0161] "User terminal" refers to an electronic device used to receive and display advertising data, and generally refers to a smartphone or tablet.

[0162] The "inquiry content" is text information that the user inputs as a question or concern about the advertisement while viewing the advertisement.

[0163] A "generative AI model" is an artificial intelligence model that analyzes the content of received inquiries and generates optimal answers.

[0164] "Feedback" refers to information provided by a user by inputting their evaluation or opinion of an advertisement or its response.

[0165] "Benefits" are rewards, discounts, or other benefits offered by advertisers based on user feedback.

[0166] "Database" means an information system for storing and managing feedback and other data collected from users.

[0167] An "advertiser" is a company or individual that creates advertisements and displays them to users.

[0168] A "server" is a central computing system for providing services to user terminals over the Internet.

[0169] The "means for providing a reward" is a mechanism for generating a reward based on user feedback and transmitting it to the user terminal.

[0170] In this invention, the server, terminal, and user elements work together to respond to inquiries, collect feedback, and provide special benefits in real time. The role of each element will be specifically described below.

[0171] server

[0172] The server sends advertising data to the user's device via the Internet. This includes advertising information such as text, images, and videos. The server also receives inquiries from users, passes them to a generative AI model for analysis, and generates an optimal answer. This answer is then sent to the user's device so that the user can view it immediately. The server also collects feedback from users, stores the data in a database, and provides it to advertisers. It is also responsible for generating rewards based on the feedback and providing them to users.

[0173] Terminal

[0174] The terminal provides an interface for users to view advertisements. Ad data is sent from the server and displayed on the terminal. An inquiry button is displayed while the advertisement is displayed, and when the user clicks it, an inquiry interface opens. When the user enters a question, the content is sent to the server. An interface for displaying the answers sent from the server is also provided, allowing the user to check the answers in real time. A feedback input interface is also provided, and this is sent to the server.

[0175] User

[0176] Users watch an advertisement and click the inquiry button in the advertisement to enter a question. They can check the answer from the server in real time through the device interface. They can also enter and submit feedback on the advertisement and answer. This feedback is collected by the server and provided to the advertiser. If a reward is offered in response to the feedback, the user can receive the reward.

[0177] Specific examples

[0178] As a concrete example, imagine a user watching an advertisement for a new car on their smartphone. The user asks, "What is the fuel efficiency of this car?" The server receives the query and uses a generative AI model to generate an answer: "The average fuel efficiency of this car is 18km / L." This answer is sent to the user's smartphone and displayed immediately. If the user also provides feedback such as "The explanation in the advertisement was easy to understand," the server collects this feedback and provides it to the advertiser. The advertiser can improve the advertisement based on the feedback and send the user a discount coupon as a reward.

[0179] Prompt Sentence Examples

[0180] "I want to build an in-ad query system to ask about the fuel economy of this car. This will include generating answers in real time and displaying them to the user."

[0181] In this way, this invention allows users to make direct inquiries about advertisements and receive answers on the spot in real time. Advertisers can also quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0183] Processing Steps

[0184] Step 1:

[0185] Server: Transmits advertising data to the user terminal.

[0186] Input: Ad data stored on the server (text, images, videos, etc.).

[0187] Data processing: Converting advertising data into a format appropriate for the user's device.

[0188] Output: Advertising data sent to the user device.

[0189] Specific operation: The server program that distributes advertisements transmits advertisement data to user terminals via the Internet. At this time, it takes into consideration the user's profile information to select and transmit the most suitable advertisement.

[0190] Step 2:

[0191] Terminal: displays advertising data to the user.

[0192] Input: Advertising data sent from the server.

[0193] Data processing: Display advertising data appropriately according to the device's screen size and resolution.

[0194] Output: The advertisement displayed on the user's device display.

[0195] Specific operation: The advertisement display program on the user's device receives the advertisement data and displays the advertisement (text, image, video) on the screen. An inquiry button is also displayed while the advertisement is displayed.

[0196] Step 3:

[0197] User: Click the inquiry button and enter the inquiry details.

[0198] Input: User action after viewing the ad (clicking the inquiry button).

[0199] Data processing: Display an interface for inquiries (such as a chat window).

[0200] Output: The query entered by the user.

[0201] What it does: When a user clicks on the contact button in the ad, a chat window opens where the user can type in their question.

[0202] Step 4:

[0203] Terminal: Sends the query to the server.

[0204] Input: The query entered by the user.

[0205] Data processing: The query content is converted into a format that can be sent to the server.

[0206] Output: The query data sent to the server.

[0207] Specific operation: The query entered by the user is converted into an appropriate format and sent to the server.

[0208] Step 5:

[0209] Server: Passes the query to the generative AI model and generates an answer.

[0210] Input: The inquiry received from the user.

[0211] Data processing: Converting the query content into a format suitable for the generative AI model.

[0212] Output: The answer generated by the generative AI model.

[0213] Specific operation: The server passes the query to a generative AI model (e.g., GPT-3 (registered trademark)) to generate the optimal answer. The generated answer is then received and passed to the next step.

[0214] Step 6:

[0215] Server: Sends the generated answer to the user terminal.

[0216] Input: The answer generated by the generative AI model.

[0217] Data processing: The response data is converted into a format for sending to the user's terminal.

[0218] Output: Answer data sent to the user's device.

[0219] Specific operation: The server sends the answer data received from the generative AI model to the user's device.

[0220] Step 7:

[0221] Terminal: Display the answer to the user.

[0222] Input: The response data sent from the server.

[0223] Data processing: The response data is displayed appropriately on the user's device screen.

[0224] Output: The answer displayed on the user's device display.

[0225] Specific operation: The generated answer is displayed on the screen of the user's device so that the user can check it immediately.

[0226] Step 8:

[0227] User: Enter feedback on ads and answers.

[0228] Input: User ratings and opinions on ads and answers.

[0229] Data processing: Write feedback into the feedback interface.

[0230] Output: User-entered feedback data.

[0231] Specific operation: The user enters feedback on the advertisement or answer in the feedback interface.

[0232] Step 9:

[0233] Device: Sends feedback to the server.

[0234] Input: Feedback data entered by the user.

[0235] Data processing: The feedback data is converted into a format for sending to the server.

[0236] Output: Feedback data sent to the server.

[0237] Specific behavior: Converts the feedback entered by the user into an appropriate format and sends it to the server.

[0238] Step 10:

[0239] Server: Collects feedback and stores it in a database.

[0240] Input: Feedback data received from the user.

[0241] Data processing: Organizing the feedback data and converting it into a format that can be stored in the database.

[0242] Output: Feedback data stored in a database.

[0243] Specific operation: The server stores the feedback data received from the user in a database and makes it available for reference to advertisers.

[0244] Step 11:

[0245] Server: Generates rewards based on the feedback and sends them to the user.

[0246] Input: Feedback data stored in the database.

[0247] Data processing: Generate rewards based on the feedback and convert them into a format that can be sent to the user's device.

[0248] Output: Reward data sent to user device.

[0249] Specific operation: The server analyzes the feedback in the database, generates a reward (e.g., a discount coupon), and sends it to the user's terminal.

[0250] These steps result in a system that provides real-time answers to user inquiries about advertisements and maximizes advertising effectiveness based on collected feedback.

[0251] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0252] The present invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries, and further combines an emotion engine that recognizes the user's emotions. The system includes the following means.

[0253] First, the server sends the advertising data over the Internet to the advertising space on a web page or application, which can be in the form of images, text, video, etc.

[0254] The terminal then displays the received advertisement data on the user terminal, and the advertisement includes an inquiry button that the user can click to launch an inquiry interface.

[0255] When a user watches an advertisement and clicks on an inquiry button in the advertisement, the terminal displays an inquiry interface, providing an interface for the user to input questions or concerns. When the user inputs the inquiry content and clicks on the send button, the terminal sends the content to the server.

[0256] The server receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model analyzes the user's inquiry and generates a corresponding answer. At the same time, the emotion engine recognizes the user's emotion contained in the inquiry and analyzes the emotion data.

[0257] The server receives the answer from the generative AI model and the recognition data from the emotion engine, and adjusts the answer according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite. The server then sends the adjusted answer to the user's device, where it is displayed to the user.

[0258] The system also includes a feedback function, allowing users to enter feedback after receiving a response. The device sends this feedback to the server, which then collects the feedback and provides it to the advertiser. Furthermore, the emotion engine also collects user emotion data and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve their advertisements and offer special offers or discounts to users.

[0259] Specific examples of program processing

[0260] Scenario 1: Smartphone enquiry

[0261] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from the generative AI model, while simultaneously analyzing the user's emotions using the emotion engine. The generative AI model generates the answer, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[0262] Scenario 2: Providing feedback and sentiment data

[0263] After viewing the advertisement, User B uses the inquiry interface to input feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the emotion engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This emotion data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0264] As a result, the present invention is a system with advanced functionality that allows for the provision of answers to users' inquiries in real time, recognizes the user's emotions, adjusts the answers accordingly, and provides feedback and emotional data to advertisers.

[0265] The processing flow will be explained below.

[0266] Step 1:

[0267] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[0268] Step 2:

[0269] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[0270] Step 3:

[0271] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[0272] Step 4:

[0273] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[0274] Step 5:

[0275] User: Enter their question or concern into the inquiry interface and click the submit button.

[0276] Step 6:

[0277] Terminal: Sends the query entered by the user to the server.

[0278] Step 7:

[0279] Server: Receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model generates a corresponding answer, and the emotion engine analyzes the user's emotions.

[0280] Step 8:

[0281] Generative AI model: Analyzes the query and generates the optimal answer, such as "The battery life of this smartphone is 24 hours on average."

[0282] Step 9:

[0283] Emotion engine: Recognizes the user's emotions from the content of the inquiry. For example, if the user is dissatisfied or suspicious, the emotion engine will recognize that state.

[0284] Step 10:

[0285] Server: Receives the answers from the generative AI model and the recognition data from the emotion engine, and adjusts the answers according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite and detailed.

[0286] Step 11:

[0287] Server: Sends the adjusted answer to the user terminal.

[0288] Step 12:

[0289] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[0290] Step 13 (Optional):

[0291] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[0292] Step 14 (Optional):

[0293] Terminal: Sends the feedback entered by the user to the server.

[0294] Step 15 (Optional):

[0295] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0296] Step 16 (Optional):

[0297] Emotion engine: accumulates user emotional data and provides it to advertisers. For example, the data might say, "The user is satisfied with the information, but has doubts about the price."

[0298] Step 17 (Optional):

[0299] Advertisers: Use feedback and sentiment data to explore ways to improve their ads, including offering new offers or discounts.

[0300] Step 18 (Optional):

[0301] Server: Sends special offers and discount information from advertisers to user terminals.

[0302] Step 19 (Optional):

[0303] Terminal: Displays special offers and discount information to the user. For example, if a 10% discount coupon is offered, this information is visually presented to the user.

[0304] Example 2

[0305] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0306] Conventional advertising systems have the problem that users cannot get real-time answers when they have questions about an ad, resulting in a poor user experience. Furthermore, it is difficult to recognize user emotions and respond appropriately, which means that feedback to advertisers is not properly collected. Furthermore, there are also issues such as delays in implementing advertising improvements based on user emotion data.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0308] In this invention, the server includes means for transmitting advertising data to a user terminal, means for displaying the advertising data on the user terminal and providing an inquiry button, means for providing an interface for the user to click the inquiry button and input the inquiry content, means for transmitting the inquiry content from the user terminal to the server, means for passing the received inquiry content to a generation AI model and a sentiment analysis engine for analysis and generating a corresponding answer, means for analyzing the user's emotional data and adjusting the answer, means for transmitting the generated answer to the user terminal, and means for displaying the answer on the user terminal. This allows users to receive answers to questions raised in advertisements in real time and receive the optimal response based on their emotions. Furthermore, advertisers can evaluate and improve the effectiveness of their advertisements based on user feedback and emotional data.

[0309] "Advertising data" refers to information sent to a user terminal in the form of images, text, video, etc.

[0310] "User terminal" refers to an electronic device that allows a user to view advertising data and make inquiries.

[0311] "Contact Button" refers to an interactive element within an advertisement that a user can click to launch a contact interface.

[0312] "Inquiry interface" refers to an interface through which a user can input questions or concerns about an advertisement.

[0313] "Server" refers to a remote computer that sends advertising data, receives inquiries, passes data to generative AI models, and performs sentiment analysis.

[0314] A "generative AI model" refers to artificial intelligence that analyzes the content of a user's inquiry and generates a corresponding answer.

[0315] An "emotion analysis engine" refers to a system that analyzes emotional data from user inquiries and provides that data.

[0316] "User emotion data" refers to information about a user's emotions analyzed by the emotion analysis engine.

[0317] "Adjusted answers" refer to answers generated by a generative AI model and appropriately modified based on the user's emotional data.

[0318] "Feedback" refers to the opinions and thoughts that users enter after receiving a response.

[0319] "Advertiser" refers to an entity that provides advertising data and receives feedback from users.

[0320] The present invention is a system that allows users to make inquiries within advertisements, provides real-time answers to those inquiries, and recognizes the user's emotions. This system is composed of a server, a terminal, a generative AI model, and a sentiment analysis engine.

[0321] First, the server sends advertising data to the user terminal. This advertising data can be in the form of images, text, videos, etc. The server periodically updates the advertising data and sends it to the terminal via the Internet.

[0322] The terminal then displays the advertisement data received from the server to the user. The advertisement includes an inquiry button that the user can click to launch an inquiry interface. The inquiry interface provides a text box for the user to enter their question or inquiry.

[0323] When a user enters the inquiry content and clicks the send button, the device sends the content to the server, which then passes the received inquiry content to the generative AI model and sentiment analysis engine for processing.

[0324] The generative AI model uses natural language processing technology such as GPT-3 to analyze the inquiry and generate an appropriate response. At the same time, the sentiment analysis engine analyzes the user's emotional data contained in the inquiry. The sentiment analysis engine uses sentiment analysis technology such as the Sentiment Analysis API.

[0325] The server receives the answer from the generative AI model and the emotion data from the sentiment analysis engine, and adjusts the answer as needed. For example, if the user expresses dissatisfaction, the answer will be made more polite. The adjusted answer is sent from the server to the device and displayed to the user.

[0326] After receiving the answer, the user can enter feedback. The device will send this feedback to the server, which will then provide the feedback and sentiment data to the advertiser, who can then use it to improve their advertisement and offer rewards or discounts to the user if necessary.

[0327] Below are some examples of specific scenarios and prompts.

[0328] Scenario 1: Smartphone enquiry

[0329] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from a generative AI model (e.g., GPT-3) and simultaneously analyzes the user's emotions using a sentiment analysis engine. The generative AI model generates the answer, "The battery life of this smartphone is, on average, 24 hours," and the sentiment analysis engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[0330] Scenario 2: Providing feedback and sentiment data

[0331] After viewing the advertisement, User B uses the inquiry interface to enter feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the sentiment analysis engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This sentiment data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0332] With these scenarios, the present invention is a system that allows users to make real-time inquiries within advertisements and receive responses tailored to their emotions. It also allows advertisers to optimize the effectiveness of their advertisements based on user feedback and emotion data.

[0333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0334] Step 1:

[0335] The server sends the advertising data to the user's device. The advertising data can be in the form of images, text, videos, etc., and is sent via the Internet. The input to this process is the latest advertising data stored in the advertising database, and the output is the advertising data to be sent to the user's device. The server formats the advertising data into JSON format and sends it via an HTTP request.

[0336] Step 2:

[0337] The terminal receives the advertising data and displays it to the user. The displayed advertisement includes a query button. The input to this process is the advertising data sent from the server, and the output is an advertisement display that the user can visually confirm. The terminal renders the advertising data in HTML format and displays it in the browser or application. A JavaScript event listener is set on the query button.

[0338] Step 3:

[0339] When a user watches an ad and clicks on the inquiry button, the inquiry interface is launched. The input of this process is the user's click, and the output is the display of the inquiry interface. The user enters a question or concern in a text box. Specifically, JavaScript pops up an inquiry form and allows the user to enter text.

[0340] Step 4:

[0341] The terminal sends the inquiry entered by the user to the server. When the user enters the inquiry and clicks the send button, the terminal sends the content to the server. The input in this process is the user's inquiry, and the output is the inquiry data sent to the server. Specifically, JavaScript retrieves the content of the text box and sends it to the server via an AJAX request.

[0342] Step 5:

[0343] The server receives the query and passes it to the generative AI model and sentiment analysis engine for analysis. The input to this process is the user's query, and the output is the generative AI model's answer and the sentiment analysis engine's emotion data. Specifically, the server formats the query into JSON and sends a request to the generative AI model API and sentiment analysis engine API.

[0344] Step 6:

[0345] The generative AI model analyzes the query content and generates a corresponding answer. At the same time, the sentiment analysis engine analyzes the user's emotions. The input in this process is the query content sent from the server, and the output is the answer and emotional data. Specifically, the generative AI model (e.g., GPT-3) uses NLP technology to perform analysis and generate text. The sentiment analysis engine (e.g., Sentiment Analysis API) performs sentiment analysis.

[0346] Step 7:

[0347] The server receives the generated answer and emotion data, adjusts the answer as needed, and then sends the adjusted answer to the user's device. The input to this process is the answer from the generative AI model and the emotion data from the emotion analysis engine, and the output is the adjusted answer sent to the device. Specifically, the server implements simple logic to make adjustments, such as including additional information if there is dissatisfaction, and then sends it as JSON data to the device.

[0348] Step 8:

[0349] The terminal receives the adjusted answer and displays it to the user. The input to this process is the adjusted answer sent by the server, and the output is the answer displayed to the user. Specifically, the terminal parses the received JSON data and generates HTML to display in the browser or application.

[0350] Step 9:

[0351] After the user checks the answer, they can enter their feedback. The input in this process is the user's feedback, and the output is the feedback data sent to the terminal. Specifically, a feedback text box is displayed below the answer, and the user can enter their feedback.

[0352] Step 10:

[0353] The device sends the user's feedback to the server. The input of this process is the user's feedback, and the output is the feedback data sent to the server. Specifically, JavaScript retrieves the feedback content and sends it to the server via an AJAX request.

[0354] Step 11:

[0355] The server provides feedback and emotion data to the advertiser, who can then use this information to improve their advertisement and offer special offers or discounts to users. The input to this process is the feedback and emotion data, and the output is a report provided to the advertiser. Specifically, the server stores the feedback and emotion data in a database and periodically provides reports to the advertiser.

[0356] (Application example 2)

[0357] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0358] In conventional advertising systems, when a user has doubts or questions about an advertisement, it is difficult to respond quickly, making it difficult to maintain the user's interest. Furthermore, there is no method for maximizing the effectiveness of advertising by providing answers or feedback that take the user's emotions into consideration. Furthermore, there is a lack of means for advertisers to grasp user reactions and emotions in real time and optimize their advertising strategies. The purpose of the present invention is to solve these problems.

[0359] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0360] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for providing an emotion engine that analyzes user emotions, means for adjusting the answer based on the analyzed emotion data, means for transmitting the generated answer to the user terminal, means for receiving feedback from the user and providing the feedback and analyzed emotion data to the advertiser, and means for displaying the answer on the user terminal. This makes it possible to provide an appropriate answer to a user's inquiry about an advertisement in real time, taking the user's emotion into consideration. Furthermore, advertisers can optimize their strategies for maximizing the effectiveness of their advertisements in real time based on the user feedback and emotion data.

[0361] "Advertising data" refers to digital information in the form of images, text, videos, etc. that are displayed on a user terminal.

[0362] A "user terminal" is an electronic device such as a smartphone, tablet, or PC that a user uses to view information.

[0363] The "inquiry interface" is an interface that allows users to input questions or opinions about advertisements.

[0364] A "generative AI model" is an artificial intelligence model that analyzes the content of a user's inquiry and automatically generates a corresponding answer.

[0365] An "emotion engine" is a system or component that analyzes emotions from user input.

[0366] "Feedback" refers to information that a user inputs as an evaluation or opinion of a response or advertisement received from the system.

[0367] "Advertiser" means the person or company responsible for serving advertisements and measuring and optimizing their effectiveness.

[0368] The "means for adjusting responses" is a function that modifies the responses generated by the generative AI model to make them more appropriate based on the emotional data analyzed by the emotion engine.

[0369] An "advertising strategy" is a plan or policy that an advertiser formulates to maximize the effectiveness of advertising based on user feedback and emotional data.

[0370] The present invention provides a system for providing real-time answers to user inquiries while an advertisement is being displayed, and further adjusts the answers by analyzing the user's emotions. Specific embodiments of this system are described below.

[0371] Overall structure

[0372] This system consists of a server, user terminals, and a network. The server is equipped with a generative AI model and an emotion engine, while the user terminal displays advertising data and provides an inquiry interface. The network connects the server and user terminals via the Internet.

[0373] server

[0374] The server has the following features:

[0375] 1. Sending advertising data: Sending advertising data provided by advertisers to user devices. Advertising data can be in the form of images, text, videos, etc.

[0376] 2. Receiving the inquiry: The inquiry is received from the user terminal.

[0377] 3. Answer generation using generative AI models: Analyze the content of the received inquiry and generate an appropriate answer.

[0378] 4. Emotion analysis using an emotion engine: Analyzes emotional data from the user's inquiry and adjusts the response according to the emotion.

[0379] 5. Send answer: Send the adjusted answer to the user device.

[0380] 6. Collecting and providing feedback: Receive feedback from users and provide it to advertisers along with analyzed sentiment data.

[0381] User terminal

[0382] The user terminal has the following features:

[0383] 1. Display advertising data: Display the received advertising data.

[0384] 2. Providing an inquiry interface: Provide an interface for users to make inquiries about advertisements. When users click the inquiry button, an interface will be displayed and they can enter their questions.

[0385] 3. Sending the inquiry: The inquiry entered by the user is sent to the server.

[0386] 4. Display Answer: Display the answer received from the server to the user.

[0387] 5. Feedback collection: Collect the feedback provided by the user and send it to the server.

[0388] Software and hardware used

[0389] Hardware:

[0390] User devices such as smartphones, tablets, and PCs

[0391] Cloud server (e.g., AWS (registered trademark), GCP)

[0392] software:

[0393] Generative AI models (e.g., OpenAI GPT-4)

[0394] Emotion engine (e.g. IBM Watson®)

[0395] Server-side frameworks (e.g. Python + Flask)

[0396] Specific examples

[0397] For example, consider the case where User A is watching an advertisement for a new smartphone on their smartphone. They click the inquiry button in the advertisement and enter, "How long is the battery life of this smartphone?" The user's device sends this inquiry to the server. The server uses a generative AI model to generate an answer such as, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that "User A" does not have any specific complaints or doubts. The answer is adjusted and sent as is to the user's device, where it is displayed to the user.

[0398] Example prompt sentence:

[0399] "Your ad campaign provides a system that responds to real-time user inquiries. Please answer the following query. User asks: 'How long does the battery last on this smartphone?'"

[0400] As a result, the present invention is configured to specifically implement a system that enables providing answers to users' inquiries in real time, recognizes users' emotions, adjusts answers accordingly, and provides feedback and emotional data to advertisers.

[0401] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0402] Step 1:

[0403] The server transmits the advertising data provided by the advertiser to the user's terminal via the Internet. The advertising data includes images, text, videos, etc., and is displayed on the user's terminal.

[0404] Input: Advertisement data from advertisers

[0405] Output: Sending advertising data to the user device

[0406] Operation: The server receives data provided by the advertiser and sends it to the user's device via the Internet, which then displays the advertisement.

[0407] Step 2:

[0408] The user terminal displays the received advertisement data, which includes a query button, and the user views the advertisement.

[0409] Input: Advertising data sent from the server

[0410] Output: A display of the ad that the user sees.

[0411] Operation: The user terminal displays the received advertising data on the screen, allowing the user to view the advertisement.

[0412] Step 3:

[0413] The user clicks on the inquiry button in the ad to launch the inquiry interface, enters a question, and clicks the submit button.

[0414] Input: User click actions and entered questions

[0415] Output: Generates query content

[0416] Operation: The user terminal detects the user's click and launches the inquiry interface. When the user enters a question and clicks the submit button, the content is generated as inquiry data.

[0417] Step 4:

[0418] The user terminal transmits the generated inquiry to the server.

[0419] Input: The query entered by the user

[0420] Output: Sends query to server

[0421] Operation: The user terminal sends query data to the server and requests the server to process it.

[0422] Step 5:

[0423] The server passes the received query content to the generative AI model, which analyzes the content and generates a corresponding answer.

[0424] Input: Inquiry sent by the user

[0425] Output: The answer generated by the generative AI model

[0426] How it works: The server analyzes the query and passes it as a prompt to the generative AI model, which then generates an appropriate answer to the question.

[0427] Step 6:

[0428] The server passes the generated answer to the emotion engine, which analyzes the user's emotion. The emotion engine generates emotion data and returns it to the server.

[0429] Input: Generated answers and user queries

[0430] Output: User emotion data

[0431] How it works: The server passes the generated answer to the emotion engine, along with the user's query. The emotion engine analyzes the user's emotion and returns the result to the server.

[0432] Step 7:

[0433] The server adjusts the responses from the generative AI model based on the emotional data, for example, making the responses more polite if the user is dissatisfied.

[0434] Input: Emotion data and the generative AI model's answer

[0435] Output: Adjusted answer

[0436] How it works: The server analyzes the emotion data and modifies the generative AI model's answer as needed. An adjusted answer is generated.

[0437] Step 8:

[0438] The server sends the adjusted answer to the user terminal, which displays the answer.

[0439] Input: Adjusted answer

[0440] Output: Display the answer on the user's terminal

[0441] Operation: The server sends the adjusted answer to the user's device, which displays the answer on the screen. The user can view the answer.

[0442] Step 9:

[0443] After receiving the answer, the user inputs feedback into the system, which is then sent to the server by the user terminal.

[0444] Input: User feedback

[0445] Output: Send feedback to the server

[0446] Operation: The user terminal provides a feedback interface, and the user inputs feedback, which is then sent to the server.

[0447] Step 10:

[0448] The server provides the feedback and sentiment data received from the user to the advertiser.

[0449] Input: User feedback and sentiment data

[0450] Output: Providing feedback and sentiment data to advertisers

[0451] How it works: The server compiles user feedback and sentiment data and provides it to advertisers, who can use this data to optimize their advertising strategies.

[0452] Through the above processing steps, the present invention makes it possible to provide appropriate answers to real-time user inquiries and maximize the effectiveness of advertising by utilizing user emotion data.

[0453] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0454] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0455] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0456] [Second embodiment]

[0457] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0458] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0459] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0460] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0461] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0462] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0463] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0464] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0465] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0466] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0467] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0468] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0469] This invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries. The system includes functions for transmitting advertisement data to a user terminal, receiving inquiries about advertisements displayed on the user terminal, providing a generative AI model that analyzes the received inquiries and generates corresponding answers, transmitting the generated answers to the user terminal, and displaying the answers on the user terminal.

[0470] Explanation of program processing

[0471] Sending advertising data

[0472] Server: Sends advertising data over the Internet to advertising spaces on web pages or applications. Advertising data can be in the form of images, text, videos, etc.

[0473] User views ad and initiates inquiry

[0474] Terminal: Displays the received advertising data to the user. Displays an inquiry button while the advertisement is being displayed and sets a click event.

[0475] User: Watch the ad and if there is anything they would like to inquire about, click the inquiry button.

[0476] Enter and submit your inquiry

[0477] Terminal: Detects the click event of the inquiry button, displays an inquiry interface (chat window, etc.), and sends the inquiry content entered by the user to the server.

[0478] User: Type your question or concern into the chat window and click the send button.

[0479] Processing inquiries and generating responses

[0480] Server: Passes the inquiry received from the user to the generative AI model and requests it to analyze and generate an answer.

[0481] Generative AI model: Analyzes the content of the received inquiry and generates the optimal answer. If the question is about smartphone performance, this answer might be, "The battery of this smartphone lasts for 24 hours on average."

[0482] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0483] Show Answers

[0484] Terminal: The received reply is displayed on the user's terminal. The user can immediately check the reply displayed in the chat window.

[0485] User Feedback (Optional)

[0486] User: After receiving the answer, enter feedback on the ad they watched and the answer.

[0487] Terminal: displays the interface for feedback and sends user input to the server.

[0488] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0489] Advertisers: Based on the feedback, they can consider ways to improve their advertising and decide on special offers and discounts for users.

[0490] Specific examples

[0491] Scenario 1: Product Question

[0492] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0493] Scenario 2: Providing feedback

[0494] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0495] This system allows users to make direct inquiries about advertisements and receive answers on the spot in real time. It also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0496] The processing flow will be explained below.

[0497] Step 1:

[0498] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[0499] Step 2:

[0500] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[0501] Step 3:

[0502] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[0503] Step 4:

[0504] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[0505] Step 5:

[0506] User: Enter their question or concern into the inquiry interface and click the submit button.

[0507] Step 6:

[0508] Terminal: Sends the query entered by the user to the server.

[0509] Step 7:

[0510] Server: Receives the user's inquiry and passes it to the generative AI model to analyze and generate an answer.

[0511] Step 8:

[0512] Generative AI model: Analyzes the content of the inquiry and generates the optimal answer, such as "The battery of this smartphone lasts for an average of 24 hours."

[0513] Step 9:

[0514] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0515] Step 10:

[0516] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[0517] Step 11 (Optional):

[0518] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[0519] Step 12 (Optional):

[0520] Terminal: Sends the feedback entered by the user to the server.

[0521] Step 13 (Optional):

[0522] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0523] Step 14 (Optional):

[0524] Advertisers: Based on feedback, they consider ways to improve their advertising and decide on special offers and discounts for users.

[0525] Step 15 (Optional):

[0526] Server: Sends special offers and discount information from advertisers to user terminals.

[0527] Step 16 (Optional):

[0528] Terminal: Displays special offers and discount information to the user.

[0529] Example 1

[0530] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0531] In today's advertising systems, there is a lack of a way for users to ask questions directly within the ad and receive answers in real time. There is also no mechanism for advertisers to quickly collect user feedback to maximize advertising effectiveness. This makes it difficult to resolve user questions and improve advertising effectiveness.

[0532] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0533] In this invention, the server includes a means for transmitting advertisement data to a user terminal, a means for receiving inquiries about an advertisement displayed on the user terminal, a means for analyzing the received inquiries and providing a generative AI model for generating corresponding answers, and a means for receiving feedback from users and providing the feedback to advertisers. This allows users to receive answers to inquiries in advertisements in real time, and enables advertisers to quickly collect user feedback and maximize the effectiveness of their advertisements.

[0534] "Advertising data" refers to digital content, including the content of advertisements, that is sent to and displayed on a user's device. The format may include images, text, video, etc.

[0535] "User terminal" refers to the device on which a user displays and interacts with advertisements. Examples include computers, smartphones, and tablets.

[0536] A "generative AI model" is an artificial intelligence system that analyzes the content of incoming inquiries and generates optimal answers, for example, using natural language processing technology.

[0537] "Feedback" refers to opinions and ratings provided by users regarding advertisements and responses, which allows advertisers to evaluate the effectiveness of their advertisements and obtain information to improve them.

[0538] A "server" is a computer system that manages communication with user terminals, delivers advertising data, receives inquiries, and provides data to generative AI models.

[0539] An "inquiry interface" is a screen or window that allows users to enter inquiries while an ad is being displayed. For example, it could be a chat window.

[0540] A "prompt" is a sentence that helps the generative AI model interpret the query, allowing it to generate a more accurate answer.

[0541] "Advertiser" means a company or individual that distributes advertisements and uses user feedback to evaluate and improve their effectiveness.

[0542] The present invention is a system that allows users to ask questions within advertisements and provides real-time answers to those questions. The system consists of the following main components:

[0543] Overall system configuration

[0544] The system includes a server that transmits advertising data to user terminals, a terminal that displays advertisements and receives inquiries from users, and a generative AI model that processes the inquiries and generates answers.

[0545] Sending advertising data

[0546] The server transmits the advertising data to the user terminal via the Internet. This advertising data is in the form of images, text, video, etc. For example, server software for delivering advertisements (AdServer) is used.

[0547] User views ad and initiates inquiry

[0548] The terminal displays the received advertising data to the user and provides a query interface. The query interface sets a click event and is displayed when the user clicks the query button. The terminal generates this interface using, for example, JavaScript or native application code.

[0549] Enter and submit your inquiry

[0550] When the device detects a click event from the user, it displays an inquiry interface (e.g., a chat window). The user enters a question in the inquiry interface and clicks the send button. The entered inquiry content is sent to the server in JSON format. The device uses a UI library such as React or Vue.js.

[0551] Processing inquiries and generating responses

[0552] The server passes the query received from the user to the generative AI model. The generative AI model analyzes the query, generates a prompt sentence, and generates the optimal answer. An example of a prompt sentence is "Please tell me about the battery life." The generative AI model can use, for example, OpenAI's GPT-4.

[0553] Submitting and viewing answers

[0554] The server receives the answers returned by the generative AI model and sends them to the user's device. The device displays the received answers in a chat window, allowing the user to get answers to their questions in real time. For front-end processing, a JavaScript library is used to dynamically update the DOM.

[0555] Collect user feedback (optional)

[0556] After receiving the answer, the user can enter feedback on the advertisement or the answer. A feedback interface is also displayed on the terminal, and the user's input is sent to the server. The server stores the collected feedback in a database and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve the advertisement.

[0557] Specific examples

[0558] Scenario 1: Product Question

[0559] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0560] Scenario 2: Providing feedback

[0561] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0562] This allows users to make direct inquiries about advertisements and receive answers on the spot in real time, and also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0563] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0564] Step 1:

[0565] The server transmits advertising data to the user terminal. Specifically, the server retrieves advertising data from a database and transmits it to the user terminal via the Internet. This advertising data includes images, text, video, etc. The input is the advertising data in the database, and the output is the transmission of the advertising data to the user terminal.

[0566] Step 2:

[0567] The terminal displays the received advertising data to the user. Furthermore, while the advertisement is being displayed, an inquiry button is displayed on the screen and a click event is set for this button. If the user is interested in the advertisement and clicks the inquiry button, the process proceeds to the next step. The input is advertising data from the server, and the output is the display of the advertisement to the user and the setting of an inquiry button.

[0568] Step 3:

[0569] When a user watches an advertisement and has an inquiry, they click the inquiry button. Clicking this button displays the inquiry interface. The input is the advertisement data on the device and the inquiry button, and the output is the interface displayed on the device.

[0570] Step 4:

[0571] The device detects the user's click event and displays an interface for inquiries (for example, a chat window). The inquiry entered by the user is converted into JSON format and sent to the server. The specific operation here is to capture the click event using JavaScript or native app code, dynamically generate the interface, and send the data to the server. The input is the user's inquiry, and the output is the JSON data sent to the server.

[0572] Step 5:

[0573] The server analyzes the query received from the user. It passes this to the generative AI model and processes the data to generate a prompt and obtain the optimal answer. Specific operations include organizing the data through a normalization process and sending the query to the generative AI model. The input is the user's query, and the output is the prompt to the generative AI model.

[0574] Step 6:

[0575] The generative AI model analyzes the received prompt and generates the optimal answer. For example, if the prompt is "Please tell me about the battery life," the model generates the answer "The battery of this smartphone lasts for 24 hours on average." The input is the prompt from the server, and the output is the generated answer text.

[0576] Step 7:

[0577] The server receives the answer returned by the generative AI model and sends it to the user's device. Specific operations include receiving the answer and transferring the data. The input is the answer from the generative AI model, and the output is sending the answer to the device.

[0578] Step 8:

[0579] The device displays the response received from the server in the chat window. The user can immediately check the response. Specifically, it updates the DOM using a JavaScript library and displays the response. The input is the response data from the server, and the output is the response displayed in the chat window.

[0580] Step 9:

[0581] The user receives the answer and can input feedback on the advertisement or answer they have viewed. The input is the user's feedback comment, and the output is the generation of feedback data.

[0582] Step 10:

[0583] The terminal displays a feedback interface and sends user input to the server. Specific operations include generating a screen for feedback input and sending data. The input is user feedback, and the output is sending feedback to the server.

[0584] Step 11:

[0585] The server stores the feedback from the user in a database and provides it to the advertiser. The input is the user feedback data, and the output is storing it in the database and providing the feedback to the advertiser.

[0586] Step 12:

[0587] Advertisers can use the collected feedback to improve their ads and offer special offers or discounts. Specific operations include analyzing feedback data and formulating improvement plans. The input is feedback data, and the output is improved ads and special offer information.

[0588] (Application example 1)

[0589] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0590] Systems that enable real-time communication between users and advertisers through advertising are limited, and traditional advertising systems lack a means to instantly answer user questions. They also lack a mechanism for efficiently collecting user feedback and providing it to advertisers to help improve their advertising. Since there is insufficient effort to improve engagement through the provision of user rewards, maximizing advertising effectiveness while improving user experience is a challenge.

[0591] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0592] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for transmitting the generated answer to the user terminal, means for displaying the answer on the user terminal, means for collecting user feedback and storing the feedback in a database, means for providing the stored feedback to the advertiser, and means for providing a benefit to the user based on the feedback. This allows users to make inquiries in real time while viewing an advertisement and receive an answer that is generated immediately. Furthermore, advertisers can improve the content of their advertisements based on the feedback, thereby increasing user engagement.

[0593] "Advertising data" refers to advertising information transmitted over the Internet and displayed on a user terminal, and may be in the form of text, images, video, or the like.

[0594] "User terminal" refers to an electronic device used to receive and display advertising data, and generally refers to a smartphone or tablet.

[0595] The "inquiry content" is text information that the user inputs as a question or concern about the advertisement while viewing the advertisement.

[0596] A "generative AI model" is an artificial intelligence model that analyzes the content of received inquiries and generates optimal answers.

[0597] "Feedback" refers to information provided by a user by inputting their evaluation or opinion of an advertisement or its response.

[0598] "Benefits" are rewards, discounts, or other benefits offered by advertisers based on user feedback.

[0599] "Database" means an information system for storing and managing feedback and other data collected from users.

[0600] An "advertiser" is a company or individual that creates advertisements and displays them to users.

[0601] A "server" is a central computing system for providing services to user terminals over the Internet.

[0602] The "means for providing a reward" is a mechanism for generating a reward based on user feedback and transmitting it to the user terminal.

[0603] In this invention, the server, terminal, and user elements work together to respond to inquiries, collect feedback, and provide special benefits in real time. The role of each element will be specifically described below.

[0604] server

[0605] The server sends advertising data to the user's device via the Internet. This includes advertising information such as text, images, and videos. The server also receives inquiries from users, passes them to a generative AI model for analysis, and generates an optimal answer. This answer is then sent to the user's device so that the user can view it immediately. The server also collects feedback from users, stores the data in a database, and provides it to advertisers. It is also responsible for generating rewards based on the feedback and providing them to users.

[0606] Terminal

[0607] The terminal provides an interface for users to view advertisements. Ad data is sent from the server and displayed on the terminal. An inquiry button is displayed while the advertisement is displayed, and when the user clicks it, an inquiry interface opens. When the user enters a question, the content is sent to the server. An interface for displaying the answers sent from the server is also provided, allowing the user to check the answers in real time. A feedback input interface is also provided, and this is sent to the server.

[0608] User

[0609] Users watch an advertisement and click the inquiry button in the advertisement to enter a question. They can check the answer from the server in real time through the device interface. They can also enter and submit feedback on the advertisement and answer. This feedback is collected by the server and provided to the advertiser. If a reward is offered in response to the feedback, the user can receive the reward.

[0610] Specific examples

[0611] As a concrete example, imagine a user watching an advertisement for a new car on their smartphone. The user asks, "What is the fuel efficiency of this car?" The server receives the query and uses a generative AI model to generate an answer: "The average fuel efficiency of this car is 18km / L." This answer is sent to the user's smartphone and displayed immediately. If the user also provides feedback such as "The explanation in the advertisement was easy to understand," the server collects this feedback and provides it to the advertiser. The advertiser can improve the advertisement based on the feedback and send the user a discount coupon as a reward.

[0612] Prompt Sentence Examples

[0613] "I want to build an in-ad query system to ask about the fuel economy of this car. This will include generating answers in real time and displaying them to the user."

[0614] In this way, this invention allows users to make direct inquiries about advertisements and receive answers on the spot in real time. Advertisers can also quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0615] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0616] Processing Steps

[0617] Step 1:

[0618] Server: Transmits advertising data to the user terminal.

[0619] Input: Ad data stored on the server (text, images, videos, etc.).

[0620] Data processing: Converting advertising data into a format appropriate for the user's device.

[0621] Output: Advertising data sent to the user device.

[0622] Specific operation: The server program that distributes advertisements transmits advertisement data to user terminals via the Internet. At this time, it takes into consideration the user's profile information to select and transmit the most suitable advertisement.

[0623] Step 2:

[0624] Terminal: displays advertising data to the user.

[0625] Input: Advertising data sent from the server.

[0626] Data processing: Display advertising data appropriately according to the device's screen size and resolution.

[0627] Output: The advertisement displayed on the user's device display.

[0628] Specific operation: The advertisement display program on the user's device receives the advertisement data and displays the advertisement (text, image, video) on the screen. An inquiry button is also displayed while the advertisement is displayed.

[0629] Step 3:

[0630] User: Click the inquiry button and enter the inquiry details.

[0631] Input: User action after viewing the ad (clicking the inquiry button).

[0632] Data processing: Display an interface for inquiries (such as a chat window).

[0633] Output: The query entered by the user.

[0634] What it does: When a user clicks on the contact button in the ad, a chat window opens where the user can type in their question.

[0635] Step 4:

[0636] Terminal: Sends the query to the server.

[0637] Input: The query entered by the user.

[0638] Data processing: The query content is converted into a format that can be sent to the server.

[0639] Output: The query data sent to the server.

[0640] Specific operation: The query entered by the user is converted into an appropriate format and sent to the server.

[0641] Step 5:

[0642] Server: Passes the query to the generative AI model and generates an answer.

[0643] Input: The inquiry received from the user.

[0644] Data processing: Converting the query content into a format suitable for the generative AI model.

[0645] Output: The answer generated by the generative AI model.

[0646] Specific operation: The server passes the query to a generative AI model (e.g., GPT-3) to generate the optimal answer. The generated answer is then received and passed to the next step.

[0647] Step 6:

[0648] Server: Sends the generated answer to the user terminal.

[0649] Input: The answer generated by the generative AI model.

[0650] Data processing: The response data is converted into a format for sending to the user's terminal.

[0651] Output: Answer data sent to the user's device.

[0652] Specific operation: The server sends the answer data received from the generative AI model to the user's device.

[0653] Step 7:

[0654] Terminal: Display the answer to the user.

[0655] Input: The response data sent from the server.

[0656] Data processing: The response data is displayed appropriately on the user's device screen.

[0657] Output: The answer displayed on the user's device display.

[0658] Specific operation: The generated answer is displayed on the screen of the user's device so that the user can check it immediately.

[0659] Step 8:

[0660] User: Enter feedback on ads and answers.

[0661] Input: User ratings and opinions on ads and answers.

[0662] Data processing: Write feedback into the feedback interface.

[0663] Output: User-entered feedback data.

[0664] Specific operation: The user enters feedback on the advertisement or answer in the feedback interface.

[0665] Step 9:

[0666] Device: Sends feedback to the server.

[0667] Input: Feedback data entered by the user.

[0668] Data processing: The feedback data is converted into a format for sending to the server.

[0669] Output: Feedback data sent to the server.

[0670] Specific behavior: Converts the feedback entered by the user into an appropriate format and sends it to the server.

[0671] Step 10:

[0672] Server: Collects feedback and stores it in a database.

[0673] Input: Feedback data received from the user.

[0674] Data processing: Organizing the feedback data and converting it into a format that can be stored in the database.

[0675] Output: Feedback data stored in a database.

[0676] Specific operation: The server stores the feedback data received from the user in a database and makes it available for reference to advertisers.

[0677] Step 11:

[0678] Server: Generates rewards based on the feedback and sends them to the user.

[0679] Input: Feedback data stored in the database.

[0680] Data processing: Generate rewards based on the feedback and convert them into a format that can be sent to the user's device.

[0681] Output: Reward data sent to user device.

[0682] Specific operation: The server analyzes the feedback in the database, generates a reward (e.g., a discount coupon), and sends it to the user's terminal.

[0683] These steps result in a system that provides real-time answers to user inquiries about advertisements and maximizes advertising effectiveness based on collected feedback.

[0684] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0685] The present invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries, and further combines an emotion engine that recognizes the user's emotions. The system includes the following means.

[0686] First, the server sends the advertising data over the Internet to the advertising space on a web page or application, which can be in the form of images, text, video, etc.

[0687] The terminal then displays the received advertisement data on the user terminal, and the advertisement includes an inquiry button that the user can click to launch an inquiry interface.

[0688] When a user watches an advertisement and clicks on an inquiry button in the advertisement, the terminal displays an inquiry interface, providing an interface for the user to input questions or concerns. When the user inputs the inquiry content and clicks on the send button, the terminal sends the content to the server.

[0689] The server receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model analyzes the user's inquiry and generates a corresponding answer. At the same time, the emotion engine recognizes the user's emotion contained in the inquiry and analyzes the emotion data.

[0690] The server receives the answer from the generative AI model and the recognition data from the emotion engine, and adjusts the answer according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite. The server then sends the adjusted answer to the user's device, where it is displayed to the user.

[0691] The system also includes a feedback function, allowing users to enter feedback after receiving a response. The device sends this feedback to the server, which then collects the feedback and provides it to the advertiser. Furthermore, the emotion engine also collects user emotion data and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve their advertisements and offer special offers or discounts to users.

[0692] Specific examples of program processing

[0693] Scenario 1: Smartphone enquiry

[0694] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from the generative AI model, while simultaneously analyzing the user's emotions using the emotion engine. The generative AI model generates the answer, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[0695] Scenario 2: Providing feedback and sentiment data

[0696] After viewing the advertisement, User B uses the inquiry interface to input feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the emotion engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This emotion data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0697] As a result, the present invention is a system with advanced functionality that allows for the provision of answers to users' inquiries in real time, recognizes the user's emotions, adjusts the answers accordingly, and provides feedback and emotional data to advertisers.

[0698] The processing flow will be explained below.

[0699] Step 1:

[0700] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[0701] Step 2:

[0702] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[0703] Step 3:

[0704] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[0705] Step 4:

[0706] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[0707] Step 5:

[0708] User: Enter their question or concern into the inquiry interface and click the submit button.

[0709] Step 6:

[0710] Terminal: Sends the query entered by the user to the server.

[0711] Step 7:

[0712] Server: Receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model generates a corresponding answer, and the emotion engine analyzes the user's emotions.

[0713] Step 8:

[0714] Generative AI model: Analyzes the query and generates the optimal answer, such as "The battery life of this smartphone is 24 hours on average."

[0715] Step 9:

[0716] Emotion engine: Recognizes the user's emotions from the content of the inquiry. For example, if the user is dissatisfied or suspicious, the emotion engine will recognize that state.

[0717] Step 10:

[0718] Server: Receives the answers from the generative AI model and the recognition data from the emotion engine, and adjusts the answers according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite and detailed.

[0719] Step 11:

[0720] Server: Sends the adjusted answer to the user terminal.

[0721] Step 12:

[0722] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[0723] Step 13 (Optional):

[0724] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[0725] Step 14 (Optional):

[0726] Terminal: Sends the feedback entered by the user to the server.

[0727] Step 15 (Optional):

[0728] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0729] Step 16 (Optional):

[0730] Emotion engine: accumulates user emotional data and provides it to advertisers. For example, the data might say, "The user is satisfied with the information, but has doubts about the price."

[0731] Step 17 (Optional):

[0732] Advertisers: Use feedback and sentiment data to explore ways to improve their ads, including offering new offers or discounts.

[0733] Step 18 (Optional):

[0734] Server: Sends special offers and discount information from advertisers to user terminals.

[0735] Step 19 (Optional):

[0736] Terminal: Displays special offers and discount information to the user. For example, if a 10% discount coupon is offered, this information is visually presented to the user.

[0737] Example 2

[0738] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0739] Conventional advertising systems have the problem that users cannot get real-time answers when they have questions about an ad, resulting in a poor user experience. Furthermore, it is difficult to recognize user emotions and respond appropriately, which means that feedback to advertisers is not properly collected. Furthermore, there are also issues such as delays in implementing advertising improvements based on user emotion data.

[0740] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0741] In this invention, the server includes means for transmitting advertising data to a user terminal, means for displaying the advertising data on the user terminal and providing an inquiry button, means for providing an interface for the user to click the inquiry button and input the inquiry content, means for transmitting the inquiry content from the user terminal to the server, means for passing the received inquiry content to a generation AI model and a sentiment analysis engine for analysis and generating a corresponding answer, means for analyzing the user's emotional data and adjusting the answer, means for transmitting the generated answer to the user terminal, and means for displaying the answer on the user terminal. This allows users to receive answers to questions raised in advertisements in real time and receive the optimal response based on their emotions. Furthermore, advertisers can evaluate and improve the effectiveness of their advertisements based on user feedback and emotional data.

[0742] "Advertising data" refers to information sent to a user terminal in the form of images, text, video, etc.

[0743] "User terminal" refers to an electronic device that allows a user to view advertising data and make inquiries.

[0744] "Contact Button" refers to an interactive element within an advertisement that a user can click to launch a contact interface.

[0745] "Inquiry interface" refers to an interface through which a user can input questions or concerns about an advertisement.

[0746] "Server" refers to a remote computer that sends advertising data, receives inquiries, passes data to generative AI models, and performs sentiment analysis.

[0747] A "generative AI model" refers to artificial intelligence that analyzes the content of a user's inquiry and generates a corresponding answer.

[0748] An "emotion analysis engine" refers to a system that analyzes emotional data from user inquiries and provides that data.

[0749] "User emotion data" refers to information about a user's emotions analyzed by the emotion analysis engine.

[0750] "Adjusted answers" refer to answers generated by a generative AI model and appropriately modified based on the user's emotional data.

[0751] "Feedback" refers to the opinions and thoughts that users enter after receiving a response.

[0752] "Advertiser" refers to an entity that provides advertising data and receives feedback from users.

[0753] The present invention is a system that allows users to make inquiries within advertisements, provides real-time answers to those inquiries, and recognizes the user's emotions. This system is composed of a server, a terminal, a generative AI model, and a sentiment analysis engine.

[0754] First, the server sends advertising data to the user terminal. This advertising data can be in the form of images, text, videos, etc. The server periodically updates the advertising data and sends it to the terminal via the Internet.

[0755] The terminal then displays the advertisement data received from the server to the user. The advertisement includes an inquiry button that the user can click to launch an inquiry interface. The inquiry interface provides a text box for the user to enter their question or inquiry.

[0756] When a user enters the inquiry content and clicks the send button, the device sends the content to the server, which then passes the received inquiry content to the generative AI model and sentiment analysis engine for processing.

[0757] The generative AI model uses natural language processing technology such as GPT-3 to analyze the inquiry and generate an appropriate response. At the same time, the sentiment analysis engine analyzes the user's emotional data contained in the inquiry. The sentiment analysis engine uses sentiment analysis technology such as the Sentiment Analysis API.

[0758] The server receives the answer from the generative AI model and the emotion data from the sentiment analysis engine, and adjusts the answer as needed. For example, if the user expresses dissatisfaction, the answer will be made more polite. The adjusted answer is sent from the server to the device and displayed to the user.

[0759] After receiving the answer, the user can enter feedback. The device will send this feedback to the server, which will then provide the feedback and sentiment data to the advertiser, who can then use it to improve their advertisement and offer rewards or discounts to the user if necessary.

[0760] Below are some examples of specific scenarios and prompts.

[0761] Scenario 1: Smartphone enquiry

[0762] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from a generative AI model (e.g., GPT-3) and simultaneously analyzes the user's emotions using a sentiment analysis engine. The generative AI model generates the answer, "The battery life of this smartphone is, on average, 24 hours," and the sentiment analysis engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[0763] Scenario 2: Providing feedback and sentiment data

[0764] After viewing the advertisement, User B uses the inquiry interface to enter feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the sentiment analysis engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This sentiment data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0765] With these scenarios, the present invention is a system that allows users to make real-time inquiries within advertisements and receive responses tailored to their emotions. It also allows advertisers to optimize the effectiveness of their advertisements based on user feedback and emotion data.

[0766] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0767] Step 1:

[0768] The server sends the advertising data to the user's device. The advertising data can be in the form of images, text, videos, etc., and is sent via the Internet. The input to this process is the latest advertising data stored in the advertising database, and the output is the advertising data to be sent to the user's device. The server formats the advertising data into JSON format and sends it via an HTTP request.

[0769] Step 2:

[0770] The terminal receives the advertising data and displays it to the user. The displayed advertisement includes a query button. The input to this process is the advertising data sent from the server, and the output is an advertisement display that the user can visually confirm. The terminal renders the advertising data in HTML format and displays it in the browser or application. A JavaScript event listener is set on the query button.

[0771] Step 3:

[0772] When a user watches an ad and clicks on the inquiry button, the inquiry interface is launched. The input of this process is the user's click, and the output is the display of the inquiry interface. The user enters a question or concern in a text box. Specifically, JavaScript pops up an inquiry form and allows the user to enter text.

[0773] Step 4:

[0774] The terminal sends the inquiry entered by the user to the server. When the user enters the inquiry and clicks the send button, the terminal sends the content to the server. The input in this process is the user's inquiry, and the output is the inquiry data sent to the server. Specifically, JavaScript retrieves the content of the text box and sends it to the server via an AJAX request.

[0775] Step 5:

[0776] The server receives the query and passes it to the generative AI model and sentiment analysis engine for analysis. The input to this process is the user's query, and the output is the generative AI model's answer and the sentiment analysis engine's emotion data. Specifically, the server formats the query into JSON and sends a request to the generative AI model API and sentiment analysis engine API.

[0777] Step 6:

[0778] The generative AI model analyzes the query content and generates a corresponding answer. At the same time, the sentiment analysis engine analyzes the user's emotions. The input in this process is the query content sent from the server, and the output is the answer and emotional data. Specifically, the generative AI model (e.g., GPT-3) uses NLP technology to perform analysis and generate text. The sentiment analysis engine (e.g., Sentiment Analysis API) performs sentiment analysis.

[0779] Step 7:

[0780] The server receives the generated answer and emotion data, adjusts the answer as needed, and then sends the adjusted answer to the user's device. The input to this process is the answer from the generative AI model and the emotion data from the emotion analysis engine, and the output is the adjusted answer sent to the device. Specifically, the server implements simple logic to make adjustments, such as including additional information if there is dissatisfaction, and then sends it as JSON data to the device.

[0781] Step 8:

[0782] The terminal receives the adjusted answer and displays it to the user. The input to this process is the adjusted answer sent by the server, and the output is the answer displayed to the user. Specifically, the terminal parses the received JSON data and generates HTML to display in the browser or application.

[0783] Step 9:

[0784] After the user checks the answer, they can enter their feedback. The input in this process is the user's feedback, and the output is the feedback data sent to the terminal. Specifically, a feedback text box is displayed below the answer, and the user can enter their feedback.

[0785] Step 10:

[0786] The device sends the user's feedback to the server. The input of this process is the user's feedback, and the output is the feedback data sent to the server. Specifically, JavaScript retrieves the feedback content and sends it to the server via an AJAX request.

[0787] Step 11:

[0788] The server provides feedback and emotion data to the advertiser, who can then use this information to improve their advertisement and offer special offers or discounts to users. The input to this process is the feedback and emotion data, and the output is a report provided to the advertiser. Specifically, the server stores the feedback and emotion data in a database and periodically provides reports to the advertiser.

[0789] (Application example 2)

[0790] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0791] In conventional advertising systems, when a user has doubts or questions about an advertisement, it is difficult to respond quickly, making it difficult to maintain the user's interest. Furthermore, there is no method for maximizing the effectiveness of advertising by providing answers or feedback that take the user's emotions into consideration. Furthermore, there is a lack of means for advertisers to grasp user reactions and emotions in real time and optimize their advertising strategies. The purpose of the present invention is to solve these problems.

[0792] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0793] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for providing an emotion engine that analyzes user emotions, means for adjusting the answer based on the analyzed emotion data, means for transmitting the generated answer to the user terminal, means for receiving feedback from the user and providing the feedback and analyzed emotion data to the advertiser, and means for displaying the answer on the user terminal. This makes it possible to provide an appropriate answer to a user's inquiry about an advertisement in real time, taking the user's emotion into consideration. Furthermore, advertisers can optimize their strategies for maximizing the effectiveness of their advertisements in real time based on the user feedback and emotion data.

[0794] "Advertising data" refers to digital information in the form of images, text, videos, etc. that are displayed on a user terminal.

[0795] A "user terminal" is an electronic device such as a smartphone, tablet, or PC that a user uses to view information.

[0796] The "inquiry interface" is an interface that allows users to input questions or opinions about advertisements.

[0797] A "generative AI model" is an artificial intelligence model that analyzes the content of a user's inquiry and automatically generates a corresponding answer.

[0798] An "emotion engine" is a system or component that analyzes emotions from user input.

[0799] "Feedback" refers to information that a user inputs as an evaluation or opinion of a response or advertisement received from the system.

[0800] "Advertiser" means the person or company responsible for serving advertisements and measuring and optimizing their effectiveness.

[0801] The "means for adjusting responses" is a function that modifies the responses generated by the generative AI model to make them more appropriate based on the emotional data analyzed by the emotion engine.

[0802] An "advertising strategy" is a plan or policy that an advertiser formulates to maximize the effectiveness of advertising based on user feedback and emotional data.

[0803] The present invention provides a system for providing real-time answers to user inquiries while an advertisement is being displayed, and further adjusts the answers by analyzing the user's emotions. Specific embodiments of this system are described below.

[0804] Overall structure

[0805] This system consists of a server, user terminals, and a network. The server is equipped with a generative AI model and an emotion engine, while the user terminal displays advertising data and provides an inquiry interface. The network connects the server and user terminals via the Internet.

[0806] server

[0807] The server has the following features:

[0808] 1. Sending advertising data: Sending advertising data provided by advertisers to user devices. Advertising data can be in the form of images, text, videos, etc.

[0809] 2. Receiving the inquiry: The inquiry is received from the user terminal.

[0810] 3. Answer generation using generative AI models: Analyze the content of the received inquiry and generate an appropriate answer.

[0811] 4. Emotion analysis using an emotion engine: Analyzes emotional data from the user's inquiry and adjusts the response according to the emotion.

[0812] 5. Send answer: Send the adjusted answer to the user device.

[0813] 6. Collecting and providing feedback: Receive feedback from users and provide it to advertisers along with analyzed sentiment data.

[0814] User terminal

[0815] The user terminal has the following features:

[0816] 1. Display advertising data: Display the received advertising data.

[0817] 2. Providing an inquiry interface: Provide an interface for users to make inquiries about advertisements. When users click the inquiry button, an interface will be displayed and they can enter their questions.

[0818] 3. Sending the inquiry: The inquiry entered by the user is sent to the server.

[0819] 4. Display Answer: Display the answer received from the server to the user.

[0820] 5. Feedback collection: Collect the feedback provided by the user and send it to the server.

[0821] Software and hardware used

[0822] Hardware:

[0823] User devices such as smartphones, tablets, and PCs

[0824] Cloud server (e.g. AWS, GCP)

[0825] software:

[0826] Generative AI models (e.g., OpenAI GPT-4)

[0827] Emotion engine (e.g. IBM Watson)

[0828] Server-side frameworks (e.g. Python + Flask)

[0829] Specific examples

[0830] For example, consider the case where User A is watching an advertisement for a new smartphone on their smartphone. They click the inquiry button in the advertisement and enter, "How long is the battery life of this smartphone?" The user's device sends this inquiry to the server. The server uses a generative AI model to generate an answer such as, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that "User A" does not have any specific complaints or doubts. The answer is adjusted and sent as is to the user's device, where it is displayed to the user.

[0831] Example prompt sentence:

[0832] "Your ad campaign provides a system that responds to real-time user inquiries. Please answer the following query. User asks: 'How long does the battery last on this smartphone?'"

[0833] As a result, the present invention is configured to specifically implement a system that enables providing answers to users' inquiries in real time, recognizes users' emotions, adjusts answers accordingly, and provides feedback and emotional data to advertisers.

[0834] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0835] Step 1:

[0836] The server transmits the advertising data provided by the advertiser to the user's terminal via the Internet. The advertising data includes images, text, videos, etc., and is displayed on the user's terminal.

[0837] Input: Advertisement data from advertisers

[0838] Output: Sending advertising data to the user device

[0839] Operation: The server receives data provided by the advertiser and sends it to the user's device via the Internet, which then displays the advertisement.

[0840] Step 2:

[0841] The user terminal displays the received advertisement data, which includes a query button, and the user views the advertisement.

[0842] Input: Advertising data sent from the server

[0843] Output: A display of the ad that the user sees.

[0844] Operation: The user terminal displays the received advertising data on the screen, allowing the user to view the advertisement.

[0845] Step 3:

[0846] The user clicks on the inquiry button in the ad to launch the inquiry interface, enters a question, and clicks the submit button.

[0847] Input: User click actions and entered questions

[0848] Output: Generates query content

[0849] Operation: The user terminal detects the user's click and launches the inquiry interface. When the user enters a question and clicks the submit button, the content is generated as inquiry data.

[0850] Step 4:

[0851] The user terminal transmits the generated inquiry to the server.

[0852] Input: The query entered by the user

[0853] Output: Sends query to server

[0854] Operation: The user terminal sends query data to the server and requests the server to process it.

[0855] Step 5:

[0856] The server passes the received query content to the generative AI model, which analyzes the content and generates a corresponding answer.

[0857] Input: Inquiry sent by the user

[0858] Output: The answer generated by the generative AI model

[0859] How it works: The server analyzes the query and passes it as a prompt to the generative AI model, which then generates an appropriate answer to the question.

[0860] Step 6:

[0861] The server passes the generated answer to the emotion engine, which analyzes the user's emotion. The emotion engine generates emotion data and returns it to the server.

[0862] Input: Generated answers and user queries

[0863] Output: User emotion data

[0864] How it works: The server passes the generated answer to the emotion engine, along with the user's query. The emotion engine analyzes the user's emotion and returns the result to the server.

[0865] Step 7:

[0866] The server adjusts the responses from the generative AI model based on the emotional data, for example, making the responses more polite if the user is dissatisfied.

[0867] Input: Emotion data and the generative AI model's answer

[0868] Output: Adjusted answer

[0869] How it works: The server analyzes the emotion data and modifies the generative AI model's answer as needed. An adjusted answer is generated.

[0870] Step 8:

[0871] The server sends the adjusted answer to the user terminal, which displays the answer.

[0872] Input: Adjusted answer

[0873] Output: Display the answer on the user's terminal

[0874] Operation: The server sends the adjusted answer to the user's device, which displays the answer on the screen. The user can view the answer.

[0875] Step 9:

[0876] After receiving the answer, the user inputs feedback into the system, which is then sent to the server by the user terminal.

[0877] Input: User feedback

[0878] Output: Send feedback to the server

[0879] Operation: The user terminal provides a feedback interface, and the user inputs feedback, which is then sent to the server.

[0880] Step 10:

[0881] The server provides the feedback and sentiment data received from the user to the advertiser.

[0882] Input: User feedback and sentiment data

[0883] Output: Providing feedback and sentiment data to advertisers

[0884] How it works: The server compiles user feedback and sentiment data and provides it to advertisers, who can use this data to optimize their advertising strategies.

[0885] Through the above processing steps, the present invention makes it possible to provide appropriate answers to real-time user inquiries and maximize the effectiveness of advertising by utilizing user emotion data.

[0886] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0887] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0888] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0889] [Third embodiment]

[0890] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0891] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0892] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0893] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0894] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0895] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0896] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0897] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0898] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0899] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0900] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0901] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0902] This invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries. The system includes functions for transmitting advertisement data to a user terminal, receiving inquiries about advertisements displayed on the user terminal, providing a generative AI model that analyzes the received inquiries and generates corresponding answers, transmitting the generated answers to the user terminal, and displaying the answers on the user terminal.

[0903] Explanation of program processing

[0904] Sending advertising data

[0905] Server: Sends advertising data over the Internet to advertising spaces on web pages or applications. Advertising data can be in the form of images, text, videos, etc.

[0906] User views ad and initiates inquiry

[0907] Terminal: Displays the received advertising data to the user. Displays an inquiry button while the advertisement is being displayed and sets a click event.

[0908] User: Watch the ad and if there is anything they would like to inquire about, click the inquiry button.

[0909] Enter and submit your inquiry

[0910] Terminal: Detects the click event of the inquiry button, displays an inquiry interface (chat window, etc.), and sends the inquiry content entered by the user to the server.

[0911] User: Type your question or concern into the chat window and click the send button.

[0912] Processing inquiries and generating responses

[0913] Server: Passes the inquiry received from the user to the generative AI model and requests it to analyze and generate an answer.

[0914] Generative AI model: Analyzes the content of the received inquiry and generates the optimal answer. If the question is about smartphone performance, this answer might be, "The battery of this smartphone lasts for 24 hours on average."

[0915] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0916] Show Answers

[0917] Terminal: The received reply is displayed on the user's terminal. The user can immediately check the reply displayed in the chat window.

[0918] User Feedback (Optional)

[0919] User: After receiving the answer, enter feedback on the ad they watched and the answer.

[0920] Terminal: displays the interface for feedback and sends user input to the server.

[0921] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0922] Advertisers: Based on the feedback, they can consider ways to improve their advertising and decide on special offers and discounts for users.

[0923] Specific examples

[0924] Scenario 1: Product Question

[0925] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0926] Scenario 2: Providing feedback

[0927] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0928] This system allows users to make direct inquiries about advertisements and receive answers on the spot in real time. It also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0929] The processing flow will be explained below.

[0930] Step 1:

[0931] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[0932] Step 2:

[0933] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[0934] Step 3:

[0935] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[0936] Step 4:

[0937] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[0938] Step 5:

[0939] User: Enter their question or concern into the inquiry interface and click the submit button.

[0940] Step 6:

[0941] Terminal: Sends the query entered by the user to the server.

[0942] Step 7:

[0943] Server: Receives the user's inquiry and passes it to the generative AI model to analyze and generate an answer.

[0944] Step 8:

[0945] Generative AI model: Analyzes the content of the inquiry and generates the optimal answer, such as "The battery of this smartphone lasts for an average of 24 hours."

[0946] Step 9:

[0947] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[0948] Step 10:

[0949] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[0950] Step 11 (Optional):

[0951] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[0952] Step 12 (Optional):

[0953] Terminal: Sends the feedback entered by the user to the server.

[0954] Step 13 (Optional):

[0955] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[0956] Step 14 (Optional):

[0957] Advertisers: Based on feedback, they consider ways to improve their advertising and decide on special offers and discounts for users.

[0958] Step 15 (Optional):

[0959] Server: Sends special offers and discount information from advertisers to user terminals.

[0960] Step 16 (Optional):

[0961] Terminal: Displays special offers and discount information to the user.

[0962] Example 1

[0963] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0964] In today's advertising systems, there is a lack of a way for users to ask questions directly within the ad and receive answers in real time. There is also no mechanism for advertisers to quickly collect user feedback to maximize advertising effectiveness. This makes it difficult to resolve user questions and improve advertising effectiveness.

[0965] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0966] In this invention, the server includes a means for transmitting advertisement data to a user terminal, a means for receiving inquiries about an advertisement displayed on the user terminal, a means for analyzing the received inquiries and providing a generative AI model for generating corresponding answers, and a means for receiving feedback from users and providing the feedback to advertisers. This allows users to receive answers to inquiries in advertisements in real time, and enables advertisers to quickly collect user feedback and maximize the effectiveness of their advertisements.

[0967] "Advertising data" refers to digital content, including the content of advertisements, that is sent to and displayed on a user's device. The format may include images, text, video, etc.

[0968] "User terminal" refers to the device on which a user displays and interacts with advertisements. Examples include computers, smartphones, and tablets.

[0969] A "generative AI model" is an artificial intelligence system that analyzes the content of incoming inquiries and generates optimal answers, for example, using natural language processing technology.

[0970] "Feedback" refers to opinions and ratings provided by users regarding advertisements and responses, which allows advertisers to evaluate the effectiveness of their advertisements and obtain information to improve them.

[0971] A "server" is a computer system that manages communication with user terminals, delivers advertising data, receives inquiries, and provides data to generative AI models.

[0972] An "inquiry interface" is a screen or window that allows users to enter inquiries while an ad is being displayed. For example, it could be a chat window.

[0973] A "prompt" is a sentence that helps the generative AI model interpret the query, allowing it to generate a more accurate answer.

[0974] "Advertiser" means a company or individual that distributes advertisements and uses user feedback to evaluate and improve their effectiveness.

[0975] The present invention is a system that allows users to ask questions within advertisements and provides real-time answers to those questions. The system consists of the following main components:

[0976] Overall system configuration

[0977] The system includes a server that transmits advertising data to user terminals, a terminal that displays advertisements and receives inquiries from users, and a generative AI model that processes the inquiries and generates answers.

[0978] Sending advertising data

[0979] The server transmits the advertising data to the user terminal via the Internet. This advertising data is in the form of images, text, video, etc. For example, server software for delivering advertisements (AdServer) is used.

[0980] User views ad and initiates inquiry

[0981] The terminal displays the received advertising data to the user and provides a query interface. The query interface sets a click event and is displayed when the user clicks the query button. The terminal generates this interface using, for example, JavaScript or native application code.

[0982] Enter and submit your inquiry

[0983] When the device detects a click event from the user, it displays an inquiry interface (e.g., a chat window). The user enters a question in the inquiry interface and clicks the send button. The entered inquiry content is sent to the server in JSON format. The device uses a UI library such as React or Vue.js.

[0984] Processing inquiries and generating responses

[0985] The server passes the query received from the user to the generative AI model. The generative AI model analyzes the query, generates a prompt sentence, and generates the optimal answer. An example of a prompt sentence is "Please tell me about the battery life." The generative AI model can use, for example, OpenAI's GPT-4.

[0986] Submitting and viewing answers

[0987] The server receives the answers returned by the generative AI model and sends them to the user's device. The device displays the received answers in a chat window, allowing the user to get answers to their questions in real time. For front-end processing, a JavaScript library is used to dynamically update the DOM.

[0988] Collect user feedback (optional)

[0989] After receiving the answer, the user can enter feedback on the advertisement or the answer. A feedback interface is also displayed on the terminal, and the user's input is sent to the server. The server stores the collected feedback in a database and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve the advertisement.

[0990] Specific examples

[0991] Scenario 1: Product Question

[0992] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[0993] Scenario 2: Providing feedback

[0994] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[0995] This allows users to make direct inquiries about advertisements and receive answers on the spot in real time, and also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[0996] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0997] Step 1:

[0998] The server transmits advertising data to the user terminal. Specifically, the server retrieves advertising data from a database and transmits it to the user terminal via the Internet. This advertising data includes images, text, video, etc. The input is the advertising data in the database, and the output is the transmission of the advertising data to the user terminal.

[0999] Step 2:

[1000] The terminal displays the received advertising data to the user. Furthermore, while the advertisement is being displayed, an inquiry button is displayed on the screen and a click event is set for this button. If the user is interested in the advertisement and clicks the inquiry button, the process proceeds to the next step. The input is advertising data from the server, and the output is the display of the advertisement to the user and the setting of an inquiry button.

[1001] Step 3:

[1002] When a user watches an advertisement and has an inquiry, they click the inquiry button. Clicking this button displays the inquiry interface. The input is the advertisement data on the device and the inquiry button, and the output is the interface displayed on the device.

[1003] Step 4:

[1004] The device detects the user's click event and displays an interface for inquiries (for example, a chat window). The inquiry entered by the user is converted into JSON format and sent to the server. The specific operation here is to capture the click event using JavaScript or native app code, dynamically generate the interface, and send the data to the server. The input is the user's inquiry, and the output is the JSON data sent to the server.

[1005] Step 5:

[1006] The server analyzes the query received from the user. It passes this to the generative AI model and processes the data to generate a prompt and obtain the optimal answer. Specific operations include organizing the data through a normalization process and sending the query to the generative AI model. The input is the user's query, and the output is the prompt to the generative AI model.

[1007] Step 6:

[1008] The generative AI model analyzes the received prompt and generates the optimal answer. For example, if the prompt is "Please tell me about the battery life," the model generates the answer "The battery of this smartphone lasts for 24 hours on average." The input is the prompt from the server, and the output is the generated answer text.

[1009] Step 7:

[1010] The server receives the answer returned by the generative AI model and sends it to the user's device. Specific operations include receiving the answer and transferring the data. The input is the answer from the generative AI model, and the output is sending the answer to the device.

[1011] Step 8:

[1012] The device displays the response received from the server in the chat window. The user can immediately check the response. Specifically, it updates the DOM using a JavaScript library and displays the response. The input is the response data from the server, and the output is the response displayed in the chat window.

[1013] Step 9:

[1014] The user receives the answer and can input feedback on the advertisement or answer they have viewed. The input is the user's feedback comment, and the output is the generation of feedback data.

[1015] Step 10:

[1016] The terminal displays a feedback interface and sends user input to the server. Specific operations include generating a screen for feedback input and sending data. The input is user feedback, and the output is sending feedback to the server.

[1017] Step 11:

[1018] The server stores the feedback from the user in a database and provides it to the advertiser. The input is the user feedback data, and the output is storing it in the database and providing the feedback to the advertiser.

[1019] Step 12:

[1020] Advertisers can use the collected feedback to improve their ads and offer special offers or discounts. Specific operations include analyzing feedback data and formulating improvement plans. The input is feedback data, and the output is improved ads and special offer information.

[1021] (Application example 1)

[1022] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1023] Systems that enable real-time communication between users and advertisers through advertising are limited, and traditional advertising systems lack a means to instantly answer user questions. They also lack a mechanism for efficiently collecting user feedback and providing it to advertisers to help improve their advertising. Since there is insufficient effort to improve engagement through the provision of user rewards, maximizing advertising effectiveness while improving user experience is a challenge.

[1024] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1025] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for transmitting the generated answer to the user terminal, means for displaying the answer on the user terminal, means for collecting user feedback and storing the feedback in a database, means for providing the stored feedback to the advertiser, and means for providing a benefit to the user based on the feedback. This allows users to make inquiries in real time while viewing an advertisement and receive an answer that is generated immediately. Furthermore, advertisers can improve the content of their advertisements based on the feedback, thereby increasing user engagement.

[1026] "Advertising data" refers to advertising information transmitted over the Internet and displayed on a user terminal, and may be in the form of text, images, video, or the like.

[1027] "User terminal" refers to an electronic device used to receive and display advertising data, and generally refers to a smartphone or tablet.

[1028] The "inquiry content" is text information that the user inputs as a question or concern about the advertisement while viewing the advertisement.

[1029] A "generative AI model" is an artificial intelligence model that analyzes the content of received inquiries and generates optimal answers.

[1030] "Feedback" refers to information provided by a user by inputting their evaluation or opinion of an advertisement or its response.

[1031] "Benefits" are rewards, discounts, or other benefits offered by advertisers based on user feedback.

[1032] "Database" means an information system for storing and managing feedback and other data collected from users.

[1033] An "advertiser" is a company or individual that creates advertisements and displays them to users.

[1034] A "server" is a central computing system for providing services to user terminals over the Internet.

[1035] The "means for providing a reward" is a mechanism for generating a reward based on user feedback and transmitting it to the user terminal.

[1036] In this invention, the server, terminal, and user elements work together to respond to inquiries, collect feedback, and provide special benefits in real time. The role of each element will be specifically described below.

[1037] server

[1038] The server sends advertising data to the user's device via the Internet. This includes advertising information such as text, images, and videos. The server also receives inquiries from users, passes them to a generative AI model for analysis, and generates an optimal answer. This answer is then sent to the user's device so that the user can view it immediately. The server also collects feedback from users, stores the data in a database, and provides it to advertisers. It is also responsible for generating rewards based on the feedback and providing them to users.

[1039] Terminal

[1040] The terminal provides an interface for users to view advertisements. Ad data is sent from the server and displayed on the terminal. An inquiry button is displayed while the advertisement is displayed, and when the user clicks it, an inquiry interface opens. When the user enters a question, the content is sent to the server. An interface for displaying the answers sent from the server is also provided, allowing the user to check the answers in real time. A feedback input interface is also provided, and this is sent to the server.

[1041] User

[1042] Users watch an advertisement and click the inquiry button in the advertisement to enter a question. They can check the answer from the server in real time through the device interface. They can also enter and submit feedback on the advertisement and answer. This feedback is collected by the server and provided to the advertiser. If a reward is offered in response to the feedback, the user can receive the reward.

[1043] Specific examples

[1044] As a concrete example, imagine a user watching an advertisement for a new car on their smartphone. The user asks, "What is the fuel efficiency of this car?" The server receives the query and uses a generative AI model to generate an answer: "The average fuel efficiency of this car is 18km / L." This answer is sent to the user's smartphone and displayed immediately. If the user also provides feedback such as "The explanation in the advertisement was easy to understand," the server collects this feedback and provides it to the advertiser. The advertiser can improve the advertisement based on the feedback and send the user a discount coupon as a reward.

[1045] Prompt Sentence Examples

[1046] "I want to build an in-ad query system to ask about the fuel economy of this car. This will include generating answers in real time and displaying them to the user."

[1047] In this way, this invention allows users to make direct inquiries about advertisements and receive answers on the spot in real time. Advertisers can also quickly collect feedback from users and maximize the effectiveness of their advertisements.

[1048] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1049] Processing Steps

[1050] Step 1:

[1051] Server: Transmits advertising data to the user terminal.

[1052] Input: Ad data stored on the server (text, images, videos, etc.).

[1053] Data processing: Converting advertising data into a format appropriate for the user's device.

[1054] Output: Advertising data sent to the user device.

[1055] Specific operation: The server program that distributes advertisements transmits advertisement data to user terminals via the Internet. At this time, it takes into consideration the user's profile information to select and transmit the most suitable advertisement.

[1056] Step 2:

[1057] Terminal: displays advertising data to the user.

[1058] Input: Advertising data sent from the server.

[1059] Data processing: Display advertising data appropriately according to the device's screen size and resolution.

[1060] Output: The advertisement displayed on the user's device display.

[1061] Specific operation: The advertisement display program on the user's device receives the advertisement data and displays the advertisement (text, image, video) on the screen. An inquiry button is also displayed while the advertisement is displayed.

[1062] Step 3:

[1063] User: Click the inquiry button and enter the inquiry details.

[1064] Input: User action after viewing the ad (clicking the inquiry button).

[1065] Data processing: Display an interface for inquiries (such as a chat window).

[1066] Output: The query entered by the user.

[1067] What it does: When a user clicks on the contact button in the ad, a chat window opens where the user can type in their question.

[1068] Step 4:

[1069] Terminal: Sends the query to the server.

[1070] Input: The query entered by the user.

[1071] Data processing: The query content is converted into a format that can be sent to the server.

[1072] Output: The query data sent to the server.

[1073] Specific operation: The query entered by the user is converted into an appropriate format and sent to the server.

[1074] Step 5:

[1075] Server: Passes the query to the generative AI model and generates an answer.

[1076] Input: The inquiry received from the user.

[1077] Data processing: Converting the query content into a format suitable for the generative AI model.

[1078] Output: The answer generated by the generative AI model.

[1079] Specific operation: The server passes the query to a generative AI model (e.g., GPT-3) to generate the optimal answer. The generated answer is then received and passed to the next step.

[1080] Step 6:

[1081] Server: Sends the generated answer to the user terminal.

[1082] Input: The answer generated by the generative AI model.

[1083] Data processing: The response data is converted into a format for sending to the user's terminal.

[1084] Output: Answer data sent to the user's device.

[1085] Specific operation: The server sends the answer data received from the generative AI model to the user's device.

[1086] Step 7:

[1087] Terminal: Display the answer to the user.

[1088] Input: The response data sent from the server.

[1089] Data processing: The response data is displayed appropriately on the user's device screen.

[1090] Output: The answer displayed on the user's device display.

[1091] Specific operation: The generated answer is displayed on the screen of the user's device so that the user can check it immediately.

[1092] Step 8:

[1093] User: Enter feedback on ads and answers.

[1094] Input: User ratings and opinions on ads and answers.

[1095] Data processing: Write feedback into the feedback interface.

[1096] Output: User-entered feedback data.

[1097] Specific operation: The user enters feedback on the advertisement or answer in the feedback interface.

[1098] Step 9:

[1099] Device: Sends feedback to the server.

[1100] Input: Feedback data entered by the user.

[1101] Data processing: The feedback data is converted into a format for sending to the server.

[1102] Output: Feedback data sent to the server.

[1103] Specific behavior: Converts the feedback entered by the user into an appropriate format and sends it to the server.

[1104] Step 10:

[1105] Server: Collects feedback and stores it in a database.

[1106] Input: Feedback data received from the user.

[1107] Data processing: Organizing the feedback data and converting it into a format that can be stored in the database.

[1108] Output: Feedback data stored in a database.

[1109] Specific operation: The server stores the feedback data received from the user in a database and makes it available for reference to advertisers.

[1110] Step 11:

[1111] Server: Generates rewards based on the feedback and sends them to the user.

[1112] Input: Feedback data stored in the database.

[1113] Data processing: Generate rewards based on the feedback and convert them into a format that can be sent to the user's device.

[1114] Output: Reward data sent to user device.

[1115] Specific operation: The server analyzes the feedback in the database, generates a reward (e.g., a discount coupon), and sends it to the user's terminal.

[1116] These steps result in a system that provides real-time answers to user inquiries about advertisements and maximizes advertising effectiveness based on collected feedback.

[1117] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1118] The present invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries, and further combines an emotion engine that recognizes the user's emotions. The system includes the following means.

[1119] First, the server sends the advertising data over the Internet to the advertising space on a web page or application, which can be in the form of images, text, video, etc.

[1120] The terminal then displays the received advertisement data on the user terminal, and the advertisement includes an inquiry button that the user can click to launch an inquiry interface.

[1121] When a user watches an advertisement and clicks on an inquiry button in the advertisement, the terminal displays an inquiry interface, providing an interface for the user to input questions or concerns. When the user inputs the inquiry content and clicks on the send button, the terminal sends the content to the server.

[1122] The server receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model analyzes the user's inquiry and generates a corresponding answer. At the same time, the emotion engine recognizes the user's emotion contained in the inquiry and analyzes the emotion data.

[1123] The server receives the answer from the generative AI model and the recognition data from the emotion engine, and adjusts the answer according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite. The server then sends the adjusted answer to the user's device, where it is displayed to the user.

[1124] The system also includes a feedback function, allowing users to enter feedback after receiving a response. The device sends this feedback to the server, which then collects the feedback and provides it to the advertiser. Furthermore, the emotion engine also collects user emotion data and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve their advertisements and offer special offers or discounts to users.

[1125] Specific examples of program processing

[1126] Scenario 1: Smartphone enquiry

[1127] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from the generative AI model, while simultaneously analyzing the user's emotions using the emotion engine. The generative AI model generates the answer, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[1128] Scenario 2: Providing feedback and sentiment data

[1129] After viewing the advertisement, User B uses the inquiry interface to input feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the emotion engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This emotion data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1130] As a result, the present invention is a system with advanced functionality that allows for the provision of answers to users' inquiries in real time, recognizes the user's emotions, adjusts the answers accordingly, and provides feedback and emotional data to advertisers.

[1131] The processing flow will be explained below.

[1132] Step 1:

[1133] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[1134] Step 2:

[1135] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[1136] Step 3:

[1137] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[1138] Step 4:

[1139] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[1140] Step 5:

[1141] User: Enter their question or concern into the inquiry interface and click the submit button.

[1142] Step 6:

[1143] Terminal: Sends the query entered by the user to the server.

[1144] Step 7:

[1145] Server: Receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model generates a corresponding answer, and the emotion engine analyzes the user's emotions.

[1146] Step 8:

[1147] Generative AI model: Analyzes the query and generates the optimal answer, such as "The battery life of this smartphone is 24 hours on average."

[1148] Step 9:

[1149] Emotion engine: Recognizes the user's emotions from the content of the inquiry. For example, if the user is dissatisfied or suspicious, the emotion engine will recognize that state.

[1150] Step 10:

[1151] Server: Receives the answers from the generative AI model and the recognition data from the emotion engine, and adjusts the answers according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite and detailed.

[1152] Step 11:

[1153] Server: Sends the adjusted answer to the user terminal.

[1154] Step 12:

[1155] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[1156] Step 13 (Optional):

[1157] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[1158] Step 14 (Optional):

[1159] Terminal: Sends the feedback entered by the user to the server.

[1160] Step 15 (Optional):

[1161] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[1162] Step 16 (Optional):

[1163] Emotion engine: accumulates user emotional data and provides it to advertisers. For example, the data might say, "The user is satisfied with the information, but has doubts about the price."

[1164] Step 17 (Optional):

[1165] Advertisers: Use feedback and sentiment data to explore ways to improve their ads, including offering new offers or discounts.

[1166] Step 18 (Optional):

[1167] Server: Sends special offers and discount information from advertisers to user terminals.

[1168] Step 19 (Optional):

[1169] Terminal: Displays special offers and discount information to the user. For example, if a 10% discount coupon is offered, this information is visually presented to the user.

[1170] Example 2

[1171] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1172] Conventional advertising systems have the problem that users cannot get real-time answers when they have questions about an ad, resulting in a poor user experience. Furthermore, it is difficult to recognize user emotions and respond appropriately, which means that feedback to advertisers is not properly collected. Furthermore, there are also issues such as delays in implementing advertising improvements based on user emotion data.

[1173] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1174] In this invention, the server includes means for transmitting advertising data to a user terminal, means for displaying the advertising data on the user terminal and providing an inquiry button, means for providing an interface for the user to click the inquiry button and input the inquiry content, means for transmitting the inquiry content from the user terminal to the server, means for passing the received inquiry content to a generation AI model and a sentiment analysis engine for analysis and generating a corresponding answer, means for analyzing the user's emotional data and adjusting the answer, means for transmitting the generated answer to the user terminal, and means for displaying the answer on the user terminal. This allows users to receive answers to questions raised in advertisements in real time and receive the optimal response based on their emotions. Furthermore, advertisers can evaluate and improve the effectiveness of their advertisements based on user feedback and emotional data.

[1175] "Advertising data" refers to information sent to a user terminal in the form of images, text, video, etc.

[1176] "User terminal" refers to an electronic device that allows a user to view advertising data and make inquiries.

[1177] "Contact Button" refers to an interactive element within an advertisement that a user can click to launch a contact interface.

[1178] "Inquiry interface" refers to an interface through which a user can input questions or concerns about an advertisement.

[1179] "Server" refers to a remote computer that sends advertising data, receives inquiries, passes data to generative AI models, and performs sentiment analysis.

[1180] A "generative AI model" refers to artificial intelligence that analyzes the content of a user's inquiry and generates a corresponding answer.

[1181] An "emotion analysis engine" refers to a system that analyzes emotional data from user inquiries and provides that data.

[1182] "User emotion data" refers to information about a user's emotions analyzed by the emotion analysis engine.

[1183] "Adjusted answers" refer to answers generated by a generative AI model and appropriately modified based on the user's emotional data.

[1184] "Feedback" refers to the opinions and thoughts that users enter after receiving a response.

[1185] "Advertiser" refers to an entity that provides advertising data and receives feedback from users.

[1186] The present invention is a system that allows users to make inquiries within advertisements, provides real-time answers to those inquiries, and recognizes the user's emotions. This system is composed of a server, a terminal, a generative AI model, and a sentiment analysis engine.

[1187] First, the server sends advertising data to the user terminal. This advertising data can be in the form of images, text, videos, etc. The server periodically updates the advertising data and sends it to the terminal via the Internet.

[1188] The terminal then displays the advertisement data received from the server to the user. The advertisement includes an inquiry button that the user can click to launch an inquiry interface. The inquiry interface provides a text box for the user to enter their question or inquiry.

[1189] When a user enters the inquiry content and clicks the send button, the device sends the content to the server, which then passes the received inquiry content to the generative AI model and sentiment analysis engine for processing.

[1190] The generative AI model uses natural language processing technology such as GPT-3 to analyze the inquiry and generate an appropriate response. At the same time, the sentiment analysis engine analyzes the user's emotional data contained in the inquiry. The sentiment analysis engine uses sentiment analysis technology such as the Sentiment Analysis API.

[1191] The server receives the answer from the generative AI model and the emotion data from the sentiment analysis engine, and adjusts the answer as needed. For example, if the user expresses dissatisfaction, the answer will be made more polite. The adjusted answer is sent from the server to the device and displayed to the user.

[1192] After receiving the answer, the user can enter feedback. The device will send this feedback to the server, which will then provide the feedback and sentiment data to the advertiser, who can then use it to improve their advertisement and offer rewards or discounts to the user if necessary.

[1193] Below are some examples of specific scenarios and prompts.

[1194] Scenario 1: Smartphone enquiry

[1195] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from a generative AI model (e.g., GPT-3) and simultaneously analyzes the user's emotions using a sentiment analysis engine. The generative AI model generates the answer, "The battery life of this smartphone is, on average, 24 hours," and the sentiment analysis engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[1196] Scenario 2: Providing feedback and sentiment data

[1197] After viewing the advertisement, User B uses the inquiry interface to enter feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the sentiment analysis engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This sentiment data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1198] With these scenarios, the present invention is a system that allows users to make real-time inquiries within advertisements and receive responses tailored to their emotions. It also allows advertisers to optimize the effectiveness of their advertisements based on user feedback and emotion data.

[1199] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1200] Step 1:

[1201] The server sends the advertising data to the user's device. The advertising data can be in the form of images, text, videos, etc., and is sent via the Internet. The input to this process is the latest advertising data stored in the advertising database, and the output is the advertising data to be sent to the user's device. The server formats the advertising data into JSON format and sends it via an HTTP request.

[1202] Step 2:

[1203] The terminal receives the advertising data and displays it to the user. The displayed advertisement includes a query button. The input to this process is the advertising data sent from the server, and the output is an advertisement display that the user can visually confirm. The terminal renders the advertising data in HTML format and displays it in the browser or application. A JavaScript event listener is set on the query button.

[1204] Step 3:

[1205] When a user watches an ad and clicks on the inquiry button, the inquiry interface is launched. The input of this process is the user's click, and the output is the display of the inquiry interface. The user enters a question or concern in a text box. Specifically, JavaScript pops up an inquiry form and allows the user to enter text.

[1206] Step 4:

[1207] The terminal sends the inquiry entered by the user to the server. When the user enters the inquiry and clicks the send button, the terminal sends the content to the server. The input in this process is the user's inquiry, and the output is the inquiry data sent to the server. Specifically, JavaScript retrieves the content of the text box and sends it to the server via an AJAX request.

[1208] Step 5:

[1209] The server receives the query and passes it to the generative AI model and sentiment analysis engine for analysis. The input to this process is the user's query, and the output is the generative AI model's answer and the sentiment analysis engine's emotion data. Specifically, the server formats the query into JSON and sends a request to the generative AI model API and sentiment analysis engine API.

[1210] Step 6:

[1211] The generative AI model analyzes the query content and generates a corresponding answer. At the same time, the sentiment analysis engine analyzes the user's emotions. The input in this process is the query content sent from the server, and the output is the answer and emotional data. Specifically, the generative AI model (e.g., GPT-3) uses NLP technology to perform analysis and generate text. The sentiment analysis engine (e.g., Sentiment Analysis API) performs sentiment analysis.

[1212] Step 7:

[1213] The server receives the generated answer and emotion data, adjusts the answer as needed, and then sends the adjusted answer to the user's device. The input to this process is the answer from the generative AI model and the emotion data from the emotion analysis engine, and the output is the adjusted answer sent to the device. Specifically, the server implements simple logic to make adjustments, such as including additional information if there is dissatisfaction, and then sends it as JSON data to the device.

[1214] Step 8:

[1215] The terminal receives the adjusted answer and displays it to the user. The input to this process is the adjusted answer sent by the server, and the output is the answer displayed to the user. Specifically, the terminal parses the received JSON data and generates HTML to display in the browser or application.

[1216] Step 9:

[1217] After the user checks the answer, they can enter their feedback. The input in this process is the user's feedback, and the output is the feedback data sent to the terminal. Specifically, a feedback text box is displayed below the answer, and the user can enter their feedback.

[1218] Step 10:

[1219] The device sends the user's feedback to the server. The input of this process is the user's feedback, and the output is the feedback data sent to the server. Specifically, JavaScript retrieves the feedback content and sends it to the server via an AJAX request.

[1220] Step 11:

[1221] The server provides feedback and emotion data to the advertiser, who can then use this information to improve their advertisement and offer special offers or discounts to users. The input to this process is the feedback and emotion data, and the output is a report provided to the advertiser. Specifically, the server stores the feedback and emotion data in a database and periodically provides reports to the advertiser.

[1222] (Application example 2)

[1223] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1224] In conventional advertising systems, when a user has doubts or questions about an advertisement, it is difficult to respond quickly, making it difficult to maintain the user's interest. Furthermore, there is no method for maximizing the effectiveness of advertising by providing answers or feedback that take the user's emotions into consideration. Furthermore, there is a lack of means for advertisers to grasp user reactions and emotions in real time and optimize their advertising strategies. The purpose of the present invention is to solve these problems.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1226] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for providing an emotion engine that analyzes user emotions, means for adjusting the answer based on the analyzed emotion data, means for transmitting the generated answer to the user terminal, means for receiving feedback from the user and providing the feedback and analyzed emotion data to the advertiser, and means for displaying the answer on the user terminal. This makes it possible to provide an appropriate answer to a user's inquiry about an advertisement in real time, taking the user's emotion into consideration. Furthermore, advertisers can optimize their strategies for maximizing the effectiveness of their advertisements in real time based on the user feedback and emotion data.

[1227] "Advertising data" refers to digital information in the form of images, text, videos, etc. that are displayed on a user terminal.

[1228] A "user terminal" is an electronic device such as a smartphone, tablet, or PC that a user uses to view information.

[1229] The "inquiry interface" is an interface that allows users to input questions or opinions about advertisements.

[1230] A "generative AI model" is an artificial intelligence model that analyzes the content of a user's inquiry and automatically generates a corresponding answer.

[1231] An "emotion engine" is a system or component that analyzes emotions from user input.

[1232] "Feedback" refers to information that a user inputs as an evaluation or opinion of a response or advertisement received from the system.

[1233] "Advertiser" means the person or company responsible for serving advertisements and measuring and optimizing their effectiveness.

[1234] The "means for adjusting responses" is a function that modifies the responses generated by the generative AI model to make them more appropriate based on the emotional data analyzed by the emotion engine.

[1235] An "advertising strategy" is a plan or policy that an advertiser formulates to maximize the effectiveness of advertising based on user feedback and emotional data.

[1236] The present invention provides a system for providing real-time answers to user inquiries while an advertisement is being displayed, and further adjusts the answers by analyzing the user's emotions. Specific embodiments of this system are described below.

[1237] Overall structure

[1238] This system consists of a server, user terminals, and a network. The server is equipped with a generative AI model and an emotion engine, while the user terminal displays advertising data and provides an inquiry interface. The network connects the server and user terminals via the Internet.

[1239] server

[1240] The server has the following features:

[1241] 1. Sending advertising data: Sending advertising data provided by advertisers to user devices. Advertising data can be in the form of images, text, videos, etc.

[1242] 2. Receiving the inquiry: The inquiry is received from the user terminal.

[1243] 3. Answer generation using generative AI models: Analyze the content of the received inquiry and generate an appropriate answer.

[1244] 4. Emotion analysis using an emotion engine: Analyzes emotional data from the user's inquiry and adjusts the response according to the emotion.

[1245] 5. Send answer: Send the adjusted answer to the user device.

[1246] 6. Collecting and providing feedback: Receive feedback from users and provide it to advertisers along with analyzed sentiment data.

[1247] User terminal

[1248] The user terminal has the following features:

[1249] 1. Display advertising data: Display the received advertising data.

[1250] 2. Providing an inquiry interface: Provide an interface for users to make inquiries about advertisements. When users click the inquiry button, an interface will be displayed and they can enter their questions.

[1251] 3. Sending the inquiry: The inquiry entered by the user is sent to the server.

[1252] 4. Display Answer: Display the answer received from the server to the user.

[1253] 5. Feedback collection: Collect the feedback provided by the user and send it to the server.

[1254] Software and hardware used

[1255] Hardware:

[1256] User devices such as smartphones, tablets, and PCs

[1257] Cloud server (e.g. AWS, GCP)

[1258] software:

[1259] Generative AI models (e.g., OpenAI GPT-4)

[1260] Emotion engine (e.g. IBM Watson)

[1261] Server-side frameworks (e.g. Python + Flask)

[1262] Specific examples

[1263] For example, consider the case where User A is watching an advertisement for a new smartphone on their smartphone. They click the inquiry button in the advertisement and enter, "How long is the battery life of this smartphone?" The user's device sends this inquiry to the server. The server uses a generative AI model to generate an answer such as, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that "User A" does not have any specific complaints or doubts. The answer is adjusted and sent as is to the user's device, where it is displayed to the user.

[1264] Example prompt sentence:

[1265] "Your ad campaign provides a system that responds to real-time user inquiries. Please answer the following query. User asks: 'How long does the battery last on this smartphone?'"

[1266] As a result, the present invention is configured to specifically implement a system that enables providing answers to users' inquiries in real time, recognizes users' emotions, adjusts answers accordingly, and provides feedback and emotional data to advertisers.

[1267] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1268] Step 1:

[1269] The server transmits the advertising data provided by the advertiser to the user's terminal via the Internet. The advertising data includes images, text, videos, etc., and is displayed on the user's terminal.

[1270] Input: Advertisement data from advertisers

[1271] Output: Sending advertising data to the user device

[1272] Operation: The server receives data provided by the advertiser and sends it to the user's device via the Internet, which then displays the advertisement.

[1273] Step 2:

[1274] The user terminal displays the received advertisement data, which includes a query button, and the user views the advertisement.

[1275] Input: Advertising data sent from the server

[1276] Output: A display of the ad that the user sees.

[1277] Operation: The user terminal displays the received advertising data on the screen, allowing the user to view the advertisement.

[1278] Step 3:

[1279] The user clicks on the inquiry button in the ad to launch the inquiry interface, enters a question, and clicks the submit button.

[1280] Input: User click actions and entered questions

[1281] Output: Generates query content

[1282] Operation: The user terminal detects the user's click and launches the inquiry interface. When the user enters a question and clicks the submit button, the content is generated as inquiry data.

[1283] Step 4:

[1284] The user terminal transmits the generated inquiry to the server.

[1285] Input: The query entered by the user

[1286] Output: Sends query to server

[1287] Operation: The user terminal sends query data to the server and requests the server to process it.

[1288] Step 5:

[1289] The server passes the received query content to the generative AI model, which analyzes the content and generates a corresponding answer.

[1290] Input: Inquiry sent by the user

[1291] Output: The answer generated by the generative AI model

[1292] How it works: The server analyzes the query and passes it as a prompt to the generative AI model, which then generates an appropriate answer to the question.

[1293] Step 6:

[1294] The server passes the generated answer to the emotion engine, which analyzes the user's emotion. The emotion engine generates emotion data and returns it to the server.

[1295] Input: Generated answers and user queries

[1296] Output: User emotion data

[1297] How it works: The server passes the generated answer to the emotion engine, along with the user's query. The emotion engine analyzes the user's emotion and returns the result to the server.

[1298] Step 7:

[1299] The server adjusts the responses from the generative AI model based on the emotional data, for example, making the responses more polite if the user is dissatisfied.

[1300] Input: Emotion data and the generative AI model's answer

[1301] Output: Adjusted answer

[1302] How it works: The server analyzes the emotion data and modifies the generative AI model's answer as needed. An adjusted answer is generated.

[1303] Step 8:

[1304] The server sends the adjusted answer to the user terminal, which displays the answer.

[1305] Input: Adjusted answer

[1306] Output: Display the answer on the user's terminal

[1307] Operation: The server sends the adjusted answer to the user's device, which displays the answer on the screen. The user can view the answer.

[1308] Step 9:

[1309] After receiving the answer, the user inputs feedback into the system, which is then sent to the server by the user terminal.

[1310] Input: User feedback

[1311] Output: Send feedback to the server

[1312] Operation: The user terminal provides a feedback interface, and the user inputs feedback, which is then sent to the server.

[1313] Step 10:

[1314] The server provides the feedback and sentiment data received from the user to the advertiser.

[1315] Input: User feedback and sentiment data

[1316] Output: Providing feedback and sentiment data to advertisers

[1317] How it works: The server compiles user feedback and sentiment data and provides it to advertisers, who can use this data to optimize their advertising strategies.

[1318] Through the above processing steps, the present invention makes it possible to provide appropriate answers to real-time user inquiries and maximize the effectiveness of advertising by utilizing user emotion data.

[1319] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1320] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1321] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1322] [Fourth embodiment]

[1323] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1324] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1325] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1326] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1327] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1328] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1329] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1330] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1331] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1332] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1333] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1334] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1335] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1336] This invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries. The system includes functions for transmitting advertisement data to a user terminal, receiving inquiries about advertisements displayed on the user terminal, providing a generative AI model that analyzes the received inquiries and generates corresponding answers, transmitting the generated answers to the user terminal, and displaying the answers on the user terminal.

[1337] Explanation of program processing

[1338] Sending advertising data

[1339] Server: Sends advertising data over the Internet to advertising spaces on web pages or applications. Advertising data can be in the form of images, text, videos, etc.

[1340] User views ad and initiates inquiry

[1341] Terminal: Displays the received advertising data to the user. Displays an inquiry button while the advertisement is being displayed and sets a click event.

[1342] User: Watch the ad and if there is anything they would like to inquire about, click the inquiry button.

[1343] Enter and submit your inquiry

[1344] Terminal: Detects the click event of the inquiry button, displays an inquiry interface (chat window, etc.), and sends the inquiry content entered by the user to the server.

[1345] User: Type your question or concern into the chat window and click the send button.

[1346] Processing inquiries and generating responses

[1347] Server: Passes the inquiry received from the user to the generative AI model and requests it to analyze and generate an answer.

[1348] Generative AI model: Analyzes the content of the received inquiry and generates the optimal answer. If the question is about smartphone performance, this answer might be, "The battery of this smartphone lasts for 24 hours on average."

[1349] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[1350] Show Answers

[1351] Terminal: The received reply is displayed on the user's terminal. The user can immediately check the reply displayed in the chat window.

[1352] User Feedback (Optional)

[1353] User: After receiving the answer, enter feedback on the ad they watched and the answer.

[1354] Terminal: displays the interface for feedback and sends user input to the server.

[1355] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[1356] Advertisers: Based on the feedback, they can consider ways to improve their advertising and decide on special offers and discounts for users.

[1357] Specific examples

[1358] Scenario 1: Product Question

[1359] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[1360] Scenario 2: Providing feedback

[1361] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1362] This system allows users to make direct inquiries about advertisements and receive answers on the spot in real time. It also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[1363] The processing flow will be explained below.

[1364] Step 1:

[1365] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[1366] Step 2:

[1367] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[1368] Step 3:

[1369] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[1370] Step 4:

[1371] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[1372] Step 5:

[1373] User: Enter their question or concern into the inquiry interface and click the submit button.

[1374] Step 6:

[1375] Terminal: Sends the query entered by the user to the server.

[1376] Step 7:

[1377] Server: Receives the user's inquiry and passes it to the generative AI model to analyze and generate an answer.

[1378] Step 8:

[1379] Generative AI model: Analyzes the content of the inquiry and generates the optimal answer, such as "The battery of this smartphone lasts for an average of 24 hours."

[1380] Step 9:

[1381] Server: Receives the answer returned by the generative AI model and sends it to the user's device.

[1382] Step 10:

[1383] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[1384] Step 11 (Optional):

[1385] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[1386] Step 12 (Optional):

[1387] Terminal: Sends the feedback entered by the user to the server.

[1388] Step 13 (Optional):

[1389] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[1390] Step 14 (Optional):

[1391] Advertisers: Based on feedback, they consider ways to improve their advertising and decide on special offers and discounts for users.

[1392] Step 15 (Optional):

[1393] Server: Sends special offers and discount information from advertisers to user terminals.

[1394] Step 16 (Optional):

[1395] Terminal: Displays special offers and discount information to the user.

[1396] Example 1

[1397] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] In today's advertising systems, there is a lack of a way for users to ask questions directly within the ad and receive answers in real time. There is also no mechanism for advertisers to quickly collect user feedback to maximize advertising effectiveness. This makes it difficult to resolve user questions and improve advertising effectiveness.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1400] In this invention, the server includes a means for transmitting advertisement data to a user terminal, a means for receiving inquiries about an advertisement displayed on the user terminal, a means for analyzing the received inquiries and providing a generative AI model for generating corresponding answers, and a means for receiving feedback from users and providing the feedback to advertisers. This allows users to receive answers to inquiries in advertisements in real time, and enables advertisers to quickly collect user feedback and maximize the effectiveness of their advertisements.

[1401] "Advertising data" refers to digital content, including the content of advertisements, that is sent to and displayed on a user's device. The format may include images, text, video, etc.

[1402] "User terminal" refers to the device on which a user displays and interacts with advertisements. Examples include computers, smartphones, and tablets.

[1403] A "generative AI model" is an artificial intelligence system that analyzes the content of incoming inquiries and generates optimal answers, for example, using natural language processing technology.

[1404] "Feedback" refers to opinions and ratings provided by users regarding advertisements and responses, which allows advertisers to evaluate the effectiveness of their advertisements and obtain information to improve them.

[1405] A "server" is a computer system that manages communication with user terminals, delivers advertising data, receives inquiries, and provides data to generative AI models.

[1406] An "inquiry interface" is a screen or window that allows users to enter inquiries while an ad is being displayed. For example, it could be a chat window.

[1407] A "prompt" is a sentence that helps the generative AI model interpret the query, allowing it to generate a more accurate answer.

[1408] "Advertiser" means a company or individual that distributes advertisements and uses user feedback to evaluate and improve their effectiveness.

[1409] The present invention is a system that allows users to ask questions within advertisements and provides real-time answers to those questions. The system consists of the following main components:

[1410] Overall system configuration

[1411] The system includes a server that transmits advertising data to user terminals, a terminal that displays advertisements and receives inquiries from users, and a generative AI model that processes the inquiries and generates answers.

[1412] Sending advertising data

[1413] The server transmits the advertising data to the user terminal via the Internet. This advertising data is in the form of images, text, video, etc. For example, server software for delivering advertisements (AdServer) is used.

[1414] User views ad and initiates inquiry

[1415] The terminal displays the received advertising data to the user and provides a query interface. The query interface sets a click event and is displayed when the user clicks the query button. The terminal generates this interface using, for example, JavaScript or native application code.

[1416] Enter and submit your inquiry

[1417] When the device detects a click event from the user, it displays an inquiry interface (e.g., a chat window). The user enters a question in the inquiry interface and clicks the send button. The entered inquiry content is sent to the server in JSON format. The device uses a UI library such as React or Vue.js.

[1418] Processing inquiries and generating responses

[1419] The server passes the query received from the user to the generative AI model. The generative AI model analyzes the query, generates a prompt sentence, and generates the optimal answer. An example of a prompt sentence is "Please tell me about the battery life." The generative AI model can use, for example, OpenAI's GPT-4.

[1420] Submitting and viewing answers

[1421] The server receives the answers returned by the generative AI model and sends them to the user's device. The device displays the received answers in a chat window, allowing the user to get answers to their questions in real time. For front-end processing, a JavaScript library is used to dynamically update the DOM.

[1422] Collect user feedback (optional)

[1423] After receiving the answer, the user can enter feedback on the advertisement or the answer. A feedback interface is also displayed on the terminal, and the user's input is sent to the server. The server stores the collected feedback in a database and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve the advertisement.

[1424] Specific examples

[1425] Scenario 1: Product Question

[1426] User A visits a web page and watches an advertisement for a new smartphone. He clicks the inquiry button in the advertisement and enters the question, "How long does the battery last on this smartphone?" The device sends this question to the server, which then requests the generative AI model. The generative AI model generates the optimal answer, "The battery on this smartphone lasts for 24 hours on average," and sends it to User A's device via the server. User A can immediately check the answer.

[1427] Scenario 2: Providing feedback

[1428] After viewing the advertisement, User B uses the inquiry interface to input feedback, saying, "The advertisement's explanation was easy to understand." The device sends this feedback to the server, and the server provides the collected feedback to the advertiser. The advertiser considers improving the advertisement based on this feedback and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1429] This allows users to make direct inquiries about advertisements and receive answers on the spot in real time, and also allows advertisers to quickly collect feedback from users and maximize the effectiveness of their advertisements.

[1430] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1431] Step 1:

[1432] The server transmits advertising data to the user terminal. Specifically, the server retrieves advertising data from a database and transmits it to the user terminal via the Internet. This advertising data includes images, text, video, etc. The input is the advertising data in the database, and the output is the transmission of the advertising data to the user terminal.

[1433] Step 2:

[1434] The terminal displays the received advertising data to the user. Furthermore, while the advertisement is being displayed, an inquiry button is displayed on the screen and a click event is set for this button. If the user is interested in the advertisement and clicks the inquiry button, the process proceeds to the next step. The input is advertising data from the server, and the output is the display of the advertisement to the user and the setting of an inquiry button.

[1435] Step 3:

[1436] When a user watches an advertisement and has an inquiry, they click the inquiry button. Clicking this button displays the inquiry interface. The input is the advertisement data on the device and the inquiry button, and the output is the interface displayed on the device.

[1437] Step 4:

[1438] The device detects the user's click event and displays an interface for inquiries (for example, a chat window). The inquiry entered by the user is converted into JSON format and sent to the server. The specific operation here is to capture the click event using JavaScript or native app code, dynamically generate the interface, and send the data to the server. The input is the user's inquiry, and the output is the JSON data sent to the server.

[1439] Step 5:

[1440] The server analyzes the query received from the user. It passes this to the generative AI model and processes the data to generate a prompt and obtain the optimal answer. Specific operations include organizing the data through a normalization process and sending the query to the generative AI model. The input is the user's query, and the output is the prompt to the generative AI model.

[1441] Step 6:

[1442] The generative AI model analyzes the received prompt and generates the optimal answer. For example, if the prompt is "Please tell me about the battery life," the model generates the answer "The battery of this smartphone lasts for 24 hours on average." The input is the prompt from the server, and the output is the generated answer text.

[1443] Step 7:

[1444] The server receives the answer returned by the generative AI model and sends it to the user's device. Specific operations include receiving the answer and transferring the data. The input is the answer from the generative AI model, and the output is sending the answer to the device.

[1445] Step 8:

[1446] The device displays the response received from the server in the chat window. The user can immediately check the response. Specifically, it updates the DOM using a JavaScript library and displays the response. The input is the response data from the server, and the output is the response displayed in the chat window.

[1447] Step 9:

[1448] The user receives the answer and can input feedback on the advertisement or answer they have viewed. The input is the user's feedback comment, and the output is the generation of feedback data.

[1449] Step 10:

[1450] The terminal displays a feedback interface and sends user input to the server. Specific operations include generating a screen for feedback input and sending data. The input is user feedback, and the output is sending feedback to the server.

[1451] Step 11:

[1452] The server stores the feedback from the user in a database and provides it to the advertiser. The input is the user feedback data, and the output is storing it in the database and providing the feedback to the advertiser.

[1453] Step 12:

[1454] Advertisers can use the collected feedback to improve their ads and offer special offers or discounts. Specific operations include analyzing feedback data and formulating improvement plans. The input is feedback data, and the output is improved ads and special offer information.

[1455] (Application example 1)

[1456] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1457] Systems that enable real-time communication between users and advertisers through advertising are limited, and traditional advertising systems lack a means to instantly answer user questions. They also lack a mechanism for efficiently collecting user feedback and providing it to advertisers to help improve their advertising. Since there is insufficient effort to improve engagement through the provision of user rewards, maximizing advertising effectiveness while improving user experience is a challenge.

[1458] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1459] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for transmitting the generated answer to the user terminal, means for displaying the answer on the user terminal, means for collecting user feedback and storing the feedback in a database, means for providing the stored feedback to the advertiser, and means for providing a benefit to the user based on the feedback. This allows users to make inquiries in real time while viewing an advertisement and receive an answer that is generated immediately. Furthermore, advertisers can improve the content of their advertisements based on the feedback, thereby increasing user engagement.

[1460] "Advertising data" refers to advertising information transmitted over the Internet and displayed on a user terminal, and may be in the form of text, images, video, or the like.

[1461] "User terminal" refers to an electronic device used to receive and display advertising data, and generally refers to a smartphone or tablet.

[1462] The "inquiry content" is text information that the user inputs as a question or concern about the advertisement while viewing the advertisement.

[1463] A "generative AI model" is an artificial intelligence model that analyzes the content of received inquiries and generates optimal answers.

[1464] "Feedback" refers to information provided by a user by inputting their evaluation or opinion of an advertisement or its response.

[1465] "Benefits" are rewards, discounts, or other benefits offered by advertisers based on user feedback.

[1466] "Database" means an information system for storing and managing feedback and other data collected from users.

[1467] An "advertiser" is a company or individual that creates advertisements and displays them to users.

[1468] A "server" is a central computing system for providing services to user terminals over the Internet.

[1469] The "means for providing a reward" is a mechanism for generating a reward based on user feedback and transmitting it to the user terminal.

[1470] In this invention, the server, terminal, and user elements work together to respond to inquiries, collect feedback, and provide special benefits in real time. The role of each element will be specifically described below.

[1471] server

[1472] The server sends advertising data to the user's device via the Internet. This includes advertising information such as text, images, and videos. The server also receives inquiries from users, passes them to a generative AI model for analysis, and generates an optimal answer. This answer is then sent to the user's device so that the user can view it immediately. The server also collects feedback from users, stores the data in a database, and provides it to advertisers. It is also responsible for generating rewards based on the feedback and providing them to users.

[1473] Terminal

[1474] The terminal provides an interface for users to view advertisements. Ad data is sent from the server and displayed on the terminal. An inquiry button is displayed while the advertisement is displayed, and when the user clicks it, an inquiry interface opens. When the user enters a question, the content is sent to the server. An interface for displaying the answers sent from the server is also provided, allowing the user to check the answers in real time. A feedback input interface is also provided, and this is sent to the server.

[1475] User

[1476] Users watch an advertisement and click the inquiry button in the advertisement to enter a question. They can check the answer from the server in real time through the device interface. They can also enter and submit feedback on the advertisement and answer. This feedback is collected by the server and provided to the advertiser. If a reward is offered in response to the feedback, the user can receive the reward.

[1477] Specific examples

[1478] As a concrete example, imagine a user watching an advertisement for a new car on their smartphone. The user asks, "What is the fuel efficiency of this car?" The server receives the query and uses a generative AI model to generate an answer: "The average fuel efficiency of this car is 18km / L." This answer is sent to the user's smartphone and displayed immediately. If the user also provides feedback such as "The explanation in the advertisement was easy to understand," the server collects this feedback and provides it to the advertiser. The advertiser can improve the advertisement based on the feedback and send the user a discount coupon as a reward.

[1479] Prompt Sentence Examples

[1480] "I want to build an in-ad query system to ask about the fuel economy of this car. This will include generating answers in real time and displaying them to the user."

[1481] In this way, this invention allows users to make direct inquiries about advertisements and receive answers on the spot in real time. Advertisers can also quickly collect feedback from users and maximize the effectiveness of their advertisements.

[1482] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1483] Processing Steps

[1484] Step 1:

[1485] Server: Transmits advertising data to the user terminal.

[1486] Input: Ad data stored on the server (text, images, videos, etc.).

[1487] Data processing: Converting advertising data into a format appropriate for the user's device.

[1488] Output: Advertising data sent to the user device.

[1489] Specific operation: The server program that distributes advertisements transmits advertisement data to user terminals via the Internet. At this time, it takes into consideration the user's profile information to select and transmit the most suitable advertisement.

[1490] Step 2:

[1491] Terminal: displays advertising data to the user.

[1492] Input: Advertising data sent from the server.

[1493] Data processing: Display advertising data appropriately according to the device's screen size and resolution.

[1494] Output: The advertisement displayed on the user's device display.

[1495] Specific operation: The advertisement display program on the user's device receives the advertisement data and displays the advertisement (text, image, video) on the screen. An inquiry button is also displayed while the advertisement is displayed.

[1496] Step 3:

[1497] User: Click the inquiry button and enter the inquiry details.

[1498] Input: User action after viewing the ad (clicking the inquiry button).

[1499] Data processing: Display an interface for inquiries (such as a chat window).

[1500] Output: The query entered by the user.

[1501] What it does: When a user clicks on the contact button in the ad, a chat window opens where the user can type in their question.

[1502] Step 4:

[1503] Terminal: Sends the query to the server.

[1504] Input: The query entered by the user.

[1505] Data processing: The query content is converted into a format that can be sent to the server.

[1506] Output: The query data sent to the server.

[1507] Specific operation: The query entered by the user is converted into an appropriate format and sent to the server.

[1508] Step 5:

[1509] Server: Passes the query to the generative AI model and generates an answer.

[1510] Input: The inquiry received from the user.

[1511] Data processing: Converting the query content into a format suitable for the generative AI model.

[1512] Output: The answer generated by the generative AI model.

[1513] Specific operation: The server passes the query to a generative AI model (e.g., GPT-3) to generate the optimal answer. The generated answer is then received and passed to the next step.

[1514] Step 6:

[1515] Server: Sends the generated answer to the user terminal.

[1516] Input: The answer generated by the generative AI model.

[1517] Data processing: The response data is converted into a format for sending to the user's terminal.

[1518] Output: Answer data sent to the user's device.

[1519] Specific operation: The server sends the answer data received from the generative AI model to the user's device.

[1520] Step 7:

[1521] Terminal: Display the answer to the user.

[1522] Input: The response data sent from the server.

[1523] Data processing: The response data is displayed appropriately on the user's device screen.

[1524] Output: The answer displayed on the user's device display.

[1525] Specific operation: The generated answer is displayed on the screen of the user's device so that the user can check it immediately.

[1526] Step 8:

[1527] User: Enter feedback on ads and answers.

[1528] Input: User ratings and opinions on ads and answers.

[1529] Data processing: Write feedback into the feedback interface.

[1530] Output: User-entered feedback data.

[1531] Specific operation: The user enters feedback on the advertisement or answer in the feedback interface.

[1532] Step 9:

[1533] Device: Sends feedback to the server.

[1534] Input: Feedback data entered by the user.

[1535] Data processing: The feedback data is converted into a format for sending to the server.

[1536] Output: Feedback data sent to the server.

[1537] Specific behavior: Converts the feedback entered by the user into an appropriate format and sends it to the server.

[1538] Step 10:

[1539] Server: Collects feedback and stores it in a database.

[1540] Input: Feedback data received from the user.

[1541] Data processing: Organizing the feedback data and converting it into a format that can be stored in the database.

[1542] Output: Feedback data stored in a database.

[1543] Specific operation: The server stores the feedback data received from the user in a database and makes it available for reference to advertisers.

[1544] Step 11:

[1545] Server: Generates rewards based on the feedback and sends them to the user.

[1546] Input: Feedback data stored in the database.

[1547] Data processing: Generate rewards based on the feedback and convert them into a format that can be sent to the user's device.

[1548] Output: Reward data sent to user device.

[1549] Specific operation: The server analyzes the feedback in the database, generates a reward (e.g., a discount coupon), and sends it to the user's terminal.

[1550] These steps result in a system that provides real-time answers to user inquiries about advertisements and maximizes advertising effectiveness based on collected feedback.

[1551] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1552] The present invention is a system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries, and further combines an emotion engine that recognizes the user's emotions. The system includes the following means.

[1553] First, the server sends the advertising data over the Internet to the advertising space on a web page or application, which can be in the form of images, text, video, etc.

[1554] The terminal then displays the received advertisement data on the user terminal, and the advertisement includes an inquiry button that the user can click to launch an inquiry interface.

[1555] When a user watches an advertisement and clicks on an inquiry button in the advertisement, the terminal displays an inquiry interface, providing an interface for the user to input questions or concerns. When the user inputs the inquiry content and clicks on the send button, the terminal sends the content to the server.

[1556] The server receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model analyzes the user's inquiry and generates a corresponding answer. At the same time, the emotion engine recognizes the user's emotion contained in the inquiry and analyzes the emotion data.

[1557] The server receives the answer from the generative AI model and the recognition data from the emotion engine, and adjusts the answer according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite. The server then sends the adjusted answer to the user's device, where it is displayed to the user.

[1558] The system also includes a feedback function, allowing users to enter feedback after receiving a response. The device sends this feedback to the server, which then collects the feedback and provides it to the advertiser. Furthermore, the emotion engine also collects user emotion data and provides it to the advertiser. Based on this information, the advertiser can consider ways to improve their advertisements and offer special offers or discounts to users.

[1559] Specific examples of program processing

[1560] Scenario 1: Smartphone enquiry

[1561] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from the generative AI model, while simultaneously analyzing the user's emotions using the emotion engine. The generative AI model generates the answer, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[1562] Scenario 2: Providing feedback and sentiment data

[1563] After viewing the advertisement, User B uses the inquiry interface to input feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the emotion engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This emotion data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1564] As a result, the present invention is a system with advanced functionality that allows for the provision of answers to users' inquiries in real time, recognizes the user's emotions, adjusts the answers accordingly, and provides feedback and emotional data to advertisers.

[1565] The processing flow will be explained below.

[1566] Step 1:

[1567] Server: Transmits advertising data (images, text, videos, etc.) to advertising spaces on web pages and applications via the Internet.

[1568] Step 2:

[1569] Terminal: The received advertising data is displayed on the user terminal. The advertisement is embedded in the interface and can be viewed by the user.

[1570] Step 3:

[1571] User: Views the displayed advertisement, and if there is something they would like to inquire about, clicks on the inquiry button in the advertisement.

[1572] Step 4:

[1573] Terminal: Detects a click event on the inquiry button and displays an inquiry interface (chat window, etc.).

[1574] Step 5:

[1575] User: Enter their question or concern into the inquiry interface and click the submit button.

[1576] Step 6:

[1577] Terminal: Sends the query entered by the user to the server.

[1578] Step 7:

[1579] Server: Receives the user's inquiry and passes it to the generative AI model and emotion engine for processing. The generative AI model generates a corresponding answer, and the emotion engine analyzes the user's emotions.

[1580] Step 8:

[1581] Generative AI model: Analyzes the query and generates the optimal answer, such as "The battery life of this smartphone is 24 hours on average."

[1582] Step 9:

[1583] Emotion engine: Recognizes the user's emotions from the content of the inquiry. For example, if the user is dissatisfied or suspicious, the emotion engine will recognize that state.

[1584] Step 10:

[1585] Server: Receives the answers from the generative AI model and the recognition data from the emotion engine, and adjusts the answers according to the user's emotions. For example, if the user is dissatisfied, the answer will be more polite and detailed.

[1586] Step 11:

[1587] Server: Sends the adjusted answer to the user terminal.

[1588] Step 12:

[1589] Terminal: The answer received from the server is displayed on the user's terminal. The user checks the answer displayed in the chat window.

[1590] Step 13 (Optional):

[1591] User: After receiving the responses, use the feedback interface to enter your thoughts and opinions.

[1592] Step 14 (Optional):

[1593] Terminal: Sends the feedback entered by the user to the server.

[1594] Step 15 (Optional):

[1595] Server: Collects user feedback, stores it in a database, and provides the collected feedback to advertisers.

[1596] Step 16 (Optional):

[1597] Emotion engine: accumulates user emotional data and provides it to advertisers. For example, the data might say, "The user is satisfied with the information, but has doubts about the price."

[1598] Step 17 (Optional):

[1599] Advertisers: Use feedback and sentiment data to explore ways to improve their ads, including offering new offers or discounts.

[1600] Step 18 (Optional):

[1601] Server: Sends special offers and discount information from advertisers to user terminals.

[1602] Step 19 (Optional):

[1603] Terminal: Displays special offers and discount information to the user. For example, if a 10% discount coupon is offered, this information is visually presented to the user.

[1604] Example 2

[1605] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1606] Conventional advertising systems have the problem that users cannot get real-time answers when they have questions about an ad, resulting in a poor user experience. Furthermore, it is difficult to recognize user emotions and respond appropriately, which means that feedback to advertisers is not properly collected. Furthermore, there are also issues such as delays in implementing advertising improvements based on user emotion data.

[1607] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1608] In this invention, the server includes means for transmitting advertising data to a user terminal, means for displaying the advertising data on the user terminal and providing an inquiry button, means for providing an interface for the user to click the inquiry button and input the inquiry content, means for transmitting the inquiry content from the user terminal to the server, means for passing the received inquiry content to a generation AI model and a sentiment analysis engine for analysis and generating a corresponding answer, means for analyzing the user's emotional data and adjusting the answer, means for transmitting the generated answer to the user terminal, and means for displaying the answer on the user terminal. This allows users to receive answers to questions raised in advertisements in real time and receive the optimal response based on their emotions. Furthermore, advertisers can evaluate and improve the effectiveness of their advertisements based on user feedback and emotional data.

[1609] "Advertising data" refers to information sent to a user terminal in the form of images, text, video, etc.

[1610] "User terminal" refers to an electronic device that allows a user to view advertising data and make inquiries.

[1611] "Contact Button" refers to an interactive element within an advertisement that a user can click to launch a contact interface.

[1612] "Inquiry interface" refers to an interface through which a user can input questions or concerns about an advertisement.

[1613] "Server" refers to a remote computer that sends advertising data, receives inquiries, passes data to generative AI models, and performs sentiment analysis.

[1614] A "generative AI model" refers to artificial intelligence that analyzes the content of a user's inquiry and generates a corresponding answer.

[1615] An "emotion analysis engine" refers to a system that analyzes emotional data from user inquiries and provides that data.

[1616] "User emotion data" refers to information about a user's emotions analyzed by the emotion analysis engine.

[1617] "Adjusted answers" refer to answers generated by a generative AI model and appropriately modified based on the user's emotional data.

[1618] "Feedback" refers to the opinions and thoughts that users enter after receiving a response.

[1619] "Advertiser" refers to an entity that provides advertising data and receives feedback from users.

[1620] The present invention is a system that allows users to make inquiries within advertisements, provides real-time answers to those inquiries, and recognizes the user's emotions. This system is composed of a server, a terminal, a generative AI model, and a sentiment analysis engine.

[1621] First, the server sends advertising data to the user terminal. This advertising data can be in the form of images, text, videos, etc. The server periodically updates the advertising data and sends it to the terminal via the Internet.

[1622] The terminal then displays the advertisement data received from the server to the user. The advertisement includes an inquiry button that the user can click to launch an inquiry interface. The inquiry interface provides a text box for the user to enter their question or inquiry.

[1623] When a user enters the inquiry content and clicks the send button, the device sends the content to the server, which then passes the received inquiry content to the generative AI model and sentiment analysis engine for processing.

[1624] The generative AI model uses natural language processing technology such as GPT-3 to analyze the inquiry and generate an appropriate response. At the same time, the sentiment analysis engine analyzes the user's emotional data contained in the inquiry. The sentiment analysis engine uses sentiment analysis technology such as the Sentiment Analysis API.

[1625] The server receives the answer from the generative AI model and the emotion data from the sentiment analysis engine, and adjusts the answer as needed. For example, if the user expresses dissatisfaction, the answer will be made more polite. The adjusted answer is sent from the server to the device and displayed to the user.

[1626] After receiving the answer, the user can enter feedback. The device will send this feedback to the server, which will then provide the feedback and sentiment data to the advertiser, who can then use it to improve their advertisement and offer rewards or discounts to the user if necessary.

[1627] Below are some examples of specific scenarios and prompts.

[1628] Scenario 1: Smartphone enquiry

[1629] User A is watching an advertisement for a new smartphone on a web page. User A clicks the inquiry button in the advertisement and enters, "How long is the battery life of this smartphone?" The device sends this inquiry to the server. The server requests processing from a generative AI model (e.g., GPT-3) and simultaneously analyzes the user's emotions using a sentiment analysis engine. The generative AI model generates the answer, "The battery life of this smartphone is, on average, 24 hours," and the sentiment analysis engine recognizes that the user has no specific complaints or doubts. The server sends the answer as is to the device, which displays it to User A.

[1630] Scenario 2: Providing feedback and sentiment data

[1631] After viewing the advertisement, User B uses the inquiry interface to enter feedback such as, "The advertisement explanation was easy to understand, but I would like to know more about the price." The device sends this feedback to the server, which collects the feedback and provides it to the advertiser. Furthermore, the sentiment analysis engine analyzes it and determines that User B is satisfied with the information but has doubts about the price. This sentiment data is also provided to the advertiser. Based on this, the advertiser considers ways to improve the advertisement and decides to offer the user a 10% discount coupon as a reward. The server sends this information to the device, and the discount coupon is displayed to User B.

[1632] With these scenarios, the present invention is a system that allows users to make real-time inquiries within advertisements and receive responses tailored to their emotions. It also allows advertisers to optimize the effectiveness of their advertisements based on user feedback and emotion data.

[1633] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1634] Step 1:

[1635] The server sends the advertising data to the user's device. The advertising data can be in the form of images, text, videos, etc., and is sent via the Internet. The input to this process is the latest advertising data stored in the advertising database, and the output is the advertising data to be sent to the user's device. The server formats the advertising data into JSON format and sends it via an HTTP request.

[1636] Step 2:

[1637] The terminal receives the advertising data and displays it to the user. The displayed advertisement includes a query button. The input to this process is the advertising data sent from the server, and the output is an advertisement display that the user can visually confirm. The terminal renders the advertising data in HTML format and displays it in the browser or application. A JavaScript event listener is set on the query button.

[1638] Step 3:

[1639] When a user watches an ad and clicks on the inquiry button, the inquiry interface is launched. The input of this process is the user's click, and the output is the display of the inquiry interface. The user enters a question or concern in a text box. Specifically, JavaScript pops up an inquiry form and allows the user to enter text.

[1640] Step 4:

[1641] The terminal sends the inquiry entered by the user to the server. When the user enters the inquiry and clicks the send button, the terminal sends the content to the server. The input in this process is the user's inquiry, and the output is the inquiry data sent to the server. Specifically, JavaScript retrieves the content of the text box and sends it to the server via an AJAX request.

[1642] Step 5:

[1643] The server receives the query and passes it to the generative AI model and sentiment analysis engine for analysis. The input to this process is the user's query, and the output is the generative AI model's answer and the sentiment analysis engine's emotion data. Specifically, the server formats the query into JSON and sends a request to the generative AI model API and sentiment analysis engine API.

[1644] Step 6:

[1645] The generative AI model analyzes the query content and generates a corresponding answer. At the same time, the sentiment analysis engine analyzes the user's emotions. The input in this process is the query content sent from the server, and the output is the answer and emotional data. Specifically, the generative AI model (e.g., GPT-3) uses NLP technology to perform analysis and generate text. The sentiment analysis engine (e.g., Sentiment Analysis API) performs sentiment analysis.

[1646] Step 7:

[1647] The server receives the generated answer and emotion data, adjusts the answer as needed, and then sends the adjusted answer to the user's device. The input to this process is the answer from the generative AI model and the emotion data from the emotion analysis engine, and the output is the adjusted answer sent to the device. Specifically, the server implements simple logic to make adjustments, such as including additional information if there is dissatisfaction, and then sends it as JSON data to the device.

[1648] Step 8:

[1649] The terminal receives the adjusted answer and displays it to the user. The input to this process is the adjusted answer sent by the server, and the output is the answer displayed to the user. Specifically, the terminal parses the received JSON data and generates HTML to display in the browser or application.

[1650] Step 9:

[1651] After the user checks the answer, they can enter their feedback. The input in this process is the user's feedback, and the output is the feedback data sent to the terminal. Specifically, a feedback text box is displayed below the answer, and the user can enter their feedback.

[1652] Step 10:

[1653] The device sends the user's feedback to the server. The input of this process is the user's feedback, and the output is the feedback data sent to the server. Specifically, JavaScript retrieves the feedback content and sends it to the server via an AJAX request.

[1654] Step 11:

[1655] The server provides feedback and emotion data to the advertiser, who can then use this information to improve their advertisement and offer special offers or discounts to users. The input to this process is the feedback and emotion data, and the output is a report provided to the advertiser. Specifically, the server stores the feedback and emotion data in a database and periodically provides reports to the advertiser.

[1656] (Application example 2)

[1657] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1658] In conventional advertising systems, when a user has doubts or questions about an advertisement, it is difficult to respond quickly, making it difficult to maintain the user's interest. Furthermore, there is no method for maximizing the effectiveness of advertising by providing answers or feedback that take the user's emotions into consideration. Furthermore, there is a lack of means for advertisers to grasp user reactions and emotions in real time and optimize their advertising strategies. The purpose of the present invention is to solve these problems.

[1659] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1660] In this invention, the server includes means for transmitting advertising data to a user terminal, means for receiving inquiries about an advertisement displayed on the user terminal, means for providing a generative AI model that analyzes the received inquiries and generates a corresponding answer, means for providing an emotion engine that analyzes user emotions, means for adjusting the answer based on the analyzed emotion data, means for transmitting the generated answer to the user terminal, means for receiving feedback from the user and providing the feedback and analyzed emotion data to the advertiser, and means for displaying the answer on the user terminal. This makes it possible to provide an appropriate answer to a user's inquiry about an advertisement in real time, taking the user's emotion into consideration. Furthermore, advertisers can optimize their strategies for maximizing the effectiveness of their advertisements in real time based on the user feedback and emotion data.

[1661] "Advertising data" refers to digital information in the form of images, text, videos, etc. that are displayed on a user terminal.

[1662] A "user terminal" is an electronic device such as a smartphone, tablet, or PC that a user uses to view information.

[1663] The "inquiry interface" is an interface that allows users to input questions or opinions about advertisements.

[1664] A "generative AI model" is an artificial intelligence model that analyzes the content of a user's inquiry and automatically generates a corresponding answer.

[1665] An "emotion engine" is a system or component that analyzes emotions from user input.

[1666] "Feedback" refers to information that a user inputs as an evaluation or opinion of a response or advertisement received from the system.

[1667] "Advertiser" means the person or company responsible for serving advertisements and measuring and optimizing their effectiveness.

[1668] The "means for adjusting responses" is a function that modifies the responses generated by the generative AI model to make them more appropriate based on the emotional data analyzed by the emotion engine.

[1669] An "advertising strategy" is a plan or policy that an advertiser formulates to maximize the effectiveness of advertising based on user feedback and emotional data.

[1670] The present invention provides a system for providing real-time answers to user inquiries while an advertisement is being displayed, and further adjusts the answers by analyzing the user's emotions. Specific embodiments of this system are described below.

[1671] Overall structure

[1672] This system consists of a server, user terminals, and a network. The server is equipped with a generative AI model and an emotion engine, while the user terminal displays advertising data and provides an inquiry interface. The network connects the server and user terminals via the Internet.

[1673] server

[1674] The server has the following features:

[1675] 1. Sending advertising data: Sending advertising data provided by advertisers to user devices. Advertising data can be in the form of images, text, videos, etc.

[1676] 2. Receiving the inquiry: The inquiry is received from the user terminal.

[1677] 3. Answer generation using generative AI models: Analyze the content of the received inquiry and generate an appropriate answer.

[1678] 4. Emotion analysis using an emotion engine: Analyzes emotional data from the user's inquiry and adjusts the response according to the emotion.

[1679] 5. Send answer: Send the adjusted answer to the user device.

[1680] 6. Collecting and providing feedback: Receive feedback from users and provide it to advertisers along with analyzed sentiment data.

[1681] User terminal

[1682] The user terminal has the following features:

[1683] 1. Display advertising data: Display the received advertising data.

[1684] 2. Providing an inquiry interface: Provide an interface for users to make inquiries about advertisements. When users click the inquiry button, an interface will be displayed and they can enter their questions.

[1685] 3. Sending the inquiry: The inquiry entered by the user is sent to the server.

[1686] 4. Display Answer: Display the answer received from the server to the user.

[1687] 5. Feedback collection: Collect the feedback provided by the user and send it to the server.

[1688] Software and hardware used

[1689] Hardware:

[1690] User devices such as smartphones, tablets, and PCs

[1691] Cloud server (e.g. AWS, GCP)

[1692] software:

[1693] Generative AI models (e.g., OpenAI GPT-4)

[1694] Emotion engine (e.g. IBM Watson)

[1695] Server-side frameworks (e.g. Python + Flask)

[1696] Specific examples

[1697] For example, consider the case where User A is watching an advertisement for a new smartphone on their smartphone. They click the inquiry button in the advertisement and enter, "How long is the battery life of this smartphone?" The user's device sends this inquiry to the server. The server uses a generative AI model to generate an answer such as, "The battery life of this smartphone is 24 hours on average," and the emotion engine recognizes that "User A" does not have any specific complaints or doubts. The answer is adjusted and sent as is to the user's device, where it is displayed to the user.

[1698] Example prompt sentence:

[1699] "Your ad campaign provides a system that responds to real-time user inquiries. Please answer the following query. User asks: 'How long does the battery last on this smartphone?'"

[1700] As a result, the present invention is configured to specifically implement a system that enables providing answers to users' inquiries in real time, recognizes users' emotions, adjusts answers accordingly, and provides feedback and emotional data to advertisers.

[1701] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1702] Step 1:

[1703] The server transmits the advertising data provided by the advertiser to the user's terminal via the Internet. The advertising data includes images, text, videos, etc., and is displayed on the user's terminal.

[1704] Input: Advertisement data from advertisers

[1705] Output: Sending advertising data to the user device

[1706] Operation: The server receives data provided by the advertiser and sends it to the user's device via the Internet, which then displays the advertisement.

[1707] Step 2:

[1708] The user terminal displays the received advertisement data, which includes a query button, and the user views the advertisement.

[1709] Input: Advertising data sent from the server

[1710] Output: A display of the ad that the user sees.

[1711] Operation: The user terminal displays the received advertising data on the screen, allowing the user to view the advertisement.

[1712] Step 3:

[1713] The user clicks on the inquiry button in the ad to launch the inquiry interface, enters a question, and clicks the submit button.

[1714] Input: User click actions and entered questions

[1715] Output: Generates query content

[1716] Operation: The user terminal detects the user's click and launches the inquiry interface. When the user enters a question and clicks the submit button, the content is generated as inquiry data.

[1717] Step 4:

[1718] The user terminal transmits the generated inquiry to the server.

[1719] Input: The query entered by the user

[1720] Output: Sends query to server

[1721] Operation: The user terminal sends query data to the server and requests the server to process it.

[1722] Step 5:

[1723] The server passes the received query content to the generative AI model, which analyzes the content and generates a corresponding answer.

[1724] Input: Inquiry sent by the user

[1725] Output: The answer generated by the generative AI model

[1726] How it works: The server analyzes the query and passes it as a prompt to the generative AI model, which then generates an appropriate answer to the question.

[1727] Step 6:

[1728] The server passes the generated answer to the emotion engine, which analyzes the user's emotion. The emotion engine generates emotion data and returns it to the server.

[1729] Input: Generated answers and user queries

[1730] Output: User emotion data

[1731] How it works: The server passes the generated answer to the emotion engine, along with the user's query. The emotion engine analyzes the user's emotion and returns the result to the server.

[1732] Step 7:

[1733] The server adjusts the responses from the generative AI model based on the emotional data, for example, making the responses more polite if the user is dissatisfied.

[1734] Input: Emotion data and the generative AI model's answer

[1735] Output: Adjusted answer

[1736] How it works: The server analyzes the emotion data and modifies the generative AI model's answer as needed. An adjusted answer is generated.

[1737] Step 8:

[1738] The server sends the adjusted answer to the user terminal, which displays the answer.

[1739] Input: Adjusted answer

[1740] Output: Display the answer on the user's terminal

[1741] Operation: The server sends the adjusted answer to the user's device, which displays the answer on the screen. The user can view the answer.

[1742] Step 9:

[1743] After receiving the answer, the user inputs feedback into the system, which is then sent to the server by the user terminal.

[1744] Input: User feedback

[1745] Output: Send feedback to the server

[1746] Operation: The user terminal provides a feedback interface, and the user inputs feedback, which is then sent to the server.

[1747] Step 10:

[1748] The server provides the feedback and sentiment data received from the user to the advertiser.

[1749] Input: User feedback and sentiment data

[1750] Output: Providing feedback and sentiment data to advertisers

[1751] How it works: The server compiles user feedback and sentiment data and provides it to advertisers, who can use this data to optimize their advertising strategies.

[1752] Through the above processing steps, the present invention makes it possible to provide appropriate answers to real-time user inquiries and maximize the effectiveness of advertising by utilizing user emotion data.

[1753] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1754] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1755] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1756] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1757] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1758] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1759] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1760] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1761] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1762] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1763] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1764] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1765] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1766] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1767] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1768] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1769] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1770] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1771] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1772] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1773] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1774] The following is further disclosed regarding the above embodiment.

[1775] (Claim 1)

[1776] A system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries,

[1777] means for transmitting advertising data to a user terminal;

[1778] means for receiving inquiries about advertisements displayed on a user terminal;

[1779] a means for analyzing the received inquiry and generating a corresponding response using a generative AI model;

[1780] means for transmitting the generated answer to a user terminal;

[1781] The system includes means for displaying the answers on a user terminal.

[1782] (Claim 2)

[1783] 2. The system according to claim 1, further comprising means for allowing a user to activate a query interface and input query content while the advertisement data is being displayed.

[1784] (Claim 3)

[1785] 10. The system of claim 1, further comprising means for receiving feedback from the user and providing the feedback to the advertiser.

[1786] "Example 1"

[1787] (Claim 1)

[1788] A system for providing real-time answers to user inquiries within advertisements,

[1789] means for transmitting advertising data to a user terminal;

[1790] means for receiving inquiries about advertisements displayed on a user terminal;

[1791] a means for analyzing the received inquiry and generating a corresponding response using a generative AI model;

[1792] means for transmitting the generated answer to a user terminal;

[1793] means for displaying the answer on a user terminal;

[1794] A system including means for receiving feedback from users and providing the feedback to advertisers.

[1795] (Claim 2)

[1796] 2. The system according to claim 1, further comprising means for allowing a user to activate a query interface and input query content while the advertisement data is being displayed.

[1797] (Claim 3)

[1798] 10. The system of claim 1, further comprising means for the generative AI model to use prompt sentences in analyzing query content and generating appropriate responses.

[1799] "Application Example 1"

[1800] (Claim 1)

[1801] A system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries,

[1802] means for transmitting advertising data to a user terminal;

[1803] means for receiving inquiries about advertisements displayed on a user terminal;

[1804] a means for analyzing the received inquiry and generating a corresponding response using a generative AI model;

[1805] means for transmitting the generated answer to a user terminal;

[1806] means for displaying the answer on a user terminal;

[1807] a means for collecting user feedback and storing the feedback in a database;

[1808] a means for providing the stored feedback to the advertiser;

[1809] A system including a means for providing rewards to users based on feedback.

[1810] (Claim 2)

[1811] 2. The system according to claim 1, further comprising means for allowing a user to activate a query interface and input query content while the advertisement data is being displayed.

[1812] (Claim 3)

[1813] 10. The system of claim 1, further comprising means for receiving feedback from the user and providing the feedback to the advertiser.

[1814] "Example 2: Combining Emotion Engines"

[1815] (Claim 1)

[1816] means for transmitting advertising data to a user terminal;

[1817] means for displaying advertisement data on a user terminal and providing an inquiry button;

[1818] means for providing an interface for a user to click an inquiry button and input the inquiry content;

[1819] means for transmitting inquiry contents from a user terminal to a server;

[1820] A means for passing the received inquiry content to a generative AI model and a sentiment analysis engine for analysis and generating a corresponding answer;

[1821] means for analyzing the user's emotional data and adjusting the response;

[1822] means for transmitting the generated answer to a user terminal;

[1823] The system includes means for displaying the answers on a user terminal.

[1824] (Claim 2)

[1825] 2. The system according to claim 1, further comprising means for allowing a user to activate an inquiry interface and input inquiry content while the advertisement data is being displayed.

[1826] (Claim 3)

[1827] 10. The system of claim 1, further comprising means for receiving feedback from the user and providing the feedback to the advertiser.

[1828] "Application example 2 when combining emotion engines"

[1829] (Claim 1)

[1830] means for transmitting advertising data to a user terminal;

[1831] means for receiving inquiries about advertisements displayed on a user terminal;

[1832] a means for analyzing the received inquiry and generating a corresponding response using a generative AI model;

[1833] means for providing an emotion engine for analyzing the emotion of a user;

[1834] means for adjusting responses based on the analyzed emotion data;

[1835] means for transmitting the generated answer to a user terminal;

[1836] The system includes means for displaying the answers on a user terminal.

[1837] (Claim 2)

[1838] 2. The system according to claim 1, further comprising means for allowing a user to activate a query interface and input query content while the advertisement data is being displayed.

[1839] (Claim 3)

[1840] 10. The system of claim 1, further comprising means for receiving feedback from the user and providing the feedback and the analyzed emotional data to the advertiser. [Explanation of symbols]

[1841] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A system that allows users to make inquiries within advertisements and provides real-time answers to those inquiries, means for transmitting advertising data to a user terminal; means for receiving inquiries about advertisements displayed on a user terminal; a means for analyzing the received inquiry and generating a corresponding response using a generative AI model; means for transmitting the generated answer to a user terminal; The system includes means for displaying the answers on a user terminal.

2. 2. The system according to claim 1, further comprising means for allowing a user to activate a query interface and input query content while the advertisement data is being displayed.

3. 10. The system of claim 1, further comprising means for receiving feedback from the user and providing the feedback to the advertiser.

Images