system

Abstract

We provide the system. [Solution] A means for receiving input information from the user and generating and updating the user profile, A generation method for generating various variations of advertisements based on advertising materials, A method for selecting the most suitable advertisements based on user profiles and replacing them during broadcast, A means of collecting viewing data and analyzing that data, A means of providing reports to advertisers and broadcasters based on the analysis results, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional advertisement distribution system, since advertisements do not consider the individual situations and preferences of viewers, the viewing effect of advertisements is reduced, and there is a problem that the viewing experience is impaired by inappropriate content flowing for a specific viewer. Furthermore, it is difficult to balance privacy protection and advertisement personalization, and there are also limitations in the analysis and optimization of advertisement effects for advertisers and broadcasters.

Means for Solving the Problems

[0005] The present invention solves the above-mentioned problems by providing a system that includes means for receiving input information from users and generating and updating user profiles, means for generating various variations of advertisements based on advertising materials, means for selecting the optimal advertisement based on the user profile and replacing it during broadcast, means for collecting and analyzing viewing data, and means for providing reports to advertisers and broadcasters based on the analysis results. Furthermore, by anonymizing user profiles to protect privacy and replacing advertisements in real time using edge computing, the invention provides an advertising experience tailored to individual needs and maximizes its effectiveness.

[0006] A "user profile" is a collection of data that includes a user's basic information, preferences, and advertising preferences.

[0007] "Generation means" refers to a device or program for automatically creating various variations of advertisements from advertising materials.

[0008] "Selection means" refers to a device or program that identifies and selects the most suitable advertisement based on the user profile.

[0009] "Means of replacement" refers to a device or program for replacing an existing advertisement being broadcast with a selected advertisement.

[0010] "Viewing data" refers to information about viewing, such as the duration of time an advertisement was viewed and the user's reaction.

[0011] "Analysis means" refers to a device or program that processes collected viewing data using statistical or machine learning techniques to evaluate the effectiveness of advertising.

[0012] "Means of providing reports" refers to a device or program that provides advertisers and broadcasters with information on the effectiveness and optimization of advertising based on the analysis results.

[0013] Edge computing is a computing model that improves real-time performance by performing data processing on edge devices located away from the network's central hub.

[0014] "Anonymization" is a technique or method that removes or transforms personally identifiable information to protect privacy. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The system of this invention is built to achieve personalized and effective ad delivery. In this system, the user, server, and terminal each play a specific role, optimizing the ads through a series of operations.

[0037] First, users input their ad preferences and basic information through their devices. This information forms a user profile, which is anonymized on the device and then sent to the server. Based on this profile information, the server passes the materials provided by advertisers to a generation system to create a variety of ad variations. This makes it possible to generate ads that are best suited to each user.

[0038] The device receives program data during broadcast and, when the next advertising slot begins, selects the most suitable advertisement in real time from a list received from the server. This process utilizes edge computing, and advertisements are dynamically replaced based on the user's profile. As a result, a seamless experience is provided that does not affect sight or hearing.

[0039] For example, if a user sets their preferences to "hide ads for alcoholic beverages," the device can use this information to display different ads—such as ads for non-alcoholic beverages or other products of interest—in the slots where alcohol-related ads would normally appear. This allows users to enjoy a personalized advertising experience without feeling uncomfortable.

[0040] Furthermore, the device collects data on the user's ad viewing and sends it to a server. This viewing data is processed by an analytical tool to quantify the effectiveness of the ads. The resulting analytical data is generated as a report and provided to advertisers and broadcasters, which helps improve future ad delivery strategies.

[0041] This invention makes it possible to deliver effective advertisements that are valuable to both advertisers and broadcasters while improving the user experience.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] Users access their device and enter their basic personal information and advertising preferences through a registration form. This includes age, gender, and the selection of categories of ads to display.

[0045] Step 2:

[0046] The terminal receives information entered by the user and creates an individual user profile. This profile is anonymized and converted into a form that does not identify the individual before being sent to the server.

[0047] Step 3:

[0048] The server, based on the received user profile, passes the advertising materials provided by the advertiser to the generation system, which then generates multiple variations of the advertisement. During this process, AI considers the matching with the user profile to create the most optimal advertisement.

[0049] Step 4:

[0050] The terminal monitors program data in real time and predicts the timing of the next advertising slot. It receives a list of ad variations provided by the server and prepares for them.

[0051] Step 5:

[0052] As the device approaches an ad space, it utilizes edge computing to select the most suitable ad based on the user profile. This selection process uses AI-powered ad effectiveness prediction and matching algorithms.

[0053] Step 6:

[0054] When an ad slot is reached, the device seamlessly replaces the currently airing program with a selected ad and displays it to the user. Because the ads are tailored to the user's interests and preferences, they provide a personalized experience.

[0055] Step 7:

[0056] The device records the user's behavior while watching advertisements and collects viewing data such as viewing time and whether or not they were skipped. The collected data is anonymized again and sent to the server.

[0057] Step 8:

[0058] The server analyzes the transmitted viewing data using analytical tools to evaluate the effectiveness of the advertisements. The analysis results are compiled into reports for advertisers and broadcasters.

[0059] Step 9:

[0060] The server provides the generated reports to advertisers and broadcasters to help improve future advertising campaigns. This feedback loop allows for continuous optimization of advertising.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Modern advertising delivery systems suffer from insufficient personalization of ads based on individual user interests and preferences, as well as inadequate protection of viewer privacy. Furthermore, real-time data collection and analysis are difficult when accurately measuring the effectiveness of ads and incorporating the results into future advertising strategies. Solving these challenges is essential to improving the user experience and enabling effective ad delivery for both advertisers and media companies.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for receiving and anonymizing user input information and generating and updating user profiles; means for generating various variations of advertisements based on advertising materials, using generation AI technology; and means for selecting and replacing advertisements in real time using edge computing. This enables the provision of personalized advertisements to individual users, protects privacy, and allows for accurate measurement of advertising effectiveness and the formulation of effective advertising delivery strategies.

[0066] A "user profile" is a collection of data managed by a server, which includes user preferences and basic information regarding ad display, and is anonymized.

[0067] "Generative AI technology" is a technology that utilizes artificial intelligence to generate diverse advertising variations based on provided materials.

[0068] Edge computing is a technology that performs data processing on distributed devices and systems, enabling real-time ad selection and replacement.

[0069] "Seamlessly replacing" refers to inserting advertisements naturally without affecting the content being broadcast, in a way that does not cause any visual or auditory discomfort.

[0070] "Advertising materials" refer to the raw data of information and content provided by advertisers, which serve as the basis for advertisements processed by generation AI technology.

[0071] "Viewing data" refers to information about a user's ad viewing, including which ads were displayed and under what circumstances.

[0072] "Analysis results" refer to the measurement results regarding the effectiveness and impact of advertisements obtained after analyzing viewing data, and are provided as reports to advertisers and media companies.

[0073] This invention is a system in which users, terminals, and servers cooperate to personalize and effectively deliver advertisements.

[0074] Users enter their advertising preferences and basic information through the device's user interface. This information is anonymized on the device and sent to the server as a user profile.

[0075] Based on the received user profile, the server uses a generative AI model, such as advanced natural language processing technology, to generate various ad variations from the materials provided by the advertiser. These generated ads are optimized for each user; for example, the prompt could read, "Generate ad variations tailored to the user's preferences. The user does not like alcoholic beverages, so suggest non-alcoholic beverages or products in other areas of interest."

[0076] The device receives program data in real time, refers to an ad list provided by the server, and uses edge computing technology to select the most suitable ad to display in the next ad slot. This process is carried out without the user experiencing any visual or auditory discomfort, providing a seamless experience.

[0077] Furthermore, the device collects user ad viewing data and sends it to a server. This data is analyzed on the server to quantify the effectiveness of the ads, and the results of this analysis are provided to advertisers and media companies to be used to improve future ad delivery strategies.

[0078] In this way, the invention realizes a system that effectively delivers personalized advertisements while protecting user privacy, and at the same time provides meaningful data to advertisers and media companies.

[0079] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0080] Step 1:

[0081] Users input their advertising preferences and basic information through the device's interface. This input data includes product categories of interest and ad types they wish to hide. The device uses this information to generate an anonymized user profile by removing personally identifiable data. The output is the anonymized user profile.

[0082] Step 2:

[0083] The device sends an anonymized user profile to the server. Upon receiving this information, the server uses a generative AI model to process the advertising material provided by the advertiser. It inputs prompts into the generative AI model to generate various variations of the advertisement. In this process, the inputs are the user profile and advertising material, and the output is a list of the generated ad variations.

[0084] Step 3:

[0085] The server sends instructions to the terminal to select the most suitable advertisement for each user from the generated list of ad variations. The terminal receives program data being broadcast in real time, and based on the ad list provided by the server, selects and displays the most suitable advertisement to display in the next ad slot. Here, the input is real-time program data and the ad list, and the output is the selected advertisement.

[0086] Step 4:

[0087] The device collects data about the advertisements the user has viewed. This data includes the time the advertisement was displayed, the type of advertisement, and whether the viewing was completed. The device sends the collected viewing data to the server. The input is viewing status data, and the output is data about viewing completion and its details.

[0088] Step 5:

[0089] The server receives viewing data sent from the terminals and analyzes the effectiveness of the advertisements. The analysis quantifies which advertisements were effective and under what circumstances, and uses this information to improve future advertising strategies. The final output is an analysis report, which is provided to advertisers and media companies.

[0090] (Application Example 1)

[0091] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Traditional advertising delivery systems have struggled to personalize ads based on individual users' interests and preferences, posing a challenge in improving the user experience. Furthermore, there were limited means to accurately track the effectiveness of ad delivery and provide immediate, specific reports to advertisers and broadcasters. In addition, insufficient privacy protection was a cause for concern.

[0093] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0094] In this invention, the server includes means for receiving attribute information from users and generating and updating user models, means for generating various forms of advertisements based on advertising materials, and means for selecting the most suitable advertisement based on the user model and dynamically switching them during broadcast. This makes it possible to deliver advertisements optimized to the user's interests and preferences.

[0095] A "user model" is a data structure that represents the preferences and interests of individual users based on attribute information collected from them.

[0096] "Generation means" refers to a system or process for creating advertisements in various forms based on advertising materials.

[0097] "Advertiser" refers to an organization or individual that distributes advertisements.

[0098] A "prompt sentence" is a sentence used to instruct a generative AI model and generate advertisements that meet specific conditions or objectives.

[0099] A "generative AI model" is a type of artificial intelligence used to generate advertisements based on user interests.

[0100] Edge computing is a technology that performs data processing closer to the user than in a centralized data center.

[0101] The embodiments for carrying out the present invention will be described. A system is constructed in which a server, a terminal, and a user work together to optimize ad delivery.

[0102] First, users use their smartphones to input their basic information and advertising preferences. This information is anonymized by the device and sent to the server while protecting privacy. The server generates a user model based on the received information and updates it constantly. The user model is a data structure that reflects the user's preferences and interests.

[0103] Next, the server uses a generation mechanism to create various forms of advertisements from the materials provided by the advertisers. The generation mechanism uses software such as Python and TENSORFLOW® to generate prompt text from the advertisement materials. Instructions are given to the generation AI model via the prompt text to generate advertisements best suited to the user's interests. An example of a prompt text might be, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0104] Furthermore, the device utilizes edge computing technology to refresh advertisements in real time, taking into account historical information and user models. Technologies such as Google Cloud Platform can be used. This provides a seamless advertising experience for each user.

[0105] Viewing information is periodically sent from the user's device to the server. The server analyzes this information to quantify the user's ad viewing trends and ad effectiveness. The analysis results are provided to advertisers and broadcasters and used to improve future ad delivery strategies.

[0106] With the above configuration, the advertising distribution system of this invention enables advertising distribution that provides an optimized advertising experience for users while also being valuable for advertisers and broadcasters.

[0107] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0108] Step 1:

[0109] Users enter basic information and advertising preferences using their smartphones. The entered information is anonymized on the device for data protection. The input here consists of user attribute information, and the output is an anonymized user model.

[0110] Step 2:

[0111] The terminal sends an anonymized user model to the server. The server generates and updates the user model based on the received data. This user model is a data structure that represents the user's preferences and interests. The input is the anonymized user model, and the output is the updated user model.

[0112] Step 3:

[0113] The server uses an AI model to generate various types of advertisements based on the advertising materials received from advertisers. Here, the prompt text generated by the generation method is used as input to the AI ​​model, and the output is an advertisement tailored to the user. A specific example of this operation is a prompt text such as, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0114] Step 4:

[0115] The server sends generated advertisements to the device, and the device uses edge computing to select and replace the advertisements in real time. The input is advertisement data from the server, and the output is optimized advertisements displayed to the user. Specifically, the advertisements are selected based on a user model.

[0116] Step 5:

[0117] The device collects user ad viewing information and sends it to the server. The server analyzes the viewing information and generates data on ad effectiveness. The input is viewing information, and the output is the analysis results. The analysis results help improve ad delivery strategies.

[0118] Step 6:

[0119] The server provides the analysis results as a report to advertisers and broadcasters. The input is the analysis results of the advertising effectiveness, and the output is the report. This report is used to formulate measures for improving advertising in the future.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention aims to provide users with a more personalized advertising experience by combining an emotion engine with an advertising delivery system. This system consists of the user, server, terminal, and emotion engine components.

[0122] First, users input basic information and advertising preferences through their devices, which generates a user profile. This profile is anonymized on the device and sent to the server. In addition, an emotion engine is installed on the device to monitor the user's emotions in real time while they are watching. The emotion engine analyzes the user's emotions from facial expressions, tone of voice, eye gaze, etc., and can continuously acquire this data.

[0123] The server combines user profiles and sentiment data to generate a variety of advertisements based on materials provided by advertisers. In this process, the generation mechanism incorporates feedback from sentiment analysis to select the advertisement best suited to the user's current emotional state. This ensures that advertisements are not only based on personal data but also take into account emotional relevance.

[0124] On the device, advertisements optimized for the user are selected using the spaces between lines of the currently airing program. The selected advertisements are then replaced in real time during the broadcast and presented to the user. Edge computing minimizes latency in this process, providing a seamless advertising experience.

[0125] As a concrete example of the emotion engine, when a user displays a sad expression, ads that do not overly stimulate emotions (e.g., product ads with calming music) are selected. Similarly, when a user is smiling, ads with lively and cheerful images are presented. In this way, by optimizing ads according to emotions, a more effective and user-friendly advertising experience is achieved.

[0126] Furthermore, the device collects viewing data, including changes in the user's emotions while watching advertisements, anonymizes it again, and sends it to the server. The server analyzes this emotional data and uses it to evaluate the effectiveness of the advertisements. The analysis results are provided to advertisers and broadcasters as reports and used for future advertising campaigns.

[0127] This invention enables ad delivery based on the user's emotional state, providing a more personalized advertising experience tailored to individual emotions.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] Users log in to their devices and enter their personal information and basic preferences regarding advertising. This creates a user profile, which is then anonymized on the device.

[0131] Step 2:

[0132] The device sends an anonymized user profile to the server and simultaneously activates an emotion engine to monitor the user's facial expressions, voice, gaze, etc., and retrieves emotion data for each request.

[0133] Step 3:

[0134] Based on user profiles and sentiment data received from the terminal, the server uses materials provided by the advertiser to create multiple ad variations via a generation mechanism. In this process, the generation mechanism takes sentiment data into account and generates ads that are appropriate for the emotional state.

[0135] Step 4:

[0136] The server stores the generated ad variations and prepares ad lists that match the user's profile.

[0137] Step 5:

[0138] The terminal monitors the currently airing program and predicts the next advertising slot. As the advertising slot approaches, it selects the most suitable advertisement from the list of advertisements received from the server.

[0139] Step 6:

[0140] When an ad slot becomes available, the device checks emotional data obtained in real time from the emotion engine and makes a final confirmation that the selected ad matches that emotion.

[0141] Step 7:

[0142] When a matching ad is identified, the device seamlessly replaces the ad in the program in real time and displays it to the user. This provides a personalized advertising experience that responds to the user's emotions.

[0143] Step 8:

[0144] The device also collects data on the user's emotional changes while watching advertisements as viewing data, anonymizes this data, and sends it to the server.

[0145] Step 9:

[0146] The server analyzes the collected viewing and sentiment data to evaluate the effectiveness of the advertisements. The results are provided to advertisers and broadcasters as advertising effectiveness reports and used to optimize future campaigns.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] In modern advertising systems, it is difficult to provide more appropriate and emotionally sensitive advertisements to individual users while ensuring the security of data, including users' personal information. Furthermore, there is a need to accurately evaluate the effectiveness of advertisements and use that information to improve future campaigns. To address these challenges, a new system is needed that combines the protection of personal information with real-time sentiment analysis.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for receiving input information from the user and generating and updating a personal profile; means for generating various variations of advertisements using generative artificial intelligence by combining the personal profile with collected emotional data; and means for selecting the most suitable advertisement based on emotional analysis and replacing it within the video content. This makes it possible to deliver advertisements that respond to the user's emotions, achieving both privacy protection and effective advertisement delivery.

[0152] A "personal profile" is an anonymized dataset of basic information and preferences collected from users, and is used to personalize advertisements.

[0153] "Generative artificial intelligence" is an algorithm that generates new content and ideas based on input data, and is a technology that creates diverse variations of advertisements.

[0154] "Emotional data" refers to information obtained by measuring and analyzing the emotional state of a user from their facial expressions, tone of voice, gaze, etc., and is used for selecting advertisements and measuring their effectiveness.

[0155] "Ad generation methods" refer to the processes and technologies used to generate advertisements that are best suited to the user, based on collected personal profiles and sentiment data.

[0156] "Anonymization" is a data processing technique that protects privacy by removing or transforming information that can identify an individual.

[0157] This invention aims to provide an advertising experience that takes user emotions into consideration in an advertising delivery system. It mainly consists of a user, a terminal, a server, and an emotion analysis device.

[0158] First, users input basic information and advertising preferences through their devices. The device then generates a personal profile internally and sends this information to a server using anonymization technology. This anonymization technology utilizes methods that remove personally identifiable information and aggregate the data.

[0159] The device is equipped with an emotion analysis device that monitors the user's facial expressions, voice tone, and gaze in real time. This analysis device continuously acquires the user's emotional data and stores the analysis results. This emotional data is then analyzed using facial recognition software and voice analysis algorithms.

[0160] The server uses generative artificial intelligence based on this data to generate a variety of advertisements using material data provided by advertisers. The generated advertisements are then selected based on the user's personal profile and emotional data to determine the most appropriate one. In this process, the generative AI model optimizes itself by using prompts such as, "Choose a product that matches the user's current emotional state."

[0161] The selected advertisements are played on the device. By using edge computing, advertisements are replaced in real time with the content being broadcast, maintaining a smooth viewing experience. For example, if the user makes a sad face, an advertisement for a product with calming music will be displayed, showing advertisements that are synchronized with the user's emotions.

[0162] Furthermore, the device also collects changes in the user's emotions while viewing advertisements, anonymizes this data again, and sends it to the server. The server uses this emotional data to evaluate the effectiveness of the advertisements and reports the results to the advertisers and telecommunications companies.

[0163] An example of a prompt message is: "We want to monitor the emotional expressions the user is showing towards their device and select ads that match those emotions. Please use an emotion analysis algorithm to suggest how to generate appropriate ads."

[0164] In this way, by using this system, it is possible to provide advertising experiences tailored to the emotions of individual users and to enhance advertising effectiveness.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] Users input basic information and advertising preferences through their devices. This information is used to build a personal profile of the user. The data obtained as input includes name, age, and interest categories, and the device generates the personal profile based on this information. The generated profile is converted into a format that does not identify the individual using an anonymization algorithm and sent to the server as output.

[0168] Step 2:

[0169] The terminal uses an emotion analysis device to monitor the user's facial expressions and voice tone in real time. Inputs include video and audio data from the camera and microphone. These are processed by facial recognition software and voice analysis algorithms to analyze the user's emotional state. The output is the user's current emotional data, which is then shared within the system.

[0170] Step 3:

[0171] The server generates ad variations using generative artificial intelligence (generative AI model) based on personal profiles and sentiment data. Input includes material data provided by advertisers, anonymized personal profiles, and sentiment data. The AI ​​model analyzes this data and selects the most suitable ad variation. This ad is then adjusted to match the user's emotional state. The selected ad data is generated as output and sent to the device.

[0172] Step 4:

[0173] The device receives advertising data sent from the server and displays it in real time between video content. The input is advertising data from the server. By using edge computing technology, advertisements are played appropriately without delay. As output, personalized advertisements are seamlessly presented to the user.

[0174] Step 5:

[0175] While the user is watching an ad, the device again collects emotional data. The input is changes in facial expressions and voice during viewing. These changes are recorded to measure the effectiveness of the ad. The output is emotional change data generated during ad viewing, which is later anonymized again and sent to the server.

[0176] Step 6:

[0177] The server evaluates the effectiveness of advertisements by analyzing viewing and emotional data. The input is emotional change data during ad viewing. The analysis provides insights into the performance of the advertisements. The output is the measurement of the advertisement's effectiveness, which is provided as a report to advertisers and telecommunications companies. These results are then used to inform future advertising campaigns.

[0178] (Application Example 2)

[0179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0180] In modern advertising, a challenge exists in providing advertising experiences tailored to individual user emotions and preferences. Traditional systems are limited to ad delivery based on user profile data, making it difficult to display ads that take real-time emotional states into account. As a result, advertising effectiveness is not fully realized, and there is a need to optimize the user experience.

[0181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0182] In this invention, the server includes means for receiving input information from the user and generating and updating a user profile; means for generating advertisements with various expressions based on diverse media materials; means for analyzing the user's emotional state, selecting the most appropriate advertisement based on the user profile and emotional data, and displaying it on a visual device; means for collecting emotional data and viewing data and analyzing that data; and means for providing reports to advertisers and information distributors based on the analysis results. This makes it possible to optimize advertisements in real time according to the user's emotions and provide a more personalized advertising experience.

[0183] A "user profile" is a set of data in a database that is generated based on a user's basic information and preferences regarding advertisements.

[0184] "Media materials" refer to digital content such as video, audio, and images used as advertising materials.

[0185] "Emotional state" refers to information that indicates the user's psychological and emotional condition, analyzed from the user's facial expressions, tone of voice, gaze, etc.

[0186] A "visual device" is a device that allows a user to receive information visually, and includes smart glasses and mobile device displays.

[0187] "Emotional data" refers to data obtained by analyzing a user's emotional state, and is used to optimize advertising.

[0188] An "advertiser" refers to a company or organization that places advertisements to promote the sale of its products or services.

[0189] An "information provider" refers to a service provider that offers various information content to users.

[0190] This invention utilizes a user's smart glasses and other visual devices, a server, and edge computing software responsible for computational processing. The following describes in detail how each element functions.

[0191] The server receives input from users, generates a user profile based on this information, and continuously updates it. This profile reflects the user's personal interests and responses to advertisements. The server also incorporates diverse media materials and generates advertisements optimized for the user's characteristics.

[0192] The device, i.e., the user's visual device such as smart glasses, analyzes the user's emotional state in real time through cameras and sensors. Emotional analysis is performed through facial recognition, voice tone analysis, eye-tracking, etc., and the obtained emotional data is sent to a server.

[0193] The server selects the most appropriate advertisement based on collected sentiment data and user profiles and sends it to the device. This process utilizes a generative AI model and uses prompts to generate ad content optimized for specific emotional states. For example, prompts such as "What kind of video content is best suited for a user who is feeling relaxed?" serve as guidelines for generation.

[0194] For example, when a user visits a shopping mall, they might be presented with movie promotions or new product information tailored to their mood that day through smart glasses. By providing advertising experiences that resonate with consumers' emotions in this way, it becomes possible to increase user engagement.

[0195] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0196] Step 1:

[0197] The device collects user input information through cameras and voice sensors. This allows for the acquisition of user facial expressions, voice tone, and gaze data. This data is used for real-time emotion analysis. Input is sensor data, and output is the raw data to be analyzed.

[0198] Step 2:

[0199] The device performs sentiment analysis based on the acquired data to determine the user's emotional state. The analysis is carried out by converting signal data into emotional states using image and audio processing technologies. Inputs are camera images and audio signals, while output is data representing the user's emotions.

[0200] Step 3:

[0201] The server integrates sentiment data and user profiles sent from the terminal and initiates the ad generation process. It uses a generative AI model to generate prompt text, which serves as a guide for creating diverse ad content. The input is sentiment data and profile data, and the output is prompt text.

[0202] Step 4:

[0203] The server analyzes the advertising material based on the generated prompt text and selects the advertising content that best suits the user's emotional state. The selected content is then appropriately adjusted for the visual device. The input is the prompt text and advertising material, and the output is the optimized advertising content.

[0204] Step 5:

[0205] The device receives advertising content sent from the server and displays it on the screen within the user's field of view. This allows the user to experience real-time, optimized advertising. The input is the optimized advertising content, and the output is the display of the advertisement on the visual display.

[0206] Step 6:

[0207] While a user watches an ad, the device collects sentiment data again and sends it to the server as viewing insights. This provides the feedback data necessary for analyzing the effectiveness of the ad. The input is the sentiment data during viewing, and the output is the feedback data.

[0208] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0209] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0210] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0211] [Second Embodiment]

[0212] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0213] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0214] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0215] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0216] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0217] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0218] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0219] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0220] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0221] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0222] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0223] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0224] The system of this invention is built to achieve personalized and effective ad delivery. In this system, the user, server, and terminal each play a specific role, optimizing the ads through a series of operations.

[0225] First, users input their ad preferences and basic information through their devices. This information forms a user profile, which is anonymized on the device and then sent to the server. Based on this profile information, the server passes the materials provided by advertisers to a generation system to create a variety of ad variations. This makes it possible to generate ads that are best suited to each user.

[0226] The device receives program data during broadcast and, when the next advertising slot begins, selects the most suitable advertisement in real time from a list received from the server. This process utilizes edge computing, and advertisements are dynamically replaced based on the user's profile. As a result, a seamless experience is provided that does not affect sight or hearing.

[0227] For example, if a user sets their preferences to "hide ads for alcoholic beverages," the device can use this information to display different ads—such as ads for non-alcoholic beverages or other products of interest—in the slots where alcohol-related ads would normally appear. This allows users to enjoy a personalized advertising experience without feeling uncomfortable.

[0228] Furthermore, the device collects data on the user's ad viewing and sends it to a server. This viewing data is processed by an analytical tool to quantify the effectiveness of the ads. The resulting analytical data is generated as a report and provided to advertisers and broadcasters, which helps improve future ad delivery strategies.

[0229] This invention makes it possible to deliver effective advertisements that are valuable to both advertisers and broadcasters while improving the user experience.

[0230] The following describes the processing flow.

[0231] Step 1:

[0232] Users access their device and enter their basic personal information and advertising preferences through a registration form. This includes age, gender, and the selection of categories of ads to display.

[0233] Step 2:

[0234] The terminal receives information entered by the user and creates an individual user profile. This profile is anonymized and converted into a form that does not identify the individual before being sent to the server.

[0235] Step 3:

[0236] The server, based on the received user profile, passes the advertising materials provided by the advertiser to the generation system, which then generates multiple variations of the advertisement. During this process, AI considers the matching with the user profile to create the most optimal advertisement.

[0237] Step 4:

[0238] The terminal monitors program data in real time and predicts the timing of the next advertising slot. It receives a list of ad variations provided by the server and prepares for them.

[0239] Step 5:

[0240] As the device approaches an ad space, it utilizes edge computing to select the most suitable ad based on the user profile. This selection process uses AI-powered ad effectiveness prediction and matching algorithms.

[0241] Step 6:

[0242] When an ad slot is reached, the device seamlessly replaces the currently airing program with a selected ad and displays it to the user. Because the ads are tailored to the user's interests and preferences, they provide a personalized experience.

[0243] Step 7:

[0244] The device records the user's behavior while watching advertisements and collects viewing data such as viewing time and whether or not they were skipped. The collected data is anonymized again and sent to the server.

[0245] Step 8:

[0246] The server analyzes the transmitted viewing data using analytical tools to evaluate the effectiveness of the advertisements. The analysis results are compiled into reports for advertisers and broadcasters.

[0247] Step 9:

[0248] The server provides the generated reports to advertisers and broadcasters to help improve future advertising campaigns. This feedback loop allows for continuous optimization of advertising.

[0249] (Example 1)

[0250] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0251] Modern advertising delivery systems suffer from insufficient personalization of ads based on individual user interests and preferences, as well as inadequate protection of viewer privacy. Furthermore, real-time data collection and analysis are difficult when accurately measuring the effectiveness of ads and incorporating the results into future advertising strategies. Solving these challenges is essential to improving the user experience and enabling effective ad delivery for both advertisers and media companies.

[0252] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0253] In this invention, the server includes means for receiving and anonymizing user input information and generating and updating user profiles; means for generating various variations of advertisements based on advertising materials, using generation AI technology; and means for selecting and replacing advertisements in real time using edge computing. This enables the provision of personalized advertisements to individual users, protects privacy, and allows for accurate measurement of advertising effectiveness and the formulation of effective advertising delivery strategies.

[0254] A "user profile" is a collection of data managed by a server, which includes user preferences and basic information regarding ad display, and is anonymized.

[0255] "Generative AI technology" is a technology that utilizes artificial intelligence to generate diverse advertising variations based on provided materials.

[0256] Edge computing is a technology that performs data processing on distributed devices and systems, enabling real-time ad selection and replacement.

[0257] "Seamlessly replacing" refers to inserting advertisements naturally without affecting the content being broadcast, in a way that does not cause any visual or auditory discomfort.

[0258] "Advertising materials" refer to the raw data of information and content provided by advertisers, which serve as the basis for advertisements processed by generation AI technology.

[0259] "Viewing data" refers to information about a user's ad viewing, including which ads were displayed and under what circumstances.

[0260] "Analysis results" refer to the measurement results regarding the effectiveness and impact of advertisements obtained after analyzing viewing data, and are provided as reports to advertisers and media companies.

[0261] This invention is a system in which users, terminals, and servers cooperate to personalize and effectively deliver advertisements.

[0262] Users enter their advertising preferences and basic information through the device's user interface. This information is anonymized on the device and sent to the server as a user profile.

[0263] Based on the received user profile, the server uses a generative AI model, such as advanced natural language processing technology, to generate various ad variations from the materials provided by the advertiser. These generated ads are optimized for each user; for example, the prompt could read, "Generate ad variations tailored to the user's preferences. The user does not like alcoholic beverages, so suggest non-alcoholic beverages or products in other areas of interest."

[0264] The device receives program data in real time, refers to an ad list provided by the server, and uses edge computing technology to select the most suitable ad to display in the next ad slot. This process is carried out without the user experiencing any visual or auditory discomfort, providing a seamless experience.

[0265] Furthermore, the device collects user ad viewing data and sends it to a server. This data is analyzed on the server to quantify the effectiveness of the ads, and the results of this analysis are provided to advertisers and media companies to be used to improve future ad delivery strategies.

[0266] In this way, the invention realizes a system that effectively delivers personalized advertisements while protecting user privacy, and at the same time provides meaningful data to advertisers and media companies.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] Users input their advertising preferences and basic information through the device's interface. This input data includes product categories of interest and ad types they wish to hide. The device uses this information to generate an anonymized user profile by removing personally identifiable data. The output is the anonymized user profile.

[0270] Step 2:

[0271] The device sends an anonymized user profile to the server. Upon receiving this information, the server uses a generative AI model to process the advertising material provided by the advertiser. It inputs prompts into the generative AI model to generate various variations of the advertisement. In this process, the inputs are the user profile and advertising material, and the output is a list of the generated ad variations.

[0272] Step 3:

[0273] The server sends instructions to the terminal to select the most suitable advertisement for each user from the generated list of ad variations. The terminal receives program data being broadcast in real time, and based on the ad list provided by the server, selects and displays the most suitable advertisement to display in the next ad slot. Here, the input is real-time program data and the ad list, and the output is the selected advertisement.

[0274] Step 4:

[0275] The device collects data about the advertisements the user has viewed. This data includes the time the advertisement was displayed, the type of advertisement, and whether the viewing was completed. The device sends the collected viewing data to the server. The input is viewing status data, and the output is data about viewing completion and its details.

[0276] Step 5:

[0277] The server receives the viewing data transmitted from the terminal and analyzes the effectiveness of the advertisement. In the analysis, it quantifies which advertisement has an effect under what circumstances, and uses this to improve the advertisement distribution strategy for subsequent times. The final output is an analysis report, which is provided to advertisers and media operators.

[0278] (Application Example 1)

[0279] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0280] In a conventional advertisement distribution system, personalization of advertisements based on the interests and preferences of individual users was insufficient, and improving the user experience was an issue. Also, there was a problem in that there were limited means to accurately grasp the effectiveness of advertisement distribution and to provide immediate and specific reports to advertisers and broadcasting stations. Furthermore, insufficient privacy protection had been a concern.

[0281] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0282] In this invention, the server includes means for receiving attribute information from the user, generating and updating a user model, generating means for generating various forms of advertisements based on advertisement materials, and means for selecting an optimal advertisement based on the user model and dynamically replacing it during broadcasting. As a result, it becomes possible to distribute advertisements optimized for the interests and preferences of the user.

[0283] A "user model" is a data structure that represents the preferences and interests of individual users based on the attribute information collected from the users.

[0284] "Generating means" is a system or process for creating various forms of advertisements based on advertisement materials.

[0285] An "advertisement provider" refers to an organization or individual that is the entity for distributing advertisements.

[0286] A "prompt sentence" is a sentence used to give instructions to a generative AI model to generate an advertisement that meets specific conditions or purposes.

[0287] A "generative AI model" is a type of artificial intelligence used to generate advertisements based on user interests.

[0288] "Edge computing" is a technology that performs data processing not in a centralized data center but at a location closer to the user.

[0289] The embodiments for implementing the present invention will be described. Three elements, a server, a terminal, and a user, cooperate to construct a system for optimizing advertisement delivery.

[0290] First, the user uses a smartphone to input their basic information and preferences for advertisement display. This information is anonymized by the terminal and transmitted to the server while protecting privacy. The server generates a user model based on the received information and constantly updates it. The user model is a data structure that reflects the user's preferences and interests.

[0291] Next, the server uses generation means to create various forms of advertisements from the materials provided by the advertisement provider. Software such as Python and TensorFlow is used for the generation means to generate a prompt sentence from the advertisement materials. Instructions are given to the generative AI model via the prompt sentence to generate an advertisement most suitable for the user's interests. An example of a prompt sentence is "If the user is interested in traveling, please generate an advertisement for a related travel company."

[0292] Also, the terminal utilizes edge computing technology to replace advertisements in real time while considering historical information and the user model. Technologies such as Google Cloud Platform can be used. This provides a seamless advertisement experience for each user.

[0293] Viewing information is periodically sent from the user's device to the server. The server analyzes this information to quantify the user's ad viewing trends and ad effectiveness. The analysis results are provided to advertisers and broadcasters and used to improve future ad delivery strategies.

[0294] With the above configuration, the advertising distribution system of this invention enables advertising distribution that provides an optimized advertising experience for users while also being valuable for advertisers and broadcasters.

[0295] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0296] Step 1:

[0297] Users enter basic information and advertising preferences using their smartphones. The entered information is anonymized on the device for data protection. The input here consists of user attribute information, and the output is an anonymized user model.

[0298] Step 2:

[0299] The terminal sends an anonymized user model to the server. The server generates and updates the user model based on the received data. This user model is a data structure that represents the user's preferences and interests. The input is the anonymized user model, and the output is the updated user model.

[0300] Step 3:

[0301] The server uses an AI model to generate various types of advertisements based on the advertising materials received from advertisers. Here, the prompt text generated by the generation method is used as input to the AI ​​model, and the output is an advertisement tailored to the user. A specific example of this operation is a prompt text such as, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0302] Step 4:

[0303] The server sends the generated advertisement to the terminal, and the terminal uses edge computing to select and replace the advertisement in real time. The input is the advertisement data from the server, and the output is the optimized advertisement displayed to the user. Specifically, it is selected based on the user model.

[0304] Step 5:

[0305] The terminal collects the user's advertisement viewing information and sends it to the server. The server analyzes the viewing information to generate advertisement effect data. The input is the viewing information, and the output is the analysis result. The result of the analysis is useful for improving the advertisement delivery strategy.

[0306] Step 6:

[0307] The server provides the analysis result to the advertisement provider and the broadcasting organization as a report. The input is the analysis result of the advertisement effect, and the output is the report. Based on this report, the advertisement improvement measures for the next and subsequent times are formulated.

[0308] Furthermore, an emotion engine for estimating the user's emotion may be combined. That is, the specific processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform specific processing using the user's emotion.

[0309] The present invention realizes a more personalized advertisement experience for the user by combining an emotion engine with an advertisement delivery system. This system is composed of each component of the user, the server, the terminal, and the emotion engine.

[0310] First, users input basic information and advertising preferences through their devices, which generates a user profile. This profile is anonymized on the device and sent to the server. In addition, an emotion engine is installed on the device to monitor the user's emotions in real time while they are watching. The emotion engine analyzes the user's emotions from facial expressions, tone of voice, eye gaze, etc., and can continuously acquire this data.

[0311] The server combines user profiles and sentiment data to generate a variety of advertisements based on materials provided by advertisers. In this process, the generation mechanism incorporates feedback from sentiment analysis to select the advertisement best suited to the user's current emotional state. This ensures that advertisements are not only based on personal data but also take into account emotional relevance.

[0312] On the device, advertisements optimized for the user are selected using the spaces between lines of the currently airing program. The selected advertisements are then replaced in real time during the broadcast and presented to the user. Edge computing minimizes latency in this process, providing a seamless advertising experience.

[0313] As a concrete example of the emotion engine, when a user displays a sad expression, ads that do not overly stimulate emotions (e.g., product ads with calming music) are selected. Similarly, when a user is smiling, ads with lively and cheerful images are presented. In this way, by optimizing ads according to emotions, a more effective and user-friendly advertising experience is achieved.

[0314] Furthermore, the device collects viewing data, including changes in the user's emotions while watching advertisements, anonymizes it again, and sends it to the server. The server analyzes this emotional data and uses it to evaluate the effectiveness of the advertisements. The analysis results are provided to advertisers and broadcasters as reports and used for future advertising campaigns.

[0315] This invention enables ad delivery based on the user's emotional state, providing a more personalized advertising experience tailored to individual emotions.

[0316] The following describes the processing flow.

[0317] Step 1:

[0318] Users log in to their devices and enter their personal information and basic preferences regarding advertising. This creates a user profile, which is then anonymized on the device.

[0319] Step 2:

[0320] The device sends an anonymized user profile to the server and simultaneously activates an emotion engine to monitor the user's facial expressions, voice, gaze, etc., and retrieves emotion data for each request.

[0321] Step 3:

[0322] Based on user profiles and sentiment data received from the terminal, the server uses materials provided by the advertiser to create multiple ad variations via a generation mechanism. In this process, the generation mechanism takes sentiment data into account and generates ads that are appropriate for the emotional state.

[0323] Step 4:

[0324] The server stores the generated ad variations and prepares ad lists that match the user's profile.

[0325] Step 5:

[0326] The terminal monitors the currently airing program and predicts the next advertising slot. As the advertising slot approaches, it selects the most suitable advertisement from the list of advertisements received from the server.

[0327] Step 6:

[0328] When an ad slot becomes available, the device checks emotional data obtained in real time from the emotion engine and makes a final confirmation that the selected ad matches that emotion.

[0329] Step 7:

[0330] When a matching ad is identified, the device seamlessly replaces the ad in the program in real time and displays it to the user. This provides a personalized advertising experience that responds to the user's emotions.

[0331] Step 8:

[0332] The device also collects data on the user's emotional changes while watching advertisements as viewing data, anonymizes this data, and sends it to the server.

[0333] Step 9:

[0334] The server analyzes the collected viewing and sentiment data to evaluate the effectiveness of the advertisements. The results are provided to advertisers and broadcasters as advertising effectiveness reports and used to optimize future campaigns.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0337] In modern advertising systems, it is difficult to provide more appropriate and emotionally sensitive advertisements to individual users while ensuring the security of data, including users' personal information. Furthermore, there is a need to accurately evaluate the effectiveness of advertisements and use that information to improve future campaigns. To address these challenges, a new system is needed that combines the protection of personal information with real-time sentiment analysis.

[0338] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0339] In this invention, the server includes means for receiving input information from the user and generating and updating a personal profile; means for generating various variations of advertisements using generative artificial intelligence by combining the personal profile with collected emotional data; and means for selecting the most suitable advertisement based on emotional analysis and replacing it within the video content. This makes it possible to deliver advertisements that respond to the user's emotions, achieving both privacy protection and effective advertisement delivery.

[0340] A "personal profile" is an anonymized dataset of basic information and preferences collected from users, and is used to personalize advertisements.

[0341] "Generative artificial intelligence" is an algorithm that generates new content and ideas based on input data, and is a technology that creates diverse variations of advertisements.

[0342] "Emotional data" refers to information obtained by measuring and analyzing the emotional state of a user from their facial expressions, tone of voice, gaze, etc., and is used for selecting advertisements and measuring their effectiveness.

[0343] "Ad generation methods" refer to the processes and technologies used to generate advertisements that are best suited to the user, based on collected personal profiles and sentiment data.

[0344] "Anonymization" is a data processing technique that protects privacy by removing or transforming information that can identify an individual.

[0345] This invention aims to provide an advertising experience that takes user emotions into consideration in an advertising delivery system. It mainly consists of a user, a terminal, a server, and an emotion analysis device.

[0346] First, users input basic information and advertising preferences through their devices. The device then generates a personal profile internally and sends this information to a server using anonymization technology. This anonymization technology utilizes methods that remove personally identifiable information and aggregate the data.

[0347] The device is equipped with an emotion analysis device that monitors the user's facial expressions, voice tone, and gaze in real time. This analysis device continuously acquires the user's emotional data and stores the analysis results. This emotional data is then analyzed using facial recognition software and voice analysis algorithms.

[0348] The server uses generative artificial intelligence based on this data to generate a variety of advertisements using material data provided by advertisers. The generated advertisements are then selected based on the user's personal profile and emotional data to determine the most appropriate one. In this process, the generative AI model optimizes itself by using prompts such as, "Choose a product that matches the user's current emotional state."

[0349] The selected advertisements are played on the device. By using edge computing, advertisements are replaced in real time with the content being broadcast, maintaining a smooth viewing experience. For example, if the user makes a sad face, an advertisement for a product with calming music will be displayed, showing advertisements that are synchronized with the user's emotions.

[0350] Furthermore, the device also collects changes in the user's emotions while viewing advertisements, anonymizes this data again, and sends it to the server. The server uses this emotional data to evaluate the effectiveness of the advertisements and reports the results to the advertisers and telecommunications companies.

[0351] An example of a prompt message is: "We want to monitor the emotional expressions the user is showing towards their device and select ads that match those emotions. Please use an emotion analysis algorithm to suggest how to generate appropriate ads."

[0352] In this way, by using this system, it is possible to provide advertising experiences tailored to the emotions of individual users and to enhance advertising effectiveness.

[0353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0354] Step 1:

[0355] Users input basic information and advertising preferences through their devices. This information is used to build a personal profile of the user. The data obtained as input includes name, age, and interest categories, and the device generates the personal profile based on this information. The generated profile is converted into a format that does not identify the individual using an anonymization algorithm and sent to the server as output.

[0356] Step 2:

[0357] The terminal uses an emotion analysis device to monitor the user's facial expressions and voice tone in real time. Inputs include video and audio data from the camera and microphone. These are processed by facial recognition software and voice analysis algorithms to analyze the user's emotional state. The output is the user's current emotional data, which is then shared within the system.

[0358] Step 3:

[0359] The server generates ad variations using generative artificial intelligence (generative AI model) based on personal profiles and sentiment data. Input includes material data provided by advertisers, anonymized personal profiles, and sentiment data. The AI ​​model analyzes this data and selects the most suitable ad variation. This ad is then adjusted to match the user's emotional state. The selected ad data is generated as output and sent to the device.

[0360] Step 4:

[0361] The device receives advertising data sent from the server and displays it in real time between video content. The input is advertising data from the server. By using edge computing technology, advertisements are played appropriately without delay. As output, personalized advertisements are seamlessly presented to the user.

[0362] Step 5:

[0363] While the user is watching an ad, the device again collects emotional data. The input is changes in facial expressions and voice during viewing. These changes are recorded to measure the effectiveness of the ad. The output is emotional change data generated during ad viewing, which is later anonymized again and sent to the server.

[0364] Step 6:

[0365] The server evaluates the effectiveness of advertisements by analyzing viewing and emotional data. The input is emotional change data during ad viewing. The analysis provides insights into the performance of the advertisements. The output is the measurement of the advertisement's effectiveness, which is provided as a report to advertisers and telecommunications companies. These results are then used to inform future advertising campaigns.

[0366] (Application Example 2)

[0367] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0368] In modern advertising, a challenge exists in providing advertising experiences tailored to individual user emotions and preferences. Traditional systems are limited to ad delivery based on user profile data, making it difficult to display ads that take real-time emotional states into account. As a result, advertising effectiveness is not fully realized, and there is a need to optimize the user experience.

[0369] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0370] In this invention, the server includes means for receiving input information from the user and generating and updating a user profile; means for generating advertisements with various expressions based on diverse media materials; means for analyzing the user's emotional state, selecting the most appropriate advertisement based on the user profile and emotional data, and displaying it on a visual device; means for collecting emotional data and viewing data and analyzing that data; and means for providing reports to advertisers and information distributors based on the analysis results. This makes it possible to optimize advertisements in real time according to the user's emotions and provide a more personalized advertising experience.

[0371] A "user profile" is a set of data in a database that is generated based on a user's basic information and preferences regarding advertisements.

[0372] "Media materials" refer to digital content such as video, audio, and images used as advertising materials.

[0373] "Emotional state" refers to information that indicates the user's psychological and emotional condition, analyzed from the user's facial expressions, tone of voice, gaze, etc.

[0374] A "visual device" is a device that allows a user to receive information visually, and includes smart glasses and mobile device displays.

[0375] "Emotional data" refers to data obtained by analyzing a user's emotional state, and is used to optimize advertising.

[0376] An "advertiser" refers to a company or organization that places advertisements to promote the sale of its products or services.

[0377] An "information provider" refers to a service provider that offers various information content to users.

[0378] This invention utilizes a user's smart glasses and other visual devices, a server, and edge computing software responsible for computational processing. The following describes in detail how each element functions.

[0379] The server receives input from users, generates a user profile based on this information, and continuously updates it. This profile reflects the user's personal interests and responses to advertisements. The server also incorporates diverse media materials and generates advertisements optimized for the user's characteristics.

[0380] The device, i.e., the user's visual device such as smart glasses, analyzes the user's emotional state in real time through cameras and sensors. Emotional analysis is performed through facial recognition, voice tone analysis, eye-tracking, etc., and the obtained emotional data is sent to a server.

[0381] The server selects the most appropriate advertisement based on collected sentiment data and user profiles and sends it to the device. This process utilizes a generative AI model and uses prompts to generate ad content optimized for specific emotional states. For example, prompts such as "What kind of video content is best suited for a user who is feeling relaxed?" serve as guidelines for generation.

[0382] For example, when a user visits a shopping mall, they might be presented with movie promotions or new product information tailored to their mood that day through smart glasses. By providing advertising experiences that resonate with consumers' emotions in this way, it becomes possible to increase user engagement.

[0383] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0384] Step 1:

[0385] The device collects user input information through cameras and voice sensors. This allows for the acquisition of user facial expressions, voice tone, and gaze data. This data is used for real-time emotion analysis. Input is sensor data, and output is the raw data to be analyzed.

[0386] Step 2:

[0387] The device performs sentiment analysis based on the acquired data to determine the user's emotional state. The analysis is carried out by converting signal data into emotional states using image and audio processing technologies. Inputs are camera images and audio signals, while output is data representing the user's emotions.

[0388] Step 3:

[0389] The server integrates sentiment data and user profiles sent from the terminal and initiates the ad generation process. It uses a generative AI model to generate prompt text, which serves as a guide for creating diverse ad content. The input is sentiment data and profile data, and the output is prompt text.

[0390] Step 4:

[0391] The server analyzes the advertising material based on the generated prompt text and selects the advertising content that best suits the user's emotional state. The selected content is then appropriately adjusted for the visual device. The input is the prompt text and advertising material, and the output is the optimized advertising content.

[0392] Step 5:

[0393] The device receives advertising content sent from the server and displays it on the screen within the user's field of view. This allows the user to experience real-time, optimized advertising. The input is the optimized advertising content, and the output is the display of the advertisement on the visual display.

[0394] Step 6:

[0395] While a user watches an ad, the device collects sentiment data again and sends it to the server as viewing insights. This provides the feedback data necessary for analyzing the effectiveness of the ad. The input is the sentiment data during viewing, and the output is the feedback data.

[0396] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0397] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0398] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0399] [Third Embodiment]

[0400] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0401] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0402] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0403] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0404] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0405] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0406] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0407] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0408] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0409] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0410] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0411] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0412] The system of this invention is built to achieve personalized and effective ad delivery. In this system, the user, server, and terminal each play a specific role, optimizing the ads through a series of operations.

[0413] First, users input their ad preferences and basic information through their devices. This information forms a user profile, which is anonymized on the device and then sent to the server. Based on this profile information, the server passes the materials provided by advertisers to a generation system to create a variety of ad variations. This makes it possible to generate ads that are best suited to each user.

[0414] The device receives program data during broadcast and, when the next advertising slot begins, selects the most suitable advertisement in real time from a list received from the server. This process utilizes edge computing, and advertisements are dynamically replaced based on the user's profile. As a result, a seamless experience is provided that does not affect sight or hearing.

[0415] For example, if a user sets their preferences to "hide ads for alcoholic beverages," the device can use this information to display different ads—such as ads for non-alcoholic beverages or other products of interest—in the slots where alcohol-related ads would normally appear. This allows users to enjoy a personalized advertising experience without feeling uncomfortable.

[0416] Furthermore, the device collects data on the user's ad viewing and sends it to a server. This viewing data is processed by an analytical tool to quantify the effectiveness of the ads. The resulting analytical data is generated as a report and provided to advertisers and broadcasters, which helps improve future ad delivery strategies.

[0417] This invention makes it possible to deliver effective advertisements that are valuable to both advertisers and broadcasters while improving the user experience.

[0418] The following describes the processing flow.

[0419] Step 1:

[0420] Users access their device and enter their basic personal information and advertising preferences through a registration form. This includes age, gender, and the selection of categories of ads to display.

[0421] Step 2:

[0422] The terminal receives information entered by the user and creates an individual user profile. This profile is anonymized and converted into a form that does not identify the individual before being sent to the server.

[0423] Step 3:

[0424] The server, based on the received user profile, passes the advertising materials provided by the advertiser to the generation system, which then generates multiple variations of the advertisement. During this process, AI considers the matching with the user profile to create the most optimal advertisement.

[0425] Step 4:

[0426] The terminal monitors program data in real time and predicts the timing of the next advertising slot. It receives a list of ad variations provided by the server and prepares for them.

[0427] Step 5:

[0428] As the device approaches an ad space, it utilizes edge computing to select the most suitable ad based on the user profile. This selection process uses AI-powered ad effectiveness prediction and matching algorithms.

[0429] Step 6:

[0430] When an ad slot is reached, the device seamlessly replaces the currently airing program with a selected ad and displays it to the user. Because the ads are tailored to the user's interests and preferences, they provide a personalized experience.

[0431] Step 7:

[0432] The device records the user's behavior while watching advertisements and collects viewing data such as viewing time and whether or not they were skipped. The collected data is anonymized again and sent to the server.

[0433] Step 8:

[0434] The server analyzes the transmitted viewing data using analytical tools to evaluate the effectiveness of the advertisements. The analysis results are compiled into reports for advertisers and broadcasters.

[0435] Step 9:

[0436] The server provides the generated reports to advertisers and broadcasters to help improve future advertising campaigns. This feedback loop allows for continuous optimization of advertising.

[0437] (Example 1)

[0438] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0439] Modern advertising delivery systems suffer from insufficient personalization of ads based on individual user interests and preferences, as well as inadequate protection of viewer privacy. Furthermore, real-time data collection and analysis are difficult when accurately measuring the effectiveness of ads and incorporating the results into future advertising strategies. Solving these challenges is essential to improving the user experience and enabling effective ad delivery for both advertisers and media companies.

[0440] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0441] In this invention, the server includes means for receiving and anonymizing user input information and generating and updating user profiles; means for generating various variations of advertisements based on advertising materials, using generation AI technology; and means for selecting and replacing advertisements in real time using edge computing. This enables the provision of personalized advertisements to individual users, protects privacy, and allows for accurate measurement of advertising effectiveness and the formulation of effective advertising delivery strategies.

[0442] A "user profile" is a collection of data managed by a server, which includes user preferences and basic information regarding ad display, and is anonymized.

[0443] "Generative AI technology" is a technology that utilizes artificial intelligence to generate diverse advertising variations based on provided materials.

[0444] Edge computing is a technology that performs data processing on distributed devices and systems, enabling real-time ad selection and replacement.

[0445] "Seamlessly replacing" refers to inserting advertisements naturally without affecting the content being broadcast, in a way that does not cause any visual or auditory discomfort.

[0446] "Advertising materials" refer to the raw data of information and content provided by advertisers, which serve as the basis for advertisements processed by generation AI technology.

[0447] "Viewing data" refers to information about a user's ad viewing, including which ads were displayed and under what circumstances.

[0448] "Analysis results" refer to the measurement results regarding the effectiveness and impact of advertisements obtained after analyzing viewing data, and are provided as reports to advertisers and media companies.

[0449] This invention is a system in which users, terminals, and servers cooperate to personalize and effectively deliver advertisements.

[0450] Users enter their advertising preferences and basic information through the device's user interface. This information is anonymized on the device and sent to the server as a user profile.

[0451] Based on the received user profile, the server uses a generative AI model, such as advanced natural language processing technology, to generate various ad variations from the materials provided by the advertiser. These generated ads are optimized for each user; for example, the prompt could read, "Generate ad variations tailored to the user's preferences. The user does not like alcoholic beverages, so suggest non-alcoholic beverages or products in other areas of interest."

[0452] The device receives program data in real time, refers to an ad list provided by the server, and uses edge computing technology to select the most suitable ad to display in the next ad slot. This process is carried out without the user experiencing any visual or auditory discomfort, providing a seamless experience.

[0453] Furthermore, the device collects user ad viewing data and sends it to a server. This data is analyzed on the server to quantify the effectiveness of the ads, and the results of this analysis are provided to advertisers and media companies to be used to improve future ad delivery strategies.

[0454] In this way, the invention realizes a system that effectively delivers personalized advertisements while protecting user privacy, and at the same time provides meaningful data to advertisers and media companies.

[0455] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0456] Step 1:

[0457] Users input their advertising preferences and basic information through the device's interface. This input data includes product categories of interest and ad types they wish to hide. The device uses this information to generate an anonymized user profile by removing personally identifiable data. The output is the anonymized user profile.

[0458] Step 2:

[0459] The device sends an anonymized user profile to the server. Upon receiving this information, the server uses a generative AI model to process the advertising material provided by the advertiser. It inputs prompts into the generative AI model to generate various variations of the advertisement. In this process, the inputs are the user profile and advertising material, and the output is a list of the generated ad variations.

[0460] Step 3:

[0461] The server sends instructions to the terminal to select the most suitable advertisement for each user from the generated list of ad variations. The terminal receives program data being broadcast in real time, and based on the ad list provided by the server, selects and displays the most suitable advertisement to display in the next ad slot. Here, the input is real-time program data and the ad list, and the output is the selected advertisement.

[0462] Step 4:

[0463] The device collects data about the advertisements the user has viewed. This data includes the time the advertisement was displayed, the type of advertisement, and whether the viewing was completed. The device sends the collected viewing data to the server. The input is viewing status data, and the output is data about viewing completion and its details.

[0464] Step 5:

[0465] The server receives viewing data sent from the terminals and analyzes the effectiveness of the advertisements. The analysis quantifies which advertisements were effective and under what circumstances, and uses this information to improve future advertising strategies. The final output is an analysis report, which is provided to advertisers and media companies.

[0466] (Application Example 1)

[0467] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] Traditional advertising delivery systems have struggled to personalize ads based on individual users' interests and preferences, posing a challenge in improving the user experience. Furthermore, there were limited means to accurately track the effectiveness of ad delivery and provide immediate, specific reports to advertisers and broadcasters. In addition, insufficient privacy protection was a cause for concern.

[0469] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0470] In this invention, the server includes means for receiving attribute information from users and generating and updating user models, means for generating various forms of advertisements based on advertising materials, and means for selecting the most suitable advertisement based on the user model and dynamically switching them during broadcast. This makes it possible to deliver advertisements optimized to the user's interests and preferences.

[0471] A "user model" is a data structure that represents the preferences and interests of individual users based on attribute information collected from them.

[0472] "Generation means" refers to a system or process for creating advertisements in various forms based on advertising materials.

[0473] "Advertiser" refers to an organization or individual that distributes advertisements.

[0474] A "prompt sentence" is a sentence used to instruct a generative AI model and generate advertisements that meet specific conditions or objectives.

[0475] A "generative AI model" is a type of artificial intelligence used to generate advertisements based on user interests.

[0476] Edge computing is a technology that performs data processing closer to the user than in a centralized data center.

[0477] The embodiments for carrying out the present invention will be described. A system is constructed in which a server, a terminal, and a user work together to optimize ad delivery.

[0478] First, users use their smartphones to input their basic information and advertising preferences. This information is anonymized by the device and sent to the server while protecting privacy. The server generates a user model based on the received information and updates it constantly. The user model is a data structure that reflects the user's preferences and interests.

[0479] Next, the server uses a generation mechanism to create various forms of advertisements from the materials provided by the advertisers. The generation mechanism uses software such as Python or TensorFlow to generate prompt messages from the advertisement materials. These prompt messages instruct a generation AI model to generate advertisements best suited to the user's interests. An example prompt message might be, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0480] Furthermore, the device utilizes edge computing technology to refresh advertisements in real time, taking into account historical information and user models. Technologies such as Google Cloud Platform can be used. This provides a seamless advertising experience for each user.

[0481] Viewing information is periodically sent from the user's device to the server. The server analyzes this information to quantify the user's ad viewing trends and ad effectiveness. The analysis results are provided to advertisers and broadcasters and used to improve future ad delivery strategies.

[0482] With the above configuration, the advertising distribution system of this invention enables advertising distribution that provides an optimized advertising experience for users while also being valuable for advertisers and broadcasters.

[0483] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0484] Step 1:

[0485] Users enter basic information and advertising preferences using their smartphones. The entered information is anonymized on the device for data protection. The input here consists of user attribute information, and the output is an anonymized user model.

[0486] Step 2:

[0487] The terminal sends an anonymized user model to the server. The server generates and updates the user model based on the received data. This user model is a data structure that represents the user's preferences and interests. The input is the anonymized user model, and the output is the updated user model.

[0488] Step 3:

[0489] The server uses an AI model to generate various types of advertisements based on the advertising materials received from advertisers. Here, the prompt text generated by the generation method is used as input to the AI ​​model, and the output is an advertisement tailored to the user. A specific example of this operation is a prompt text such as, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0490] Step 4:

[0491] The server sends generated advertisements to the device, and the device uses edge computing to select and replace the advertisements in real time. The input is advertisement data from the server, and the output is optimized advertisements displayed to the user. Specifically, the advertisements are selected based on a user model.

[0492] Step 5:

[0493] The device collects user ad viewing information and sends it to the server. The server analyzes the viewing information and generates data on ad effectiveness. The input is viewing information, and the output is the analysis results. The analysis results help improve ad delivery strategies.

[0494] Step 6:

[0495] The server provides the analysis results as a report to advertisers and broadcasters. The input is the analysis results of the advertising effectiveness, and the output is the report. This report is used to formulate measures for improving advertising in the future.

[0496] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0497] This invention aims to provide users with a more personalized advertising experience by combining an emotion engine with an advertising delivery system. This system consists of the user, server, terminal, and emotion engine components.

[0498] First, users input basic information and advertising preferences through their devices, which generates a user profile. This profile is anonymized on the device and sent to the server. In addition, an emotion engine is installed on the device to monitor the user's emotions in real time while they are watching. The emotion engine analyzes the user's emotions from facial expressions, tone of voice, eye gaze, etc., and can continuously acquire this data.

[0499] The server combines user profiles and sentiment data to generate a variety of advertisements based on materials provided by advertisers. In this process, the generation mechanism incorporates feedback from sentiment analysis to select the advertisement best suited to the user's current emotional state. This ensures that advertisements are not only based on personal data but also take into account emotional relevance.

[0500] On the device, advertisements optimized for the user are selected using the spaces between lines of the currently airing program. The selected advertisements are then replaced in real time during the broadcast and presented to the user. Edge computing minimizes latency in this process, providing a seamless advertising experience.

[0501] As a concrete example of the emotion engine, when a user displays a sad expression, ads that do not overly stimulate emotions (e.g., product ads with calming music) are selected. Similarly, when a user is smiling, ads with lively and cheerful images are presented. In this way, by optimizing ads according to emotions, a more effective and user-friendly advertising experience is achieved.

[0502] Furthermore, the device collects viewing data, including changes in the user's emotions while watching advertisements, anonymizes it again, and sends it to the server. The server analyzes this emotional data and uses it to evaluate the effectiveness of the advertisements. The analysis results are provided to advertisers and broadcasters as reports and used for future advertising campaigns.

[0503] This invention enables ad delivery based on the user's emotional state, providing a more personalized advertising experience tailored to individual emotions.

[0504] The following describes the processing flow.

[0505] Step 1:

[0506] Users log in to their devices and enter their personal information and basic preferences regarding advertising. This creates a user profile, which is then anonymized on the device.

[0507] Step 2:

[0508] The device sends an anonymized user profile to the server and simultaneously activates an emotion engine to monitor the user's facial expressions, voice, gaze, etc., and retrieves emotion data for each request.

[0509] Step 3:

[0510] Based on user profiles and sentiment data received from the terminal, the server uses materials provided by the advertiser to create multiple ad variations via a generation mechanism. In this process, the generation mechanism takes sentiment data into account and generates ads that are appropriate for the emotional state.

[0511] Step 4:

[0512] The server stores the generated ad variations and prepares ad lists that match the user's profile.

[0513] Step 5:

[0514] The terminal monitors the currently airing program and predicts the next advertising slot. As the advertising slot approaches, it selects the most suitable advertisement from the list of advertisements received from the server.

[0515] Step 6:

[0516] When an ad slot becomes available, the device checks emotional data obtained in real time from the emotion engine and makes a final confirmation that the selected ad matches that emotion.

[0517] Step 7:

[0518] When a matching ad is identified, the device seamlessly replaces the ad in the program in real time and displays it to the user. This provides a personalized advertising experience that responds to the user's emotions.

[0519] Step 8:

[0520] The device also collects data on the user's emotional changes while watching advertisements as viewing data, anonymizes this data, and sends it to the server.

[0521] Step 9:

[0522] The server analyzes the collected viewing and sentiment data to evaluate the effectiveness of the advertisements. The results are provided to advertisers and broadcasters as advertising effectiveness reports and used to optimize future campaigns.

[0523] (Example 2)

[0524] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0525] In modern advertising systems, it is difficult to provide more appropriate and emotionally sensitive advertisements to individual users while ensuring the security of data, including users' personal information. Furthermore, there is a need to accurately evaluate the effectiveness of advertisements and use that information to improve future campaigns. To address these challenges, a new system is needed that combines the protection of personal information with real-time sentiment analysis.

[0526] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0527] In this invention, the server includes means for receiving input information from the user and generating and updating a personal profile; means for generating various variations of advertisements using generative artificial intelligence by combining the personal profile with collected emotional data; and means for selecting the most suitable advertisement based on emotional analysis and replacing it within the video content. This makes it possible to deliver advertisements that respond to the user's emotions, achieving both privacy protection and effective advertisement delivery.

[0528] A "personal profile" is an anonymized dataset of basic information and preferences collected from users, and is used to personalize advertisements.

[0529] "Generative artificial intelligence" is an algorithm that generates new content and ideas based on input data, and is a technology that creates diverse variations of advertisements.

[0530] "Emotional data" refers to information obtained by measuring and analyzing the emotional state of a user from their facial expressions, tone of voice, gaze, etc., and is used for selecting advertisements and measuring their effectiveness.

[0531] "Ad generation methods" refer to the processes and technologies used to generate advertisements that are best suited to the user, based on collected personal profiles and sentiment data.

[0532] "Anonymization" is a data processing technique that protects privacy by removing or transforming information that can identify an individual.

[0533] This invention aims to provide an advertising experience that takes user emotions into consideration in an advertising delivery system. It mainly consists of a user, a terminal, a server, and an emotion analysis device.

[0534] First, users input basic information and advertising preferences through their devices. The device then generates a personal profile internally and sends this information to a server using anonymization technology. This anonymization technology utilizes methods that remove personally identifiable information and aggregate the data.

[0535] The device is equipped with an emotion analysis device that monitors the user's facial expressions, voice tone, and gaze in real time. This analysis device continuously acquires the user's emotional data and stores the analysis results. This emotional data is then analyzed using facial recognition software and voice analysis algorithms.

[0536] The server uses generative artificial intelligence based on this data to generate a variety of advertisements using material data provided by advertisers. The generated advertisements are then selected based on the user's personal profile and emotional data to determine the most appropriate one. In this process, the generative AI model optimizes itself by using prompts such as, "Choose a product that matches the user's current emotional state."

[0537] The selected advertisements are played on the device. By using edge computing, advertisements are replaced in real time with the content being broadcast, maintaining a smooth viewing experience. For example, if the user makes a sad face, an advertisement for a product with calming music will be displayed, showing advertisements that are synchronized with the user's emotions.

[0538] Furthermore, the device also collects changes in the user's emotions while viewing advertisements, anonymizes this data again, and sends it to the server. The server uses this emotional data to evaluate the effectiveness of the advertisements and reports the results to the advertisers and telecommunications companies.

[0539] An example of a prompt message is: "We want to monitor the emotional expressions the user is showing towards their device and select ads that match those emotions. Please use an emotion analysis algorithm to suggest how to generate appropriate ads."

[0540] In this way, by using this system, it is possible to provide advertising experiences tailored to the emotions of individual users and to enhance advertising effectiveness.

[0541] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0542] Step 1:

[0543] Users input basic information and advertising preferences through their devices. This information is used to build a personal profile of the user. The data obtained as input includes name, age, and interest categories, and the device generates the personal profile based on this information. The generated profile is converted into a format that does not identify the individual using an anonymization algorithm and sent to the server as output.

[0544] Step 2:

[0545] The terminal uses an emotion analysis device to monitor the user's facial expressions and voice tone in real time. Inputs include video and audio data from the camera and microphone. These are processed by facial recognition software and voice analysis algorithms to analyze the user's emotional state. The output is the user's current emotional data, which is then shared within the system.

[0546] Step 3:

[0547] The server generates ad variations using generative artificial intelligence (generative AI model) based on personal profiles and sentiment data. Input includes material data provided by advertisers, anonymized personal profiles, and sentiment data. The AI ​​model analyzes this data and selects the most suitable ad variation. This ad is then adjusted to match the user's emotional state. The selected ad data is generated as output and sent to the device.

[0548] Step 4:

[0549] The device receives advertising data sent from the server and displays it in real time between video content. The input is advertising data from the server. By using edge computing technology, advertisements are played appropriately without delay. As output, personalized advertisements are seamlessly presented to the user.

[0550] Step 5:

[0551] While the user is watching an ad, the device again collects emotional data. The input is changes in facial expressions and voice during viewing. These changes are recorded to measure the effectiveness of the ad. The output is emotional change data generated during ad viewing, which is later anonymized again and sent to the server.

[0552] Step 6:

[0553] The server evaluates the effectiveness of advertisements by analyzing viewing and emotional data. The input is emotional change data during ad viewing. The analysis provides insights into the performance of the advertisements. The output is the measurement of the advertisement's effectiveness, which is provided as a report to advertisers and telecommunications companies. These results are then used to inform future advertising campaigns.

[0554] (Application Example 2)

[0555] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0556] In modern advertising, a challenge exists in providing advertising experiences tailored to individual user emotions and preferences. Traditional systems are limited to ad delivery based on user profile data, making it difficult to display ads that take real-time emotional states into account. As a result, advertising effectiveness is not fully realized, and there is a need to optimize the user experience.

[0557] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0558] In this invention, the server includes means for receiving input information from the user and generating and updating a user profile; means for generating advertisements with various expressions based on diverse media materials; means for analyzing the user's emotional state, selecting the most appropriate advertisement based on the user profile and emotional data, and displaying it on a visual device; means for collecting emotional data and viewing data and analyzing that data; and means for providing reports to advertisers and information distributors based on the analysis results. This makes it possible to optimize advertisements in real time according to the user's emotions and provide a more personalized advertising experience.

[0559] A "user profile" is a set of data in a database that is generated based on a user's basic information and preferences regarding advertisements.

[0560] "Media materials" refer to digital content such as video, audio, and images used as advertising materials.

[0561] "Emotional state" refers to information that indicates the user's psychological and emotional condition, analyzed from the user's facial expressions, tone of voice, gaze, etc.

[0562] A "visual device" is a device that allows a user to receive information visually, and includes smart glasses and mobile device displays.

[0563] "Emotional data" refers to data obtained by analyzing a user's emotional state, and is used to optimize advertising.

[0564] An "advertiser" refers to a company or organization that places advertisements to promote the sale of its products or services.

[0565] An "information provider" refers to a service provider that offers various information content to users.

[0566] This invention utilizes a user's smart glasses and other visual devices, a server, and edge computing software responsible for computational processing. The following describes in detail how each element functions.

[0567] The server receives input from users, generates a user profile based on this information, and continuously updates it. This profile reflects the user's personal interests and responses to advertisements. The server also incorporates diverse media materials and generates advertisements optimized for the user's characteristics.

[0568] The device, i.e., the user's visual device such as smart glasses, analyzes the user's emotional state in real time through cameras and sensors. Emotional analysis is performed through facial recognition, voice tone analysis, eye-tracking, etc., and the obtained emotional data is sent to a server.

[0569] The server selects the most appropriate advertisement based on collected sentiment data and user profiles and sends it to the device. This process utilizes a generative AI model and uses prompts to generate ad content optimized for specific emotional states. For example, prompts such as "What kind of video content is best suited for a user who is feeling relaxed?" serve as guidelines for generation.

[0570] For example, when a user visits a shopping mall, they might be presented with movie promotions or new product information tailored to their mood that day through smart glasses. By providing advertising experiences that resonate with consumers' emotions in this way, it becomes possible to increase user engagement.

[0571] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0572] Step 1:

[0573] The device collects user input information through cameras and voice sensors. This allows for the acquisition of user facial expressions, voice tone, and gaze data. This data is used for real-time emotion analysis. Input is sensor data, and output is the raw data to be analyzed.

[0574] Step 2:

[0575] The device performs sentiment analysis based on the acquired data to determine the user's emotional state. The analysis is carried out by converting signal data into emotional states using image and audio processing technologies. Inputs are camera images and audio signals, while output is data representing the user's emotions.

[0576] Step 3:

[0577] The server integrates sentiment data and user profiles sent from the terminal and initiates the ad generation process. It uses a generative AI model to generate prompt text, which serves as a guide for creating diverse ad content. The input is sentiment data and profile data, and the output is prompt text.

[0578] Step 4:

[0579] The server analyzes the advertising material based on the generated prompt text and selects the advertising content that best suits the user's emotional state. The selected content is then appropriately adjusted for the visual device. The input is the prompt text and advertising material, and the output is the optimized advertising content.

[0580] Step 5:

[0581] The device receives advertising content sent from the server and displays it on the screen within the user's field of view. This allows the user to experience real-time, optimized advertising. The input is the optimized advertising content, and the output is the display of the advertisement on the visual display.

[0582] Step 6:

[0583] While a user watches an ad, the device collects sentiment data again and sends it to the server as viewing insights. This provides the feedback data necessary for analyzing the effectiveness of the ad. The input is the sentiment data during viewing, and the output is the feedback data.

[0584] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0585] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0586] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0587] [Fourth Embodiment]

[0588] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0589] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0590] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0591] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0592] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0593] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0594] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0595] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0596] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0597] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0598] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0599] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0600] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0601] The system of this invention is built to achieve personalized and effective ad delivery. In this system, the user, server, and terminal each play a specific role, optimizing the ads through a series of operations.

[0602] First, users input their ad preferences and basic information through their devices. This information forms a user profile, which is anonymized on the device and then sent to the server. Based on this profile information, the server passes the materials provided by advertisers to a generation system to create a variety of ad variations. This makes it possible to generate ads that are best suited to each user.

[0603] The device receives program data during broadcast and, when the next advertising slot begins, selects the most suitable advertisement in real time from a list received from the server. This process utilizes edge computing, and advertisements are dynamically replaced based on the user's profile. As a result, a seamless experience is provided that does not affect sight or hearing.

[0604] For example, if a user sets their preferences to "hide ads for alcoholic beverages," the device can use this information to display different ads—such as ads for non-alcoholic beverages or other products of interest—in the slots where alcohol-related ads would normally appear. This allows users to enjoy a personalized advertising experience without feeling uncomfortable.

[0605] Furthermore, the device collects data on the user's ad viewing and sends it to a server. This viewing data is processed by an analytical tool to quantify the effectiveness of the ads. The resulting analytical data is generated as a report and provided to advertisers and broadcasters, which helps improve future ad delivery strategies.

[0606] This invention makes it possible to deliver effective advertisements that are valuable to both advertisers and broadcasters while improving the user experience.

[0607] The following describes the processing flow.

[0608] Step 1:

[0609] Users access their device and enter their basic personal information and advertising preferences through a registration form. This includes age, gender, and the selection of categories of ads to display.

[0610] Step 2:

[0611] The terminal receives information entered by the user and creates an individual user profile. This profile is anonymized and converted into a form that does not identify the individual before being sent to the server.

[0612] Step 3:

[0613] The server, based on the received user profile, passes the advertising materials provided by the advertiser to the generation system, which then generates multiple variations of the advertisement. During this process, AI considers the matching with the user profile to create the most optimal advertisement.

[0614] Step 4:

[0615] The terminal monitors program data in real time and predicts the timing of the next advertising slot. It receives a list of ad variations provided by the server and prepares for them.

[0616] Step 5:

[0617] As the device approaches an ad space, it utilizes edge computing to select the most suitable ad based on the user profile. This selection process uses AI-powered ad effectiveness prediction and matching algorithms.

[0618] Step 6:

[0619] When an ad slot is reached, the device seamlessly replaces the currently airing program with a selected ad and displays it to the user. Because the ads are tailored to the user's interests and preferences, they provide a personalized experience.

[0620] Step 7:

[0621] The device records the user's behavior while watching advertisements and collects viewing data such as viewing time and whether or not they were skipped. The collected data is anonymized again and sent to the server.

[0622] Step 8:

[0623] The server analyzes the transmitted viewing data using analytical tools to evaluate the effectiveness of the advertisements. The analysis results are compiled into reports for advertisers and broadcasters.

[0624] Step 9:

[0625] The server provides the generated reports to advertisers and broadcasters to help improve future advertising campaigns. This feedback loop allows for continuous optimization of advertising.

[0626] (Example 1)

[0627] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0628] Modern advertising delivery systems suffer from insufficient personalization of ads based on individual user interests and preferences, as well as inadequate protection of viewer privacy. Furthermore, real-time data collection and analysis are difficult when accurately measuring the effectiveness of ads and incorporating the results into future advertising strategies. Solving these challenges is essential to improving the user experience and enabling effective ad delivery for both advertisers and media companies.

[0629] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0630] In this invention, the server includes means for receiving and anonymizing user input information and generating and updating user profiles; means for generating various variations of advertisements based on advertising materials, using generation AI technology; and means for selecting and replacing advertisements in real time using edge computing. This enables the provision of personalized advertisements to individual users, protects privacy, and allows for accurate measurement of advertising effectiveness and the formulation of effective advertising delivery strategies.

[0631] A "user profile" is a collection of data managed by a server, which includes user preferences and basic information regarding ad display, and is anonymized.

[0632] "Generative AI technology" is a technology that utilizes artificial intelligence to generate diverse advertising variations based on provided materials.

[0633] Edge computing is a technology that performs data processing on distributed devices and systems, enabling real-time ad selection and replacement.

[0634] "Seamlessly replacing" refers to inserting advertisements naturally without affecting the content being broadcast, in a way that does not cause any visual or auditory discomfort.

[0635] "Advertising materials" refer to the raw data of information and content provided by advertisers, which serve as the basis for advertisements processed by generation AI technology.

[0636] "Viewing data" refers to information about a user's ad viewing, including which ads were displayed and under what circumstances.

[0637] "Analysis results" refer to the measurement results regarding the effectiveness and impact of advertisements obtained after analyzing viewing data, and are provided as reports to advertisers and media companies.

[0638] This invention is a system in which users, terminals, and servers cooperate to personalize and effectively deliver advertisements.

[0639] Users enter their advertising preferences and basic information through the device's user interface. This information is anonymized on the device and sent to the server as a user profile.

[0640] Based on the received user profile, the server uses a generative AI model, such as advanced natural language processing technology, to generate various ad variations from the materials provided by the advertiser. These generated ads are optimized for each user; for example, the prompt could read, "Generate ad variations tailored to the user's preferences. The user does not like alcoholic beverages, so suggest non-alcoholic beverages or products in other areas of interest."

[0641] The device receives program data in real time, refers to an ad list provided by the server, and uses edge computing technology to select the most suitable ad to display in the next ad slot. This process is carried out without the user experiencing any visual or auditory discomfort, providing a seamless experience.

[0642] Furthermore, the device collects user ad viewing data and sends it to a server. This data is analyzed on the server to quantify the effectiveness of the ads, and the results of this analysis are provided to advertisers and media companies to be used to improve future ad delivery strategies.

[0643] In this way, the invention realizes a system that effectively delivers personalized advertisements while protecting user privacy, and at the same time provides meaningful data to advertisers and media companies.

[0644] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0645] Step 1:

[0646] Users input their advertising preferences and basic information through the device's interface. This input data includes product categories of interest and ad types they wish to hide. The device uses this information to generate an anonymized user profile by removing personally identifiable data. The output is the anonymized user profile.

[0647] Step 2:

[0648] The device sends an anonymized user profile to the server. Upon receiving this information, the server uses a generative AI model to process the advertising material provided by the advertiser. It inputs prompts into the generative AI model to generate various variations of the advertisement. In this process, the inputs are the user profile and advertising material, and the output is a list of the generated ad variations.

[0649] Step 3:

[0650] The server sends instructions to the terminal to select the most suitable advertisement for each user from the generated list of ad variations. The terminal receives program data being broadcast in real time, and based on the ad list provided by the server, selects and displays the most suitable advertisement to display in the next ad slot. Here, the input is real-time program data and the ad list, and the output is the selected advertisement.

[0651] Step 4:

[0652] The device collects data about the advertisements the user has viewed. This data includes the time the advertisement was displayed, the type of advertisement, and whether the viewing was completed. The device sends the collected viewing data to the server. The input is viewing status data, and the output is data about viewing completion and its details.

[0653] Step 5:

[0654] The server receives viewing data sent from the terminals and analyzes the effectiveness of the advertisements. The analysis quantifies which advertisements were effective and under what circumstances, and uses this information to improve future advertising strategies. The final output is an analysis report, which is provided to advertisers and media companies.

[0655] (Application Example 1)

[0656] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0657] Traditional advertising delivery systems have struggled to personalize ads based on individual users' interests and preferences, posing a challenge in improving the user experience. Furthermore, there were limited means to accurately track the effectiveness of ad delivery and provide immediate, specific reports to advertisers and broadcasters. In addition, insufficient privacy protection was a cause for concern.

[0658] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0659] In this invention, the server includes means for receiving attribute information from users and generating and updating user models, means for generating various forms of advertisements based on advertising materials, and means for selecting the most suitable advertisement based on the user model and dynamically switching them during broadcast. This makes it possible to deliver advertisements optimized to the user's interests and preferences.

[0660] A "user model" is a data structure that represents the preferences and interests of individual users based on attribute information collected from them.

[0661] "Generation means" refers to a system or process for creating advertisements in various forms based on advertising materials.

[0662] "Advertiser" refers to an organization or individual that distributes advertisements.

[0663] A "prompt sentence" is a sentence used to instruct a generative AI model and generate advertisements that meet specific conditions or objectives.

[0664] A "generative AI model" is a type of artificial intelligence used to generate advertisements based on user interests.

[0665] Edge computing is a technology that performs data processing closer to the user than in a centralized data center.

[0666] The embodiments for carrying out the present invention will be described. A system is constructed in which a server, a terminal, and a user work together to optimize ad delivery.

[0667] First, users use their smartphones to input their basic information and advertising preferences. This information is anonymized by the device and sent to the server while protecting privacy. The server generates a user model based on the received information and updates it constantly. The user model is a data structure that reflects the user's preferences and interests.

[0668] Next, the server uses a generation mechanism to create various forms of advertisements from the materials provided by the advertisers. The generation mechanism uses software such as Python or TensorFlow to generate prompt messages from the advertisement materials. These prompt messages instruct a generation AI model to generate advertisements best suited to the user's interests. An example prompt message might be, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0669] Furthermore, the device utilizes edge computing technology to refresh advertisements in real time, taking into account historical information and user models. Technologies such as Google Cloud Platform can be used. This provides a seamless advertising experience for each user.

[0670] Viewing information is periodically sent from the user's device to the server. The server analyzes this information to quantify the user's ad viewing trends and ad effectiveness. The analysis results are provided to advertisers and broadcasters and used to improve future ad delivery strategies.

[0671] With the above configuration, the advertising distribution system of this invention enables advertising distribution that provides an optimized advertising experience for users while also being valuable for advertisers and broadcasters.

[0672] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0673] Step 1:

[0674] Users enter basic information and advertising preferences using their smartphones. The entered information is anonymized on the device for data protection. The input here consists of user attribute information, and the output is an anonymized user model.

[0675] Step 2:

[0676] The terminal sends an anonymized user model to the server. The server generates and updates the user model based on the received data. This user model is a data structure that represents the user's preferences and interests. The input is the anonymized user model, and the output is the updated user model.

[0677] Step 3:

[0678] The server uses an AI model to generate various types of advertisements based on the advertising materials received from advertisers. Here, the prompt text generated by the generation method is used as input to the AI ​​model, and the output is an advertisement tailored to the user. A specific example of this operation is a prompt text such as, "If the user is interested in travel, generate advertisements for relevant travel agencies."

[0679] Step 4:

[0680] The server sends generated advertisements to the device, and the device uses edge computing to select and replace the advertisements in real time. The input is advertisement data from the server, and the output is optimized advertisements displayed to the user. Specifically, the advertisements are selected based on a user model.

[0681] Step 5:

[0682] The device collects user ad viewing information and sends it to the server. The server analyzes the viewing information and generates data on ad effectiveness. The input is viewing information, and the output is the analysis results. The analysis results help improve ad delivery strategies.

[0683] Step 6:

[0684] The server provides the analysis results as a report to advertisers and broadcasters. The input is the analysis results of the advertising effectiveness, and the output is the report. This report is used to formulate measures for improving advertising in the future.

[0685] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0686] This invention aims to provide users with a more personalized advertising experience by combining an emotion engine with an advertising delivery system. This system consists of the user, server, terminal, and emotion engine components.

[0687] First, users input basic information and advertising preferences through their devices, which generates a user profile. This profile is anonymized on the device and sent to the server. In addition, an emotion engine is installed on the device to monitor the user's emotions in real time while they are watching. The emotion engine analyzes the user's emotions from facial expressions, tone of voice, eye gaze, etc., and can continuously acquire this data.

[0688] The server combines user profiles and sentiment data to generate a variety of advertisements based on materials provided by advertisers. In this process, the generation mechanism incorporates feedback from sentiment analysis to select the advertisement best suited to the user's current emotional state. This ensures that advertisements are not only based on personal data but also take into account emotional relevance.

[0689] On the device, advertisements optimized for the user are selected using the spaces between lines of the currently airing program. The selected advertisements are then replaced in real time during the broadcast and presented to the user. Edge computing minimizes latency in this process, providing a seamless advertising experience.

[0690] As a concrete example of the emotion engine, when a user displays a sad expression, ads that do not overly stimulate emotions (e.g., product ads with calming music) are selected. Similarly, when a user is smiling, ads with lively and cheerful images are presented. In this way, by optimizing ads according to emotions, a more effective and user-friendly advertising experience is achieved.

[0691] Furthermore, the device collects viewing data, including changes in the user's emotions while watching advertisements, anonymizes it again, and sends it to the server. The server analyzes this emotional data and uses it to evaluate the effectiveness of the advertisements. The analysis results are provided to advertisers and broadcasters as reports and used for future advertising campaigns.

[0692] This invention enables ad delivery based on the user's emotional state, providing a more personalized advertising experience tailored to individual emotions.

[0693] The following describes the processing flow.

[0694] Step 1:

[0695] Users log in to their devices and enter their personal information and basic preferences regarding advertising. This creates a user profile, which is then anonymized on the device.

[0696] Step 2:

[0697] The device sends an anonymized user profile to the server and simultaneously activates an emotion engine to monitor the user's facial expressions, voice, gaze, etc., and retrieves emotion data for each request.

[0698] Step 3:

[0699] Based on user profiles and sentiment data received from the terminal, the server uses materials provided by the advertiser to create multiple ad variations via a generation mechanism. In this process, the generation mechanism takes sentiment data into account and generates ads that are appropriate for the emotional state.

[0700] Step 4:

[0701] The server stores the generated ad variations and prepares ad lists that match the user's profile.

[0702] Step 5:

[0703] The terminal monitors the currently airing program and predicts the next advertising slot. As the advertising slot approaches, it selects the most suitable advertisement from the list of advertisements received from the server.

[0704] Step 6:

[0705] When an ad slot becomes available, the device checks emotional data obtained in real time from the emotion engine and makes a final confirmation that the selected ad matches that emotion.

[0706] Step 7:

[0707] When a matching ad is identified, the device seamlessly replaces the ad in the program in real time and displays it to the user. This provides a personalized advertising experience that responds to the user's emotions.

[0708] Step 8:

[0709] The device also collects data on the user's emotional changes while watching advertisements as viewing data, anonymizes this data, and sends it to the server.

[0710] Step 9:

[0711] The server analyzes the collected viewing and sentiment data to evaluate the effectiveness of the advertisements. The results are provided to advertisers and broadcasters as advertising effectiveness reports and used to optimize future campaigns.

[0712] (Example 2)

[0713] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0714] In modern advertising systems, it is difficult to provide more appropriate and emotionally sensitive advertisements to individual users while ensuring the security of data, including users' personal information. Furthermore, there is a need to accurately evaluate the effectiveness of advertisements and use that information to improve future campaigns. To address these challenges, a new system is needed that combines the protection of personal information with real-time sentiment analysis.

[0715] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0716] In this invention, the server includes means for receiving input information from the user and generating and updating a personal profile; means for generating various variations of advertisements using generative artificial intelligence by combining the personal profile with collected emotional data; and means for selecting the most suitable advertisement based on emotional analysis and replacing it within the video content. This makes it possible to deliver advertisements that respond to the user's emotions, achieving both privacy protection and effective advertisement delivery.

[0717] A "personal profile" is an anonymized dataset of basic information and preferences collected from users, and is used to personalize advertisements.

[0718] "Generative artificial intelligence" is an algorithm that generates new content and ideas based on input data, and is a technology that creates diverse variations of advertisements.

[0719] "Emotional data" refers to information obtained by measuring and analyzing the emotional state of a user from their facial expressions, tone of voice, gaze, etc., and is used for selecting advertisements and measuring their effectiveness.

[0720] "Ad generation methods" refer to the processes and technologies used to generate advertisements that are best suited to the user, based on collected personal profiles and sentiment data.

[0721] "Anonymization" is a data processing technique that protects privacy by removing or transforming information that can identify an individual.

[0722] This invention aims to provide an advertising experience that takes user emotions into consideration in an advertising delivery system. It mainly consists of a user, a terminal, a server, and an emotion analysis device.

[0723] First, users input basic information and advertising preferences through their devices. The device then generates a personal profile internally and sends this information to a server using anonymization technology. This anonymization technology utilizes methods that remove personally identifiable information and aggregate the data.

[0724] The device is equipped with an emotion analysis device that monitors the user's facial expressions, voice tone, and gaze in real time. This analysis device continuously acquires the user's emotional data and stores the analysis results. This emotional data is then analyzed using facial recognition software and voice analysis algorithms.

[0725] The server uses generative artificial intelligence based on this data to generate a variety of advertisements using material data provided by advertisers. The generated advertisements are then selected based on the user's personal profile and emotional data to determine the most appropriate one. In this process, the generative AI model optimizes itself by using prompts such as, "Choose a product that matches the user's current emotional state."

[0726] The selected advertisements are played on the device. By using edge computing, advertisements are replaced in real time with the content being broadcast, maintaining a smooth viewing experience. For example, if the user makes a sad face, an advertisement for a product with calming music will be displayed, showing advertisements that are synchronized with the user's emotions.

[0727] Furthermore, the device also collects changes in the user's emotions while viewing advertisements, anonymizes this data again, and sends it to the server. The server uses this emotional data to evaluate the effectiveness of the advertisements and reports the results to the advertisers and telecommunications companies.

[0728] An example of a prompt message is: "We want to monitor the emotional expressions the user is showing towards their device and select ads that match those emotions. Please use an emotion analysis algorithm to suggest how to generate appropriate ads."

[0729] In this way, by using this system, it is possible to provide advertising experiences tailored to the emotions of individual users and to enhance advertising effectiveness.

[0730] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0731] Step 1:

[0732] Users input basic information and advertising preferences through their devices. This information is used to build a personal profile of the user. The data obtained as input includes name, age, and interest categories, and the device generates the personal profile based on this information. The generated profile is converted into a format that does not identify the individual using an anonymization algorithm and sent to the server as output.

[0733] Step 2:

[0734] The terminal uses an emotion analysis device to monitor the user's facial expressions and voice tone in real time. Inputs include video and audio data from the camera and microphone. These are processed by facial recognition software and voice analysis algorithms to analyze the user's emotional state. The output is the user's current emotional data, which is then shared within the system.

[0735] Step 3:

[0736] The server generates ad variations using generative artificial intelligence (generative AI model) based on personal profiles and sentiment data. Input includes material data provided by advertisers, anonymized personal profiles, and sentiment data. The AI ​​model analyzes this data and selects the most suitable ad variation. This ad is then adjusted to match the user's emotional state. The selected ad data is generated as output and sent to the device.

[0737] Step 4:

[0738] The device receives advertising data sent from the server and displays it in real time between video content. The input is advertising data from the server. By using edge computing technology, advertisements are played appropriately without delay. As output, personalized advertisements are seamlessly presented to the user.

[0739] Step 5:

[0740] While the user is watching an ad, the device again collects emotional data. The input is changes in facial expressions and voice during viewing. These changes are recorded to measure the effectiveness of the ad. The output is emotional change data generated during ad viewing, which is later anonymized again and sent to the server.

[0741] Step 6:

[0742] The server evaluates the effectiveness of advertisements by analyzing viewing and emotional data. The input is emotional change data during ad viewing. The analysis provides insights into the performance of the advertisements. The output is the measurement of the advertisement's effectiveness, which is provided as a report to advertisers and telecommunications companies. These results are then used to inform future advertising campaigns.

[0743] (Application Example 2)

[0744] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0745] In modern advertising, a challenge exists in providing advertising experiences tailored to individual user emotions and preferences. Traditional systems are limited to ad delivery based on user profile data, making it difficult to display ads that take real-time emotional states into account. As a result, advertising effectiveness is not fully realized, and there is a need to optimize the user experience.

[0746] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0747] In this invention, the server includes means for receiving input information from the user and generating and updating a user profile; means for generating advertisements with various expressions based on diverse media materials; means for analyzing the user's emotional state, selecting the most appropriate advertisement based on the user profile and emotional data, and displaying it on a visual device; means for collecting emotional data and viewing data and analyzing that data; and means for providing reports to advertisers and information distributors based on the analysis results. This makes it possible to optimize advertisements in real time according to the user's emotions and provide a more personalized advertising experience.

[0748] A "user profile" is a set of data in a database that is generated based on a user's basic information and preferences regarding advertisements.

[0749] "Media materials" refer to digital content such as video, audio, and images used as advertising materials.

[0750] "Emotional state" refers to information that indicates the user's psychological and emotional condition, analyzed from the user's facial expressions, tone of voice, gaze, etc.

[0751] A "visual device" is a device that allows a user to receive information visually, and includes smart glasses and mobile device displays.

[0752] "Emotional data" refers to data obtained by analyzing a user's emotional state, and is used to optimize advertising.

[0753] An "advertiser" refers to a company or organization that places advertisements to promote the sale of its products or services.

[0754] An "information provider" refers to a service provider that offers various information content to users.

[0755] This invention utilizes a user's smart glasses and other visual devices, a server, and edge computing software responsible for computational processing. The following describes in detail how each element functions.

[0756] The server receives input from users, generates a user profile based on this information, and continuously updates it. This profile reflects the user's personal interests and responses to advertisements. The server also incorporates diverse media materials and generates advertisements optimized for the user's characteristics.

[0757] The device, i.e., the user's visual device such as smart glasses, analyzes the user's emotional state in real time through cameras and sensors. Emotional analysis is performed through facial recognition, voice tone analysis, eye-tracking, etc., and the obtained emotional data is sent to a server.

[0758] The server selects the most appropriate advertisement based on collected sentiment data and user profiles and sends it to the device. This process utilizes a generative AI model and uses prompts to generate ad content optimized for specific emotional states. For example, prompts such as "What kind of video content is best suited for a user who is feeling relaxed?" serve as guidelines for generation.

[0759] For example, when a user visits a shopping mall, they might be presented with movie promotions or new product information tailored to their mood that day through smart glasses. By providing advertising experiences that resonate with consumers' emotions in this way, it becomes possible to increase user engagement.

[0760] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0761] Step 1:

[0762] The device collects user input information through cameras and voice sensors. This allows for the acquisition of user facial expressions, voice tone, and gaze data. This data is used for real-time emotion analysis. Input is sensor data, and output is the raw data to be analyzed.

[0763] Step 2:

[0764] The device performs sentiment analysis based on the acquired data to determine the user's emotional state. The analysis is carried out by converting signal data into emotional states using image and audio processing technologies. Inputs are camera images and audio signals, while output is data representing the user's emotions.

[0765] Step 3:

[0766] The server integrates sentiment data and user profiles sent from the terminal and initiates the ad generation process. It uses a generative AI model to generate prompt text, which serves as a guide for creating diverse ad content. The input is sentiment data and profile data, and the output is prompt text.

[0767] Step 4:

[0768] The server analyzes the advertising material based on the generated prompt text and selects the advertising content that best suits the user's emotional state. The selected content is then appropriately adjusted for the visual device. The input is the prompt text and advertising material, and the output is the optimized advertising content.

[0769] Step 5:

[0770] The device receives advertising content sent from the server and displays it on the screen within the user's field of view. This allows the user to experience real-time, optimized advertising. The input is the optimized advertising content, and the output is the display of the advertisement on the visual display.

[0771] Step 6:

[0772] While a user watches an ad, the device collects sentiment data again and sends it to the server as viewing insights. This provides the feedback data necessary for analyzing the effectiveness of the ad. The input is the sentiment data during viewing, and the output is the feedback data.

[0773] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0774] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0775] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0776] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0777] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0778] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0779] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0780] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0781] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0782] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0783] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0784] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0785] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0786] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0787] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0788] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0789] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0790] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0791] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0792] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0793] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0794] The following is further disclosed regarding the embodiments described above.

[0795] (Claim 1)

[0796] A means for receiving input information from the user and generating and updating the user profile,

[0797] A generation method for generating various variations of advertisements based on advertising materials,

[0798] A method for selecting the most suitable advertisements based on user profiles and replacing them during broadcast,

[0799] A means of collecting viewing data and analyzing that data,

[0800] A means of providing reports to advertisers and broadcasters based on the analysis results,

[0801] A system that includes this.

[0802] (Claim 2)

[0803] The system according to claim 1, further comprising means for anonymizing user profiles and protecting privacy.

[0804] (Claim 3)

[0805] The system according to claim 1, further comprising means for replacing advertisements in real time using edge computing.

[0806] "Example 1"

[0807] (Claim 1)

[0808] A means for receiving and anonymizing user input information, and for generating and updating user profiles,

[0809] A means for generating various variations of advertisements based on advertising materials, comprising a means for using generation AI technology,

[0810] A method for selecting the most suitable advertisements based on user profiles and seamlessly replacing them during broadcast,

[0811] A means of quantifying advertising effectiveness by collecting and analyzing viewing data,

[0812] A means of providing reports to advertisers and media companies based on the analysis results,

[0813] A system that includes this.

[0814] (Claim 2)

[0815] The system according to claim 1, further comprising means for anonymizing user profiles and ensuring the protection of personal information.

[0816] (Claim 3)

[0817] The system according to claim 1, further comprising means for selecting and replacing advertisements in real time using edge computing.

[0818] "Application Example 1"

[0819] (Claim 1)

[0820] A means for receiving attribute information from users and generating and updating user models,

[0821] A generation method for generating various forms of advertisements based on advertising materials,

[0822] A method for selecting the most suitable advertisements based on the user model and dynamically replacing them during broadcast,

[0823] A means of collecting and analyzing viewing information,

[0824] A means of reporting the analysis results to advertisers and broadcasters,

[0825] A method using a generative AI model to generate prompt text for selecting advertisements based on user interests,

[0826] A system that includes this.

[0827] (Claim 2)

[0828] The system according to claim 1, further comprising means for anonymizing the user model and protecting personal information.

[0829] (Claim 3)

[0830] The system according to claim 1, further comprising means for changing advertisements in real time using edge computing.

[0831] "Example 2 of combining an emotion engine"

[0832] (Claim 1)

[0833] A means for receiving user input information and generating and updating personal profiles,

[0834] A generation method that combines emotional data collected from personal profiles with generative artificial intelligence to generate various variations of advertisements,

[0835] A method of selecting the most suitable advertisement based on sentiment analysis and replacing it within the video content,

[0836] A method for collecting viewing data using emotional data in conjunction with personal profiles, and for analyzing that data,

[0837] A means of evaluating the effectiveness of advertising based on the analysis results and providing reports to the provider and the telecommunications company,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, further comprising means for anonymizing personal profile and emotional data and protecting personal information.

[0841] (Claim 3)

[0842] The system according to claim 1, further comprising means for replacing advertisements in real time using an information terminal.

[0843] "Application example 2 when combining with an emotional engine"

[0844] (Claim 1)

[0845] A means for receiving input information from the user and generating and updating the user profile,

[0846] A generation method for generating advertisements with various forms of expression based on diverse media materials,

[0847] A means for analyzing the user's emotional state, selecting the most suitable advertisement based on the user profile and emotional data, and displaying it on a visual device,

[0848] A means for collecting emotional data and viewing data, and for analyzing that data,

[0849] A means of providing reports to advertisers and information distributors based on the analysis results,

[0850] A system that includes this.

[0851] (Claim 2)

[0852] The system according to claim 1, further comprising means for anonymizing user profiles and protecting personal information.

[0853] (Claim 3)

[0854] The system according to claim 1, further comprising means for displaying advertisements in real time through visual devices. [Explanation of Symbols]

[0855] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for receiving input information from the user and generating and updating the user profile, A generation method for generating various variations of advertisements based on advertising materials, A method for selecting the most suitable advertisements based on user profiles and replacing them during broadcast, A means of collecting viewing data and analyzing that data, A means of providing reports to advertisers and broadcasters based on the analysis results, A system that includes this.

2. The system according to claim 1, further comprising means for anonymizing user profiles and protecting privacy.

3. The system according to claim 1, further comprising means for replacing advertisements in real time using edge computing.

Images