System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting activity data of a user; means for analyzing the collected activity data to identify preferences, interests, and behavioral patterns of the user; means for generating advertising content based on the identified preferences of the user; and means for delivering the generated advertising content to a terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional advertising systems had problems in providing personalized ads for each user, and were unable to accurately reflect the user's interests and preferences. As a result, ad click-through rates and conversion rates were low, limiting the effectiveness for advertisers. In addition, insufficient analysis of collected data made it difficult to identify user behavior patterns and preferences. This made it impossible to select the optimal timing and content for ad delivery, leading to an increase in the number of annoying ads displayed to users. [Means for solving the problem]

[0005] The present invention provides a system for collecting user activity data and analyzing the data to identify user preferences, interests, and behavioral patterns. To achieve this, the present invention includes the following means.

[0006] 1. Provide a means to collect user activity data.

[0007] 2. Provide a means to analyze collected activity data and identify user preferences, interests, and behavioral patterns.

[0008] 3. Providing a means for generating advertising content based on the identified user preferences.

[0009] 4. Provide a means for delivering the generated advertising content to the user's terminal.

[0010] 5. Providing a means for receiving and displaying advertising content on the user's terminal and a means for recording the user's response to the advertisement.

[0011] 6. Provide a means for transmitting the recorded reaction data to a server.

[0012] These methods can effectively deliver ads that match users' preferences and interests, improving click-through rates and conversion rates. Furthermore, by removing noise from collected data and generating user profiles based on clean, integrated data, more accurate analysis and personalization become possible.

[0013] "User activity data" refers to data collected about the actions a user takes on the Internet, including, for example, web browsing history, purchase history, social media posting history, and search history.

[0014] "Data analysis" refers to the process of processing collected user activity data to identify user preferences, interests, and behavioral patterns.

[0015] "User Preferences" refers to the preferences or tastes a User has for particular products or services.

[0016] "User Interests" refers to the interest or concern a User has in a particular field or topic.

[0017] "Behavioral patterns" refer to the tendencies and habits of behavior that users repeatedly exhibit on the Internet.

[0018] "Advertising content" refers to information such as text, images, videos, banners, etc. of advertisements delivered to users.

[0019] A "terminal" is a physical device that a user uses to access the Internet, and specifically includes a smartphone, tablet, or PC.

[0020] "Response data" refers to data that records the user's response to an advertisement, and specifically includes information such as clicks, viewing time, and purchases.

[0021] "Server" refers to a computing device that processes and stores large amounts of data and provides various services over a network. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention relates to a system for effectively collecting and analyzing user activity data and generating and delivering customized advertisements based on the collected data. An embodiment of this system is described in detail below.

[0044] Server-side processing

[0045] Data collection

[0046] The server collects the user's web browsing history. Specifically, it acquires data such as the websites visited, keywords searched, and links clicked. It also obtains the user's purchase history from the database, and furthermore, it uses social media APIs to collect activity data such as the user's posts and "likes."

[0047] Data analysis

[0048] The server first cleans the collected data, removing noise and outliers. Next, it integrates data from different sources to generate a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms are used for analysis, revealing the content the user prefers and the times when they are most active.

[0049] Advertising content generation

[0050] The server uses AI to generate ad text optimized for the user's preferences based on the analysis results. For example, if a user is interested in camera features, the server generates ad text such as "Smartphones with the latest high-resolution cameras." Related images and videos are also generated at the same time.

[0051] Ad serving

[0052] The server calculates and sets the optimal time to deliver the generated ad. For example, if the user is most active after 8:00 PM, the server sets the ad to be delivered during that time. The ad is then sent to the user's device at the set time.

[0053] Terminal side processing

[0054] Advertisement Receipt and Display

[0055] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[0056] Obtaining user responses

[0057] When a user clicks on an ad, the device records that information, along with other details such as the time spent viewing the ad and where the user scrolled, which allows the device to determine how long the ad held the user's attention.

[0058] Sending reaction data

[0059] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[0060] User response

[0061] Ad viewing

[0062] Users are more likely to engage with ads because they are tailored to their interests. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[0063] Ad Actions

[0064] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[0065] Specific examples

[0066] For example, if a user is looking for a new smartphone, the server collects the user's past search history, purchase history, review viewing history, etc. and analyzes the data. The server determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera" based on that. The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server. By repeating this cycle, it is possible to provide a beneficial advertising experience for the user and maximize the effectiveness of the advertisement.

[0067] In this manner, the present invention, which provides a method for generating and delivering personalized advertisements based on user interests and preferences, can improve advertising click-through rates and conversion rates.

[0068] The processing flow will be explained below.

[0069] Step 1:

[0070] The server collects your web browsing history, specifically recording information such as the websites you visit, the keywords you search for, and the links you click.

[0071] Step 2:

[0072] The server retrieves the user's purchase history, which includes a list of past purchases, purchase dates, and purchase amounts.

[0073] Step 3:

[0074] The server uses social media APIs to collect user activity data such as posts and "likes."

[0075] Step 4:

[0076] The server cleans the collected data, specifically removing noise and outliers to improve the quality of the data.

[0077] Step 5:

[0078] The server aggregates the cleaned data to generate a single user profile, combining information from different data sources to create a comprehensive data set.

[0079] Step 6:

[0080] The server analyzes the combined data to identify user preferences, interests, and behavioral patterns, using machine learning algorithms.

[0081] Step 7:

[0082] The server uses generative AI to generate ad text based on user preferences, for example, creating an ad for a "smartphone with the latest high-resolution camera" for a user interested in camera features.

[0083] Step 8:

[0084] The server analyzes the user's most active times and calculates the optimal time to deliver ads.

[0085] Step 9:

[0086] The server transmits the generated advertisement to the user's terminal at the set time.

[0087] Step 10:

[0088] The terminal receives the advertisement sent from the server.

[0089] Step 11:

[0090] The device displays ads where appropriate, such as in the sidebar of a web page or in a banner within an app.

[0091] Step 12:

[0092] The user views the displayed advertisement.

[0093] Step 13:

[0094] When a user clicks on an ad, the device records the click information.

[0095] Step 14:

[0096] The terminal transmits the recorded user reaction data to the server.

[0097] Step 15:

[0098] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In today's internet advertising environment, many users are exposed to countless advertisements, yet providing effectively personalized advertisements to users is considered a challenge. Despite technological advances in recommendation engines and user profiling, many challenges remain in optimizing the accuracy of collected data and the timing of advertisement display. This results in a large number of advertisements that do not appeal to users, reducing effectiveness for advertisers. Furthermore, collecting user responses in real time and utilizing them to generate subsequent advertisements is extremely complex and requires advanced technological capabilities. Given this background, there is a need to establish a system that can effectively collect and analyze user activity data and deliver optimal advertisements tailored to users at the appropriate time.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes: means for collecting user activity data; means for cleaning the collected activity data and removing noise data and outliers; means for integrating the filtered data to generate a user profile; means for analyzing the user's preferences, interests, and behavioral patterns from the integrated data using a machine learning algorithm; means for generating advertising content using a generative AI model based on the analysis results; and means for setting the optimal timing for delivering the generated advertising content and delivering it to the user's device. This makes it possible to provide highly personalized advertisements to users at the appropriate time and maximize their effectiveness. In addition, by collecting user response data and utilizing it in generating and delivering the next advertisement, the accuracy and effectiveness of the advertisement can be continuously improved.

[0104] "User activity data" is information that records a user's online behavior, such as web browsing history, search keywords, links clicked, purchase history, and social media activity data.

[0105] "Cleaning" is the process of removing noise data and outliers from collected data and extracting only valid data.

[0106] "Data integration" is the process of combining data collected from different sources into one cohesive data set.

[0107] A "user profile" is information created based on integrated data that indicates a user's characteristics, such as preferences, interests, and behavioral patterns.

[0108] A "machine learning algorithm" is a computer program that analyzes collected data and finds patterns and trends.

[0109] A "generative AI model" is an artificial intelligence system that automatically generates useful advertising content such as text, images, and videos from large amounts of data.

[0110] "Advertising content" refers to information that is delivered to users and consists of advertising messages and related images, videos, etc.

[0111] The "optimal delivery time" is the time when users are predicted to be most active and interested in the ad.

[0112] "User terminal" refers to an electronic device, such as a smartphone, personal computer, or tablet, on which a user receives and views advertisements.

[0113] "Response data" refers to information about the actions a user takes in response to an advertisement, such as clicks, viewing time, and scroll position.

[0114] MODE FOR CARRYING OUT THE INVENTION

[0115] The present invention relates to a system that effectively collects and analyzes user activity data and generates and delivers customized advertisements based on the collected data. This system utilizes machine learning algorithms and generative AI models to provide highly personalized advertisements to users at optimal times.

[0116] Server-side processing

[0117] Data collection

[0118] When a user browses the web, the server collects their browsing history. Specifically, it obtains information such as the URLs of websites visited, keywords searched, and links clicked. It also collects the user's purchase history and social media activity data (e.g., posts and "likes"). This data is collected using web scripts, APIs (e.g., social media APIs), etc.

[0119] Data Cleaning

[0120] The server cleans the collected data, using Python scripts to remove noise and outliers, and only valid data is extracted and stored in the database.

[0121] Data integration and user profile generation

[0122] The server consolidates data from different sources and generates a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms (e.g., Scikit-Learn) are used to analyze the data and reveal the content the user prefers and the times of day when they are most active.

[0123] Advertising content generation

[0124] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile, using the following prompt:

[0125] "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history."

[0126] The generative AI model generates ad copy such as, "Get 20% off the latest smartphone with a high-resolution camera right now!", along with related images and videos.

[0127] Ad delivery optimization

[0128] The server identifies the time period when the user is most active and configures the settings to deliver advertisements during that time period. For example, if the user is most active after 8:00 PM, the server configures the settings to deliver advertisements during that time period. The advertisements are then sent to the user's device at the configured time.

[0129] Terminal side processing

[0130] Advertisement Receipt and Display

[0131] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time.

[0132] Collecting user response data

[0133] When a user clicks on an ad, that information is recorded by the device, along with other details such as the time spent viewing the ad and where they scrolled.

[0134] Sending reaction data to the server

[0135] The device sends the collected user response data to the server, which uses this data to optimize the next ad generation and delivery.

[0136] User response

[0137] Ad viewing

[0138] Users can click on ads that interest them to find out more information.

[0139] Purchase Action

[0140] When a user clicks on an ad, accesses a product information page, and purchases the product, this information is sent from the device to the server and used to generate and distribute the next ad.

[0141] Through these processes, it becomes possible to provide optimized advertisements to users and maximize their effectiveness. The present invention can significantly improve the click-through rate and engagement rate of advertisements.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] System program processing steps

[0144] Step 1: Data collection

[0145] Input: User's web browsing history, search keywords, click information, purchase history, social media activity data

[0146] Output: Collected activity data

[0147] When a user browses the web, the server collects their browsing history (URLs of websites visited, search keywords, and clicked links). It also retrieves the user's purchase history from a database and uses social media APIs to collect activity data such as user posts and "likes." For example, if a user searches for "the latest smartphone with a camera," the server saves the page information related to that keyword.

[0148] Step 2: Data cleaning

[0149] Input: Collected activity data

[0150] Output: Cleaned data (noise data, data with outliers removed)

[0151] The server cleans the collected data. This process uses Python scripts to remove noise and outliers. For example, if a user's search history contains obviously misspelled or meaningless keywords, they will be removed. This ensures that only valid data is extracted and stored in the database.

[0152] Step 3: Data integration and user profile generation

[0153] Input: Cleaned data

[0154] Output: User profile (information identifying user preferences, interests, and behavioral patterns)

[0155] The server then combines the cleaned data to generate a user profile. Specifically, it combines data from different sources (search keywords, visit history, social media activity, etc.) into a single dataset. It then uses machine learning algorithms (such as Scikit-Learn) to identify the content that the user is interested in and the times of day when they are most active. For example, if a user frequently visits the "camera smartphone" page, that interest will be reflected in the profile.

[0156] Step 4: Advertising content generation

[0157] Input: User profile

[0158] Output: Generated ad content

[0159] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile. For example, it uses a prompt such as, "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history." The generative AI model generates ad copy such as, "Get 20% off smartphones with the latest high-resolution cameras right now!" and also generates related images and videos as needed.

[0160] Step 5: Optimizing ad delivery

[0161] Input: Generated ad content, user profile (most active times)

[0162] Output: Ads that are set to be delivered

[0163] The server identifies the time periods when the user is most active and sets the time to deliver ads during those periods. For example, if analysis data shows that the user is most active after 8:00 PM, the server sets the time to deliver ads during that period. The generated advertising content is then sent to the user's device at the set time.

[0164] Step 6: Receiving and displaying advertisements

[0165] Input: Served ad content

[0166] Output: Served ad

[0167] The device receives the advertisements sent from the server and displays them while the user is browsing a news site or in an appropriate location within the app. For example, when a user is browsing a news site, an advertisement for a "smartphone with the latest high-resolution camera" appears in the sidebar.

[0168] Step 7: Collect user response data

[0169] Input: User response to the ad

[0170] Output: Collected reaction data

[0171] When a user clicks on an ad, the device records that information, along with other details such as how long the ad was displayed and where the user scrolled, which can tell how long the ad held the user's attention.

[0172] Step 8: Sending reaction data to the server

[0173] Input: Collected reaction data

[0174] Output: Response data sent to the server

[0175] The device sends the recorded user response data to a server, which uses this data to optimize the generation and distribution of the next ad. For example, the server can extract the characteristics of ad content with a high click rate and reflect them in the generation of the next ad, thereby providing a more effective ad.

[0176] Step 9: View the ad

[0177] Input: Displayed ad

[0178] Output: User's interest in the ad

[0179] Users view the ads displayed on their devices, and if they are interested, they can click on the ad to view more information, which increases their interest.

[0180] Step 10: Purchase Action

[0181] Input: Clicked Ad

[0182] Output: Purchase data

[0183] When a user clicks on an ad, accesses a page with detailed information, and purchases a product, this purchase data is sent from the device to a server and used to generate and deliver the next ad. The data on completed purchases is used to set up and generate the next ad, thereby continuously improving the accuracy and effectiveness of ads.

[0184] (Application example 1)

[0185] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0186] While technologies for providing personalized advertisements based on user activity data have existed for some time, they have not yet been developed to provide advertisements that take into account the user's visual experience and real-time behavior. This makes it difficult to display advertisements that are in line with the user's current interests in real time, making it difficult to maximize advertising effectiveness.

[0187] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0188] In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data to identify user preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, and means for capturing user visual data and displaying relevant advertisements based on specific objects or locations, thereby enabling the provision of personalized advertisements based on the user's real-time visual experience.

[0189] "User activity data" refers to any data generated by users during their web browsing, purchases, social media use, etc.

[0190] "Collection means" refers to the methods and functions for collecting user activity data and visual data.

[0191] "Means of analysis" refers to the methods and technologies used to analyze collected data and identify user preferences and behavioral patterns.

[0192] "Means for generating advertising content" refers to methods and technologies for creating personalized advertising based on a user's interests and preferences.

[0193] "Means of delivery" refers to the methods and technologies used to deliver the generated advertisement to the user's device.

[0194] "Visual data" refers to information about scenery and objects that a user sees through a device such as smart glasses.

[0195] "Means of capturing and displaying relevant ads based on specific objects or locations" refers to methods or technologies for identifying a user's visual focus and displaying relevant ads based on that information.

[0196] This invention is a system that collects and analyzes user activity data and generates and delivers advertisements based on the user's preferences, and proposes a form that can be applied particularly to displaying advertisements using smart glasses.

[0197] Server Processing

[0198] Data collection

[0199] The servers collect all activity data generated by users during web browsing, purchases, and social media use, including website visit history, search keywords, clicked links, purchase history, social media posts and "likes," etc. This data is collected using browser extensions and various APIs.

[0200] Data analysis

[0201] The server cleans the collected data, removing noise and outliers. It then integrates data from different sources to generate a user profile. It uses machine learning algorithms (e.g., Scikit-learn and TensorFlow) to identify user preferences and behavioral patterns. For example, if a user frequently searches for information about cameras, their preference will be classified as "interested in cameras."

[0202] Advertising content generation

[0203] The server uses a generative AI model (e.g., GPT-4) to generate ad text optimized for the user's preferences. For example, if a user is interested in camera features, the server generates ad text such as "A smartphone equipped with the latest high-resolution camera." Related images and videos are also generated using a generative AI model (e.g., DALL-E).

[0204] Terminal handling

[0205] Ad serving

[0206] The server then distributes the generated advertisements at the optimal time. For example, if a user is most active after 8:00 PM, the server can set the advertisements to be distributed during that time period. The advertisements are then sent to the smart glasses at the set times.

[0207] Advertisement display

[0208] The device (smart glasses) receives advertisements sent from the server and displays them in the user's field of view in real time. For example, if a user sees a sign for a new camera shop while walking down the street, the device captures that information and displays a relevant advertisement.

[0209] Obtaining user responses and sending data

[0210] When a user clicks on an ad, the device records the response data, including the time spent viewing the ad and the number of clicks. The device then sends this data to the server and uses it to generate and distribute the next ad.

[0211] Specific examples

[0212] For example, suppose a user is looking for a new smartphone. The server collects and analyzes the user's past search history, purchase history, review viewing history, etc. As a result, it determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera." The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server.

[0213] Prompt Sentence Examples

[0214] User Interest: High-resolution cameras

[0215] Generate ad text: Today only, get 20% off high-res cameras!

[0216] In this way, it is possible to generate and deliver personalized ads based on users' interests and preferences, improving ad click-through rates and conversion rates.

[0217] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0218] Step 1:

[0219] Data collection

[0220] The server collects activity data generated by users during web browsing, purchases, and social media use. Specifically, it uses browser extensions and APIs to obtain data such as websites visited, keywords searched, links clicked, purchase history, social media posts and "likes," etc. The input is activity data from web browsers, social media APIs, etc., and the output is a list of the collected activity data.

[0221] Step 2:

[0222] Data Cleaning

[0223] The server cleans the activity data collected in step 1. It uses data filtering techniques to remove noise data and outliers. The input is the activity data collected in step 1, and the output is the clean activity data.

[0224] Step 3:

[0225] Data Integration

[0226] The server integrates the activity data after data cleaning. It aggregates data from different sources and generates a user profile. The input is the cleaned data, and the output is the integrated data and the generated user profile.

[0227] Step 4:

[0228] Behavioral pattern analysis

[0229] The server uses machine learning algorithms (e.g., Scikit-learn or TensorFlow) to analyze user behavior patterns. For example, if a user frequently searches for information about "cameras," the user's preferences are analyzed as "interested in cameras." The input is the integrated data and user profile, and the output is the identified user preferences and behavior patterns.

[0230] Step 5:

[0231] Ad Generation

[0232] The server generates ad text based on user preferences using a generative AI model (e.g., GPT-4). Additionally, it generates related images and videos using a generative AI model (e.g., DALL-E). The input is user preferences and behavioral patterns, and the output is the generated ad text and images / videos.

[0233] Step 6:

[0234] Ad serving

[0235] The server calculates the optimal time to deliver the generated ad. For example, if the user is most active after 8 PM, it sets the ad to be sent during that time period. The input is the generated ad text, image / video, and user activity time data, and the output is the ad data to be delivered.

[0236] Step 7:

[0237] Advertisement display

[0238] The smart glasses receive advertisements sent from the server and display them in the user's field of view in real time. The input is the delivered advertisement data, and the output is the presented advertisement. For example, if a user is walking down the street and sees a sign for a particular store, an advertisement related to that store will be displayed.

[0239] Step 8:

[0240] Obtaining user responses

[0241] The device records the user's response to the ad, such as the time spent viewing the ad, scroll position, number of clicks, etc. The input is the ad being displayed and the user's response, and the output is the recorded response data.

[0242] Step 9:

[0243] Reaction data transmission

[0244] The device sends the recorded user response data to the server, which then generates and distributes the next ad. The input is the recorded response data, and the output is the response data sent to the server.

[0245] This series of steps makes it possible to provide personalized ads based on the user's visual experience, which is expected to improve ad click-through rates and conversion rates.

[0246] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0247] This invention combines an emotion engine with a system that collects and analyzes user activity data and generates and delivers personalized advertisements based on that data to further improve accuracy. This system recognizes user emotions and reflects them in the personalization of advertisements, thereby achieving more effective advertisement delivery.

[0248] Server-side processing

[0249] Data collection

[0250] The server collects activity data such as browsing history, purchase history, and social media posts when a user uses the Internet. In addition, an emotion engine is used to detect emotions from a user's facial photograph and voice data. For example, emotional information such as whether the user is happy, sad, or surprised can be obtained from data collected through a webcam or microphone.

[0251] Data analysis

[0252] The server first cleans the collected activity and emotion data, removing noise and outliers to improve data quality. It then integrates this data to generate a comprehensive user profile. Machine learning algorithms are used to analyze the data and identify not only user preferences, interests, and behavioral patterns, but also emotional trends.

[0253] Advertising content generation

[0254] The server uses generative AI based on the analysis results to generate ad text optimized for the user's preferences and emotions. For example, if the user is interested in camera functions and recent emotional data shows a high level of joy, the server generates ad text such as "Capture your precious moments with a smartphone equipped with the latest high-resolution camera." Related images and videos are also generated at the same time.

[0255] Ad serving

[0256] The server calculates and sets the optimal time to deliver the generated advertisement. For example, it analyzes that users are most active after 8:00 PM, and sets the advertisement to be delivered during that time. The advertisement is then sent to the user's device at the set time.

[0257] Terminal side processing

[0258] Advertisement Receipt and Display

[0259] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[0260] Obtaining user responses

[0261] When a user clicks on an ad, the device records that click, along with detailed data such as the time spent viewing the ad, scroll position, and emotional changes, allowing the device to determine how long the ad held the user's attention and how their emotions changed before and after viewing the ad.

[0262] Sending reaction data

[0263] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[0264] User response

[0265] Ad viewing

[0266] Users are more likely to be interested in ads because they are tailored to their preferences and emotions. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[0267] Ad Actions

[0268] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[0269] Specific examples

[0270] For example, suppose a user is looking for a new smartphone and has recently browsed many smartphone reviews. The server collects the user's activity data and also uses an emotion engine to obtain the user's emotion data from the webcam. Data analysis reveals that the user is very interested in the camera function and has recently detected a lot of happy emotion.

[0271] Based on this, the server generates an ad that says, "Capture your precious moments with a smartphone equipped with the latest high-resolution camera," and delivers it to the user's device at the optimal time. When a user clicks on an ad, accesses a detail page, and purchases a product, their response data is sent to the server and used to further personalize future ads.

[0272] This system makes it possible to effectively deliver advertisements that accurately reflect users' preferences and emotions, thereby improving ad click rates and conversion rates.

[0273] The processing flow will be explained below.

[0274] Step 1:

[0275] The server collects information about your web browsing history, such as the websites you visit, the keywords you search for, and the links you click.

[0276] Step 2:

[0277] The server retrieves the user's purchase history, which includes details such as the products the user has previously purchased, the purchase date, the purchase location, and the purchase amount.

[0278] Step 3:

[0279] The server collects activity data such as user posts, comments, and likes through social media APIs.

[0280] Step 4:

[0281] The server uses an emotion engine to detect emotions from the user's facial photos and voice data, including the ability to analyze the user's emotions in real time using a webcam and microphone.

[0282] Step 5:

[0283] The server cleans the collected data, specifically removing noise and outliers from the collected data to ensure data consistency.

[0284] Step 6:

[0285] The server aggregates the cleaned data to create a single, comprehensive user profile, combining information from different data sources to create a comprehensive data set.

[0286] Step 7:

[0287] The server analyzes the combined data to identify user preferences, interests, behavioral patterns, and emotional trends, and uses machine learning algorithms to refine the data.

[0288] Step 8:

[0289] The server uses generative AI to generate ad text based on the user's preferences and emotions. For example, if the user is interested in camera features and the emotion of joy is detected frequently, the server generates ad copy such as "Capture joyful moments with a smartphone equipped with the latest high-resolution camera."

[0290] Step 9:

[0291] The server calculates the optimal time to deliver ads and schedules them accordingly. For example, it may determine that users are most active after 8 PM, and deliver ads during that time.

[0292] Step 10:

[0293] The server sends the generated advertisement to the user's device at the set time.

[0294] Step 11:

[0295] The terminal receives the advertisement sent from the server.

[0296] Step 12:

[0297] The device will then display the received advertisement in an appropriate location, such as in the sidebar of a web page or as a banner within an app.

[0298] Step 13:

[0299] The user views the displayed advertisement.

[0300] Step 14:

[0301] When a user clicks on an ad, the device records that click, along with detailed data such as the amount of time spent viewing the ad, where they scrolled, and changes in their emotions before and after viewing the ad.

[0302] Step 15:

[0303] The terminal transmits the recorded user reaction data to the server.

[0304] Step 16:

[0305] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[0306] Example 2

[0307] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0308] Conventional ad delivery systems personalized ads based solely on user activity data, making it impossible to deliver ads that took into account the user's emotional state. As a result, user engagement declined and advertising effectiveness was inadequate. Furthermore, noise and outliers in the collected data reduced data accuracy, making it difficult to optimize ad delivery.

[0309] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for cleaning the user's activity data and emotion data and removing noise data and outliers, means for integrating the cleaned data and generating a user profile, means for analyzing the user's emotional state using the collected emotion data, means for generating advertising content using a generative AI model based on the analysis results, and means for delivering the generated advertising content to the user's terminal. This makes it possible to integrate the user's activity data and emotion data to generate a highly accurate user profile and deliver effective advertisements based on it.

[0310] "Activity data" is information about a user's online activities, such as browsing history, purchase history, and social media posts.

[0311] "Emotional data" is information that represents a user's emotional state, detected from a user's facial photograph and voice data collected through input devices such as a webcam and microphone.

[0312] "Cleaning" is the process of removing noise data and outliers from collected data, and is a process carried out to improve the quality of the data.

[0313] "Synthesis" is the process of combining cleaned activity data with emotion data to generate a single, comprehensive user profile.

[0314] A "user profile" is information that indicates a user's preferences, interests, behavioral patterns, and emotional tendencies, generated based on integrated activity data and emotional data.

[0315] An "emotion engine" is software or an algorithm that detects emotions from a user's facial photo and voice data and determines their state.

[0316] A "generative AI model" is an artificial intelligence model that generates content based on user preferences and emotions. Examples include language models that generate text and generative models that generate images.

[0317] "Advertising content" refers to media content such as advertising text, images, and videos delivered to users.

[0318] A "generative AI prompt" is a text input (prompt) that provides instructions to a generative AI model, and is a sentence that provides basic information for the AI ​​model to generate appropriate advertising content based on this.

[0319] "Response data" is data that records a user's reaction to an ad, such as click information, display time, scroll position, and emotional changes.

[0320] "Delivery" refers to the process of sending advertising content generated by the server to the user's terminal and displaying it.

[0321] This invention relates to a system that collects and analyzes user activity data and emotion data, and generates and delivers personalized advertisements based on the collected data. This system recognizes user emotions and reflects them in the personalized advertisements, thereby achieving more effective advertisement delivery.

[0322] System Configuration

[0323] Server Roles

[0324] The server does the following:

[0325] 1. Data Collection:

[0326] The server collects data about users' browsing history, purchase history, social media posts, and other activity data while they are online, for example, using browser history APIs and online store APIs.

[0327] The emotion engine also analyzes facial photos and audio data collected from the webcam and microphone to detect the user's emotions. For example, the Emotion API can be used to recognize emotions such as "happiness," "sadness," and "surprise."

[0328] 2. Data cleaning and integration:

[0329] The server cleans the collected data using Python's Pandas library and removes noise data and outliers.

[0330] The cleaned data is then combined to create a single, comprehensive user profile, which is then stored in a database such as MongoDB.

[0331] 3. User preference and sentiment analysis:

[0332] The server uses machine learning libraries (e.g., scikit-learn and TensorFlow) to analyze the user's preferences, interests, behavioral patterns, and emotional tendencies from the integrated data, for example, by using collaborative filtering algorithms to predict the user's future behavior.

[0333] 4. Advertising content generation:

[0334] Using a generative AI model (e.g., OpenAI's GPT-4), advertising content is generated based on the analysis results. The generated advertising content includes text, images, and videos optimized for user preferences and emotions.

[0335] An example of a prompt sentence that is generated is "Capture your precious moments with a smartphone equipped with the latest high-resolution camera."

[0336] 5. Advertisement Delivery:

[0337] The server calculates and sets the optimal timing for ad delivery based on the analysis data. For example, it analyzes the time periods when users are most active and sets the time periods to deliver ads.

[0338] Advertisements are sent to users' devices at set times using the Google Ads API or other ad serving APIs.

[0339] Device Role

[0340] The terminal does the following:

[0341] 1. Receiving and Displaying Advertisements:

[0342] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate times, for example, using JavaScript or HTML5.

[0343] 2. Obtaining user responses:

[0344] When a user clicks on an ad, the device records the click information and collects detailed data such as the display time, scroll position, and emotional changes, which allows for detailed analysis of the effectiveness of the ad.

[0345] 3. Reaction data transmission:

[0346] The device converts the collected user response data into JSON format and sends it to the server, which then obtains data that will be useful for generating and delivering the next ad.

[0347] User Roles

[0348] The user performs the following actions:

[0349] 1. Viewing Ads:

[0350] For example, if a user is interested in smartphones, they may see an advertisement for a new model equipped with a high-resolution camera.

[0351] 2. Advertising Actions:

[0352] When a user clicks on an advertisement that interests them, they are taken to a page with detailed product information, where they can then purchase the product. Once the purchase is complete, the data is sent from the device to a server, which will be used to deliver advertisements to the user in the future.

[0353] Through the above process, the system can integrate user activity data and emotion data to generate highly accurate user profiles, and then deliver effective advertisements based on those profiles. Specific examples of prompts include:

[0354] "Based on recent user sentiment and activity data, generate ad text using the following information:

[0355] Users are very interested in the camera function

[0356] The user's emotional state is delight

[0357] The prompt given to the generative AI model: 'Take photos of your precious moments with your smartphone, equipped with the latest high-resolution camera.'

[0358] This system makes it possible to generate and deliver advertisements that accurately match users' preferences and emotions, thereby improving ad click rates and conversion rates.

[0359] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0360] Step 1: Data collection

[0361] Input: Your browsing history, purchase history, social media posts, webcam and microphone data.

[0362] How it works: The server calls the browser history API or online store API to retrieve the user's browsing history and purchase history. At the same time, it uses an emotion engine (e.g., Emotion API) to analyze the video and audio data collected through the webcam and microphone, and detects the user's emotional state (happiness, sadness, surprise, etc.) from their facial image and audio data.

[0363] Output: User activity and emotion data is generated and stored in a database (e.g., MongoDB) on the server.

[0364] Step 2: Data cleaning and integration

[0365] Input: Collected activity and emotion data.

[0366] How it works: The server uses the Python Pandas library to clean the collected activity and emotion data, specifically removing noise and outliers to improve data quality, and then integrates the cleaned data to generate user profiles.

[0367] Output: A clean and consolidated user profile is generated and stored in the database.

[0368] Step 3: Analyze user preferences and sentiment

[0369] Input: The cleaned user profile.

[0370] How it works: The server uses scikit-learn and TensorFlow to analyze the cleaned and integrated data to determine user preferences, interests, behavioral patterns, and emotional trends, for example by using collaborative filtering algorithms to predict future user behavior and preferences.

[0371] Output: The analysis produces detailed data about user preferences and sentiment.

[0372] Step 4: Advertising content generation

[0373] Input: Analysis results on user preferences and sentiment.

[0374] Specific operation: The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate advertising content based on the analysis results. Specifically, it inputs the prompt "Capture precious moments with a smartphone equipped with the latest high-resolution camera" and runs the generative AI model to generate optimized advertising text. It then uses an image generation model such as DALL-E to create images and videos corresponding to the generated text.

[0375] Output: Personalized ad content (text, images, video) is generated.

[0376] Step 5: Ad delivery settings and sending

[0377] Input: Personalized advertising content and user activity data.

[0378] How it works: The server calculates the optimal timing for ad delivery based on analytical data. For example, it analyzes that users are most active after 8:00 PM, and sets the ad delivery time to be during that time. Specifically, it uses the Google Ads API or Facebook Marketing API to send the ad to the user's device at the set time.

[0379] Output: The ad is sent to the user's device.

[0380] Step 6: Receiving and displaying advertisements

[0381] Input: Ad content sent by the server.

[0382] How it works: The device uses scripts such as JavaScript to receive ads sent from the server and display them on web pages or within apps at the appropriate times. For example, an ad might be displayed in the sidebar while the user is browsing a news site.

[0383] Output: The ad is displayed on the user's device.

[0384] Step 7: Get user responses

[0385] Input: User behavior data when the ad is displayed.

[0386] How it works: When a user clicks on an ad, the device records the click information. At the same time, detailed response data such as the time spent viewing the ad, scroll position, and emotional changes are collected. This is done using log files on the device and browser event listeners.

[0387] Output: Collected user response data.

[0388] Step 8: Reaction data submission and analysis

[0389] Input: Collected user response data.

[0390] How it works: The device converts the collected response data into JSON format and sends it to the server, which then analyzes the response data to obtain information that will be useful for generating and delivering future ads.

[0391] Output: The analyzed response data is used to generate the next ad.

[0392] This series of processing steps realizes a system that can integrate user activity data and emotional data and effectively deliver highly accurate personalized advertisements based on that data.

[0393] (Application example 2)

[0394] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0395] While conventional ad delivery systems have performed personalization based on user preferences and behavioral patterns, it has been difficult to achieve highly personalized advertising that reflects the user's emotional state. In particular, by taking into account not only the user's summer interests but also their emotional state at that moment, it is expected that ad engagement rates and conversion rates will be further improved.

[0396] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data and identifying the user's preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, means for acquiring user emotion data, and means for personalizing the advertising content based on the acquired emotion data. This makes it possible to personalize advertisements that reflect the user's emotional state.

[0397] "User activity data" refers to information such as browsing history, purchase history, and social media posts made while a user is using the Internet.

[0398] "Means for analyzing collected activity data" means means for analyzing collected user activity data and using technologies and algorithms to identify user preferences, interests, and behavioral patterns.

[0399] "Means for generating advertising content based on user preferences" refers to technology that automatically creates advertising messages and visuals that correspond to a user's preferences and interests.

[0400] "Means for delivering generated advertising content to a user's device" means a method or system for sending personalized advertising to a device used by a user.

[0401] "Means of acquiring user emotional data" refers to technology that recognizes and detects emotions in real time from the user's facial expressions, voice, etc.

[0402] "Means for personalizing advertising content based on acquired emotional data" means a method or system that adjusts the content and timing of advertising based on the emotional state of a user.

[0403] "Means for receiving and displaying advertising content on a user's device" refers to the technology or method for displaying advertising messages on a user's device.

[0404] "Means for recording users' responses to advertisements" refers to technology that tracks users' responses, such as clicking on advertisements, and stores them as data.

[0405] "Means for transmitting recorded reaction data to a server" refers to the method for identifying a user's reaction information and transmitting it to a server for storage and analysis.

[0406] "Video and audio capture means" refers to devices such as cameras and microphones used to collect user emotion data.

[0407] "Means for generating a user profile integrated with emotional data" refers to techniques and methods for creating a comprehensive user profile that includes emotional information.

[0408] The present invention is a system for generating and delivering personalized advertisements using user activity data and emotion data. This system is composed of a means for collecting and analyzing users' internet browsing history, purchase history, and social media postings, and an emotion engine for recognizing users' emotions.

[0409] First, the server collects user activity data, such as internet browser history, purchase history on online shopping sites, social media posts, etc. This is typically done using cookies and tracking pixels.

[0410] The server then analyzes the collected activity data using the following software:

[0411] Python: General data processing

[0412] Pandas: Data Frame Operations

[0413] scikit-learn: machine learning algorithms

[0414] The collected data is first cleaned to filter out noise and outliers, and then the filtered data is integrated to generate a comprehensive user profile, which analyzes the user's preferences, interests, and behavioral patterns.

[0415] In parallel, the server collects emotional data by analyzing the video and audio data collected through the user's webcam and microphone with an emotion recognition module (e.g., EmotionRecognizer), which detects the user's emotional state in real time, such as whether they are happy, sad, or surprised.

[0416] Based on the analysis results, the server uses a generative AI model to generate advertising content. For example, if a user has recently visited a camera review site and "joy" is detected from their facial expression, the server will generate an advertisement such as "Buy a smartphone with the latest high-resolution camera."

[0417] The server then delivers the generated ad to the user's device at the optimal time. The timing of ad delivery is determined based on the user's active time period analyzed from their activity data. For example, if it is known that the user is most active after 8:00 PM, the ad will be delivered during that time period.

[0418] The device receives the advertisement sent from the server and displays it on the web page or within the app at the appropriate time. After the advertisement is displayed, the user's response (clicks, viewing time, changes in emotion, etc.) is recorded again by the device and sent to the server. This response data is used to generate and deliver advertisements from the next time onwards, achieving more accurate personalization.

[0419] (Example)

[0420] For example, if a user has recently visited many camera review sites and their facial expression shows "joy," the server can generate an ad with the message "Capture your precious moments with a smartphone equipped with the latest high-resolution camera" based on this information, and deliver the ad to them after 8:00 PM, based on the analysis results that show that users are most active at that time.

[0421] (Example of a generative AI model prompt)

[0422] A user has recently been browsing many camera review sites and the "Happy" facial expression is detected from their webcam. Generate the best ad text for the user.

[0423] Example of generated ad:

[0424] Capture your precious moments with a smartphone equipped with the latest high-resolution camera.

[0425] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0426] Step 1: Data collection

[0427] The server collects the user's internet browser history, purchase history on online shopping sites, and social media posts. It receives the user's browsing data, purchase data, and social media data as input and stores this data in a database on the server. Cookies and tracking pixels are used to collect the data.

[0428] Step 2: Collecting Emotional Data

[0429] The server captures video and audio data from the user's webcam and microphone, and analyzes it with an emotion recognition module. It receives the user's facial photo and audio data as input, and analyzes them with the emotion recognition module. The output is an emotion label, such as "happiness" or "sadness." This emotion data is also stored on the server.

[0430] Step 3: Data cleaning

[0431] The server cleans the collected activity and emotion data, removing noise and outliers. It receives various activity and emotion data as input and cleans the data using data processing libraries such as Pand and scikit-learn. The output is a clean dataset.

[0432] Step 4: Create a user profile

[0433] The server generates a user profile based on the cleaned data set. It takes the cleaned activity data and emotion data as input and integrates them to identify the user's preferences, interests, behavioral patterns, and emotional tendencies. The output is a comprehensive user profile.

[0434] Step 5: Advertising content generation

[0435] The server generates advertising content using a generative AI model based on the user profile. Using the generated user profile as input, the server sends an advertising generation request in the form of a prompt sentence to the generative AI model. The output is personalized advertising text and associated visual content.

[0436] Step 6: Ad serving

[0437] The server delivers the generated advertising content to the user's device at the optimal time. It receives personalized advertising content and the optimal delivery time analyzed from user activity data as input and determines the ad delivery schedule. As output, the advertisement is sent to the user's device and displayed at a specific time.

[0438] Step 7: Receiving and displaying advertisements

[0439] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time. It receives the advertisement content from the server as input and displays the advertisement at the time when it predicts that the user will be most interested in the advertisement.

[0440] Step 8: Recording User Responses

[0441] The device records the user's reactions, such as clicking on an ad, and sends the data to a server. As input, it collects reaction data such as user clicks, the time spent viewing the ad, and changes in emotions, and sends it to the server as output for storage.

[0442] Step 9: Utilizing reaction data

[0443] The server uses the collected response data to generate and deliver the next ad. It receives and analyzes user response data as input and optimizes the ad to improve engagement and conversion rates.

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0447] [Second embodiment]

[0448] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0449] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0454] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0455] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0456] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0459] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0460] The present invention relates to a system for effectively collecting and analyzing user activity data and generating and delivering customized advertisements based on the collected data. An embodiment of this system is described in detail below.

[0461] Server-side processing

[0462] Data collection

[0463] The server collects the user's web browsing history. Specifically, it acquires data such as the websites visited, keywords searched, and links clicked. It also obtains the user's purchase history from the database, and furthermore, it uses social media APIs to collect activity data such as the user's posts and "likes."

[0464] Data analysis

[0465] The server first cleans the collected data, removing noise and outliers. Next, it integrates data from different sources to generate a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms are used for analysis, revealing the content the user prefers and the times when they are most active.

[0466] Advertising content generation

[0467] The server uses AI to generate ad text optimized for the user's preferences based on the analysis results. For example, if a user is interested in camera features, the server generates ad text such as "Smartphones with the latest high-resolution cameras." Related images and videos are also generated at the same time.

[0468] Ad serving

[0469] The server calculates and sets the optimal time to deliver the generated ad. For example, if the user is most active after 8:00 PM, the server sets the ad to be delivered during that time. The ad is then sent to the user's device at the set time.

[0470] Terminal side processing

[0471] Advertisement Receipt and Display

[0472] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[0473] Obtaining user responses

[0474] When a user clicks on an ad, the device records that information, along with other details such as the time spent viewing the ad and where the user scrolled, which allows the device to determine how long the ad held the user's attention.

[0475] Sending reaction data

[0476] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[0477] User response

[0478] Ad viewing

[0479] Users are more likely to engage with ads because they are tailored to their interests. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[0480] Ad Actions

[0481] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[0482] Specific examples

[0483] For example, if a user is looking for a new smartphone, the server collects the user's past search history, purchase history, review viewing history, etc. and analyzes the data. The server determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera" based on that. The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server. By repeating this cycle, it is possible to provide a beneficial advertising experience for the user and maximize the effectiveness of the advertisement.

[0484] In this manner, the present invention, which provides a method for generating and delivering personalized advertisements based on user interests and preferences, can improve advertising click-through rates and conversion rates.

[0485] The processing flow will be explained below.

[0486] Step 1:

[0487] The server collects your web browsing history, specifically recording information such as the websites you visit, the keywords you search for, and the links you click.

[0488] Step 2:

[0489] The server retrieves the user's purchase history, which includes a list of past purchases, purchase dates, and purchase amounts.

[0490] Step 3:

[0491] The server uses social media APIs to collect user activity data such as posts and "likes."

[0492] Step 4:

[0493] The server cleans the collected data, specifically removing noise and outliers to improve the quality of the data.

[0494] Step 5:

[0495] The server aggregates the cleaned data to generate a single user profile, combining information from different data sources to create a comprehensive data set.

[0496] Step 6:

[0497] The server analyzes the combined data to identify user preferences, interests, and behavioral patterns, using machine learning algorithms.

[0498] Step 7:

[0499] The server uses generative AI to generate ad text based on user preferences, for example, creating an ad for a "smartphone with the latest high-resolution camera" for a user interested in camera features.

[0500] Step 8:

[0501] The server analyzes the user's most active times and calculates the optimal time to deliver ads.

[0502] Step 9:

[0503] The server transmits the generated advertisement to the user's terminal at the set time.

[0504] Step 10:

[0505] The terminal receives the advertisement sent from the server.

[0506] Step 11:

[0507] The device displays ads where appropriate, such as in the sidebar of a web page or in a banner within an app.

[0508] Step 12:

[0509] The user views the displayed advertisement.

[0510] Step 13:

[0511] When a user clicks on an ad, the device records the click information.

[0512] Step 14:

[0513] The terminal transmits the recorded user reaction data to the server.

[0514] Step 15:

[0515] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[0516] Example 1

[0517] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0518] In today's internet advertising environment, many users are exposed to countless advertisements, yet providing effectively personalized advertisements to users is considered a challenge. Despite technological advances in recommendation engines and user profiling, many challenges remain in optimizing the accuracy of collected data and the timing of advertisement display. This results in a large number of advertisements that do not appeal to users, reducing effectiveness for advertisers. Furthermore, collecting user responses in real time and utilizing them to generate subsequent advertisements is extremely complex and requires advanced technological capabilities. Given this background, there is a need to establish a system that can effectively collect and analyze user activity data and deliver optimal advertisements tailored to users at the appropriate time.

[0519] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0520] In this invention, the server includes: means for collecting user activity data; means for cleaning the collected activity data and removing noise data and outliers; means for integrating the filtered data to generate a user profile; means for analyzing the user's preferences, interests, and behavioral patterns from the integrated data using a machine learning algorithm; means for generating advertising content using a generative AI model based on the analysis results; and means for setting the optimal timing for delivering the generated advertising content and delivering it to the user's device. This makes it possible to provide highly personalized advertisements to users at the appropriate time and maximize their effectiveness. In addition, by collecting user response data and utilizing it in generating and delivering the next advertisement, the accuracy and effectiveness of the advertisement can be continuously improved.

[0521] "User activity data" is information that records a user's online behavior, such as web browsing history, search keywords, links clicked, purchase history, and social media activity data.

[0522] "Cleaning" is the process of removing noise data and outliers from collected data and extracting only valid data.

[0523] "Data integration" is the process of combining data collected from different sources into one cohesive data set.

[0524] A "user profile" is information created based on integrated data that indicates a user's characteristics, such as preferences, interests, and behavioral patterns.

[0525] A "machine learning algorithm" is a computer program that analyzes collected data and finds patterns and trends.

[0526] A "generative AI model" is an artificial intelligence system that automatically generates useful advertising content such as text, images, and videos from large amounts of data.

[0527] "Advertising content" refers to information that is delivered to users and consists of advertising messages and related images, videos, etc.

[0528] The "optimal delivery time" is the time when users are predicted to be most active and interested in the ad.

[0529] "User terminal" refers to an electronic device, such as a smartphone, personal computer, or tablet, on which a user receives and views advertisements.

[0530] "Response data" refers to information about the actions a user takes in response to an advertisement, such as clicks, viewing time, and scroll position.

[0531] MODE FOR CARRYING OUT THE INVENTION

[0532] The present invention relates to a system that effectively collects and analyzes user activity data and generates and delivers customized advertisements based on the collected data. This system utilizes machine learning algorithms and generative AI models to provide highly personalized advertisements to users at optimal times.

[0533] Server-side processing

[0534] Data collection

[0535] When a user browses the web, the server collects their browsing history. Specifically, it obtains information such as the URLs of websites visited, keywords searched, and links clicked. It also collects the user's purchase history and social media activity data (e.g., posts and "likes"). This data is collected using web scripts, APIs (e.g., social media APIs), etc.

[0536] Data Cleaning

[0537] The server cleans the collected data, using Python scripts to remove noise and outliers, and only valid data is extracted and stored in the database.

[0538] Data integration and user profile generation

[0539] The server consolidates data from different sources and generates a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms (e.g., Scikit-Learn) are used to analyze the data and reveal the content the user prefers and the times of day when they are most active.

[0540] Advertising content generation

[0541] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile, using the following prompt:

[0542] "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history."

[0543] The generative AI model generates ad copy such as, "Get 20% off the latest smartphone with a high-resolution camera right now!", along with related images and videos.

[0544] Ad delivery optimization

[0545] The server identifies the time period when the user is most active and configures the settings to deliver advertisements during that time period. For example, if the user is most active after 8:00 PM, the server configures the settings to deliver advertisements during that time period. The advertisements are then sent to the user's device at the configured time.

[0546] Terminal side processing

[0547] Advertisement Receipt and Display

[0548] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time.

[0549] Collecting user response data

[0550] When a user clicks on an ad, that information is recorded by the device, along with other details such as the time spent viewing the ad and where they scrolled.

[0551] Sending reaction data to the server

[0552] The device sends the collected user response data to the server, which uses this data to optimize the next ad generation and delivery.

[0553] User response

[0554] Ad viewing

[0555] Users can click on ads that interest them to find out more information.

[0556] Purchase Action

[0557] When a user clicks on an ad, accesses a product information page, and purchases the product, this information is sent from the device to the server and used to generate and distribute the next ad.

[0558] Through these processes, it becomes possible to provide optimized advertisements to users and maximize their effectiveness. The present invention can significantly improve the click-through rate and engagement rate of advertisements.

[0559] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0560] System program processing steps

[0561] Step 1: Data collection

[0562] Input: User's web browsing history, search keywords, click information, purchase history, social media activity data

[0563] Output: Collected activity data

[0564] When a user browses the web, the server collects their browsing history (URLs of websites visited, search keywords, and clicked links). It also retrieves the user's purchase history from a database and uses social media APIs to collect activity data such as user posts and "likes." For example, if a user searches for "the latest smartphone with a camera," the server saves the page information related to that keyword.

[0565] Step 2: Data cleaning

[0566] Input: Collected activity data

[0567] Output: Cleaned data (noise data, data with outliers removed)

[0568] The server cleans the collected data. This process uses Python scripts to remove noise and outliers. For example, if a user's search history contains obviously misspelled or meaningless keywords, they will be removed. This ensures that only valid data is extracted and stored in the database.

[0569] Step 3: Data integration and user profile generation

[0570] Input: Cleaned data

[0571] Output: User profile (information identifying user preferences, interests, and behavioral patterns)

[0572] The server then combines the cleaned data to generate a user profile. Specifically, it combines data from different sources (search keywords, visit history, social media activity, etc.) into a single dataset. It then uses machine learning algorithms (such as Scikit-Learn) to identify the content that the user is interested in and the times of day when they are most active. For example, if a user frequently visits the "camera smartphone" page, that interest will be reflected in the profile.

[0573] Step 4: Advertising content generation

[0574] Input: User profile

[0575] Output: Generated ad content

[0576] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile. For example, it uses a prompt such as, "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history." The generative AI model generates ad copy such as, "Get 20% off smartphones with the latest high-resolution cameras right now!" and also generates related images and videos as needed.

[0577] Step 5: Optimizing ad delivery

[0578] Input: Generated ad content, user profile (most active times)

[0579] Output: Ads that are set to be delivered

[0580] The server identifies the time periods when the user is most active and sets the time to deliver ads during those periods. For example, if analysis data shows that the user is most active after 8:00 PM, the server sets the time to deliver ads during that period. The generated advertising content is then sent to the user's device at the set time.

[0581] Step 6: Receiving and displaying advertisements

[0582] Input: Served ad content

[0583] Output: Served ad

[0584] The device receives the advertisements sent from the server and displays them while the user is browsing a news site or in an appropriate location within the app. For example, when a user is browsing a news site, an advertisement for a "smartphone with the latest high-resolution camera" appears in the sidebar.

[0585] Step 7: Collect user response data

[0586] Input: User response to the ad

[0587] Output: Collected reaction data

[0588] When a user clicks on an ad, the device records that information, along with other details such as how long the ad was displayed and where the user scrolled, which can tell how long the ad held the user's attention.

[0589] Step 8: Sending reaction data to the server

[0590] Input: Collected reaction data

[0591] Output: Response data sent to the server

[0592] The device sends the recorded user response data to a server, which uses this data to optimize the generation and distribution of the next ad. For example, the server can extract the characteristics of ad content with a high click rate and reflect them in the generation of the next ad, thereby providing a more effective ad.

[0593] Step 9: View the ad

[0594] Input: Displayed ad

[0595] Output: User's interest in the ad

[0596] Users view the ads displayed on their devices, and if they are interested, they can click on the ad to view more information, which increases their interest.

[0597] Step 10: Purchase Action

[0598] Input: Clicked Ad

[0599] Output: Purchase data

[0600] When a user clicks on an ad, accesses a page with detailed information, and purchases a product, this purchase data is sent from the device to a server and used to generate and deliver the next ad. The data on completed purchases is used to set up and generate the next ad, thereby continuously improving the accuracy and effectiveness of ads.

[0601] (Application example 1)

[0602] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0603] While technologies for providing personalized advertisements based on user activity data have existed for some time, they have not yet been developed to provide advertisements that take into account the user's visual experience and real-time behavior. This makes it difficult to display advertisements that are in line with the user's current interests in real time, making it difficult to maximize advertising effectiveness.

[0604] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0605] In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data to identify user preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, and means for capturing user visual data and displaying relevant advertisements based on specific objects or locations, thereby enabling the provision of personalized advertisements based on the user's real-time visual experience.

[0606] "User activity data" refers to any data generated by users during their web browsing, purchases, social media use, etc.

[0607] "Collection means" refers to the methods and functions for collecting user activity data and visual data.

[0608] "Means of analysis" refers to the methods and technologies used to analyze collected data and identify user preferences and behavioral patterns.

[0609] "Means for generating advertising content" refers to methods and technologies for creating personalized advertising based on a user's interests and preferences.

[0610] "Means of delivery" refers to the methods and technologies used to deliver the generated advertisement to the user's device.

[0611] "Visual data" refers to information about scenery and objects that a user sees through a device such as smart glasses.

[0612] "Means of capturing and displaying relevant ads based on specific objects or locations" refers to methods or technologies for identifying a user's visual focus and displaying relevant ads based on that information.

[0613] This invention is a system that collects and analyzes user activity data and generates and delivers advertisements based on the user's preferences, and proposes a form that can be applied particularly to displaying advertisements using smart glasses.

[0614] Server Processing

[0615] Data collection

[0616] The servers collect all activity data generated by users during web browsing, purchases, and social media use, including website visit history, search keywords, clicked links, purchase history, social media posts and "likes," etc. This data is collected using browser extensions and various APIs.

[0617] Data analysis

[0618] The server cleans the collected data, removing noise and outliers. It then integrates data from different sources to generate a user profile. It uses machine learning algorithms (e.g., Scikit-learn and TensorFlow) to identify user preferences and behavioral patterns. For example, if a user frequently searches for information about cameras, their preference will be classified as "interested in cameras."

[0619] Advertising content generation

[0620] The server uses a generative AI model (e.g., GPT-4) to generate ad text optimized for the user's preferences. For example, if a user is interested in camera features, the server generates ad text such as "A smartphone equipped with the latest high-resolution camera." Related images and videos are also generated using a generative AI model (e.g., DALL-E).

[0621] Terminal handling

[0622] Ad serving

[0623] The server then distributes the generated advertisements at the optimal time. For example, if a user is most active after 8:00 PM, the server can set the advertisements to be distributed during that time period. The advertisements are then sent to the smart glasses at the set times.

[0624] Advertisement display

[0625] The device (smart glasses) receives advertisements sent from the server and displays them in the user's field of view in real time. For example, if a user sees a sign for a new camera shop while walking down the street, the device captures that information and displays a relevant advertisement.

[0626] Obtaining user responses and sending data

[0627] When a user clicks on an ad, the device records the response data, including the time spent viewing the ad and the number of clicks. The device then sends this data to the server and uses it to generate and distribute the next ad.

[0628] Specific examples

[0629] For example, suppose a user is looking for a new smartphone. The server collects and analyzes the user's past search history, purchase history, review viewing history, etc. As a result, it determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera." The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server.

[0630] Prompt Sentence Examples

[0631] User Interest: High-resolution cameras

[0632] Generate ad text: Today only, get 20% off high-res cameras!

[0633] In this way, it is possible to generate and deliver personalized ads based on users' interests and preferences, improving ad click-through rates and conversion rates.

[0634] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0635] Step 1:

[0636] Data collection

[0637] The server collects activity data generated by users during web browsing, purchases, and social media use. Specifically, it uses browser extensions and APIs to obtain data such as websites visited, keywords searched, links clicked, purchase history, social media posts and "likes," etc. The input is activity data from web browsers, social media APIs, etc., and the output is a list of the collected activity data.

[0638] Step 2:

[0639] Data Cleaning

[0640] The server cleans the activity data collected in step 1. It uses data filtering techniques to remove noise data and outliers. The input is the activity data collected in step 1, and the output is the clean activity data.

[0641] Step 3:

[0642] Data Integration

[0643] The server integrates the activity data after data cleaning. It aggregates data from different sources and generates a user profile. The input is the cleaned data, and the output is the integrated data and the generated user profile.

[0644] Step 4:

[0645] Behavioral pattern analysis

[0646] The server uses machine learning algorithms (e.g., Scikit-learn or TensorFlow) to analyze user behavior patterns. For example, if a user frequently searches for information about "cameras," the user's preferences are analyzed as "interested in cameras." The input is the integrated data and user profile, and the output is the identified user preferences and behavior patterns.

[0647] Step 5:

[0648] Ad Generation

[0649] The server generates ad text based on user preferences using a generative AI model (e.g., GPT-4). Additionally, it generates related images and videos using a generative AI model (e.g., DALL-E). The input is user preferences and behavioral patterns, and the output is the generated ad text and images / videos.

[0650] Step 6:

[0651] Ad serving

[0652] The server calculates the optimal time to deliver the generated ad. For example, if the user is most active after 8 PM, it sets the ad to be sent during that time period. The input is the generated ad text, image / video, and user activity time data, and the output is the ad data to be delivered.

[0653] Step 7:

[0654] Advertisement display

[0655] The smart glasses receive advertisements sent from the server and display them in the user's field of view in real time. The input is the delivered advertisement data, and the output is the presented advertisement. For example, if a user is walking down the street and sees a sign for a particular store, an advertisement related to that store will be displayed.

[0656] Step 8:

[0657] Obtaining user responses

[0658] The device records the user's response to the ad, such as the time spent viewing the ad, scroll position, number of clicks, etc. The input is the ad being displayed and the user's response, and the output is the recorded response data.

[0659] Step 9:

[0660] Reaction data transmission

[0661] The device sends the recorded user response data to the server, which then generates and distributes the next ad. The input is the recorded response data, and the output is the response data sent to the server.

[0662] This series of steps makes it possible to provide personalized ads based on the user's visual experience, which is expected to improve ad click-through rates and conversion rates.

[0663] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0664] This invention combines an emotion engine with a system that collects and analyzes user activity data and generates and delivers personalized advertisements based on that data to further improve accuracy. This system recognizes user emotions and reflects them in the personalization of advertisements, thereby achieving more effective advertisement delivery.

[0665] Server-side processing

[0666] Data collection

[0667] The server collects activity data such as browsing history, purchase history, and social media posts when a user uses the Internet. In addition, an emotion engine is used to detect emotions from a user's facial photograph and voice data. For example, emotional information such as whether the user is happy, sad, or surprised can be obtained from data collected through a webcam or microphone.

[0668] Data analysis

[0669] The server first cleans the collected activity and emotion data, removing noise and outliers to improve data quality. It then integrates this data to generate a comprehensive user profile. Machine learning algorithms are used to analyze the data and identify not only user preferences, interests, and behavioral patterns, but also emotional trends.

[0670] Advertising content generation

[0671] The server uses generative AI based on the analysis results to generate ad text optimized for the user's preferences and emotions. For example, if the user is interested in camera functions and recent emotional data shows a high level of joy, the server generates ad text such as "Capture your precious moments with a smartphone equipped with the latest high-resolution camera." Related images and videos are also generated at the same time.

[0672] Ad serving

[0673] The server calculates and sets the optimal time to deliver the generated advertisement. For example, it analyzes that users are most active after 8:00 PM, and sets the advertisement to be delivered during that time. The advertisement is then sent to the user's device at the set time.

[0674] Terminal side processing

[0675] Advertisement Receipt and Display

[0676] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[0677] Obtaining user responses

[0678] When a user clicks on an ad, the device records that click, along with detailed data such as the time spent viewing the ad, scroll position, and emotional changes, allowing the device to determine how long the ad held the user's attention and how their emotions changed before and after viewing the ad.

[0679] Sending reaction data

[0680] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[0681] User response

[0682] Ad viewing

[0683] Users are more likely to be interested in ads because they are tailored to their preferences and emotions. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[0684] Ad Actions

[0685] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[0686] Specific examples

[0687] For example, suppose a user is looking for a new smartphone and has recently browsed many smartphone reviews. The server collects the user's activity data and also uses an emotion engine to obtain the user's emotion data from the webcam. Data analysis reveals that the user is very interested in the camera function and has recently detected a lot of happy emotion.

[0688] Based on this, the server generates an ad that says, "Capture your precious moments with a smartphone equipped with the latest high-resolution camera," and delivers it to the user's device at the optimal time. When a user clicks on an ad, accesses a detail page, and purchases a product, their response data is sent to the server and used to further personalize future ads.

[0689] This system makes it possible to effectively deliver advertisements that accurately reflect users' preferences and emotions, thereby improving ad click rates and conversion rates.

[0690] The processing flow will be explained below.

[0691] Step 1:

[0692] The server collects information about your web browsing history, such as the websites you visit, the keywords you search for, and the links you click.

[0693] Step 2:

[0694] The server retrieves the user's purchase history, which includes details such as the products the user has previously purchased, the purchase date, the purchase location, and the purchase amount.

[0695] Step 3:

[0696] The server collects activity data such as user posts, comments, and likes through social media APIs.

[0697] Step 4:

[0698] The server uses an emotion engine to detect emotions from the user's facial photos and voice data, including the ability to analyze the user's emotions in real time using a webcam and microphone.

[0699] Step 5:

[0700] The server cleans the collected data, specifically removing noise and outliers from the collected data to ensure data consistency.

[0701] Step 6:

[0702] The server aggregates the cleaned data to create a single, comprehensive user profile, combining information from different data sources to create a comprehensive data set.

[0703] Step 7:

[0704] The server analyzes the combined data to identify user preferences, interests, behavioral patterns, and emotional trends, and uses machine learning algorithms to refine the data.

[0705] Step 8:

[0706] The server uses generative AI to generate ad text based on the user's preferences and emotions. For example, if the user is interested in camera features and the emotion of joy is detected frequently, the server generates ad copy such as "Capture joyful moments with a smartphone equipped with the latest high-resolution camera."

[0707] Step 9:

[0708] The server calculates the optimal time to deliver ads and schedules them accordingly. For example, it may determine that users are most active after 8 PM, and deliver ads during that time.

[0709] Step 10:

[0710] The server sends the generated advertisement to the user's device at the set time.

[0711] Step 11:

[0712] The terminal receives the advertisement sent from the server.

[0713] Step 12:

[0714] The device will then display the received advertisement in an appropriate location, such as in the sidebar of a web page or as a banner within an app.

[0715] Step 13:

[0716] The user views the displayed advertisement.

[0717] Step 14:

[0718] When a user clicks on an ad, the device records that click, along with detailed data such as the amount of time spent viewing the ad, where they scrolled, and changes in their emotions before and after viewing the ad.

[0719] Step 15:

[0720] The terminal transmits the recorded user reaction data to the server.

[0721] Step 16:

[0722] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[0723] Example 2

[0724] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0725] Conventional ad delivery systems personalized ads based solely on user activity data, making it impossible to deliver ads that took into account the user's emotional state. As a result, user engagement declined and advertising effectiveness was inadequate. Furthermore, noise and outliers in the collected data reduced data accuracy, making it difficult to optimize ad delivery.

[0726] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for cleaning the user's activity data and emotion data and removing noise data and outliers, means for integrating the cleaned data and generating a user profile, means for analyzing the user's emotional state using the collected emotion data, means for generating advertising content using a generative AI model based on the analysis results, and means for delivering the generated advertising content to the user's terminal. This makes it possible to integrate the user's activity data and emotion data to generate a highly accurate user profile and deliver effective advertisements based on it.

[0727] "Activity data" is information about a user's online activities, such as browsing history, purchase history, and social media posts.

[0728] "Emotional data" is information that represents a user's emotional state, detected from a user's facial photograph and voice data collected through input devices such as a webcam and microphone.

[0729] "Cleaning" is the process of removing noise data and outliers from collected data, and is a process carried out to improve the quality of the data.

[0730] "Synthesis" is the process of combining cleaned activity data with emotion data to generate a single, comprehensive user profile.

[0731] A "user profile" is information that indicates a user's preferences, interests, behavioral patterns, and emotional tendencies, generated based on integrated activity data and emotional data.

[0732] An "emotion engine" is software or an algorithm that detects emotions from a user's facial photo and voice data and determines their state.

[0733] A "generative AI model" is an artificial intelligence model that generates content based on user preferences and emotions. Examples include language models that generate text and generative models that generate images.

[0734] "Advertising content" refers to media content such as advertising text, images, and videos delivered to users.

[0735] A "generative AI prompt" is a text input (prompt) that provides instructions to a generative AI model, and is a sentence that provides basic information for the AI ​​model to generate appropriate advertising content based on this.

[0736] "Response data" is data that records a user's reaction to an ad, such as click information, display time, scroll position, and emotional changes.

[0737] "Delivery" refers to the process of sending advertising content generated by the server to the user's terminal and displaying it.

[0738] This invention relates to a system that collects and analyzes user activity data and emotion data, and generates and delivers personalized advertisements based on the collected data. This system recognizes user emotions and reflects them in the personalized advertisements, thereby achieving more effective advertisement delivery.

[0739] System Configuration

[0740] Server Roles

[0741] The server does the following:

[0742] 1. Data Collection:

[0743] The server collects data about users' browsing history, purchase history, social media posts, and other activity data while they are online, for example, using browser history APIs and online store APIs.

[0744] The emotion engine also analyzes facial photos and audio data collected from the webcam and microphone to detect the user's emotions. For example, the Emotion API can be used to recognize emotions such as "happiness," "sadness," and "surprise."

[0745] 2. Data cleaning and integration:

[0746] The server cleans the collected data using Python's Pandas library and removes noise data and outliers.

[0747] The cleaned data is then combined to create a single, comprehensive user profile, which is then stored in a database such as MongoDB.

[0748] 3. User preference and sentiment analysis:

[0749] The server uses machine learning libraries (e.g., scikit-learn and TensorFlow) to analyze the user's preferences, interests, behavioral patterns, and emotional tendencies from the integrated data, for example, by using collaborative filtering algorithms to predict the user's future behavior.

[0750] 4. Advertising content generation:

[0751] Using a generative AI model (e.g., OpenAI's GPT-4), advertising content is generated based on the analysis results. The generated advertising content includes text, images, and videos optimized for user preferences and emotions.

[0752] An example of a prompt sentence that is generated is "Capture your precious moments with a smartphone equipped with the latest high-resolution camera."

[0753] 5. Advertisement Delivery:

[0754] The server calculates and sets the optimal timing for ad delivery based on the analysis data. For example, it analyzes the time periods when users are most active and sets the time periods to deliver ads.

[0755] Advertisements are sent to users' devices at set times using the Google Ads API or other ad serving APIs.

[0756] Device Role

[0757] The terminal does the following:

[0758] 1. Receiving and Displaying Advertisements:

[0759] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate times, for example, using JavaScript or HTML5.

[0760] 2. Obtaining user responses:

[0761] When a user clicks on an ad, the device records the click information and collects detailed data such as the display time, scroll position, and emotional changes, which allows for detailed analysis of the effectiveness of the ad.

[0762] 3. Reaction data transmission:

[0763] The device converts the collected user response data into JSON format and sends it to the server, which then obtains data that will be useful for generating and delivering the next ad.

[0764] User Roles

[0765] The user performs the following actions:

[0766] 1. Viewing Ads:

[0767] For example, if a user is interested in smartphones, they may see an advertisement for a new model equipped with a high-resolution camera.

[0768] 2. Advertising Actions:

[0769] When a user clicks on an advertisement that interests them, they are taken to a page with detailed product information, where they can then purchase the product. Once the purchase is complete, the data is sent from the device to a server, which will be used to deliver advertisements to the user in the future.

[0770] Through the above process, the system can integrate user activity data and emotion data to generate highly accurate user profiles, and then deliver effective advertisements based on those profiles. Specific examples of prompts include:

[0771] "Based on recent user sentiment and activity data, generate ad text using the following information:

[0772] Users are very interested in the camera function

[0773] The user's emotional state is delight

[0774] The prompt given to the generative AI model: 'Take photos of your precious moments with your smartphone, equipped with the latest high-resolution camera.'

[0775] This system makes it possible to generate and deliver advertisements that accurately match users' preferences and emotions, thereby improving ad click rates and conversion rates.

[0776] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0777] Step 1: Data collection

[0778] Input: Your browsing history, purchase history, social media posts, webcam and microphone data.

[0779] How it works: The server calls the browser history API or online store API to retrieve the user's browsing history and purchase history. At the same time, it uses an emotion engine (e.g., Emotion API) to analyze the video and audio data collected through the webcam and microphone, and detects the user's emotional state (happiness, sadness, surprise, etc.) from their facial image and audio data.

[0780] Output: User activity and emotion data is generated and stored in a database (e.g., MongoDB) on the server.

[0781] Step 2: Data cleaning and integration

[0782] Input: Collected activity and emotion data.

[0783] How it works: The server uses the Python Pandas library to clean the collected activity and emotion data, specifically removing noise and outliers to improve data quality, and then integrates the cleaned data to generate user profiles.

[0784] Output: A clean and consolidated user profile is generated and stored in the database.

[0785] Step 3: Analyze user preferences and sentiment

[0786] Input: The cleaned user profile.

[0787] How it works: The server uses scikit-learn and TensorFlow to analyze the cleaned and integrated data to determine user preferences, interests, behavioral patterns, and emotional trends, for example by using collaborative filtering algorithms to predict future user behavior and preferences.

[0788] Output: The analysis produces detailed data about user preferences and sentiment.

[0789] Step 4: Advertising content generation

[0790] Input: Analysis results on user preferences and sentiment.

[0791] Specific operation: The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate advertising content based on the analysis results. Specifically, it inputs the prompt "Capture precious moments with a smartphone equipped with the latest high-resolution camera" and runs the generative AI model to generate optimized advertising text. It then uses an image generation model such as DALL-E to create images and videos corresponding to the generated text.

[0792] Output: Personalized ad content (text, images, video) is generated.

[0793] Step 5: Ad delivery settings and sending

[0794] Input: Personalized advertising content and user activity data.

[0795] How it works: The server calculates the optimal timing for ad delivery based on analytical data. For example, it analyzes that users are most active after 8:00 PM, and sets the ad delivery time to be during that time. Specifically, it uses the Google Ads API or Facebook Marketing API to send the ad to the user's device at the set time.

[0796] Output: The ad is sent to the user's device.

[0797] Step 6: Receiving and displaying advertisements

[0798] Input: Ad content sent by the server.

[0799] How it works: The device uses scripts such as JavaScript to receive ads sent from the server and display them on web pages or within apps at the appropriate times. For example, an ad might be displayed in the sidebar while the user is browsing a news site.

[0800] Output: The ad is displayed on the user's device.

[0801] Step 7: Get user responses

[0802] Input: User behavior data when the ad is displayed.

[0803] How it works: When a user clicks on an ad, the device records the click information. At the same time, detailed response data such as the time spent viewing the ad, scroll position, and emotional changes are collected. This is done using log files on the device and browser event listeners.

[0804] Output: Collected user response data.

[0805] Step 8: Reaction data submission and analysis

[0806] Input: Collected user response data.

[0807] How it works: The device converts the collected response data into JSON format and sends it to the server, which then analyzes the response data to obtain information that will be useful for generating and delivering future ads.

[0808] Output: The analyzed response data is used to generate the next ad.

[0809] This series of processing steps realizes a system that can integrate user activity data and emotional data and effectively deliver highly accurate personalized advertisements based on that data.

[0810] (Application example 2)

[0811] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0812] While conventional ad delivery systems have performed personalization based on user preferences and behavioral patterns, it has been difficult to achieve highly personalized advertising that reflects the user's emotional state. In particular, by taking into account not only the user's summer interests but also their emotional state at that moment, it is expected that ad engagement rates and conversion rates will be further improved.

[0813] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data and identifying the user's preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, means for acquiring user emotion data, and means for personalizing the advertising content based on the acquired emotion data. This makes it possible to personalize advertisements that reflect the user's emotional state.

[0814] "User activity data" refers to information such as browsing history, purchase history, and social media posts made while a user is using the Internet.

[0815] "Means for analyzing collected activity data" means means for analyzing collected user activity data and using technologies and algorithms to identify user preferences, interests, and behavioral patterns.

[0816] "Means for generating advertising content based on user preferences" refers to technology that automatically creates advertising messages and visuals that correspond to a user's preferences and interests.

[0817] "Means for delivering generated advertising content to a user's device" means a method or system for sending personalized advertising to a device used by a user.

[0818] "Means of acquiring user emotional data" refers to technology that recognizes and detects emotions in real time from the user's facial expressions, voice, etc.

[0819] "Means for personalizing advertising content based on acquired emotional data" means a method or system that adjusts the content and timing of advertising based on the emotional state of a user.

[0820] "Means for receiving and displaying advertising content on a user's device" refers to the technology or method for displaying advertising messages on a user's device.

[0821] "Means for recording users' responses to advertisements" refers to technology that tracks users' responses, such as clicking on advertisements, and stores them as data.

[0822] "Means for transmitting recorded reaction data to a server" refers to the method for identifying a user's reaction information and transmitting it to a server for storage and analysis.

[0823] "Video and audio capture means" refers to devices such as cameras and microphones used to collect user emotion data.

[0824] "Means for generating a user profile integrated with emotional data" refers to techniques and methods for creating a comprehensive user profile that includes emotional information.

[0825] The present invention is a system for generating and delivering personalized advertisements using user activity data and emotion data. This system is composed of a means for collecting and analyzing users' internet browsing history, purchase history, and social media postings, and an emotion engine for recognizing users' emotions.

[0826] First, the server collects user activity data, such as internet browser history, purchase history on online shopping sites, social media posts, etc. This is typically done using cookies and tracking pixels.

[0827] The server then analyzes the collected activity data using the following software:

[0828] Python: General data processing

[0829] Pandas: Data Frame Operations

[0830] scikit-learn: machine learning algorithms

[0831] The collected data is first cleaned to filter out noise and outliers, and then the filtered data is integrated to generate a comprehensive user profile, which analyzes the user's preferences, interests, and behavioral patterns.

[0832] In parallel, the server collects emotional data by analyzing the video and audio data collected through the user's webcam and microphone with an emotion recognition module (e.g., EmotionRecognizer), which detects the user's emotional state in real time, such as whether they are happy, sad, or surprised.

[0833] Based on the analysis results, the server uses a generative AI model to generate advertising content. For example, if a user has recently visited a camera review site and "joy" is detected from their facial expression, the server will generate an advertisement such as "Buy a smartphone with the latest high-resolution camera."

[0834] The server then delivers the generated ad to the user's device at the optimal time. The timing of ad delivery is determined based on the user's active time period analyzed from their activity data. For example, if it is known that the user is most active after 8:00 PM, the ad will be delivered during that time period.

[0835] The device receives the advertisement sent from the server and displays it on the web page or within the app at the appropriate time. After the advertisement is displayed, the user's response (clicks, viewing time, changes in emotion, etc.) is recorded again by the device and sent to the server. This response data is used to generate and deliver advertisements from the next time onwards, achieving more accurate personalization.

[0836] (Example)

[0837] For example, if a user has recently visited many camera review sites and their facial expression shows "joy," the server can generate an ad with the message "Capture your precious moments with a smartphone equipped with the latest high-resolution camera" based on this information, and deliver the ad to them after 8:00 PM, based on the analysis results that show that users are most active at that time.

[0838] (Example of a generative AI model prompt)

[0839] A user has recently been browsing many camera review sites and the "Happy" facial expression is detected from their webcam. Generate the best ad text for the user.

[0840] Example of generated ad:

[0841] Capture your precious moments with a smartphone equipped with the latest high-resolution camera.

[0842] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0843] Step 1: Data collection

[0844] The server collects the user's internet browser history, purchase history on online shopping sites, and social media posts. It receives the user's browsing data, purchase data, and social media data as input and stores this data in a database on the server. Cookies and tracking pixels are used to collect the data.

[0845] Step 2: Collecting Emotional Data

[0846] The server captures video and audio data from the user's webcam and microphone, and analyzes it with an emotion recognition module. It receives the user's facial photo and audio data as input, and analyzes them with the emotion recognition module. The output is an emotion label, such as "happiness" or "sadness." This emotion data is also stored on the server.

[0847] Step 3: Data cleaning

[0848] The server cleans the collected activity and emotion data, removing noise and outliers. It receives various activity and emotion data as input and cleans the data using data processing libraries such as Pand and scikit-learn. The output is a clean dataset.

[0849] Step 4: Create a user profile

[0850] The server generates a user profile based on the cleaned data set. It takes the cleaned activity data and emotion data as input and integrates them to identify the user's preferences, interests, behavioral patterns, and emotional tendencies. The output is a comprehensive user profile.

[0851] Step 5: Advertising content generation

[0852] The server generates advertising content using a generative AI model based on the user profile. Using the generated user profile as input, the server sends an advertising generation request in the form of a prompt sentence to the generative AI model. The output is personalized advertising text and associated visual content.

[0853] Step 6: Ad serving

[0854] The server delivers the generated advertising content to the user's device at the optimal time. It receives personalized advertising content and the optimal delivery time analyzed from user activity data as input and determines the ad delivery schedule. As output, the advertisement is sent to the user's device and displayed at a specific time.

[0855] Step 7: Receiving and displaying advertisements

[0856] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time. It receives the advertisement content from the server as input and displays the advertisement at the time when it predicts that the user will be most interested in the advertisement.

[0857] Step 8: Recording User Responses

[0858] The device records the user's reactions, such as clicking on an ad, and sends the data to a server. As input, it collects reaction data such as user clicks, the time spent viewing the ad, and changes in emotions, and sends it to the server as output for storage.

[0859] Step 9: Utilizing reaction data

[0860] The server uses the collected response data to generate and deliver the next ad. It receives and analyzes user response data as input and optimizes the ad to improve engagement and conversion rates.

[0861] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0862] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0863] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0864] [Third embodiment]

[0865] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0866] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0867] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0868] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0869] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0870] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0871] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0872] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0873] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0874] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0875] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0876] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0877] The present invention relates to a system for effectively collecting and analyzing user activity data and generating and delivering customized advertisements based on the collected data. An embodiment of this system is described in detail below.

[0878] Server-side processing

[0879] Data collection

[0880] The server collects the user's web browsing history. Specifically, it acquires data such as the websites visited, keywords searched, and links clicked. It also obtains the user's purchase history from the database, and furthermore, it uses social media APIs to collect activity data such as the user's posts and "likes."

[0881] Data analysis

[0882] The server first cleans the collected data, removing noise and outliers. Next, it integrates data from different sources to generate a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms are used for analysis, revealing the content the user prefers and the times when they are most active.

[0883] Advertising content generation

[0884] The server uses AI to generate ad text optimized for the user's preferences based on the analysis results. For example, if a user is interested in camera features, the server generates ad text such as "Smartphones with the latest high-resolution cameras." Related images and videos are also generated at the same time.

[0885] Ad serving

[0886] The server calculates and sets the optimal time to deliver the generated ad. For example, if the user is most active after 8:00 PM, the server sets the ad to be delivered during that time. The ad is then sent to the user's device at the set time.

[0887] Terminal side processing

[0888] Advertisement Receipt and Display

[0889] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[0890] Obtaining user responses

[0891] When a user clicks on an ad, the device records that information, along with other details such as the time spent viewing the ad and where the user scrolled, which allows the device to determine how long the ad held the user's attention.

[0892] Sending reaction data

[0893] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[0894] User response

[0895] Ad viewing

[0896] Users are more likely to engage with ads because they are tailored to their interests. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[0897] Ad Actions

[0898] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[0899] Specific examples

[0900] For example, if a user is looking for a new smartphone, the server collects the user's past search history, purchase history, review viewing history, etc. and analyzes the data. The server determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera" based on that. The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server. By repeating this cycle, it is possible to provide a beneficial advertising experience for the user and maximize the effectiveness of the advertisement.

[0901] In this manner, the present invention, which provides a method for generating and delivering personalized advertisements based on user interests and preferences, can improve advertising click-through rates and conversion rates.

[0902] The processing flow will be explained below.

[0903] Step 1:

[0904] The server collects your web browsing history, specifically recording information such as the websites you visit, the keywords you search for, and the links you click.

[0905] Step 2:

[0906] The server retrieves the user's purchase history, which includes a list of past purchases, purchase dates, and purchase amounts.

[0907] Step 3:

[0908] The server uses social media APIs to collect user activity data such as posts and "likes."

[0909] Step 4:

[0910] The server cleans the collected data, specifically removing noise and outliers to improve the quality of the data.

[0911] Step 5:

[0912] The server aggregates the cleaned data to generate a single user profile, combining information from different data sources to create a comprehensive data set.

[0913] Step 6:

[0914] The server analyzes the combined data to identify user preferences, interests, and behavioral patterns, using machine learning algorithms.

[0915] Step 7:

[0916] The server uses generative AI to generate ad text based on user preferences, for example, creating an ad for a "smartphone with the latest high-resolution camera" for a user interested in camera features.

[0917] Step 8:

[0918] The server analyzes the user's most active times and calculates the optimal time to deliver ads.

[0919] Step 9:

[0920] The server transmits the generated advertisement to the user's terminal at the set time.

[0921] Step 10:

[0922] The terminal receives the advertisement sent from the server.

[0923] Step 11:

[0924] The device displays ads where appropriate, such as in the sidebar of a web page or in a banner within an app.

[0925] Step 12:

[0926] The user views the displayed advertisement.

[0927] Step 13:

[0928] When a user clicks on an ad, the device records the click information.

[0929] Step 14:

[0930] The terminal transmits the recorded user reaction data to the server.

[0931] Step 15:

[0932] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[0933] Example 1

[0934] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0935] In today's internet advertising environment, many users are exposed to countless advertisements, yet providing effectively personalized advertisements to users is considered a challenge. Despite technological advances in recommendation engines and user profiling, many challenges remain in optimizing the accuracy of collected data and the timing of advertisement display. This results in a large number of advertisements that do not appeal to users, reducing effectiveness for advertisers. Furthermore, collecting user responses in real time and utilizing them to generate subsequent advertisements is extremely complex and requires advanced technological capabilities. Given this background, there is a need to establish a system that can effectively collect and analyze user activity data and deliver optimal advertisements tailored to users at the appropriate time.

[0936] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0937] In this invention, the server includes: means for collecting user activity data; means for cleaning the collected activity data and removing noise data and outliers; means for integrating the filtered data to generate a user profile; means for analyzing the user's preferences, interests, and behavioral patterns from the integrated data using a machine learning algorithm; means for generating advertising content using a generative AI model based on the analysis results; and means for setting the optimal timing for delivering the generated advertising content and delivering it to the user's device. This makes it possible to provide highly personalized advertisements to users at the appropriate time and maximize their effectiveness. In addition, by collecting user response data and utilizing it in generating and delivering the next advertisement, the accuracy and effectiveness of the advertisement can be continuously improved.

[0938] "User activity data" is information that records a user's online behavior, such as web browsing history, search keywords, links clicked, purchase history, and social media activity data.

[0939] "Cleaning" is the process of removing noise data and outliers from collected data and extracting only valid data.

[0940] "Data integration" is the process of combining data collected from different sources into one cohesive data set.

[0941] A "user profile" is information created based on integrated data that indicates a user's characteristics, such as preferences, interests, and behavioral patterns.

[0942] A "machine learning algorithm" is a computer program that analyzes collected data and finds patterns and trends.

[0943] A "generative AI model" is an artificial intelligence system that automatically generates useful advertising content such as text, images, and videos from large amounts of data.

[0944] "Advertising content" refers to information that is delivered to users and consists of advertising messages and related images, videos, etc.

[0945] The "optimal delivery time" is the time when users are predicted to be most active and interested in the ad.

[0946] "User terminal" refers to an electronic device, such as a smartphone, personal computer, or tablet, on which a user receives and views advertisements.

[0947] "Response data" refers to information about the actions a user takes in response to an advertisement, such as clicks, viewing time, and scroll position.

[0948] MODE FOR CARRYING OUT THE INVENTION

[0949] The present invention relates to a system that effectively collects and analyzes user activity data and generates and delivers customized advertisements based on the collected data. This system utilizes machine learning algorithms and generative AI models to provide highly personalized advertisements to users at optimal times.

[0950] Server-side processing

[0951] Data collection

[0952] When a user browses the web, the server collects their browsing history. Specifically, it obtains information such as the URLs of websites visited, keywords searched, and links clicked. It also collects the user's purchase history and social media activity data (e.g., posts and "likes"). This data is collected using web scripts, APIs (e.g., social media APIs), etc.

[0953] Data Cleaning

[0954] The server cleans the collected data, using Python scripts to remove noise and outliers, and only valid data is extracted and stored in the database.

[0955] Data integration and user profile generation

[0956] The server consolidates data from different sources and generates a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms (e.g., Scikit-Learn) are used to analyze the data and reveal the content the user prefers and the times of day when they are most active.

[0957] Advertising content generation

[0958] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile, using the following prompt:

[0959] "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history."

[0960] The generative AI model generates ad copy such as, "Get 20% off the latest smartphone with a high-resolution camera right now!", along with related images and videos.

[0961] Ad delivery optimization

[0962] The server identifies the time period when the user is most active and configures the settings to deliver advertisements during that time period. For example, if the user is most active after 8:00 PM, the server configures the settings to deliver advertisements during that time period. The advertisements are then sent to the user's device at the configured time.

[0963] Terminal side processing

[0964] Advertisement Receipt and Display

[0965] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time.

[0966] Collecting user response data

[0967] When a user clicks on an ad, that information is recorded by the device, along with other details such as the time spent viewing the ad and where they scrolled.

[0968] Sending reaction data to the server

[0969] The device sends the collected user response data to the server, which uses this data to optimize the next ad generation and delivery.

[0970] User response

[0971] Ad viewing

[0972] Users can click on ads that interest them to find out more information.

[0973] Purchase Action

[0974] When a user clicks on an ad, accesses a product information page, and purchases the product, this information is sent from the device to the server and used to generate and distribute the next ad.

[0975] Through these processes, it becomes possible to provide optimized advertisements to users and maximize their effectiveness. The present invention can significantly improve the click-through rate and engagement rate of advertisements.

[0976] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0977] System program processing steps

[0978] Step 1: Data collection

[0979] Input: User's web browsing history, search keywords, click information, purchase history, social media activity data

[0980] Output: Collected activity data

[0981] When a user browses the web, the server collects their browsing history (URLs of websites visited, search keywords, and clicked links). It also retrieves the user's purchase history from a database and uses social media APIs to collect activity data such as user posts and "likes." For example, if a user searches for "the latest smartphone with a camera," the server saves the page information related to that keyword.

[0982] Step 2: Data cleaning

[0983] Input: Collected activity data

[0984] Output: Cleaned data (noise data, data with outliers removed)

[0985] The server cleans the collected data. This process uses Python scripts to remove noise and outliers. For example, if a user's search history contains obviously misspelled or meaningless keywords, they will be removed. This ensures that only valid data is extracted and stored in the database.

[0986] Step 3: Data integration and user profile generation

[0987] Input: Cleaned data

[0988] Output: User profile (information identifying user preferences, interests, and behavioral patterns)

[0989] The server then combines the cleaned data to generate a user profile. Specifically, it combines data from different sources (search keywords, visit history, social media activity, etc.) into a single dataset. It then uses machine learning algorithms (such as Scikit-Learn) to identify the content that the user is interested in and the times of day when they are most active. For example, if a user frequently visits the "camera smartphone" page, that interest will be reflected in the profile.

[0990] Step 4: Advertising content generation

[0991] Input: User profile

[0992] Output: Generated ad content

[0993] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile. For example, it uses a prompt such as, "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history." The generative AI model generates ad copy such as, "Get 20% off smartphones with the latest high-resolution cameras right now!" and also generates related images and videos as needed.

[0994] Step 5: Optimizing ad delivery

[0995] Input: Generated ad content, user profile (most active times)

[0996] Output: Ads that are set to be delivered

[0997] The server identifies the time periods when the user is most active and sets the time to deliver ads during those periods. For example, if analysis data shows that the user is most active after 8:00 PM, the server sets the time to deliver ads during that period. The generated advertising content is then sent to the user's device at the set time.

[0998] Step 6: Receiving and displaying advertisements

[0999] Input: Served ad content

[1000] Output: Served ad

[1001] The device receives the advertisements sent from the server and displays them while the user is browsing a news site or in an appropriate location within the app. For example, when a user is browsing a news site, an advertisement for a "smartphone with the latest high-resolution camera" appears in the sidebar.

[1002] Step 7: Collect user response data

[1003] Input: User response to the ad

[1004] Output: Collected reaction data

[1005] When a user clicks on an ad, the device records that information, along with other details such as how long the ad was displayed and where the user scrolled, which can tell how long the ad held the user's attention.

[1006] Step 8: Sending reaction data to the server

[1007] Input: Collected reaction data

[1008] Output: Response data sent to the server

[1009] The device sends the recorded user response data to a server, which uses this data to optimize the generation and distribution of the next ad. For example, the server can extract the characteristics of ad content with a high click rate and reflect them in the generation of the next ad, thereby providing a more effective ad.

[1010] Step 9: View the ad

[1011] Input: Displayed ad

[1012] Output: User's interest in the ad

[1013] Users view the ads displayed on their devices, and if they are interested, they can click on the ad to view more information, which increases their interest.

[1014] Step 10: Purchase Action

[1015] Input: Clicked Ad

[1016] Output: Purchase data

[1017] When a user clicks on an ad, accesses a page with detailed information, and purchases a product, this purchase data is sent from the device to a server and used to generate and deliver the next ad. The data on completed purchases is used to set up and generate the next ad, thereby continuously improving the accuracy and effectiveness of ads.

[1018] (Application example 1)

[1019] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1020] While technologies for providing personalized advertisements based on user activity data have existed for some time, they have not yet been developed to provide advertisements that take into account the user's visual experience and real-time behavior. This makes it difficult to display advertisements that are in line with the user's current interests in real time, making it difficult to maximize advertising effectiveness.

[1021] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1022] In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data to identify user preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, and means for capturing user visual data and displaying relevant advertisements based on specific objects or locations, thereby enabling the provision of personalized advertisements based on the user's real-time visual experience.

[1023] "User activity data" refers to any data generated by users during their web browsing, purchases, social media use, etc.

[1024] "Collection means" refers to the methods and functions for collecting user activity data and visual data.

[1025] "Means of analysis" refers to the methods and technologies used to analyze collected data and identify user preferences and behavioral patterns.

[1026] "Means for generating advertising content" refers to methods and technologies for creating personalized advertising based on a user's interests and preferences.

[1027] "Means of delivery" refers to the methods and technologies used to deliver the generated advertisement to the user's device.

[1028] "Visual data" refers to information about scenery and objects that a user sees through a device such as smart glasses.

[1029] "Means of capturing and displaying relevant ads based on specific objects or locations" refers to methods or technologies for identifying a user's visual focus and displaying relevant ads based on that information.

[1030] This invention is a system that collects and analyzes user activity data and generates and delivers advertisements based on the user's preferences, and proposes a form that can be applied particularly to displaying advertisements using smart glasses.

[1031] Server Processing

[1032] Data collection

[1033] The servers collect all activity data generated by users during web browsing, purchases, and social media use, including website visit history, search keywords, clicked links, purchase history, social media posts and "likes," etc. This data is collected using browser extensions and various APIs.

[1034] Data analysis

[1035] The server cleans the collected data, removing noise and outliers. It then integrates data from different sources to generate a user profile. It uses machine learning algorithms (e.g., Scikit-learn and TensorFlow) to identify user preferences and behavioral patterns. For example, if a user frequently searches for information about cameras, their preference will be classified as "interested in cameras."

[1036] Advertising content generation

[1037] The server uses a generative AI model (e.g., GPT-4) to generate ad text optimized for the user's preferences. For example, if a user is interested in camera features, the server generates ad text such as "A smartphone equipped with the latest high-resolution camera." Related images and videos are also generated using a generative AI model (e.g., DALL-E).

[1038] Terminal handling

[1039] Ad serving

[1040] The server then distributes the generated advertisements at the optimal time. For example, if a user is most active after 8:00 PM, the server can set the advertisements to be distributed during that time period. The advertisements are then sent to the smart glasses at the set times.

[1041] Advertisement display

[1042] The device (smart glasses) receives advertisements sent from the server and displays them in the user's field of view in real time. For example, if a user sees a sign for a new camera shop while walking down the street, the device captures that information and displays a relevant advertisement.

[1043] Obtaining user responses and sending data

[1044] When a user clicks on an ad, the device records the response data, including the time spent viewing the ad and the number of clicks. The device then sends this data to the server and uses it to generate and distribute the next ad.

[1045] Specific examples

[1046] For example, suppose a user is looking for a new smartphone. The server collects and analyzes the user's past search history, purchase history, review viewing history, etc. As a result, it determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera." The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server.

[1047] Prompt Sentence Examples

[1048] User Interest: High-resolution cameras

[1049] Generate ad text: Today only, get 20% off high-res cameras!

[1050] In this way, it is possible to generate and deliver personalized ads based on users' interests and preferences, improving ad click-through rates and conversion rates.

[1051] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1052] Step 1:

[1053] Data collection

[1054] The server collects activity data generated by users during web browsing, purchases, and social media use. Specifically, it uses browser extensions and APIs to obtain data such as websites visited, keywords searched, links clicked, purchase history, social media posts and "likes," etc. The input is activity data from web browsers, social media APIs, etc., and the output is a list of the collected activity data.

[1055] Step 2:

[1056] Data Cleaning

[1057] The server cleans the activity data collected in step 1. It uses data filtering techniques to remove noise data and outliers. The input is the activity data collected in step 1, and the output is the clean activity data.

[1058] Step 3:

[1059] Data Integration

[1060] The server integrates the activity data after data cleaning. It aggregates data from different sources and generates a user profile. The input is the cleaned data, and the output is the integrated data and the generated user profile.

[1061] Step 4:

[1062] Behavioral pattern analysis

[1063] The server uses machine learning algorithms (e.g., Scikit-learn or TensorFlow) to analyze user behavior patterns. For example, if a user frequently searches for information about "cameras," the user's preferences are analyzed as "interested in cameras." The input is the integrated data and user profile, and the output is the identified user preferences and behavior patterns.

[1064] Step 5:

[1065] Ad Generation

[1066] The server generates ad text based on user preferences using a generative AI model (e.g., GPT-4). Additionally, it generates related images and videos using a generative AI model (e.g., DALL-E). The input is user preferences and behavioral patterns, and the output is the generated ad text and images / videos.

[1067] Step 6:

[1068] Ad serving

[1069] The server calculates the optimal time to deliver the generated ad. For example, if the user is most active after 8 PM, it sets the ad to be sent during that time period. The input is the generated ad text, image / video, and user activity time data, and the output is the ad data to be delivered.

[1070] Step 7:

[1071] Advertisement display

[1072] The smart glasses receive advertisements sent from the server and display them in the user's field of view in real time. The input is the delivered advertisement data, and the output is the presented advertisement. For example, if a user is walking down the street and sees a sign for a particular store, an advertisement related to that store will be displayed.

[1073] Step 8:

[1074] Obtaining user responses

[1075] The device records the user's response to the ad, such as the time spent viewing the ad, scroll position, number of clicks, etc. The input is the ad being displayed and the user's response, and the output is the recorded response data.

[1076] Step 9:

[1077] Reaction data transmission

[1078] The device sends the recorded user response data to the server, which then generates and distributes the next ad. The input is the recorded response data, and the output is the response data sent to the server.

[1079] This series of steps makes it possible to provide personalized ads based on the user's visual experience, which is expected to improve ad click-through rates and conversion rates.

[1080] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1081] This invention combines an emotion engine with a system that collects and analyzes user activity data and generates and delivers personalized advertisements based on that data to further improve accuracy. This system recognizes user emotions and reflects them in the personalization of advertisements, thereby achieving more effective advertisement delivery.

[1082] Server-side processing

[1083] Data collection

[1084] The server collects activity data such as browsing history, purchase history, and social media posts when a user uses the Internet. In addition, an emotion engine is used to detect emotions from a user's facial photograph and voice data. For example, emotional information such as whether the user is happy, sad, or surprised can be obtained from data collected through a webcam or microphone.

[1085] Data analysis

[1086] The server first cleans the collected activity and emotion data, removing noise and outliers to improve data quality. It then integrates this data to generate a comprehensive user profile. Machine learning algorithms are used to analyze the data and identify not only user preferences, interests, and behavioral patterns, but also emotional trends.

[1087] Advertising content generation

[1088] The server uses generative AI based on the analysis results to generate ad text optimized for the user's preferences and emotions. For example, if the user is interested in camera functions and recent emotional data shows a high level of joy, the server generates ad text such as "Capture your precious moments with a smartphone equipped with the latest high-resolution camera." Related images and videos are also generated at the same time.

[1089] Ad serving

[1090] The server calculates and sets the optimal time to deliver the generated advertisement. For example, it analyzes that users are most active after 8:00 PM, and sets the advertisement to be delivered during that time. The advertisement is then sent to the user's device at the set time.

[1091] Terminal side processing

[1092] Advertisement Receipt and Display

[1093] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[1094] Obtaining user responses

[1095] When a user clicks on an ad, the device records that click, along with detailed data such as the time spent viewing the ad, scroll position, and emotional changes, allowing the device to determine how long the ad held the user's attention and how their emotions changed before and after viewing the ad.

[1096] Sending reaction data

[1097] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[1098] User response

[1099] Ad viewing

[1100] Users are more likely to be interested in ads because they are tailored to their preferences and emotions. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[1101] Ad Actions

[1102] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[1103] Specific examples

[1104] For example, suppose a user is looking for a new smartphone and has recently browsed many smartphone reviews. The server collects the user's activity data and also uses an emotion engine to obtain the user's emotion data from the webcam. Data analysis reveals that the user is very interested in the camera function and has recently detected a lot of happy emotion.

[1105] Based on this, the server generates an ad that says, "Capture your precious moments with a smartphone equipped with the latest high-resolution camera," and delivers it to the user's device at the optimal time. When a user clicks on an ad, accesses a detail page, and purchases a product, their response data is sent to the server and used to further personalize future ads.

[1106] This system makes it possible to effectively deliver advertisements that accurately reflect users' preferences and emotions, thereby improving ad click rates and conversion rates.

[1107] The processing flow will be explained below.

[1108] Step 1:

[1109] The server collects information about your web browsing history, such as the websites you visit, the keywords you search for, and the links you click.

[1110] Step 2:

[1111] The server retrieves the user's purchase history, which includes details such as the products the user has previously purchased, the purchase date, the purchase location, and the purchase amount.

[1112] Step 3:

[1113] The server collects activity data such as user posts, comments, and likes through social media APIs.

[1114] Step 4:

[1115] The server uses an emotion engine to detect emotions from the user's facial photos and voice data, including the ability to analyze the user's emotions in real time using a webcam and microphone.

[1116] Step 5:

[1117] The server cleans the collected data, specifically removing noise and outliers from the collected data to ensure data consistency.

[1118] Step 6:

[1119] The server aggregates the cleaned data to create a single, comprehensive user profile, combining information from different data sources to create a comprehensive data set.

[1120] Step 7:

[1121] The server analyzes the combined data to identify user preferences, interests, behavioral patterns, and emotional trends, and uses machine learning algorithms to refine the data.

[1122] Step 8:

[1123] The server uses generative AI to generate ad text based on the user's preferences and emotions. For example, if the user is interested in camera features and the emotion of joy is detected frequently, the server generates ad copy such as "Capture joyful moments with a smartphone equipped with the latest high-resolution camera."

[1124] Step 9:

[1125] The server calculates the optimal time to deliver ads and schedules them accordingly. For example, it may determine that users are most active after 8 PM, and deliver ads during that time.

[1126] Step 10:

[1127] The server sends the generated advertisement to the user's device at the set time.

[1128] Step 11:

[1129] The terminal receives the advertisement sent from the server.

[1130] Step 12:

[1131] The device will then display the received advertisement in an appropriate location, such as in the sidebar of a web page or as a banner within an app.

[1132] Step 13:

[1133] The user views the displayed advertisement.

[1134] Step 14:

[1135] When a user clicks on an ad, the device records that click, along with detailed data such as the amount of time spent viewing the ad, where they scrolled, and changes in their emotions before and after viewing the ad.

[1136] Step 15:

[1137] The terminal transmits the recorded user reaction data to the server.

[1138] Step 16:

[1139] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[1140] Example 2

[1141] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1142] Conventional ad delivery systems personalized ads based solely on user activity data, making it impossible to deliver ads that took into account the user's emotional state. As a result, user engagement declined and advertising effectiveness was inadequate. Furthermore, noise and outliers in the collected data reduced data accuracy, making it difficult to optimize ad delivery.

[1143] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for cleaning the user's activity data and emotion data and removing noise data and outliers, means for integrating the cleaned data and generating a user profile, means for analyzing the user's emotional state using the collected emotion data, means for generating advertising content using a generative AI model based on the analysis results, and means for delivering the generated advertising content to the user's terminal. This makes it possible to integrate the user's activity data and emotion data to generate a highly accurate user profile and deliver effective advertisements based on it.

[1144] "Activity data" is information about a user's online activities, such as browsing history, purchase history, and social media posts.

[1145] "Emotional data" is information that represents a user's emotional state, detected from a user's facial photograph and voice data collected through input devices such as a webcam and microphone.

[1146] "Cleaning" is the process of removing noise data and outliers from collected data, and is a process carried out to improve the quality of the data.

[1147] "Synthesis" is the process of combining cleaned activity data with emotion data to generate a single, comprehensive user profile.

[1148] A "user profile" is information that indicates a user's preferences, interests, behavioral patterns, and emotional tendencies, generated based on integrated activity data and emotional data.

[1149] An "emotion engine" is software or an algorithm that detects emotions from a user's facial photo and voice data and determines their state.

[1150] A "generative AI model" is an artificial intelligence model that generates content based on user preferences and emotions. Examples include language models that generate text and generative models that generate images.

[1151] "Advertising content" refers to media content such as advertising text, images, and videos delivered to users.

[1152] A "generative AI prompt" is a text input (prompt) that provides instructions to a generative AI model, and is a sentence that provides basic information for the AI ​​model to generate appropriate advertising content based on this.

[1153] "Response data" is data that records a user's reaction to an ad, such as click information, display time, scroll position, and emotional changes.

[1154] "Delivery" refers to the process of sending advertising content generated by the server to the user's terminal and displaying it.

[1155] This invention relates to a system that collects and analyzes user activity data and emotion data, and generates and delivers personalized advertisements based on the collected data. This system recognizes user emotions and reflects them in the personalized advertisements, thereby achieving more effective advertisement delivery.

[1156] System Configuration

[1157] Server Roles

[1158] The server does the following:

[1159] 1. Data Collection:

[1160] The server collects data about users' browsing history, purchase history, social media posts, and other activity data while they are online, for example, using browser history APIs and online store APIs.

[1161] The emotion engine also analyzes facial photos and audio data collected from the webcam and microphone to detect the user's emotions. For example, the Emotion API can be used to recognize emotions such as "happiness," "sadness," and "surprise."

[1162] 2. Data cleaning and integration:

[1163] The server cleans the collected data using Python's Pandas library and removes noise data and outliers.

[1164] The cleaned data is then combined to create a single, comprehensive user profile, which is then stored in a database such as MongoDB.

[1165] 3. User preference and sentiment analysis:

[1166] The server uses machine learning libraries (e.g., scikit-learn and TensorFlow) to analyze the user's preferences, interests, behavioral patterns, and emotional tendencies from the integrated data, for example, by using collaborative filtering algorithms to predict the user's future behavior.

[1167] 4. Advertising content generation:

[1168] Using a generative AI model (e.g., OpenAI's GPT-4), advertising content is generated based on the analysis results. The generated advertising content includes text, images, and videos optimized for user preferences and emotions.

[1169] An example of a prompt sentence that is generated is "Capture your precious moments with a smartphone equipped with the latest high-resolution camera."

[1170] 5. Advertisement Delivery:

[1171] The server calculates and sets the optimal timing for ad delivery based on the analysis data. For example, it analyzes the time periods when users are most active and sets the time periods to deliver ads.

[1172] Advertisements are sent to users' devices at set times using the Google Ads API or other ad serving APIs.

[1173] Device Role

[1174] The terminal does the following:

[1175] 1. Receiving and Displaying Advertisements:

[1176] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate times, for example, using JavaScript or HTML5.

[1177] 2. Obtaining user responses:

[1178] When a user clicks on an ad, the device records the click information and collects detailed data such as the display time, scroll position, and emotional changes, which allows for detailed analysis of the effectiveness of the ad.

[1179] 3. Reaction data transmission:

[1180] The device converts the collected user response data into JSON format and sends it to the server, which then obtains data that will be useful for generating and delivering the next ad.

[1181] User Roles

[1182] The user performs the following actions:

[1183] 1. Viewing Ads:

[1184] For example, if a user is interested in smartphones, they may see an advertisement for a new model equipped with a high-resolution camera.

[1185] 2. Advertising Actions:

[1186] When a user clicks on an advertisement that interests them, they are taken to a page with detailed product information, where they can then purchase the product. Once the purchase is complete, the data is sent from the device to a server, which will be used to deliver advertisements to the user in the future.

[1187] Through the above process, the system can integrate user activity data and emotion data to generate highly accurate user profiles, and then deliver effective advertisements based on those profiles. Specific examples of prompts include:

[1188] "Based on recent user sentiment and activity data, generate ad text using the following information:

[1189] Users are very interested in the camera function

[1190] The user's emotional state is delight

[1191] The prompt given to the generative AI model: 'Take photos of your precious moments with your smartphone, equipped with the latest high-resolution camera.'

[1192] This system makes it possible to generate and deliver advertisements that accurately match users' preferences and emotions, thereby improving ad click rates and conversion rates.

[1193] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1194] Step 1: Data collection

[1195] Input: Your browsing history, purchase history, social media posts, webcam and microphone data.

[1196] How it works: The server calls the browser history API or online store API to retrieve the user's browsing history and purchase history. At the same time, it uses an emotion engine (e.g., Emotion API) to analyze the video and audio data collected through the webcam and microphone, and detects the user's emotional state (happiness, sadness, surprise, etc.) from their facial image and audio data.

[1197] Output: User activity and emotion data is generated and stored in a database (e.g., MongoDB) on the server.

[1198] Step 2: Data cleaning and integration

[1199] Input: Collected activity and emotion data.

[1200] How it works: The server uses the Python Pandas library to clean the collected activity and emotion data, specifically removing noise and outliers to improve data quality, and then integrates the cleaned data to generate user profiles.

[1201] Output: A clean and consolidated user profile is generated and stored in the database.

[1202] Step 3: Analyze user preferences and sentiment

[1203] Input: The cleaned user profile.

[1204] How it works: The server uses scikit-learn and TensorFlow to analyze the cleaned and integrated data to determine user preferences, interests, behavioral patterns, and emotional trends, for example by using collaborative filtering algorithms to predict future user behavior and preferences.

[1205] Output: The analysis produces detailed data about user preferences and sentiment.

[1206] Step 4: Advertising content generation

[1207] Input: Analysis results on user preferences and sentiment.

[1208] Specific operation: The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate advertising content based on the analysis results. Specifically, it inputs the prompt "Capture precious moments with a smartphone equipped with the latest high-resolution camera" and runs the generative AI model to generate optimized advertising text. It then uses an image generation model such as DALL-E to create images and videos corresponding to the generated text.

[1209] Output: Personalized ad content (text, images, video) is generated.

[1210] Step 5: Ad delivery settings and sending

[1211] Input: Personalized advertising content and user activity data.

[1212] How it works: The server calculates the optimal timing for ad delivery based on analytical data. For example, it analyzes that users are most active after 8:00 PM, and sets the ad delivery time to be during that time. Specifically, it uses the Google Ads API or Facebook Marketing API to send the ad to the user's device at the set time.

[1213] Output: The ad is sent to the user's device.

[1214] Step 6: Receiving and displaying advertisements

[1215] Input: Ad content sent by the server.

[1216] How it works: The device uses scripts such as JavaScript to receive ads sent from the server and display them on web pages or within apps at the appropriate times. For example, an ad might be displayed in the sidebar while the user is browsing a news site.

[1217] Output: The ad is displayed on the user's device.

[1218] Step 7: Get user responses

[1219] Input: User behavior data when the ad is displayed.

[1220] How it works: When a user clicks on an ad, the device records the click information. At the same time, detailed response data such as the time spent viewing the ad, scroll position, and emotional changes are collected. This is done using log files on the device and browser event listeners.

[1221] Output: Collected user response data.

[1222] Step 8: Reaction data submission and analysis

[1223] Input: Collected user response data.

[1224] How it works: The device converts the collected response data into JSON format and sends it to the server, which then analyzes the response data to obtain information that will be useful for generating and delivering future ads.

[1225] Output: The analyzed response data is used to generate the next ad.

[1226] This series of processing steps realizes a system that can integrate user activity data and emotional data and effectively deliver highly accurate personalized advertisements based on that data.

[1227] (Application example 2)

[1228] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1229] While conventional ad delivery systems have performed personalization based on user preferences and behavioral patterns, it has been difficult to achieve highly personalized advertising that reflects the user's emotional state. In particular, by taking into account not only the user's summer interests but also their emotional state at that moment, it is expected that ad engagement rates and conversion rates will be further improved.

[1230] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data and identifying the user's preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, means for acquiring user emotion data, and means for personalizing the advertising content based on the acquired emotion data. This makes it possible to personalize advertisements that reflect the user's emotional state.

[1231] "User activity data" refers to information such as browsing history, purchase history, and social media posts made while a user is using the Internet.

[1232] "Means for analyzing collected activity data" means means for analyzing collected user activity data and using technologies and algorithms to identify user preferences, interests, and behavioral patterns.

[1233] "Means for generating advertising content based on user preferences" refers to technology that automatically creates advertising messages and visuals that correspond to a user's preferences and interests.

[1234] "Means for delivering generated advertising content to a user's device" means a method or system for sending personalized advertising to a device used by a user.

[1235] "Means of acquiring user emotional data" refers to technology that recognizes and detects emotions in real time from the user's facial expressions, voice, etc.

[1236] "Means for personalizing advertising content based on acquired emotional data" means a method or system that adjusts the content and timing of advertising based on the emotional state of a user.

[1237] "Means for receiving and displaying advertising content on a user's device" refers to the technology or method for displaying advertising messages on a user's device.

[1238] "Means for recording users' responses to advertisements" refers to technology that tracks users' responses, such as clicking on advertisements, and stores them as data.

[1239] "Means for transmitting recorded reaction data to a server" refers to the method for identifying a user's reaction information and transmitting it to a server for storage and analysis.

[1240] "Video and audio capture means" refers to devices such as cameras and microphones used to collect user emotion data.

[1241] "Means for generating a user profile integrated with emotional data" refers to techniques and methods for creating a comprehensive user profile that includes emotional information.

[1242] The present invention is a system for generating and delivering personalized advertisements using user activity data and emotion data. This system is composed of a means for collecting and analyzing users' internet browsing history, purchase history, and social media postings, and an emotion engine for recognizing users' emotions.

[1243] First, the server collects user activity data, such as internet browser history, purchase history on online shopping sites, social media posts, etc. This is typically done using cookies and tracking pixels.

[1244] The server then analyzes the collected activity data using the following software:

[1245] Python: General data processing

[1246] Pandas: Data Frame Operations

[1247] scikit-learn: machine learning algorithms

[1248] The collected data is first cleaned to filter out noise and outliers, and then the filtered data is integrated to generate a comprehensive user profile, which analyzes the user's preferences, interests, and behavioral patterns.

[1249] In parallel, the server collects emotional data by analyzing the video and audio data collected through the user's webcam and microphone with an emotion recognition module (e.g., EmotionRecognizer), which detects the user's emotional state in real time, such as whether they are happy, sad, or surprised.

[1250] Based on the analysis results, the server uses a generative AI model to generate advertising content. For example, if a user has recently visited a camera review site and "joy" is detected from their facial expression, the server will generate an advertisement such as "Buy a smartphone with the latest high-resolution camera."

[1251] The server then delivers the generated ad to the user's device at the optimal time. The timing of ad delivery is determined based on the user's active time period analyzed from their activity data. For example, if it is known that the user is most active after 8:00 PM, the ad will be delivered during that time period.

[1252] The device receives the advertisement sent from the server and displays it on the web page or within the app at the appropriate time. After the advertisement is displayed, the user's response (clicks, viewing time, changes in emotion, etc.) is recorded again by the device and sent to the server. This response data is used to generate and deliver advertisements from the next time onwards, achieving more accurate personalization.

[1253] (Example)

[1254] For example, if a user has recently visited many camera review sites and their facial expression shows "joy," the server can generate an ad with the message "Capture your precious moments with a smartphone equipped with the latest high-resolution camera" based on this information, and deliver the ad to them after 8:00 PM, based on the analysis results that show that users are most active at that time.

[1255] (Example of a generative AI model prompt)

[1256] A user has recently been browsing many camera review sites and the "Happy" facial expression is detected from their webcam. Generate the best ad text for the user.

[1257] Example of generated ad:

[1258] Capture your precious moments with a smartphone equipped with the latest high-resolution camera.

[1259] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1260] Step 1: Data collection

[1261] The server collects the user's internet browser history, purchase history on online shopping sites, and social media posts. It receives the user's browsing data, purchase data, and social media data as input and stores this data in a database on the server. Cookies and tracking pixels are used to collect the data.

[1262] Step 2: Collecting Emotional Data

[1263] The server captures video and audio data from the user's webcam and microphone, and analyzes it with an emotion recognition module. It receives the user's facial photo and audio data as input, and analyzes them with the emotion recognition module. The output is an emotion label, such as "happiness" or "sadness." This emotion data is also stored on the server.

[1264] Step 3: Data cleaning

[1265] The server cleans the collected activity and emotion data, removing noise and outliers. It receives various activity and emotion data as input and cleans the data using data processing libraries such as Pand and scikit-learn. The output is a clean dataset.

[1266] Step 4: Create a user profile

[1267] The server generates a user profile based on the cleaned data set. It takes the cleaned activity data and emotion data as input and integrates them to identify the user's preferences, interests, behavioral patterns, and emotional tendencies. The output is a comprehensive user profile.

[1268] Step 5: Advertising content generation

[1269] The server generates advertising content using a generative AI model based on the user profile. Using the generated user profile as input, the server sends an advertising generation request in the form of a prompt sentence to the generative AI model. The output is personalized advertising text and associated visual content.

[1270] Step 6: Ad serving

[1271] The server delivers the generated advertising content to the user's device at the optimal time. It receives personalized advertising content and the optimal delivery time analyzed from user activity data as input and determines the ad delivery schedule. As output, the advertisement is sent to the user's device and displayed at a specific time.

[1272] Step 7: Receiving and displaying advertisements

[1273] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time. It receives the advertisement content from the server as input and displays the advertisement at the time when it predicts that the user will be most interested in the advertisement.

[1274] Step 8: Recording User Responses

[1275] The device records the user's reactions, such as clicking on an ad, and sends the data to a server. As input, it collects reaction data such as user clicks, the time spent viewing the ad, and changes in emotions, and sends it to the server as output for storage.

[1276] Step 9: Utilizing reaction data

[1277] The server uses the collected response data to generate and deliver the next ad. It receives and analyzes user response data as input and optimizes the ad to improve engagement and conversion rates.

[1278] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1279] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1280] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1281] [Fourth embodiment]

[1282] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1283] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1284] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1285] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1286] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1287] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1288] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1289] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1290] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1291] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1292] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1293] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1294] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1295] The present invention relates to a system for effectively collecting and analyzing user activity data and generating and delivering customized advertisements based on the collected data. An embodiment of this system is described in detail below.

[1296] Server-side processing

[1297] Data collection

[1298] The server collects the user's web browsing history. Specifically, it acquires data such as the websites visited, keywords searched, and links clicked. It also obtains the user's purchase history from the database, and furthermore, it uses social media APIs to collect activity data such as the user's posts and "likes."

[1299] Data analysis

[1300] The server first cleans the collected data, removing noise and outliers. Next, it integrates data from different sources to generate a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms are used for analysis, revealing the content the user prefers and the times when they are most active.

[1301] Advertising content generation

[1302] The server uses AI to generate ad text optimized for the user's preferences based on the analysis results. For example, if a user is interested in camera features, the server generates ad text such as "Smartphones with the latest high-resolution cameras." Related images and videos are also generated at the same time.

[1303] Ad serving

[1304] The server calculates and sets the optimal time to deliver the generated ad. For example, if the user is most active after 8:00 PM, the server sets the ad to be delivered during that time. The ad is then sent to the user's device at the set time.

[1305] Terminal side processing

[1306] Advertisement Receipt and Display

[1307] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[1308] Obtaining user responses

[1309] When a user clicks on an ad, the device records that information, along with other details such as the time spent viewing the ad and where the user scrolled, which allows the device to determine how long the ad held the user's attention.

[1310] Sending reaction data

[1311] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[1312] User response

[1313] Ad viewing

[1314] Users are more likely to engage with ads because they are tailored to their interests. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[1315] Ad Actions

[1316] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[1317] Specific examples

[1318] For example, if a user is looking for a new smartphone, the server collects the user's past search history, purchase history, review viewing history, etc. and analyzes the data. The server determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera" based on that. The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server. By repeating this cycle, it is possible to provide a beneficial advertising experience for the user and maximize the effectiveness of the advertisement.

[1319] In this manner, the present invention, which provides a method for generating and delivering personalized advertisements based on user interests and preferences, can improve advertising click-through rates and conversion rates.

[1320] The processing flow will be explained below.

[1321] Step 1:

[1322] The server collects your web browsing history, specifically recording information such as the websites you visit, the keywords you search for, and the links you click.

[1323] Step 2:

[1324] The server retrieves the user's purchase history, which includes a list of past purchases, purchase dates, and purchase amounts.

[1325] Step 3:

[1326] The server uses social media APIs to collect user activity data such as posts and "likes."

[1327] Step 4:

[1328] The server cleans the collected data, specifically removing noise and outliers to improve the quality of the data.

[1329] Step 5:

[1330] The server aggregates the cleaned data to generate a single user profile, combining information from different data sources to create a comprehensive data set.

[1331] Step 6:

[1332] The server analyzes the combined data to identify user preferences, interests, and behavioral patterns, using machine learning algorithms.

[1333] Step 7:

[1334] The server uses generative AI to generate ad text based on user preferences, for example, creating an ad for a "smartphone with the latest high-resolution camera" for a user interested in camera features.

[1335] Step 8:

[1336] The server analyzes the user's most active times and calculates the optimal time to deliver ads.

[1337] Step 9:

[1338] The server transmits the generated advertisement to the user's terminal at the set time.

[1339] Step 10:

[1340] The terminal receives the advertisement sent from the server.

[1341] Step 11:

[1342] The device displays ads where appropriate, such as in the sidebar of a web page or in a banner within an app.

[1343] Step 12:

[1344] The user views the displayed advertisement.

[1345] Step 13:

[1346] When a user clicks on an ad, the device records the click information.

[1347] Step 14:

[1348] The terminal transmits the recorded user reaction data to the server.

[1349] Step 15:

[1350] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[1351] Example 1

[1352] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1353] In today's internet advertising environment, many users are exposed to countless advertisements, yet providing effectively personalized advertisements to users is considered a challenge. Despite technological advances in recommendation engines and user profiling, many challenges remain in optimizing the accuracy of collected data and the timing of advertisement display. This results in a large number of advertisements that do not appeal to users, reducing effectiveness for advertisers. Furthermore, collecting user responses in real time and utilizing them to generate subsequent advertisements is extremely complex and requires advanced technological capabilities. Given this background, there is a need to establish a system that can effectively collect and analyze user activity data and deliver optimal advertisements tailored to users at the appropriate time.

[1354] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1355] In this invention, the server includes: means for collecting user activity data; means for cleaning the collected activity data and removing noise data and outliers; means for integrating the filtered data to generate a user profile; means for analyzing the user's preferences, interests, and behavioral patterns from the integrated data using a machine learning algorithm; means for generating advertising content using a generative AI model based on the analysis results; and means for setting the optimal timing for delivering the generated advertising content and delivering it to the user's device. This makes it possible to provide highly personalized advertisements to users at the appropriate time and maximize their effectiveness. In addition, by collecting user response data and utilizing it in generating and delivering the next advertisement, the accuracy and effectiveness of the advertisement can be continuously improved.

[1356] "User activity data" is information that records a user's online behavior, such as web browsing history, search keywords, links clicked, purchase history, and social media activity data.

[1357] "Cleaning" is the process of removing noise data and outliers from collected data and extracting only valid data.

[1358] "Data integration" is the process of combining data collected from different sources into one cohesive data set.

[1359] A "user profile" is information created based on integrated data that indicates a user's characteristics, such as preferences, interests, and behavioral patterns.

[1360] A "machine learning algorithm" is a computer program that analyzes collected data and finds patterns and trends.

[1361] A "generative AI model" is an artificial intelligence system that automatically generates useful advertising content such as text, images, and videos from large amounts of data.

[1362] "Advertising content" refers to information that is delivered to users and consists of advertising messages and related images, videos, etc.

[1363] The "optimal delivery time" is the time when users are predicted to be most active and interested in the ad.

[1364] "User terminal" refers to an electronic device, such as a smartphone, personal computer, or tablet, on which a user receives and views advertisements.

[1365] "Response data" refers to information about the actions a user takes in response to an advertisement, such as clicks, viewing time, and scroll position.

[1366] MODE FOR CARRYING OUT THE INVENTION

[1367] The present invention relates to a system that effectively collects and analyzes user activity data and generates and delivers customized advertisements based on the collected data. This system utilizes machine learning algorithms and generative AI models to provide highly personalized advertisements to users at optimal times.

[1368] Server-side processing

[1369] Data collection

[1370] When a user browses the web, the server collects their browsing history. Specifically, it obtains information such as the URLs of websites visited, keywords searched, and links clicked. It also collects the user's purchase history and social media activity data (e.g., posts and "likes"). This data is collected using web scripts, APIs (e.g., social media APIs), etc.

[1371] Data Cleaning

[1372] The server cleans the collected data, using Python scripts to remove noise and outliers, and only valid data is extracted and stored in the database.

[1373] Data integration and user profile generation

[1374] The server consolidates data from different sources and generates a user profile, which identifies the user's preferences, interests, and behavioral patterns. Machine learning algorithms (e.g., Scikit-Learn) are used to analyze the data and reveal the content the user prefers and the times of day when they are most active.

[1375] Advertising content generation

[1376] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile, using the following prompt:

[1377] "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history."

[1378] The generative AI model generates ad copy such as, "Get 20% off the latest smartphone with a high-resolution camera right now!", along with related images and videos.

[1379] Ad delivery optimization

[1380] The server identifies the time period when the user is most active and configures the settings to deliver advertisements during that time period. For example, if the user is most active after 8:00 PM, the server configures the settings to deliver advertisements during that time period. The advertisements are then sent to the user's device at the configured time.

[1381] Terminal side processing

[1382] Advertisement Receipt and Display

[1383] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time.

[1384] Collecting user response data

[1385] When a user clicks on an ad, that information is recorded by the device, along with other details such as the time spent viewing the ad and where they scrolled.

[1386] Sending reaction data to the server

[1387] The device sends the collected user response data to the server, which uses this data to optimize the next ad generation and delivery.

[1388] User response

[1389] Ad viewing

[1390] Users can click on ads that interest them to find out more information.

[1391] Purchase Action

[1392] When a user clicks on an ad, accesses a product information page, and purchases the product, this information is sent from the device to the server and used to generate and distribute the next ad.

[1393] Through these processes, it becomes possible to provide optimized advertisements to users and maximize their effectiveness. The present invention can significantly improve the click-through rate and engagement rate of advertisements.

[1394] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1395] System program processing steps

[1396] Step 1: Data collection

[1397] Input: User's web browsing history, search keywords, click information, purchase history, social media activity data

[1398] Output: Collected activity data

[1399] When a user browses the web, the server collects their browsing history (URLs of websites visited, search keywords, and clicked links). It also retrieves the user's purchase history from a database and uses social media APIs to collect activity data such as user posts and "likes." For example, if a user searches for "the latest smartphone with a camera," the server saves the page information related to that keyword.

[1400] Step 2: Data cleaning

[1401] Input: Collected activity data

[1402] Output: Cleaned data (noise data, data with outliers removed)

[1403] The server cleans the collected data. This process uses Python scripts to remove noise and outliers. For example, if a user's search history contains obviously misspelled or meaningless keywords, they will be removed. This ensures that only valid data is extracted and stored in the database.

[1404] Step 3: Data integration and user profile generation

[1405] Input: Cleaned data

[1406] Output: User profile (information identifying user preferences, interests, and behavioral patterns)

[1407] The server then combines the cleaned data to generate a user profile. Specifically, it combines data from different sources (search keywords, visit history, social media activity, etc.) into a single dataset. It then uses machine learning algorithms (such as Scikit-Learn) to identify the content that the user is interested in and the times of day when they are most active. For example, if a user frequently visits the "camera smartphone" page, that interest will be reflected in the profile.

[1408] Step 4: Advertising content generation

[1409] Input: User profile

[1410] Output: Generated ad content

[1411] The server uses a generative AI model (e.g., GPT-4) to generate ad text based on the user profile. For example, it uses a prompt such as, "Generate ad copy for smartphones with the latest high-resolution cameras based on past search and purchase history." The generative AI model generates ad copy such as, "Get 20% off smartphones with the latest high-resolution cameras right now!" and also generates related images and videos as needed.

[1412] Step 5: Optimizing ad delivery

[1413] Input: Generated ad content, user profile (most active times)

[1414] Output: Ads that are set to be delivered

[1415] The server identifies the time periods when the user is most active and sets the time to deliver ads during those periods. For example, if analysis data shows that the user is most active after 8:00 PM, the server sets the time to deliver ads during that period. The generated advertising content is then sent to the user's device at the set time.

[1416] Step 6: Receiving and displaying advertisements

[1417] Input: Served ad content

[1418] Output: Served ad

[1419] The device receives the advertisements sent from the server and displays them while the user is browsing a news site or in an appropriate location within the app. For example, when a user is browsing a news site, an advertisement for a "smartphone with the latest high-resolution camera" appears in the sidebar.

[1420] Step 7: Collect user response data

[1421] Input: User response to the ad

[1422] Output: Collected reaction data

[1423] When a user clicks on an ad, the device records that information, along with other details such as how long the ad was displayed and where the user scrolled, which can tell how long the ad held the user's attention.

[1424] Step 8: Sending reaction data to the server

[1425] Input: Collected reaction data

[1426] Output: Response data sent to the server

[1427] The device sends the recorded user response data to a server, which uses this data to optimize the generation and distribution of the next ad. For example, the server can extract the characteristics of ad content with a high click rate and reflect them in the generation of the next ad, thereby providing a more effective ad.

[1428] Step 9: View the ad

[1429] Input: Displayed ad

[1430] Output: User's interest in the ad

[1431] Users view the ads displayed on their devices, and if they are interested, they can click on the ad to view more information, which increases their interest.

[1432] Step 10: Purchase Action

[1433] Input: Clicked Ad

[1434] Output: Purchase data

[1435] When a user clicks on an ad, accesses a page with detailed information, and purchases a product, this purchase data is sent from the device to a server and used to generate and deliver the next ad. The data on completed purchases is used to set up and generate the next ad, thereby continuously improving the accuracy and effectiveness of ads.

[1436] (Application example 1)

[1437] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1438] While technologies for providing personalized advertisements based on user activity data have existed for some time, they have not yet been developed to provide advertisements that take into account the user's visual experience and real-time behavior. This makes it difficult to display advertisements that are in line with the user's current interests in real time, making it difficult to maximize advertising effectiveness.

[1439] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1440] In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data to identify user preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, and means for capturing user visual data and displaying relevant advertisements based on specific objects or locations, thereby enabling the provision of personalized advertisements based on the user's real-time visual experience.

[1441] "User activity data" refers to any data generated by users during their web browsing, purchases, social media use, etc.

[1442] "Collection means" refers to the methods and functions for collecting user activity data and visual data.

[1443] "Means of analysis" refers to the methods and technologies used to analyze collected data and identify user preferences and behavioral patterns.

[1444] "Means for generating advertising content" refers to methods and technologies for creating personalized advertising based on a user's interests and preferences.

[1445] "Means of delivery" refers to the methods and technologies used to deliver the generated advertisement to the user's device.

[1446] "Visual data" refers to information about scenery and objects that a user sees through a device such as smart glasses.

[1447] "Means of capturing and displaying relevant ads based on specific objects or locations" refers to methods or technologies for identifying a user's visual focus and displaying relevant ads based on that information.

[1448] This invention is a system that collects and analyzes user activity data and generates and delivers advertisements based on the user's preferences, and proposes a form that can be applied particularly to displaying advertisements using smart glasses.

[1449] Server Processing

[1450] Data collection

[1451] The servers collect all activity data generated by users during web browsing, purchases, and social media use, including website visit history, search keywords, clicked links, purchase history, social media posts and "likes," etc. This data is collected using browser extensions and various APIs.

[1452] Data analysis

[1453] The server cleans the collected data, removing noise and outliers. It then integrates data from different sources to generate a user profile. It uses machine learning algorithms (e.g., Scikit-learn and TensorFlow) to identify user preferences and behavioral patterns. For example, if a user frequently searches for information about cameras, their preference will be classified as "interested in cameras."

[1454] Advertising content generation

[1455] The server uses a generative AI model (e.g., GPT-4) to generate ad text optimized for the user's preferences. For example, if a user is interested in camera features, the server generates ad text such as "A smartphone equipped with the latest high-resolution camera." Related images and videos are also generated using a generative AI model (e.g., DALL-E).

[1456] Terminal handling

[1457] Ad serving

[1458] The server then distributes the generated advertisements at the optimal time. For example, if a user is most active after 8:00 PM, the server can set the advertisements to be distributed during that time period. The advertisements are then sent to the smart glasses at the set times.

[1459] Advertisement display

[1460] The device (smart glasses) receives advertisements sent from the server and displays them in the user's field of view in real time. For example, if a user sees a sign for a new camera shop while walking down the street, the device captures that information and displays a relevant advertisement.

[1461] Obtaining user responses and sending data

[1462] When a user clicks on an ad, the device records the response data, including the time spent viewing the ad and the number of clicks. The device then sends this data to the server and uses it to generate and distribute the next ad.

[1463] Specific examples

[1464] For example, suppose a user is looking for a new smartphone. The server collects and analyzes the user's past search history, purchase history, review viewing history, etc. As a result, it determines that the user is particularly interested in camera features and generates an advertisement for a "smartphone equipped with the latest high-resolution camera." The server sets the advertisement to be delivered during the user's most active time. The device displays the advertisement at the appropriate time, and when the user clicks on the advertisement, the response data is sent to the server.

[1465] Prompt Sentence Examples

[1466] User Interest: High-resolution cameras

[1467] Generate ad text: Today only, get 20% off high-res cameras!

[1468] In this way, it is possible to generate and deliver personalized ads based on users' interests and preferences, improving ad click-through rates and conversion rates.

[1469] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1470] Step 1:

[1471] Data collection

[1472] The server collects activity data generated by users during web browsing, purchases, and social media use. Specifically, it uses browser extensions and APIs to obtain data such as websites visited, keywords searched, links clicked, purchase history, social media posts and "likes," etc. The input is activity data from web browsers, social media APIs, etc., and the output is a list of the collected activity data.

[1473] Step 2:

[1474] Data Cleaning

[1475] The server cleans the activity data collected in step 1. It uses data filtering techniques to remove noise data and outliers. The input is the activity data collected in step 1, and the output is the clean activity data.

[1476] Step 3:

[1477] Data Integration

[1478] The server integrates the activity data after data cleaning. It aggregates data from different sources and generates a user profile. The input is the cleaned data, and the output is the integrated data and the generated user profile.

[1479] Step 4:

[1480] Behavioral pattern analysis

[1481] The server uses machine learning algorithms (e.g., Scikit-learn or TensorFlow) to analyze user behavior patterns. For example, if a user frequently searches for information about "cameras," the user's preferences are analyzed as "interested in cameras." The input is the integrated data and user profile, and the output is the identified user preferences and behavior patterns.

[1482] Step 5:

[1483] Ad Generation

[1484] The server generates ad text based on user preferences using a generative AI model (e.g., GPT-4). Additionally, it generates related images and videos using a generative AI model (e.g., DALL-E). The input is user preferences and behavioral patterns, and the output is the generated ad text and images / videos.

[1485] Step 6:

[1486] Ad serving

[1487] The server calculates the optimal time to deliver the generated ad. For example, if the user is most active after 8 PM, it sets the ad to be sent during that time period. The input is the generated ad text, image / video, and user activity time data, and the output is the ad data to be delivered.

[1488] Step 7:

[1489] Advertisement display

[1490] The smart glasses receive advertisements sent from the server and display them in the user's field of view in real time. The input is the delivered advertisement data, and the output is the presented advertisement. For example, if a user is walking down the street and sees a sign for a particular store, an advertisement related to that store will be displayed.

[1491] Step 8:

[1492] Obtaining user responses

[1493] The device records the user's response to the ad, such as the time spent viewing the ad, scroll position, number of clicks, etc. The input is the ad being displayed and the user's response, and the output is the recorded response data.

[1494] Step 9:

[1495] Reaction data transmission

[1496] The device sends the recorded user response data to the server, which then generates and distributes the next ad. The input is the recorded response data, and the output is the response data sent to the server.

[1497] This series of steps makes it possible to provide personalized ads based on the user's visual experience, which is expected to improve ad click-through rates and conversion rates.

[1498] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1499] This invention combines an emotion engine with a system that collects and analyzes user activity data and generates and delivers personalized advertisements based on that data to further improve accuracy. This system recognizes user emotions and reflects them in the personalization of advertisements, thereby achieving more effective advertisement delivery.

[1500] Server-side processing

[1501] Data collection

[1502] The server collects activity data such as browsing history, purchase history, and social media posts when a user uses the Internet. In addition, an emotion engine is used to detect emotions from a user's facial photograph and voice data. For example, emotional information such as whether the user is happy, sad, or surprised can be obtained from data collected through a webcam or microphone.

[1503] Data analysis

[1504] The server first cleans the collected activity and emotion data, removing noise and outliers to improve data quality. It then integrates this data to generate a comprehensive user profile. Machine learning algorithms are used to analyze the data and identify not only user preferences, interests, and behavioral patterns, but also emotional trends.

[1505] Advertising content generation

[1506] The server uses generative AI based on the analysis results to generate ad text optimized for the user's preferences and emotions. For example, if the user is interested in camera functions and recent emotional data shows a high level of joy, the server generates ad text such as "Capture your precious moments with a smartphone equipped with the latest high-resolution camera." Related images and videos are also generated at the same time.

[1507] Ad serving

[1508] The server calculates and sets the optimal time to deliver the generated advertisement. For example, it analyzes that users are most active after 8:00 PM, and sets the advertisement to be delivered during that time. The advertisement is then sent to the user's device at the set time.

[1509] Terminal side processing

[1510] Advertisement Receipt and Display

[1511] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time, for example, displaying an advertisement in the sidebar while the user is browsing a news site.

[1512] Obtaining user responses

[1513] When a user clicks on an ad, the device records that click, along with detailed data such as the time spent viewing the ad, scroll position, and emotional changes, allowing the device to determine how long the ad held the user's attention and how their emotions changed before and after viewing the ad.

[1514] Sending reaction data

[1515] The device sends the recorded user response data to a server, which uses this data to generate and deliver the next ad. This enables repeated personalization, improving ad engagement rates.

[1516] User response

[1517] Ad viewing

[1518] Users are more likely to be interested in ads because they are tailored to their preferences and emotions. For example, a user interested in the latest smartphones will be more interested in ads for new models with high-resolution cameras.

[1519] Ad Actions

[1520] When a user clicks on an advertisement that interests them, they can access a page with detailed product information. They can also purchase the product on that page. Once a purchase is completed, the data is sent from the device to a server, which will be used to deliver advertisements to them in the future.

[1521] Specific examples

[1522] For example, suppose a user is looking for a new smartphone and has recently browsed many smartphone reviews. The server collects the user's activity data and also uses an emotion engine to obtain the user's emotion data from the webcam. Data analysis reveals that the user is very interested in the camera function and has recently detected a lot of happy emotion.

[1523] Based on this, the server generates an ad that says, "Capture your precious moments with a smartphone equipped with the latest high-resolution camera," and delivers it to the user's device at the optimal time. When a user clicks on an ad, accesses a detail page, and purchases a product, their response data is sent to the server and used to further personalize future ads.

[1524] This system makes it possible to effectively deliver advertisements that accurately reflect users' preferences and emotions, thereby improving ad click rates and conversion rates.

[1525] The processing flow will be explained below.

[1526] Step 1:

[1527] The server collects information about your web browsing history, such as the websites you visit, the keywords you search for, and the links you click.

[1528] Step 2:

[1529] The server retrieves the user's purchase history, which includes details such as the products the user has previously purchased, the purchase date, the purchase location, and the purchase amount.

[1530] Step 3:

[1531] The server collects activity data such as user posts, comments, and likes through social media APIs.

[1532] Step 4:

[1533] The server uses an emotion engine to detect emotions from the user's facial photos and voice data, including the ability to analyze the user's emotions in real time using a webcam and microphone.

[1534] Step 5:

[1535] The server cleans the collected data, specifically removing noise and outliers from the collected data to ensure data consistency.

[1536] Step 6:

[1537] The server aggregates the cleaned data to create a single, comprehensive user profile, combining information from different data sources to create a comprehensive data set.

[1538] Step 7:

[1539] The server analyzes the combined data to identify user preferences, interests, behavioral patterns, and emotional trends, and uses machine learning algorithms to refine the data.

[1540] Step 8:

[1541] The server uses generative AI to generate ad text based on the user's preferences and emotions. For example, if the user is interested in camera features and the emotion of joy is detected frequently, the server generates ad copy such as "Capture joyful moments with a smartphone equipped with the latest high-resolution camera."

[1542] Step 9:

[1543] The server calculates the optimal time to deliver ads and schedules them accordingly. For example, it may determine that users are most active after 8 PM, and deliver ads during that time.

[1544] Step 10:

[1545] The server sends the generated advertisement to the user's device at the set time.

[1546] Step 11:

[1547] The terminal receives the advertisement sent from the server.

[1548] Step 12:

[1549] The device will then display the received advertisement in an appropriate location, such as in the sidebar of a web page or as a banner within an app.

[1550] Step 13:

[1551] The user views the displayed advertisement.

[1552] Step 14:

[1553] When a user clicks on an ad, the device records that click, along with detailed data such as the amount of time spent viewing the ad, where they scrolled, and changes in their emotions before and after viewing the ad.

[1554] Step 15:

[1555] The terminal transmits the recorded user reaction data to the server.

[1556] Step 16:

[1557] The server receives the user's response data and uses it to generate and distribute advertisements in the future.

[1558] Example 2

[1559] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] Conventional ad delivery systems personalized ads based solely on user activity data, making it impossible to deliver ads that took into account the user's emotional state. As a result, user engagement declined and advertising effectiveness was inadequate. Furthermore, noise and outliers in the collected data reduced data accuracy, making it difficult to optimize ad delivery.

[1561] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for cleaning the user's activity data and emotion data and removing noise data and outliers, means for integrating the cleaned data and generating a user profile, means for analyzing the user's emotional state using the collected emotion data, means for generating advertising content using a generative AI model based on the analysis results, and means for delivering the generated advertising content to the user's terminal. This makes it possible to integrate the user's activity data and emotion data to generate a highly accurate user profile and deliver effective advertisements based on it.

[1562] "Activity data" is information about a user's online activities, such as browsing history, purchase history, and social media posts.

[1563] "Emotional data" is information that represents a user's emotional state, detected from a user's facial photograph and voice data collected through input devices such as a webcam and microphone.

[1564] "Cleaning" is the process of removing noise data and outliers from collected data, and is a process carried out to improve the quality of the data.

[1565] "Synthesis" is the process of combining cleaned activity data with emotion data to generate a single, comprehensive user profile.

[1566] A "user profile" is information that indicates a user's preferences, interests, behavioral patterns, and emotional tendencies, generated based on integrated activity data and emotional data.

[1567] An "emotion engine" is software or an algorithm that detects emotions from a user's facial photo and voice data and determines their state.

[1568] A "generative AI model" is an artificial intelligence model that generates content based on user preferences and emotions. Examples include language models that generate text and generative models that generate images.

[1569] "Advertising content" refers to media content such as advertising text, images, and videos delivered to users.

[1570] A "generative AI prompt" is a text input (prompt) that provides instructions to a generative AI model, and is a sentence that provides basic information for the AI ​​model to generate appropriate advertising content based on this.

[1571] "Response data" is data that records a user's reaction to an ad, such as click information, display time, scroll position, and emotional changes.

[1572] "Delivery" refers to the process of sending advertising content generated by the server to the user's terminal and displaying it.

[1573] This invention relates to a system that collects and analyzes user activity data and emotion data, and generates and delivers personalized advertisements based on the collected data. This system recognizes user emotions and reflects them in the personalized advertisements, thereby achieving more effective advertisement delivery.

[1574] System Configuration

[1575] Server Roles

[1576] The server does the following:

[1577] 1. Data Collection:

[1578] The server collects data about users' browsing history, purchase history, social media posts, and other activity data while they are online, for example, using browser history APIs and online store APIs.

[1579] The emotion engine also analyzes facial photos and audio data collected from the webcam and microphone to detect the user's emotions. For example, the Emotion API can be used to recognize emotions such as "happiness," "sadness," and "surprise."

[1580] 2. Data cleaning and integration:

[1581] The server cleans the collected data using Python's Pandas library and removes noise data and outliers.

[1582] The cleaned data is then combined to create a single, comprehensive user profile, which is then stored in a database such as MongoDB.

[1583] 3. User preference and sentiment analysis:

[1584] The server uses machine learning libraries (e.g., scikit-learn and TensorFlow) to analyze the user's preferences, interests, behavioral patterns, and emotional tendencies from the integrated data, for example, by using collaborative filtering algorithms to predict the user's future behavior.

[1585] 4. Advertising content generation:

[1586] Using a generative AI model (e.g., OpenAI's GPT-4), advertising content is generated based on the analysis results. The generated advertising content includes text, images, and videos optimized for user preferences and emotions.

[1587] An example of a prompt sentence that is generated is "Capture your precious moments with a smartphone equipped with the latest high-resolution camera."

[1588] 5. Advertisement Delivery:

[1589] The server calculates and sets the optimal timing for ad delivery based on the analysis data. For example, it analyzes the time periods when users are most active and sets the time periods to deliver ads.

[1590] Advertisements are sent to users' devices at set times using the Google Ads API or other ad serving APIs.

[1591] Device Role

[1592] The terminal does the following:

[1593] 1. Receiving and Displaying Advertisements:

[1594] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate times, for example, using JavaScript or HTML5.

[1595] 2. Obtaining user responses:

[1596] When a user clicks on an ad, the device records the click information and collects detailed data such as the display time, scroll position, and emotional changes, which allows for detailed analysis of the effectiveness of the ad.

[1597] 3. Reaction data transmission:

[1598] The device converts the collected user response data into JSON format and sends it to the server, which then obtains data that will be useful for generating and delivering the next ad.

[1599] User Roles

[1600] The user performs the following actions:

[1601] 1. Viewing Ads:

[1602] For example, if a user is interested in smartphones, they may see an advertisement for a new model equipped with a high-resolution camera.

[1603] 2. Advertising Actions:

[1604] When a user clicks on an advertisement that interests them, they are taken to a page with detailed product information, where they can then purchase the product. Once the purchase is complete, the data is sent from the device to a server, which will be used to deliver advertisements to the user in the future.

[1605] Through the above process, the system can integrate user activity data and emotion data to generate highly accurate user profiles, and then deliver effective advertisements based on those profiles. Specific examples of prompts include:

[1606] "Based on recent user sentiment and activity data, generate ad text using the following information:

[1607] Users are very interested in the camera function

[1608] The user's emotional state is delight

[1609] The prompt given to the generative AI model: 'Take photos of your precious moments with your smartphone, equipped with the latest high-resolution camera.'

[1610] This system makes it possible to generate and deliver advertisements that accurately match users' preferences and emotions, thereby improving ad click rates and conversion rates.

[1611] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1612] Step 1: Data collection

[1613] Input: Your browsing history, purchase history, social media posts, webcam and microphone data.

[1614] How it works: The server calls the browser history API or online store API to retrieve the user's browsing history and purchase history. At the same time, it uses an emotion engine (e.g., Emotion API) to analyze the video and audio data collected through the webcam and microphone, and detects the user's emotional state (happiness, sadness, surprise, etc.) from their facial image and audio data.

[1615] Output: User activity and emotion data is generated and stored in a database (e.g., MongoDB) on the server.

[1616] Step 2: Data cleaning and integration

[1617] Input: Collected activity and emotion data.

[1618] How it works: The server uses the Python Pandas library to clean the collected activity and emotion data, specifically removing noise and outliers to improve data quality, and then integrates the cleaned data to generate user profiles.

[1619] Output: A clean and consolidated user profile is generated and stored in the database.

[1620] Step 3: Analyze user preferences and sentiment

[1621] Input: The cleaned user profile.

[1622] How it works: The server uses scikit-learn and TensorFlow to analyze the cleaned and integrated data to determine user preferences, interests, behavioral patterns, and emotional trends, for example by using collaborative filtering algorithms to predict future user behavior and preferences.

[1623] Output: The analysis produces detailed data about user preferences and sentiment.

[1624] Step 4: Advertising content generation

[1625] Input: Analysis results on user preferences and sentiment.

[1626] Specific operation: The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate advertising content based on the analysis results. Specifically, it inputs the prompt "Capture precious moments with a smartphone equipped with the latest high-resolution camera" and runs the generative AI model to generate optimized advertising text. It then uses an image generation model such as DALL-E to create images and videos corresponding to the generated text.

[1627] Output: Personalized ad content (text, images, video) is generated.

[1628] Step 5: Ad delivery settings and sending

[1629] Input: Personalized advertising content and user activity data.

[1630] How it works: The server calculates the optimal timing for ad delivery based on analytical data. For example, it analyzes that users are most active after 8:00 PM, and sets the ad delivery time to be during that time. Specifically, it uses the Google Ads API or Facebook Marketing API to send the ad to the user's device at the set time.

[1631] Output: The ad is sent to the user's device.

[1632] Step 6: Receiving and displaying advertisements

[1633] Input: Ad content sent by the server.

[1634] How it works: The device uses scripts such as JavaScript to receive ads sent from the server and display them on web pages or within apps at the appropriate times. For example, an ad might be displayed in the sidebar while the user is browsing a news site.

[1635] Output: The ad is displayed on the user's device.

[1636] Step 7: Get user responses

[1637] Input: User behavior data when the ad is displayed.

[1638] How it works: When a user clicks on an ad, the device records the click information. At the same time, detailed response data such as the time spent viewing the ad, scroll position, and emotional changes are collected. This is done using log files on the device and browser event listeners.

[1639] Output: Collected user response data.

[1640] Step 8: Reaction data submission and analysis

[1641] Input: Collected user response data.

[1642] How it works: The device converts the collected response data into JSON format and sends it to the server, which then analyzes the response data to obtain information that will be useful for generating and delivering future ads.

[1643] Output: The analyzed response data is used to generate the next ad.

[1644] This series of processing steps realizes a system that can integrate user activity data and emotional data and effectively deliver highly accurate personalized advertisements based on that data.

[1645] (Application example 2)

[1646] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1647] While conventional ad delivery systems have performed personalization based on user preferences and behavioral patterns, it has been difficult to achieve highly personalized advertising that reflects the user's emotional state. In particular, by taking into account not only the user's summer interests but also their emotional state at that moment, it is expected that ad engagement rates and conversion rates will be further improved.

[1648] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user activity data, means for analyzing the collected activity data and identifying the user's preferences, interests, and behavioral patterns, means for generating advertising content based on the identified user preferences, means for delivering the generated advertising content to the user's terminal, means for acquiring user emotion data, and means for personalizing the advertising content based on the acquired emotion data. This makes it possible to personalize advertisements that reflect the user's emotional state.

[1649] "User activity data" refers to information such as browsing history, purchase history, and social media posts made while a user is using the Internet.

[1650] "Means for analyzing collected activity data" means means for analyzing collected user activity data and using technologies and algorithms to identify user preferences, interests, and behavioral patterns.

[1651] "Means for generating advertising content based on user preferences" refers to technology that automatically creates advertising messages and visuals that correspond to a user's preferences and interests.

[1652] "Means for delivering generated advertising content to a user's device" means a method or system for sending personalized advertising to a device used by a user.

[1653] "Means of acquiring user emotional data" refers to technology that recognizes and detects emotions in real time from the user's facial expressions, voice, etc.

[1654] "Means for personalizing advertising content based on acquired emotional data" means a method or system that adjusts the content and timing of advertising based on the emotional state of a user.

[1655] "Means for receiving and displaying advertising content on a user's device" refers to the technology or method for displaying advertising messages on a user's device.

[1656] "Means for recording users' responses to advertisements" refers to technology that tracks users' responses, such as clicking on advertisements, and stores them as data.

[1657] "Means for transmitting recorded reaction data to a server" refers to the method for identifying a user's reaction information and transmitting it to a server for storage and analysis.

[1658] "Video and audio capture means" refers to devices such as cameras and microphones used to collect user emotion data.

[1659] "Means for generating a user profile integrated with emotional data" refers to techniques and methods for creating a comprehensive user profile that includes emotional information.

[1660] The present invention is a system for generating and delivering personalized advertisements using user activity data and emotion data. This system is composed of a means for collecting and analyzing users' internet browsing history, purchase history, and social media postings, and an emotion engine for recognizing users' emotions.

[1661] First, the server collects user activity data, such as internet browser history, purchase history on online shopping sites, social media posts, etc. This is typically done using cookies and tracking pixels.

[1662] The server then analyzes the collected activity data using the following software:

[1663] Python: General data processing

[1664] Pandas: Data Frame Operations

[1665] scikit-learn: machine learning algorithms

[1666] The collected data is first cleaned to filter out noise and outliers, and then the filtered data is integrated to generate a comprehensive user profile, which analyzes the user's preferences, interests, and behavioral patterns.

[1667] In parallel, the server collects emotional data by analyzing the video and audio data collected through the user's webcam and microphone with an emotion recognition module (e.g., EmotionRecognizer), which detects the user's emotional state in real time, such as whether they are happy, sad, or surprised.

[1668] Based on the analysis results, the server uses a generative AI model to generate advertising content. For example, if a user has recently visited a camera review site and "joy" is detected from their facial expression, the server will generate an advertisement such as "Buy a smartphone with the latest high-resolution camera."

[1669] The server then delivers the generated ad to the user's device at the optimal time. The timing of ad delivery is determined based on the user's active time period analyzed from their activity data. For example, if it is known that the user is most active after 8:00 PM, the ad will be delivered during that time period.

[1670] The device receives the advertisement sent from the server and displays it on the web page or within the app at the appropriate time. After the advertisement is displayed, the user's response (clicks, viewing time, changes in emotion, etc.) is recorded again by the device and sent to the server. This response data is used to generate and deliver advertisements from the next time onwards, achieving more accurate personalization.

[1671] (Example)

[1672] For example, if a user has recently visited many camera review sites and their facial expression shows "joy," the server can generate an ad with the message "Capture your precious moments with a smartphone equipped with the latest high-resolution camera" based on this information, and deliver the ad to them after 8:00 PM, based on the analysis results that show that users are most active at that time.

[1673] (Example of a generative AI model prompt)

[1674] A user has recently been browsing many camera review sites and the "Happy" facial expression is detected from their webcam. Generate the best ad text for the user.

[1675] Example of generated ad:

[1676] Capture your precious moments with a smartphone equipped with the latest high-resolution camera.

[1677] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1678] Step 1: Data collection

[1679] The server collects the user's internet browser history, purchase history on online shopping sites, and social media posts. It receives the user's browsing data, purchase data, and social media data as input and stores this data in a database on the server. Cookies and tracking pixels are used to collect the data.

[1680] Step 2: Collecting Emotional Data

[1681] The server captures video and audio data from the user's webcam and microphone, and analyzes it with an emotion recognition module. It receives the user's facial photo and audio data as input, and analyzes them with the emotion recognition module. The output is an emotion label, such as "happiness" or "sadness." This emotion data is also stored on the server.

[1682] Step 3: Data cleaning

[1683] The server cleans the collected activity and emotion data, removing noise and outliers. It receives various activity and emotion data as input and cleans the data using data processing libraries such as Pand and scikit-learn. The output is a clean dataset.

[1684] Step 4: Create a user profile

[1685] The server generates a user profile based on the cleaned data set. It takes the cleaned activity data and emotion data as input and integrates them to identify the user's preferences, interests, behavioral patterns, and emotional tendencies. The output is a comprehensive user profile.

[1686] Step 5: Advertising content generation

[1687] The server generates advertising content using a generative AI model based on the user profile. Using the generated user profile as input, the server sends an advertising generation request in the form of a prompt sentence to the generative AI model. The output is personalized advertising text and associated visual content.

[1688] Step 6: Ad serving

[1689] The server delivers the generated advertising content to the user's device at the optimal time. It receives personalized advertising content and the optimal delivery time analyzed from user activity data as input and determines the ad delivery schedule. As output, the advertisement is sent to the user's device and displayed at a specific time.

[1690] Step 7: Receiving and displaying advertisements

[1691] The device receives the advertisements sent from the server and displays them on web pages or within apps at the appropriate time. It receives the advertisement content from the server as input and displays the advertisement at the time when it predicts that the user will be most interested in the advertisement.

[1692] Step 8: Recording User Responses

[1693] The device records the user's reactions, such as clicking on an ad, and sends the data to a server. As input, it collects reaction data such as user clicks, the time spent viewing the ad, and changes in emotions, and sends it to the server as output for storage.

[1694] Step 9: Utilizing reaction data

[1695] The server uses the collected response data to generate and deliver the next ad. It receives and analyzes user response data as input and optimizes the ad to improve engagement and conversion rates.

[1696] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1697] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1698] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1699] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1700] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1701] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1702] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1703] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1704] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1705] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1706] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1707] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1708] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1709] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1710] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1711] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1712] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1713] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1714] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1715] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1716] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1717] The following is further disclosed regarding the above embodiment.

[1718] (Claim 1)

[1719] means of collecting user activity data;

[1720] Analyzing collected activity data to identify user preferences, interests, and behavioral patterns; and

[1721] means for generating advertising content based on the identified user preferences;

[1722] means for delivering the generated advertising content to a user's device;

[1723] A system including:

[1724] (Claim 2)

[1725] means for receiving and displaying advertising content on a user's device;

[1726] a means for recording user responses to advertisements;

[1727] means for transmitting the recorded reaction data to a server;

[1728] The system of claim 1 further comprising:

[1729] (Claim 3)

[1730] A means for filtering noise data and outliers from the collected activity data;

[1731] a means for aggregating the filtered data to generate a user profile;

[1732] means for analyzing user preferences and behavior patterns from the combined data;

[1733] The system of claim 1 further comprising:

[1734] "Example 1"

[1735] (Claim 1)

[1736] means of collecting user activity data;

[1737] A means for cleaning the collected activity data to remove noise data and outliers;

[1738] a means for aggregating the filtered data to generate a user profile;

[1739] A means of analyzing user preferences, interests, and behavioral patterns from the integrated data using machine learning algorithms; and

[1740] A means for generating advertising content using a generative AI model based on the analysis results;

[1741] A means for setting the optimal timing for delivering the generated advertising content and delivering it to the user's device;

[1742] A system including:

[1743] (Claim 2)

[1744] means for receiving and displaying advertising content on a user's device;

[1745] A means of recording users' responses to advertisements and collecting detailed data such as viewing time, click information, and scroll position;

[1746] means for transmitting the recorded reaction data to a server;

[1747] The system of claim 1 further comprising:

[1748] (Claim 3)

[1749] Specific examples of collecting user activity data include web browsing history, search keywords, clicked links, purchase history, and social media activity data;

[1750] A means of individually customizing the generated advertising content based on the user profile and the most active times of the day;

[1751] A means for analyzing the collected response data and reflecting it in the next advertisement generation and distribution;

[1752] The system of claim 1 further comprising:

[1753] "Application Example 1"

[1754] (Claim 1)

[1755] means of collecting user activity data;

[1756] Analyzing collected activity data to identify user preferences, interests, and behavioral patterns; and

[1757] means for generating advertising content based on the identified user preferences;

[1758] means for delivering the generated advertising content to a user's device;

[1759] a means of capturing a user's visual data to display relevant advertisements based on specific objects or locations;

[1760] A system including:

[1761] (Claim 2)

[1762] means for receiving and displaying advertising content on a user's device;

[1763] a means for recording user responses to advertisements;

[1764] means for transmitting the recorded reaction data to a server;

[1765] It also includes a means for recognizing an object that a user is paying attention to and displaying advertisements.

[1766] 10. The system of claim 1.

[1767] (Claim 3)

[1768] A means for filtering noise data and outliers from the collected activity data;

[1769] a means for aggregating the filtered data to generate a user profile;

[1770] means for analyzing user preferences and behavior patterns from the combined data;

[1771] Includes a means to analyze user visual data and dynamically change advertising content based on that data

[1772] 10. The system of claim 1.

[1773] "Example 2: Combining Emotion Engines"

[1774] (Claim 1)

[1775] means of collecting user activity data;

[1776] a means for cleaning the collected activity data and emotion data to remove noise data and outliers;

[1777] a means for integrating the cleaned data and generating a user profile;

[1778] means for analyzing the user's emotional state using the collected emotional data;

[1779] A means for generating advertising content using a generative AI model based on the analysis results;

[1780] means for delivering the generated advertising content to a user's device;

[1781] A system including:

[1782] (Claim 2)

[1783] means for receiving and displaying advertising content on a user's device;

[1784] a means for recording user responses to advertisements;

[1785] means for transmitting the recorded reaction data to a server;

[1786] The system of claim 1 further comprising:

[1787] (Claim 3)

[1788] a means for cleaning the collected activity data and emotion data to filter out noise data and outliers;

[1789] a means for aggregating the filtered data to generate a user profile;

[1790] means for analyzing user preferences and behavior patterns from the combined data;

[1791] A means to input prompt sentences into the generative AI model based on the analysis results and generate optimal advertising content;

[1792] The system of claim 1 further comprising:

[1793] "Application example 2 when combining emotion engines"

[1794] (Claim 1)

[1795] means of collecting user activity data;

[1796] Analyzing collected activity data to identify user preferences, interests, and behavioral patterns; and

[1797] means for generating advertising content based on the identified user preferences;

[1798] means for delivering the generated advertising content to a user's device;

[1799] A means for acquiring user emotion data;

[1800] a means for personalizing advertising content based on the acquired emotional data;

[1801] A system including:

[1802] (Claim 2)

[1803] means for receiving and displaying advertising content on a user's device;

[1804] a means for recording user responses to advertisements;

[1805] means for transmitting the recorded reaction data to a server;

[1806] video and audio capture means for collecting emotion data;

[1807] The system of claim 1 further comprising:

[1808] (Claim 3)

[1809] A means for filtering noise data and outliers from the collected activity data;

[1810] a means for aggregating the filtered data to generate a user profile;

[1811] means for analyzing user preferences and behavior patterns from the combined data;

[1812] means for generating a user profile integrated with emotion data;

[1813] The system of claim 1 further comprising: [Explanation of symbols]

[1814] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means of collecting user activity data; Analyzing collected activity data to identify user preferences, interests, and behavioral patterns; and means for generating advertising content based on the identified user preferences; means for delivering the generated advertising content to a user's device; A system including:

2. means for receiving and displaying advertising content on a user's device; a means for recording user responses to advertisements; means for transmitting the recorded reaction data to a server; The system of claim 1 further comprising:

3. A means for filtering noise data and outliers from the collected activity data; a means for aggregating the filtered data to generate a user profile; means for analyzing user preferences and behavior patterns from the combined data; The system of claim 1 further comprising:

Images