System

Abstract

A system is provided.SOLUTION: A system comprising: means for generating an advertisement format; means for distributing the generated advertisement format to an external operator; means for selling the advertisement format; and means for converting the advertisement format into another form.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional ad distribution methods lacked consistency in ad formats, making it difficult for advertisers to effectively place ads across various platforms. Furthermore, users often found the displayed ads annoying, resulting in reduced advertising effectiveness. Furthermore, changing and optimizing ad formats was cumbersome, making it difficult to respond efficiently. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that includes a means for generating advertising formats, a means for distributing the generated advertising formats to external businesses, a means for selling the advertising formats, and a means for converting the advertising formats into other formats. Specifically, by consistently generating advertising formats and distributing them to external businesses, advertisers can effectively place advertisements on various platforms. Furthermore, the system improves the user experience by displaying advertising data received on user terminals at appropriate times and recording and transmitting the history. Furthermore, by collecting user feedback and optimizing the display method of the advertising formats, advertising effectiveness can be maximized.

[0006] An "ad format" is a layout and design standard for displaying advertising content.

[0007] "Generating means" refers to the function and method for creating new advertising formats.

[0008] The "distribution means" refers to a method and device for providing the generated advertisement format to other businesses or systems.

[0009] The "means of sale" is a business process for providing the created advertisement format to an advertiser and receiving payment.

[0010] "Means for converting" refers to functions and methods for converting an existing advertising format into a different format or specification.

[0011] "Advertising data" refers to information including specific advertising content created based on an advertising format.

[0012] "User terminal" refers to the device (e.g., smartphone, tablet, PC, etc.) that a user uses to view an advertisement.

[0013] The "display means" refers to a function and method for displaying advertising data on the screen of a user terminal at an appropriate timing.

[0014] "Means for recording history" refers to a function and method for saving information about displayed advertisements (e.g., display time, user interaction information, etc.).

[0015] "Feedback" refers to information including opinions and impressions provided by users regarding advertisements.

[0016] "Optimization means" refers to functions and methods for improving the display method and content of advertisements based on collected feedback, thereby increasing advertising effectiveness. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, and a user. The roles and processes of each are described in detail below.

[0039] 1. Server Processing

[0040] Ad format generation

[0041] The server generates various ad formats, including 15- and 30-second video ads, magazine cover ads, online rectangle ads, and in-stream ads, using template design and layout generation algorithms.

[0042] Ad format distribution

[0043] The server provides the format data via API or download link to distribute the generated ad formats to AI operators, allowing them to create ads using standardized ad formats.

[0044] Advertising sales

[0045] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[0046] Format Conversion

[0047] The server uses AI to change the size and format of ad formats in response to advertiser requests. For example, it may change a 15-second ad to a 30-second one, or convert a rectangular ad into an in-stream ad. These converted formats are then redistributed to AI providers and advertisers.

[0048] 2. Terminal Processing

[0049] Receiving and storing advertising data

[0050] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, which includes the advertisement format generated and converted by the server.

[0051] Displaying ads

[0052] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[0053] Recording and sending advertising history

[0054] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[0055] 3. User Actions

[0056] Watching an ad

[0057] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[0058] Providing feedback

[0059] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, by sending it to the server via their device. This feedback is used to optimize future ad formats.

[0060] Specific examples

[0061] Server processing example

[0062] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[0063] Terminal processing example

[0064] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0065] User processing example

[0066] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0067] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0068] The processing flow will be explained below.

[0069] 1. Server Processing

[0070] Step 1:

[0071] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats.

[0072] Step 2:

[0073] The server distributes the generated ad format to external businesses and transfers the format data to the external businesses' systems via API.

[0074] Step 3:

[0075] The server sells the generated ad formats to advertisers, providing a dashboard for advertisers to select and purchase the formats they need.

[0076] Step 4:

[0077] The server uses AI to convert ad formats based on requests from advertisers, such as changing a video ad from 15 seconds to 30 seconds.

[0078] Step 5:

[0079] The server then distributes the converted ad format back to external businesses and advertisers, providing format data via API or download link.

[0080] 2. Terminal Processing

[0081] Step 1:

[0082] The terminal receives the advertisement data from the server, the advertisement data including the generated and converted format.

[0083] Step 2:

[0084] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[0085] Step 3:

[0086] When a user uses a video service, the device displays an advertisement before the main content. The advertisement data is called from within the video service application and displayed in the video player.

[0087] Step 4:

[0088] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[0089] Step 5:

[0090] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[0091] 3. User Actions

[0092] Step 1:

[0093] The user watches advertisements provided through the terminal, such as advertisements played before the main video of a video service or advertisements displayed around the cover of an electronic magazine.

[0094] Step 2:

[0095] The user provides feedback on the advertisement, for example, whether the advertisement is interesting, relevant, annoying, etc., by using the feedback function of the device.

[0096] Step 3:

[0097] User feedback is sent from the device to the server, which uses the collected feedback to optimize future ads.

[0098] Specific examples

[0099] Server processing example

[0100] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells the ads to advertisers, who then use the purchased format.

[0101] Terminal processing example

[0102] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0103] User processing example

[0104] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0105] Example 1

[0106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0107] In conventional ad delivery systems, the creation, distribution, sales, and optimization of ad formats are performed piecemeal, resulting in inefficiencies for both advertisers and users. Conversion between different ad formats is also performed manually, which is time-consuming and costly. Furthermore, even if user feedback is collected, there is a lack of a way to effectively incorporate it, making ad optimization difficult.

[0108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0109] In this invention, the server includes a means for generating ad formats, a means for distributing the generated ad formats to external businesses via an API or a download link, a means for providing a dashboard for advertisers to sell the ad formats, and a means for converting the size and format of the ad formats using a machine learning algorithm in response to requests from advertisers. This makes it possible to consistently and efficiently perform processes from generating, distributing, selling, and optimizing ad formats.

[0110] "Ad format" refers to the template or design layout used to display advertising content.

[0111] "External businesses" are companies or organizations that use advertising formats to create the actual advertising content.

[0112] "API" stands for Application Programming Interface, an interface for exchanging functions and data between software programs.

[0113] A "download link" is a URL that allows a user to save a specific file or data from the Internet to their device.

[0114] A "dashboard" is a user interface that allows a user to visually display and manipulate the information and data they need.

[0115] A "machine learning algorithm" is a program or method that allows a computer to automatically learn using large amounts of data to perform specific tasks.

[0116] "Local storage" refers to a data storage device or storage medium installed within a terminal.

[0117] A "user" is a consumer who uses a video service or an electronic magazine.

[0118] "Feedback" refers to the evaluations and opinions of advertisements provided by users who have viewed them.

[0119] The present invention provides an advertisement distribution system that is efficient and effective for both advertisers and users by generating, distributing, selling, and converting advertisement formats. Detailed embodiments of the system are described below.

[0120] Server embodiment

[0121] Ad format generation

[0122] The server uses template design and layout generation algorithms to generate various ad formats, such as 15-second and 30-second video ads, magazine cover ads, online rectangle ads, in-stream ads, etc. This generation is done using image processing libraries such as Python's OpenCV and Pillow.

[0123] Ad format distribution

[0124] The server distributes the generated ad formats to external businesses via API or download links. This distribution utilizes a RESTful API, allowing external businesses to efficiently provide the ad format data they require.

[0125] Advertising sales

[0126] The server provides a dashboard for advertisers, allowing them to select and purchase the ad formats they need. This dashboard is built using React and Vue.js, making it intuitive and easy for users to use.

[0127] Ad format conversion

[0128] The server uses machine learning algorithms (e.g., TensorFlow, PyTorch) to convert the size and format of the ad format according to the advertiser's request. For example, if a 15-second video ad is extended to 30 seconds, additional content is generated based on the original ad content. This converted ad format is also distributed via API.

[0129] Terminal embodiment

[0130] Receiving and storing advertising data

[0131] The device receives the advertising data sent from the server and saves it in local storage (e.g., SQLite database). The receiving process is performed periodically, and new advertising data is downloaded and saved as needed.

[0132] Displaying ads

[0133] The device displays the saved advertisements when the user uses video services or digital magazines. This display uses HTML5 and JavaScript, and in video services, the advertisements are displayed before the main video. In digital magazines, advertisements are inserted around the cover while the user is reading.

[0134] Recording and sending advertising history

[0135] The device records the history of the displayed ads (playback time, user interaction information, etc.) and periodically sends it to the server. This history information is collected in JSON format and sent to the server via a REST API.

[0136] User's embodiment

[0137] Watching an ad

[0138] Users watch ads delivered through their devices. The UI is intuitive and designed to allow users to watch ads naturally. For example, users are presented with a 15-second full-screen ad before the main content of a video service.

[0139] Providing feedback

[0140] Users send feedback about ads to the server via their devices. For example, they can send their opinions on whether the ad was interesting, relevant, or annoying through a simple UI within the app. The feedback information is stored in a database and used to optimize future ads.

[0141] Examples and prompts

[0142] Server processing example

[0143] The server generates 15-second video ad formats and distributes them to third parties, who then use them to create advertising content. The server then sells the ads to advertisers, who then use the purchased formats in their own campaigns.

[0144] Terminal processing example

[0145] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0146] User processing example

[0147] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0148] Prompt sentence for generative AI model

[0149] "Generate a 15-second video ad format and distribute it to third parties."

[0150] "Collect user viewing history and feedback information to optimize advertising effectiveness."

[0151] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1:

[0154] The server loads the template design data and starts generating ad formats. Specifically, it uses Python's Pillow library to generate image-based templates and OpenCV to assemble video frames to create 15-second and 30-second video ads. It uses the template design data as input and obtains the generated ad format files as output.

[0155] Step 2:

[0156] The server distributes the generated ad formats to external businesses via APIs and download links. Specifically, a RESTful API is built so that external businesses can download format data by sending an HTTP request. The generated ad format file is used as input, and a download link is obtained as output, which is shared with external businesses.

[0157] Step 3:

[0158] The server sells ad formats through a dashboard for advertisers. Specifically, we designed a user interface using React and Vue.js to allow advertisers to easily select and purchase the format they need. The server receives an advertiser's purchase request as input and obtains the purchased format data and a receipt as output.

[0159] Step 4:

[0160] The server uses machine learning algorithms to convert the size and format of ad formats based on the advertiser's requests. Specifically, it uses TensorFlow and PyTorch to perform processes such as extending a 15-second video ad to 30 seconds. The server uses the advertiser's conversion request and the original ad format as input, and obtains the converted ad format as output.

[0161] Step 5:

[0162] The device receives the advertising data sent from the server and stores it in local storage (e.g., SQLite database). Specifically, it periodically downloads new advertising data from the server and stores it appropriately in local storage. It uses the advertising data from the server as input and obtains the stored advertising data as output.

[0163] Step 6:

[0164] The device displays the stored advertising data when the user uses video services or digital magazines. Specifically, it uses HTML5 and JavaScript to play and display advertisements to the user. It uses the advertising data from local storage as input and obtains the displayed advertisement as output.

[0165] Step 7:

[0166] The device records the history of displayed ads and periodically sends it to the server. Specifically, it collects display time and user interaction information in JSON format and sends it to the server via REST API. It uses user interaction information and display history as input and obtains the ad history data sent to the server as output.

[0167] Step 8:

[0168] Users view advertisements provided through their devices. The UI is intuitive and designed to allow users to view advertisements naturally. The input is the advertisement content provided by the device, and the output is the information about the advertisements viewed.

[0169] Step 9:

[0170] Users send feedback about ads to the server via their devices. Specifically, they use a simple UI within the app to input their opinions, such as "interesting," "useful," or "unpleasant." The system uses user feedback as input and obtains the feedback information sent to the server as output.

[0171] (Application example 1)

[0172] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0173] Conventional ad delivery systems lack sufficient personalization based on user interests and behavior, limiting the effectiveness of ads. Furthermore, if the timing or content of ads is inappropriate for the user, they can be annoying. Therefore, there is a need for a system that efficiently utilizes user behavior data and feedback to deliver ads with optimal timing and content.

[0174] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0175] In this invention, the server includes means for generating ad formats, means for distributing the generated ad formats to external businesses, means for selling ad formats, means for converting ad formats into other formats, means for personalizing ad formats based on user behavior data and feedback, and means for selecting an optimal ad format based on user behavior analysis using an AI model, thereby making it possible to provide users with personalized ads that are optimized for them.

[0176] An "ad format" is a template or design layout that defines the form and structure of advertising content.

[0177] An "external business" is a third-party business that creates advertising content using the advertising format.

[0178] "Personalization" refers to individually optimizing advertising content based on user behavioral data and feedback.

[0179] An "AI model" is a computational model that uses machine learning technology to analyze and learn from data, and make predictions and optimizations.

[0180] A "prompt sentence" is input text that provides specific instructions or explanations to a generative AI model.

[0181] "Server" means a centralized management system that generates, distributes, sells, and converts advertising formats.

[0182] A "user terminal" is a device (e.g., a smartphone, a tablet, etc.) that a user uses to view advertising content.

[0183] "Behavioral data" refers to records of the actions and reactions users take on advertisements and content.

[0184] "Feedback" refers to ratings and opinions provided by users who have viewed an advertisement.

[0185] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system of the present invention operates based on the roles of a server, a user terminal, and a user.

[0186] Server Processing

[0187] Ad format generation

[0188] The server generates various ad formats using template designs and layout generation algorithms, resulting in ad content in a variety of formats, including video ads and banner ads.

[0189] Ad format distribution

[0190] The generated ad formats are distributed to external businesses via APIs or download links via the server, allowing external businesses to create effective ad content using standardized ad formats.

[0191] Collecting user behavior data and feedback

[0192] The server collects and analyzes the behavioral data and feedback sent by users using an AI model (e.g., TensorFlow). This analysis extracts the user's behavioral patterns and areas of interest.

[0193] Ad personalization

[0194] Based on the analyzed data, the server provides personalized advertisements to users, including the ability to dynamically generate input prompts for the generative AI model.

[0195] User terminal processing

[0196] Receiving and displaying advertising data

[0197] The user device receives advertising data from the server and displays the advertisements at the appropriate time. For example, when a user uses a video service, a 15- or 30-second video advertisement is played before the main content, and when a user is reading an e-magazine, an advertisement is displayed around the cover.

[0198] Recording and sending advertising history and responses

[0199] The user device records the history of the ads displayed (such as the display time and user interaction information) and periodically sends this information to the server. If the user reacts to the ad, this information is also recorded and reflected in the next ad display.

[0200] User Action

[0201] Ad viewing and feedback

[0202] Users can view ads delivered through their devices and provide feedback on the ads. The feedback includes opinions on whether the ad was interesting, relevant, annoying, etc., and is sent from the device to the server. This feedback is used to optimize future ad formats.

[0203] Specific examples

[0204] Ad delivery process

[0205] 1. Ad generation and distribution: The 15-second video ad format generated by the server is then used by an external company to create a video for a new product introduction campaign. The generated ad format is then delivered to smartphones and saved in the "AdSmart" app.

[0206] 2. User behavior analysis: A user's browsing history of products in a specific category on an online shopping site is sent to the server, where a machine learning model analyzes this data and predicts the user's interests.

[0207] 3. Display Ads: When a user opens a video viewing app, AdSmart plays an appropriate 15-second video ad. After the ad ends, if the user shows interest, they will be provided with a link to a page with more details.

[0208] Prompt Sentence Examples

[0209] "Create a 15-second video ad based on a template design and layout generation. This ad is for a new product promotion campaign. Make sure it's visually appealing and compels users to click."

[0210] The above is a specific embodiment for carrying out the invention, which makes it possible to provide a personalized advertisement optimized for a user.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The server uses the template design and layout generation algorithm to generate ad formats, including various video and banner ads. The input is the template design configuration information, and the output is the generated ad format.

[0214] Step 2:

[0215] The server distributes the generated ad formats to external businesses via APIs or download links. The input is ad format data, and the output is an ad format that can be used by external businesses. Specifically, the ad format is retrieved from the server's storage and sent to the external business.

[0216] Step 3:

[0217] The user device receives a request from the server, retrieves the advertising data, and stores it locally. The input is the advertising data sent from the server, and the output is the advertising data stored on the user device. Specifically, the device receives the data using its receiving function and stores it in its internal storage.

[0218] Step 4:

[0219] The server collects user behavioral data and feedback and analyzes it using an AI model (e.g., TensorFlow). The input is user behavioral data and feedback information, and the output is the user's behavioral patterns and areas of interest. Specifically, the collected data is input into the AI ​​model and the analysis results are obtained.

[0220] Step 5:

[0221] The server provides the generative AI model with prompt sentences to generate personalized ads for users based on the analysis results. The inputs are the analysis results and a basic prompt sentence template, and the output is a personalized prompt sentence. Specifically, the prompt sentence template is dynamically edited based on the analysis results.

[0222] Step 6:

[0223] The user device displays the generated personalized advertisement to the user at an appropriate time. The input is the personalized advertisement data and the user's current operation status, and the output is the advertisement displayed to the user. Specifically, the advertisement is displayed before the main content while watching a video or when changing pages in an electronic magazine.

[0224] Step 7:

[0225] The user terminal records the history of displayed advertisements and the user's reactions, and periodically transmits them to the server. The input is the advertisement viewing history and reaction data, and the output is the history data sent to the server. Specifically, interaction information is collected when the advertisement ends and uploaded to the server.

[0226] Step 8:

[0227] The server optimizes the next ad display based on the collected feedback and ad viewing history. The input is the collected viewing history and feedback data, and the output is an improved setting for the next ad display. Specifically, it analyzes past data and optimizes the timing and content of the next ad to be displayed.

[0228] The above are the specific processing steps for realizing an advertising ecosystem.

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] The present invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, a user, and an emotion engine that recognizes the user's emotions. Each role and process will be explained in detail below.

[0231] 1. Server Processing

[0232] Ad format generation

[0233] The server generates various ad formats, including 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template design and layout generation algorithms.

[0234] Ad format distribution

[0235] The server provides format data via API or download link to distribute the generated ad formats to external businesses, allowing AI businesses to create ads using standardized ad formats.

[0236] Advertising sales

[0237] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[0238] Format Conversion

[0239] The server uses AI to convert ad formats in response to advertiser requests, such as changing a 15-second ad to a 30-second one. It can also optimize ad formats based on user sentiment data.

[0240] 2. Terminal Processing

[0241] Receiving and storing advertising data

[0242] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, including the generated and converted formats.

[0243] Displaying ads

[0244] When a user watches a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[0245] Recording and sending advertising history

[0246] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[0247] 3. User Actions

[0248] Watching an ad

[0249] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[0250] Providing feedback

[0251] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, using the device's feedback function. This feedback is used to optimize future ad formats.

[0252] 4. Emotion Engine Processing

[0253] User Emotion Recognition

[0254] The device uses an emotion engine to recognize the user's emotions in real time, which analyzes the user's facial expressions, vocal tone, and other physical responses to generate emotion data.

[0255] Sending emotional data

[0256] The device sends the recognized emotion data to a server, which uses this data to optimize ad formats. For example, ad formats that the user finds interesting can be displayed preferentially, or ad formats that the user finds annoying can be eliminated.

[0257] Specific examples

[0258] Server processing example

[0259] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[0260] Terminal processing example

[0261] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0262] User processing example

[0263] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0264] Emotion engine processing example

[0265] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0266] In this way, the present invention provides a system that integrates the creation, distribution, sale, and optimization of advertisements, and realizes advertisement delivery that takes user emotions into consideration, thereby realizing an advertisement delivery environment that is valuable to both advertisers and users.

[0267] The processing flow will be explained below.

[0268] 1. Server Processing

[0269] Step 1:

[0270] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template designs and layout generation algorithms.

[0271] Step 2:

[0272] The server distributes the generated ad format to external businesses, and transfers the format data to the external businesses' systems via API, allowing them to use it.

[0273] Step 3:

[0274] The server sells the generated ad formats to advertisers, who can select and purchase the formats they need using a dashboard provided by the server.

[0275] Step 4:

[0276] The server uses AI to convert ad formats based on requests from advertisers, for example, changing a 15-second ad format to a 30-second one.

[0277] Step 5:

[0278] The server then distributes the converted ad format back to external businesses and advertisers, providing the converted format data using an API.

[0279] Step 6:

[0280] The server receives the user's emotion data sent from the emotion engine and optimizes the ad format based on this data, for example by setting it to preferentially display ad formats that the user is interested in.

[0281] 2. Terminal Processing

[0282] Step 1:

[0283] The terminal receives the advertisement data sent from the server, including the generated and converted format.

[0284] Step 2:

[0285] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[0286] Step 3:

[0287] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. The ad data is called up and displayed in the video player.

[0288] Step 4:

[0289] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[0290] Step 5:

[0291] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[0292] 3. User Actions

[0293] Step 1:

[0294] Users view advertisements provided through their terminals, such as advertisements played before the main content of a video service or advertisements displayed around the cover of an electronic magazine.

[0295] Step 2:

[0296] The user can provide feedback on the advertisement, for example, by inputting into the terminal whether the advertisement is interesting, relevant, annoying, etc.

[0297] 4. Emotion Engine Processing

[0298] Step 1:

[0299] The device uses an emotion engine to recognize the user's emotions in real time, for example by analyzing the user's facial expressions with a camera and using voice tones and physical reactions to generate emotion data.

[0300] Step 2:

[0301] The emotion data recognized by the emotion engine is sent from the device to the server. The emotion data includes whether the user was interested, enjoyed, or displeased.

[0302] Step 3:

[0303] The server optimizes the display of ad formats based on the emotion data. For example, it prioritizes displaying formats that the user has previously shown interest in, and avoids displaying formats that the user has previously shown discomfort with.

[0304] Specific examples

[0305] Server processing example

[0306] The server generates a 15-second video ad format and distributes it to external businesses via API. The external businesses use this format to create ad content and provide it to the server. The server sells the ads to advertisers, who then use the purchased format.

[0307] Terminal processing example

[0308] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0309] User processing example

[0310] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0311] Emotion engine processing example

[0312] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0313] Example 2

[0314] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0315] In the modern advertising ecosystem, there is a need to maximize convenience and effectiveness for both advertisers and users. However, in existing systems, ad format generation, distribution, sales, and conversion are carried out independently, resulting in a lack of consistency. Furthermore, technology to utilize user sentiment data to achieve more effective ad delivery is not yet fully developed. Furthermore, there is a lack of means to properly manage and utilize ad display timing and feedback, making it difficult to optimize ad delivery.

[0316] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating advertisement formats, a means for distributing the generated advertisement formats to external businesses, a means for selling advertisement formats, a means for converting advertisement formats into other formats, and a means for collecting user emotion data and optimizing advertisement formats based on the data. This enables the consistent generation, distribution, sale, conversion, and optimization of advertisement formats, realizing an effective advertising ecosystem for both advertisers and users. Furthermore, by optimizing advertisements based on user emotion data collected by the server, advertisement delivery that is less irritating to users can be achieved. Furthermore, a system is provided that can measure and optimize advertisement effectiveness in an integrated manner, by appropriately receiving advertisement data, managing display timing, recording and transmitting advertisement history to the server, and collecting and optimizing user feedback.

[0317] "Ad Format" means the prescribed format and structure for delivering advertising content.

[0318] An "external business operator" is a third party that uses the advertising format provided by the server, and refers to an organization or individual that distributes advertisements, such as advertisers.

[0319] An "advertiser" is a company or individual that posts advertisements for its products or services and provides advertising content to appeal to users.

[0320] "Emotional data" refers to data that reflects a user's emotions, and refers to information obtained by analyzing facial expressions, vocal tone, and other physical reactions.

[0321] An "emotion engine" is a device or software that recognizes a user's emotions in real time and generates emotion data.

[0322] "Device" refers to the electronic device (e.g., smartphone, tablet, or PC) used by a user to view advertising content.

[0323] "Advertisement history" is a record of information about advertisements viewed by a user, including display time and interaction information.

[0324] "Feedback" refers to opinions and ratings provided by users regarding advertisements.

[0325] MODE FOR CARRYING OUT THE INVENTION

[0326] This invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system operates based on the roles of the server, terminal, user, and emotion engine that recognizes the user's emotions. Specific ways in which the present invention is implemented are described below.

[0327] Server Processing

[0328] The server generates ad formats using template designs and layout generation algorithms, typically using software like Adobe Illustrator or Canva. For example, the server creates 15-second and 30-second video ad formats, magazine cover ad formats, and web rectangle ad formats.

[0329] The generated ad formats are distributed to external businesses via the server using APIs and download links, allowing AI businesses to create ad content using standardized ad formats.

[0330] The server also provides a dashboard for advertisers, allowing them to purchase the ad formats they need. Online payments are made using payment gateways such as Stripe or PayPal. After purchase, advertisers apply their ad content to the formats using the provided online tools (e.g., Google Ads, Facebook Ads Manager).

[0331] The server then uses AI models (e.g., TensorFlow, PyTorch) to convert ad formats, such as changing a 15-second ad to a 30-second one, and optimizes ad formats based on user sentiment data using IBM Watson's sentiment analysis API.

[0332] Terminal handling

[0333] The device receives the advertising data sent from the server and saves it in local storage (e.g., an SQLite database). The received advertising data is displayed at the appropriate time while the user is using a video service or reading an e-magazine. Specifically, a 15-second video advertisement is played using the device's video player (e.g., an HTML5 video tag). In addition, e-magazine apps use a PDF rendering library to display advertisements around the cover.

[0334] The device records the history of the ads displayed and uploads it to a server at regular intervals. Information such as the display time and user clicks is saved and used to measure the effectiveness of the ads.

[0335] User Action

[0336] Users view advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while viewing. For example, advertisements displayed before the main content while using a video service are provided in a size and quality that is appropriate for the user.

[0337] Users can provide feedback on ads, for example, by entering comments about whether they found the ad interesting or annoying using the device's feedback function. This feedback is sent to the server in real time and used to optimize future ad formats.

[0338] Emotion engine processing

[0339] The device uses an emotion engine to recognize the user's emotions in real time, analyzing facial expressions, vocal tone, and other bodily responses to generate emotion data, using technologies such as FaceReader and IBM Watson Tone Analyzer.

[0340] The recognized emotion data is sent from the device to the server. This data is used to optimize ad formats, resulting in more effective ad display. For example, ad formats that the user finds interesting can be prioritized, or ad formats that the user finds annoying can be eliminated.

[0341] Examples of specific examples and prompts

[0342] Server processing example

[0343] The server generates 15-second video ad formats and distributes them to external businesses. The external businesses use these formats to create ad content and provide it to the server. The server sells these ads to advertisers, who then use the purchased formats in their own campaigns.

[0344] Terminal processing example

[0345] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0346] User processing example

[0347] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0348] Emotion engine processing example

[0349] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0350] Example prompt sentence:

[0351] "Optimize your ad formats based on users' emotional data. Emotional data includes facial expressions, vocal tone, and other bodily responses."

[0352] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0353] Step 1: Generate Ad Formats

[0354] The server starts generating ad formats. As input, it is given a template design and a layout generation algorithm. Specific software used is Adobe Illustrator or Canva. The server uses these tools to create 15-second and 30-second video ad formats and magazine cover ad formats. As output, it generates the generated ad formats.

[0355] Step 2: Distributing Ad Formats

[0356] The server distributes the generated ad formats to external providers. The generated ad formats and a list of external providers are given as input. In this step, the format data is sent via API. Alternatively, the server generates a file download link and notifies the provider. Specifically, the data is sent using a RESTful API. As output, the external provider receives the ad formats.

[0357] Step 3: Selling ads

[0358] The server provides a dashboard for advertisers. As input, it receives the ad format, advertiser information, and payment information. The advertiser selects the required ad format on the dashboard and makes an online payment (e.g., Stripe, PayPal). After completing the purchase, the advertiser applies their ad content to the format using the provided online tools (e.g., Google Ads, Facebook Ads Manager). As output, the advertiser receives the ad format and uses it in their campaign.

[0359] Step 4: Convert the format

[0360] The server uses an AI model to convert ad formats. It receives advertiser requests and existing ad formats as input. It uses TensorFlow or PyTorch to convert, for example, a 15-second ad to a 30-second one. It also uses IBM Watson's sentiment analysis API to optimize the ad format based on user sentiment data. The converted ad format is generated as output.

[0361] Step 5: Receiving and storing advertising data

[0362] The device receives advertising data sent from the server. As input, the advertising data sent from the server is given. The device saves this in local storage (e.g., SQLite database). Specifically, the device downloads the data using an HTTP request and saves it in SQLite. As output, the advertising data is saved on the device.

[0363] Step 6: Displaying the Ad

[0364] The device displays the advertisement. As input, it receives the locally stored advertisement data. When the user uses a video service or digital magazine, it plays a 15-second video advertisement using a video player (e.g., HTML5 video tag). It also uses a PDF rendering library to display an advertisement around the cover of the digital magazine. As output, the advertisement is displayed to the user.

[0365] Step 7: Record and submit your ad history

[0366] The device records the history of displayed ads. As input, it receives information about the displayed ads (e.g., display time, user click information). The device stores this information in an SQLite database. Periodically, it uploads the history data to the server using HTTP POST. As output, the ad history is sent to the server and used for performance measurement.

[0367] Step 8: Watch an ad

[0368] The user watches the advertisement through the device. As input, an advertisement played on the device's video player or an advertisement on the cover of an electronic magazine is given. The user watches the advertisement and performs a specific interaction (e.g., click, skip). As output, the advertisement is watched by the user.

[0369] Step 9: Provide feedback

[0370] Users provide feedback on ads. As input, a feedback form about the ads they watched is provided. Users enter their opinions and ratings in the form and click the submit button. As output, the feedback data is sent to the server and used to optimize future ad formats.

[0371] Step 10: Emotion Recognition

[0372] The device uses an emotion engine to recognize the user's emotions. The input is the user's facial expressions and vocal tone. The emotion engine analyzes this data and generates emotion data. Specifically, it uses the built-in camera and microphone, and employs FaceReader and IBM Watson Tone Analyzer. The output is the recognized emotion data.

[0373] Step 11: Sending Emotion Data

[0374] The device sends the recognized emotion data to the server. As input, the recognized emotion data is given. This data is sent to the server using an HTTP POST request. As output, the emotion data is stored on the server and used to optimize ad formats.

[0375] Through these processing steps, the system can consistently generate, optimize, distribute, and sell ad formats, as well as display ads using user emotion data, thereby realizing a valuable advertising ecosystem for both advertisers and users.

[0376] (Application example 2)

[0377] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0378] Conventional advertising systems do not take into account the user's emotional state, making it difficult to maximize advertising effectiveness. Furthermore, ads are not displayed based on the user's interests or emotions, resulting in a poor user experience. Furthermore, real-time data collection and ad display optimization are insufficient, making it difficult for advertisers to measure advertising effectiveness.

[0379] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating an advertisement format, means for distributing the generated advertisement format to an external business, means for selling the advertisement format, means for converting the advertisement format into another format, emotion recognition means for recognizing a user's emotion in real time and transmitting the data to the server, and means for optimizing the display method of the advertisement format based on the user's emotion data. This makes it possible to display advertisements taking into account the user's emotion and maximize the effectiveness of the advertisements. In addition, real-time data collection and optimization of advertisement display are realized, making it easier for advertisers to measure effectiveness.

[0380] An "advertising format" is a standard format that defines the layout and design of advertising content.

[0381] "Third Party" means any other company or individual that creates and distributes advertising content using the Ad Format.

[0382] An "emotion recognition means" is a device or software that recognizes a user's emotional state in real time by analyzing the user's facial expressions, vocal tone, and other bodily responses.

[0383] "Server" means the computer system that generates, distributes, sells, and converts advertising formats, and receives and analyzes emotional data.

[0384] "User terminal" refers to a device on which a user views advertisements, such as a smartphone, smart glasses, or head-mounted display.

[0385] "Feedback" refers to opinions and impressions that users provide about advertisements.

[0386] "Ad display optimization" refers to the process of adjusting how ads are displayed to maximize their effectiveness based on collected data and feedback.

[0387] "Real-time" means near-immediate processing and response with little delay.

[0388] "Ad display history" refers to a record of the type, time, and interaction information of the ads displayed by the user.

[0389] The present invention is a system for generating, distributing, selling, converting, and optimizing advertising formats based on user emotion data. This system mainly operates in cooperation with a server, a terminal, a user, and an emotion recognition means.

[0390] Server Processing

[0391] The server has the ability to generate ad formats and distribute them to external parties. The server provides a dashboard for advertisers to purchase, apply, and change ad formats. The server also has the ability to convert ad formats into other formats, for example, converting a 15-second ad into a 30-second ad.

[0392] Furthermore, the emotional data of the user is received through the emotion recognition means, and the display method of the advertisement format is optimized based on the received data. The emotional data when the user watches the advertisement is analyzed in real time, and the content and display method of the advertisement are adjusted based on the specific emotional response.

[0393] Terminal handling

[0394] The device receives the advertisement data sent from the server and displays it to the user at the appropriate time. For example, if the user is wearing smart glasses, the device displays the advertisement through the glasses. The device also records the history of the displayed advertisements and sends it to the server.

[0395] The device uses an emotion recognition means to recognize the user's emotions in real time. This emotion data is generated by analyzing facial expressions, vocal tone, and other physical reactions, and is transmitted to a server for use in optimizing advertisement display.

[0396] User Action

[0397] Users can view advertisements provided through their devices and provide feedback on the advertisements, including opinions on whether the advertisements were interesting, relevant, annoying, etc. This feedback is collected through the devices and transmitted to a server.

[0398] Hardware and software used

[0399] Hardware:

[0400] Smart glasses (with camera)

[0401] Regular smartphones and tablets

[0402] software:

[0403] OpenCV: Used for emotion recognition.

[0404] requests: Used to communicate with the server.

[0405] Pre-trained Emotion Recognition Model.

[0406] Specific examples

[0407] While wearing the smart glasses, a user walks around town and a new advertisement is displayed every five minutes. While the user is looking at the advertisement, a camera analyzes their facial expressions and sends them to a server. The server uses this data to optimize the next advertisement to be displayed.

[0408] For example, if the smart glasses screen displays a message saying, "New drink campaign! Click here!" and the user responds, this information will also be sent to the server. Based on the emotional data, the next advertisement display will be further optimized.

[0409] Example of input prompt for generative AI model

[0410] I would like to develop an application for smart glasses that recognizes the user's facial expressions in real time while watching advertisements and sends the emotion data to a server. The content of the advertisements will be optimized based on the user's emotion data and displayed at regular intervals. Please explain in detail how to build this application.

[0411] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0412] Step 1:

[0413] The server generates ad formats. The input is basic ad materials (images, videos, text) provided by the advertiser, and the output is a formatted ad format. Specifically, the server uses template design and layout generation algorithms to generate 15-second or 30-second video ads, magazine cover ads, online rectangular ads, etc.

[0414] Step 2:

[0415] The server distributes the generated ad format. The input is the ad format generated in step 1, and the output is the format data provided to external businesses. Specifically, the server provides the ad format to external businesses via an API or download link.

[0416] Step 3:

[0417] The server sells ad formats. The input is the advertiser's purchase request, and the output is the sold ad format. Specifically, the server allows advertisers to purchase the ad formats they need through a dashboard, and advertisers use online tools to apply their ad content to the formats.

[0418] Step 4:

[0419] The server converts the ad format. The input is the advertiser's conversion request and emotional data, and the output is the converted ad format. Specifically, the server uses AI to change the ad format according to the advertiser's request, for example, changing a 15-second ad to 30 seconds. The server also optimizes the ad format based on the user's emotional data.

[0420] Step 5:

[0421] The device receives advertising data from the server. The input is the advertising data sent from the server, and the output is the locally stored advertising data. Specifically, the device periodically communicates with the server and downloads new advertising data.

[0422] Step 6:

[0423] The device displays advertisements to the user. The input is the saved advertisement data, and the output is the displayed advertisement. Specifically, when a user uses a video service, the device plays a 15- or 30-second advertisement before the main content, and inserts a cover advertisement while reading an e-magazine.

[0424] Step 7:

[0425] The device records the history of ads displayed and sends it to the server. The input is the user's ad viewing data, and the output is the history data sent to the server. Specifically, the device records information such as the time the ad was displayed and the user's interactions, and periodically sends this information to the server.

[0426] Step 8:

[0427] The device uses emotion recognition means to recognize the user's emotions in real time. The input is the user's facial expressions and voice, and the output is emotion data. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes them using an emotion recognition algorithm.

[0428] Step 9:

[0429] The device sends the recognized emotion data to the server. The input is the emotion data analyzed in real time, and the output is the emotion data sent to the server. Specifically, the device sends the recognized emotion data to the server via the network.

[0430] Step 10:

[0431] The user provides feedback on the advertisement. The input is the feedback information provided by the user, and the output is the feedback data recorded on the terminal. Specifically, the user uses the feedback function of the terminal to input their opinion on whether the advertisement was interesting, useful, or unpleasant.

[0432] Step 11:

[0433] The terminal sends the collected feedback to the server. The input is the user's feedback data, and the output is the feedback data sent to the server. Specifically, the terminal periodically sends the collected feedback to the server.

[0434] Step 12:

[0435] The server optimizes the display method of the ad format based on the collected emotional data and feedback. The input is the emotional data and feedback, and the output is the optimized ad display method. Specifically, the server analyzes the emotional data and feedback, evaluates which format and content of the ad is effective, and reflects this in the next ad display.

[0436] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0437] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0438] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0439] [Second embodiment]

[0440] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0441] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0442] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0443] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0444] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0445] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0446] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0447] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0448] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0449] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0450] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0451] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0452] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, and a user. The roles and processes of each are described in detail below.

[0453] 1. Server Processing

[0454] Ad format generation

[0455] The server generates various ad formats, including 15- and 30-second video ads, magazine cover ads, online rectangle ads, and in-stream ads, using template design and layout generation algorithms.

[0456] Ad format distribution

[0457] The server provides the format data via API or download link to distribute the generated ad formats to AI operators, allowing them to create ads using standardized ad formats.

[0458] Advertising sales

[0459] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[0460] Format Conversion

[0461] The server uses AI to change the size and format of ad formats in response to advertiser requests. For example, it may change a 15-second ad to a 30-second one, or convert a rectangular ad into an in-stream ad. These converted formats are then redistributed to AI providers and advertisers.

[0462] 2. Terminal Processing

[0463] Receiving and storing advertising data

[0464] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, which includes the advertisement format generated and converted by the server.

[0465] Displaying ads

[0466] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[0467] Recording and sending advertising history

[0468] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[0469] 3. User Actions

[0470] Watching an ad

[0471] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[0472] Providing feedback

[0473] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, by sending it to the server via their device. This feedback is used to optimize future ad formats.

[0474] Specific examples

[0475] Server processing example

[0476] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[0477] Terminal processing example

[0478] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0479] User processing example

[0480] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0481] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0482] The processing flow will be explained below.

[0483] 1. Server Processing

[0484] Step 1:

[0485] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats.

[0486] Step 2:

[0487] The server distributes the generated ad format to external businesses and transfers the format data to the external businesses' systems via API.

[0488] Step 3:

[0489] The server sells the generated ad formats to advertisers, providing a dashboard for advertisers to select and purchase the formats they need.

[0490] Step 4:

[0491] The server uses AI to convert ad formats based on requests from advertisers, such as changing a video ad from 15 seconds to 30 seconds.

[0492] Step 5:

[0493] The server then distributes the converted ad format back to external businesses and advertisers, providing format data via API or download link.

[0494] 2. Terminal Processing

[0495] Step 1:

[0496] The terminal receives the advertisement data from the server, the advertisement data including the generated and converted format.

[0497] Step 2:

[0498] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[0499] Step 3:

[0500] When a user uses a video service, the device displays an advertisement before the main content. The advertisement data is called from within the video service application and displayed in the video player.

[0501] Step 4:

[0502] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[0503] Step 5:

[0504] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[0505] 3. User Actions

[0506] Step 1:

[0507] The user watches advertisements provided through the terminal, such as advertisements played before the main video of a video service or advertisements displayed around the cover of an electronic magazine.

[0508] Step 2:

[0509] The user provides feedback on the advertisement, for example, whether the advertisement is interesting, relevant, annoying, etc., by using the feedback function of the device.

[0510] Step 3:

[0511] User feedback is sent from the device to the server, which uses the collected feedback to optimize future ads.

[0512] Specific examples

[0513] Server processing example

[0514] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells the ads to advertisers, who then use the purchased format.

[0515] Terminal processing example

[0516] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0517] User processing example

[0518] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0519] Example 1

[0520] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0521] In conventional ad delivery systems, the creation, distribution, sales, and optimization of ad formats are performed piecemeal, resulting in inefficiencies for both advertisers and users. Conversion between different ad formats is also performed manually, which is time-consuming and costly. Furthermore, even if user feedback is collected, there is a lack of a way to effectively incorporate it, making ad optimization difficult.

[0522] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0523] In this invention, the server includes a means for generating ad formats, a means for distributing the generated ad formats to external businesses via an API or a download link, a means for providing a dashboard for advertisers to sell the ad formats, and a means for converting the size and format of the ad formats using a machine learning algorithm in response to requests from advertisers. This makes it possible to consistently and efficiently perform processes from generating, distributing, selling, and optimizing ad formats.

[0524] "Ad format" refers to the template or design layout used to display advertising content.

[0525] "External businesses" are companies or organizations that use advertising formats to create the actual advertising content.

[0526] "API" stands for Application Programming Interface, an interface for exchanging functions and data between software programs.

[0527] A "download link" is a URL that allows a user to save a specific file or data from the Internet to their device.

[0528] A "dashboard" is a user interface that allows a user to visually display and manipulate the information and data they need.

[0529] A "machine learning algorithm" is a program or method that allows a computer to automatically learn using large amounts of data to perform specific tasks.

[0530] "Local storage" refers to a data storage device or storage medium installed within a terminal.

[0531] A "user" is a consumer who uses a video service or an electronic magazine.

[0532] "Feedback" refers to the evaluations and opinions of advertisements provided by users who have viewed them.

[0533] The present invention provides an advertisement distribution system that is efficient and effective for both advertisers and users by generating, distributing, selling, and converting advertisement formats. Detailed embodiments of the system are described below.

[0534] Server embodiment

[0535] Ad format generation

[0536] The server uses template design and layout generation algorithms to generate various ad formats, such as 15-second and 30-second video ads, magazine cover ads, online rectangle ads, in-stream ads, etc. This generation is done using image processing libraries such as Python's OpenCV and Pillow.

[0537] Ad format distribution

[0538] The server distributes the generated ad formats to external businesses via API or download links. This distribution utilizes a RESTful API, allowing external businesses to efficiently provide the ad format data they require.

[0539] Advertising sales

[0540] The server provides a dashboard for advertisers, allowing them to select and purchase the ad formats they need. This dashboard is built using React and Vue.js, making it intuitive and easy for users to use.

[0541] Ad format conversion

[0542] The server uses machine learning algorithms (e.g., TensorFlow, PyTorch) to convert the size and format of the ad format according to the advertiser's request. For example, if a 15-second video ad is extended to 30 seconds, additional content is generated based on the original ad content. This converted ad format is also distributed via API.

[0543] Terminal embodiment

[0544] Receiving and storing advertising data

[0545] The device receives the advertising data sent from the server and saves it in local storage (e.g., SQLite database). The receiving process is performed periodically, and new advertising data is downloaded and saved as needed.

[0546] Displaying ads

[0547] The device displays the saved advertisements when the user uses video services or digital magazines. This display uses HTML5 and JavaScript, and in video services, the advertisements are displayed before the main video. In digital magazines, advertisements are inserted around the cover while the user is reading.

[0548] Recording and sending advertising history

[0549] The device records the history of the displayed ads (playback time, user interaction information, etc.) and periodically sends it to the server. This history information is collected in JSON format and sent to the server via a REST API.

[0550] User's embodiment

[0551] Watching an ad

[0552] Users watch ads delivered through their devices. The UI is intuitive and designed to allow users to watch ads naturally. For example, users are presented with a 15-second full-screen ad before the main content of a video service.

[0553] Providing feedback

[0554] Users send feedback about ads to the server via their devices. For example, they can send their opinions on whether the ad was interesting, relevant, or annoying through a simple UI within the app. The feedback information is stored in a database and used to optimize future ads.

[0555] Examples and prompts

[0556] Server processing example

[0557] The server generates 15-second video ad formats and distributes them to third parties, who then use them to create advertising content. The server then sells the ads to advertisers, who then use the purchased formats in their own campaigns.

[0558] Terminal processing example

[0559] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0560] User processing example

[0561] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0562] Prompt sentence for generative AI model

[0563] "Generate a 15-second video ad format and distribute it to third parties."

[0564] "Collect user viewing history and feedback information to optimize advertising effectiveness."

[0565] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0566] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0567] Step 1:

[0568] The server loads the template design data and starts generating ad formats. Specifically, it uses Python's Pillow library to generate image-based templates and OpenCV to assemble video frames to create 15-second and 30-second video ads. It uses the template design data as input and obtains the generated ad format files as output.

[0569] Step 2:

[0570] The server distributes the generated ad formats to external businesses via APIs and download links. Specifically, a RESTful API is built so that external businesses can download format data by sending an HTTP request. The generated ad format file is used as input, and a download link is obtained as output, which is shared with external businesses.

[0571] Step 3:

[0572] The server sells ad formats through a dashboard for advertisers. Specifically, we designed a user interface using React and Vue.js to allow advertisers to easily select and purchase the format they need. The server receives an advertiser's purchase request as input and obtains the purchased format data and a receipt as output.

[0573] Step 4:

[0574] The server uses machine learning algorithms to convert the size and format of ad formats based on the advertiser's requests. Specifically, it uses TensorFlow and PyTorch to perform processes such as extending a 15-second video ad to 30 seconds. The server uses the advertiser's conversion request and the original ad format as input, and obtains the converted ad format as output.

[0575] Step 5:

[0576] The device receives the advertising data sent from the server and stores it in local storage (e.g., SQLite database). Specifically, it periodically downloads new advertising data from the server and stores it appropriately in local storage. It uses the advertising data from the server as input and obtains the stored advertising data as output.

[0577] Step 6:

[0578] The device displays the stored advertising data when the user uses video services or digital magazines. Specifically, it uses HTML5 and JavaScript to play and display advertisements to the user. It uses the advertising data from local storage as input and obtains the displayed advertisement as output.

[0579] Step 7:

[0580] The device records the history of displayed ads and periodically sends it to the server. Specifically, it collects display time and user interaction information in JSON format and sends it to the server via REST API. It uses user interaction information and display history as input and obtains the ad history data sent to the server as output.

[0581] Step 8:

[0582] Users view advertisements provided through their devices. The UI is intuitive and designed to allow users to view advertisements naturally. The input is the advertisement content provided by the device, and the output is the information about the advertisements viewed.

[0583] Step 9:

[0584] Users send feedback about ads to the server via their devices. Specifically, they use a simple UI within the app to input their opinions, such as "interesting," "useful," or "unpleasant." The system uses user feedback as input and obtains the feedback information sent to the server as output.

[0585] (Application example 1)

[0586] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0587] Conventional ad delivery systems lack sufficient personalization based on user interests and behavior, limiting the effectiveness of ads. Furthermore, if the timing or content of ads is inappropriate for the user, they can be annoying. Therefore, there is a need for a system that efficiently utilizes user behavior data and feedback to deliver ads with optimal timing and content.

[0588] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0589] In this invention, the server includes means for generating ad formats, means for distributing the generated ad formats to external businesses, means for selling ad formats, means for converting ad formats into other formats, means for personalizing ad formats based on user behavior data and feedback, and means for selecting an optimal ad format based on user behavior analysis using an AI model, thereby making it possible to provide users with personalized ads that are optimized for them.

[0590] An "ad format" is a template or design layout that defines the form and structure of advertising content.

[0591] An "external business" is a third-party business that creates advertising content using the advertising format.

[0592] "Personalization" refers to individually optimizing advertising content based on user behavioral data and feedback.

[0593] An "AI model" is a computational model that uses machine learning technology to analyze and learn from data, and make predictions and optimizations.

[0594] A "prompt sentence" is input text that provides specific instructions or explanations to a generative AI model.

[0595] "Server" means a centralized management system that generates, distributes, sells, and converts advertising formats.

[0596] A "user terminal" is a device (e.g., a smartphone, a tablet, etc.) that a user uses to view advertising content.

[0597] "Behavioral data" refers to records of the actions and reactions users take on advertisements and content.

[0598] "Feedback" refers to ratings and opinions provided by users who have viewed an advertisement.

[0599] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system of the present invention operates based on the roles of a server, a user terminal, and a user.

[0600] Server Processing

[0601] Ad format generation

[0602] The server generates various ad formats using template designs and layout generation algorithms, resulting in ad content in a variety of formats, including video ads and banner ads.

[0603] Ad format distribution

[0604] The generated ad formats are distributed to external businesses via APIs or download links via the server, allowing external businesses to create effective ad content using standardized ad formats.

[0605] Collecting user behavior data and feedback

[0606] The server collects and analyzes the behavioral data and feedback sent by users using an AI model (e.g., TensorFlow). This analysis extracts the user's behavioral patterns and areas of interest.

[0607] Ad personalization

[0608] Based on the analyzed data, the server provides personalized advertisements to users, including the ability to dynamically generate input prompts for the generative AI model.

[0609] User terminal processing

[0610] Receiving and displaying advertising data

[0611] The user device receives advertising data from the server and displays the advertisements at the appropriate time. For example, when a user uses a video service, a 15- or 30-second video advertisement is played before the main content, and when a user is reading an e-magazine, an advertisement is displayed around the cover.

[0612] Recording and sending advertising history and responses

[0613] The user device records the history of the ads displayed (such as the display time and user interaction information) and periodically sends this information to the server. If the user reacts to the ad, this information is also recorded and reflected in the next ad display.

[0614] User Action

[0615] Ad viewing and feedback

[0616] Users can view ads delivered through their devices and provide feedback on the ads. The feedback includes opinions on whether the ad was interesting, relevant, annoying, etc., and is sent from the device to the server. This feedback is used to optimize future ad formats.

[0617] Specific examples

[0618] Ad delivery process

[0619] 1. Ad generation and distribution: The 15-second video ad format generated by the server is then used by an external company to create a video for a new product introduction campaign. The generated ad format is then delivered to smartphones and saved in the "AdSmart" app.

[0620] 2. User behavior analysis: A user's browsing history of products in a specific category on an online shopping site is sent to the server, where a machine learning model analyzes this data and predicts the user's interests.

[0621] 3. Display Ads: When a user opens a video viewing app, AdSmart plays an appropriate 15-second video ad. After the ad ends, if the user shows interest, they will be provided with a link to a page with more details.

[0622] Prompt Sentence Examples

[0623] "Create a 15-second video ad based on a template design and layout generation. This ad is for a new product promotion campaign. Make sure it's visually appealing and compels users to click."

[0624] The above is a specific embodiment for carrying out the invention, which makes it possible to provide a personalized advertisement optimized for a user.

[0625] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0626] Step 1:

[0627] The server uses the template design and layout generation algorithm to generate ad formats, including various video and banner ads. The input is the template design configuration information, and the output is the generated ad format.

[0628] Step 2:

[0629] The server distributes the generated ad formats to external businesses via APIs or download links. The input is ad format data, and the output is an ad format that can be used by external businesses. Specifically, the ad format is retrieved from the server's storage and sent to the external business.

[0630] Step 3:

[0631] The user device receives a request from the server, retrieves the advertising data, and stores it locally. The input is the advertising data sent from the server, and the output is the advertising data stored on the user device. Specifically, the device receives the data using its receiving function and stores it in its internal storage.

[0632] Step 4:

[0633] The server collects user behavioral data and feedback and analyzes it using an AI model (e.g., TensorFlow). The input is user behavioral data and feedback information, and the output is the user's behavioral patterns and areas of interest. Specifically, the collected data is input into the AI ​​model and the analysis results are obtained.

[0634] Step 5:

[0635] The server provides the generative AI model with prompt sentences to generate personalized ads for users based on the analysis results. The inputs are the analysis results and a basic prompt sentence template, and the output is a personalized prompt sentence. Specifically, the prompt sentence template is dynamically edited based on the analysis results.

[0636] Step 6:

[0637] The user device displays the generated personalized advertisement to the user at an appropriate time. The input is the personalized advertisement data and the user's current operation status, and the output is the advertisement displayed to the user. Specifically, the advertisement is displayed before the main content while watching a video or when changing pages in an electronic magazine.

[0638] Step 7:

[0639] The user terminal records the history of displayed advertisements and the user's reactions, and periodically transmits them to the server. The input is the advertisement viewing history and reaction data, and the output is the history data sent to the server. Specifically, interaction information is collected when the advertisement ends and uploaded to the server.

[0640] Step 8:

[0641] The server optimizes the next ad display based on the collected feedback and ad viewing history. The input is the collected viewing history and feedback data, and the output is an improved setting for the next ad display. Specifically, it analyzes past data and optimizes the timing and content of the next ad to be displayed.

[0642] The above are the specific processing steps for realizing an advertising ecosystem.

[0643] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0644] The present invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, a user, and an emotion engine that recognizes the user's emotions. Each role and process will be explained in detail below.

[0645] 1. Server Processing

[0646] Ad format generation

[0647] The server generates various ad formats, including 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template design and layout generation algorithms.

[0648] Ad format distribution

[0649] The server provides format data via API or download link to distribute the generated ad formats to external businesses, allowing AI businesses to create ads using standardized ad formats.

[0650] Advertising sales

[0651] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[0652] Format Conversion

[0653] The server uses AI to convert ad formats in response to advertiser requests, such as changing a 15-second ad to a 30-second one. It can also optimize ad formats based on user sentiment data.

[0654] 2. Terminal Processing

[0655] Receiving and storing advertising data

[0656] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, including the generated and converted formats.

[0657] Displaying ads

[0658] When a user watches a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[0659] Recording and sending advertising history

[0660] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[0661] 3. User Actions

[0662] Watching an ad

[0663] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[0664] Providing feedback

[0665] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, using the device's feedback function. This feedback is used to optimize future ad formats.

[0666] 4. Emotion Engine Processing

[0667] User Emotion Recognition

[0668] The device uses an emotion engine to recognize the user's emotions in real time, which analyzes the user's facial expressions, vocal tone, and other physical responses to generate emotion data.

[0669] Sending emotional data

[0670] The device sends the recognized emotion data to a server, which uses this data to optimize ad formats. For example, ad formats that the user finds interesting can be displayed preferentially, or ad formats that the user finds annoying can be eliminated.

[0671] Specific examples

[0672] Server processing example

[0673] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[0674] Terminal processing example

[0675] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0676] User processing example

[0677] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0678] Emotion engine processing example

[0679] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0680] In this way, the present invention provides a system that integrates the creation, distribution, sale, and optimization of advertisements, and realizes advertisement delivery that takes user emotions into consideration, thereby realizing an advertisement delivery environment that is valuable to both advertisers and users.

[0681] The processing flow will be explained below.

[0682] 1. Server Processing

[0683] Step 1:

[0684] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template designs and layout generation algorithms.

[0685] Step 2:

[0686] The server distributes the generated ad format to external businesses, and transfers the format data to the external businesses' systems via API, allowing them to use it.

[0687] Step 3:

[0688] The server sells the generated ad formats to advertisers, who can select and purchase the formats they need using a dashboard provided by the server.

[0689] Step 4:

[0690] The server uses AI to convert ad formats based on requests from advertisers, for example, changing a 15-second ad format to a 30-second one.

[0691] Step 5:

[0692] The server then distributes the converted ad format back to external businesses and advertisers, providing the converted format data using an API.

[0693] Step 6:

[0694] The server receives the user's emotion data sent from the emotion engine and optimizes the ad format based on this data, for example by setting it to preferentially display ad formats that the user is interested in.

[0695] 2. Terminal Processing

[0696] Step 1:

[0697] The terminal receives the advertisement data sent from the server, including the generated and converted format.

[0698] Step 2:

[0699] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[0700] Step 3:

[0701] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. The ad data is called up and displayed in the video player.

[0702] Step 4:

[0703] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[0704] Step 5:

[0705] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[0706] 3. User Actions

[0707] Step 1:

[0708] Users view advertisements provided through their terminals, such as advertisements played before the main content of a video service or advertisements displayed around the cover of an electronic magazine.

[0709] Step 2:

[0710] The user can provide feedback on the advertisement, for example, by inputting into the terminal whether the advertisement is interesting, relevant, annoying, etc.

[0711] 4. Emotion Engine Processing

[0712] Step 1:

[0713] The device uses an emotion engine to recognize the user's emotions in real time, for example by analyzing the user's facial expressions with a camera and using voice tones and physical reactions to generate emotion data.

[0714] Step 2:

[0715] The emotion data recognized by the emotion engine is sent from the device to the server. The emotion data includes whether the user was interested, enjoyed, or displeased.

[0716] Step 3:

[0717] The server optimizes the display of ad formats based on the emotion data. For example, it prioritizes displaying formats that the user has previously shown interest in, and avoids displaying formats that the user has previously shown discomfort with.

[0718] Specific examples

[0719] Server processing example

[0720] The server generates a 15-second video ad format and distributes it to external businesses via API. The external businesses use this format to create ad content and provide it to the server. The server sells the ads to advertisers, who then use the purchased format.

[0721] Terminal processing example

[0722] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0723] User processing example

[0724] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0725] Emotion engine processing example

[0726] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0727] Example 2

[0728] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0729] In the modern advertising ecosystem, there is a need to maximize convenience and effectiveness for both advertisers and users. However, in existing systems, ad format generation, distribution, sales, and conversion are carried out independently, resulting in a lack of consistency. Furthermore, technology to utilize user sentiment data to achieve more effective ad delivery is not yet fully developed. Furthermore, there is a lack of means to properly manage and utilize ad display timing and feedback, making it difficult to optimize ad delivery.

[0730] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating advertisement formats, a means for distributing the generated advertisement formats to external businesses, a means for selling advertisement formats, a means for converting advertisement formats into other formats, and a means for collecting user emotion data and optimizing advertisement formats based on the data. This enables the consistent generation, distribution, sale, conversion, and optimization of advertisement formats, realizing an effective advertising ecosystem for both advertisers and users. Furthermore, by optimizing advertisements based on user emotion data collected by the server, advertisement delivery that is less irritating to users can be achieved. Furthermore, a system is provided that can measure and optimize advertisement effectiveness in an integrated manner, by appropriately receiving advertisement data, managing display timing, recording and transmitting advertisement history to the server, and collecting and optimizing user feedback.

[0731] "Ad Format" means the prescribed format and structure for delivering advertising content.

[0732] An "external business operator" is a third party that uses the advertising format provided by the server, and refers to an organization or individual that distributes advertisements, such as advertisers.

[0733] An "advertiser" is a company or individual that posts advertisements for its products or services and provides advertising content to appeal to users.

[0734] "Emotional data" refers to data that reflects a user's emotions, and refers to information obtained by analyzing facial expressions, vocal tone, and other physical reactions.

[0735] An "emotion engine" is a device or software that recognizes a user's emotions in real time and generates emotion data.

[0736] "Device" refers to the electronic device (e.g., smartphone, tablet, or PC) used by a user to view advertising content.

[0737] "Advertisement history" is a record of information about advertisements viewed by a user, including display time and interaction information.

[0738] "Feedback" refers to opinions and ratings provided by users regarding advertisements.

[0739] MODE FOR CARRYING OUT THE INVENTION

[0740] This invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system operates based on the roles of the server, terminal, user, and emotion engine that recognizes the user's emotions. Specific ways in which the present invention is implemented are described below.

[0741] Server Processing

[0742] The server generates ad formats using template designs and layout generation algorithms, typically using software like Adobe Illustrator or Canva. For example, the server creates 15-second and 30-second video ad formats, magazine cover ad formats, and web rectangle ad formats.

[0743] The generated ad formats are distributed to external businesses via the server using APIs and download links, allowing AI businesses to create ad content using standardized ad formats.

[0744] The server also provides a dashboard for advertisers, allowing them to purchase the ad formats they need. Online payments are made using payment gateways such as Stripe or PayPal. After purchase, advertisers apply their ad content to the formats using the provided online tools (e.g., Google Ads, Facebook Ads Manager).

[0745] The server then uses AI models (e.g., TensorFlow, PyTorch) to convert ad formats, such as changing a 15-second ad to a 30-second one, and optimizes ad formats based on user sentiment data using IBM Watson's sentiment analysis API.

[0746] Terminal handling

[0747] The device receives the advertising data sent from the server and saves it in local storage (e.g., an SQLite database). The received advertising data is displayed at the appropriate time while the user is using a video service or reading an e-magazine. Specifically, a 15-second video advertisement is played using the device's video player (e.g., an HTML5 video tag). In addition, e-magazine apps use a PDF rendering library to display advertisements around the cover.

[0748] The device records the history of the ads displayed and uploads it to a server at regular intervals. Information such as the display time and user clicks is saved and used to measure the effectiveness of the ads.

[0749] User Action

[0750] Users view advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while viewing. For example, advertisements displayed before the main content while using a video service are provided in a size and quality that is appropriate for the user.

[0751] Users can provide feedback on ads, for example, by entering comments about whether they found the ad interesting or annoying using the device's feedback function. This feedback is sent to the server in real time and used to optimize future ad formats.

[0752] Emotion engine processing

[0753] The device uses an emotion engine to recognize the user's emotions in real time, analyzing facial expressions, vocal tone, and other bodily responses to generate emotion data, using technologies such as FaceReader and IBM Watson Tone Analyzer.

[0754] The recognized emotion data is sent from the device to the server. This data is used to optimize ad formats, resulting in more effective ad display. For example, ad formats that the user finds interesting can be prioritized, or ad formats that the user finds annoying can be eliminated.

[0755] Examples of specific examples and prompts

[0756] Server processing example

[0757] The server generates 15-second video ad formats and distributes them to external businesses. The external businesses use these formats to create ad content and provide it to the server. The server sells these ads to advertisers, who then use the purchased formats in their own campaigns.

[0758] Terminal processing example

[0759] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0760] User processing example

[0761] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0762] Emotion engine processing example

[0763] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[0764] Example prompt sentence:

[0765] "Optimize your ad formats based on users' emotional data. Emotional data includes facial expressions, vocal tone, and other bodily responses."

[0766] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0767] Step 1: Generate Ad Formats

[0768] The server starts generating ad formats. As input, it is given a template design and a layout generation algorithm. Specific software used is Adobe Illustrator or Canva. The server uses these tools to create 15-second and 30-second video ad formats and magazine cover ad formats. As output, it generates the generated ad formats.

[0769] Step 2: Distributing Ad Formats

[0770] The server distributes the generated ad formats to external providers. The generated ad formats and a list of external providers are given as input. In this step, the format data is sent via API. Alternatively, the server generates a file download link and notifies the provider. Specifically, the data is sent using a RESTful API. As output, the external provider receives the ad formats.

[0771] Step 3: Selling ads

[0772] The server provides a dashboard for advertisers. As input, it receives the ad format, advertiser information, and payment information. The advertiser selects the required ad format on the dashboard and makes an online payment (e.g., Stripe, PayPal). After completing the purchase, the advertiser applies their ad content to the format using the provided online tools (e.g., Google Ads, Facebook Ads Manager). As output, the advertiser receives the ad format and uses it in their campaign.

[0773] Step 4: Convert the format

[0774] The server uses an AI model to convert ad formats. It receives advertiser requests and existing ad formats as input. It uses TensorFlow or PyTorch to convert, for example, a 15-second ad to a 30-second one. It also uses IBM Watson's sentiment analysis API to optimize the ad format based on user sentiment data. The converted ad format is generated as output.

[0775] Step 5: Receiving and storing advertising data

[0776] The device receives advertising data sent from the server. As input, the advertising data sent from the server is given. The device saves this in local storage (e.g., SQLite database). Specifically, the device downloads the data using an HTTP request and saves it in SQLite. As output, the advertising data is saved on the device.

[0777] Step 6: Displaying the Ad

[0778] The device displays the advertisement. As input, it receives the locally stored advertisement data. When the user uses a video service or digital magazine, it plays a 15-second video advertisement using a video player (e.g., HTML5 video tag). It also uses a PDF rendering library to display an advertisement around the cover of the digital magazine. As output, the advertisement is displayed to the user.

[0779] Step 7: Record and submit your ad history

[0780] The device records the history of displayed ads. As input, it receives information about the displayed ads (e.g., display time, user click information). The device stores this information in an SQLite database. Periodically, it uploads the history data to the server using HTTP POST. As output, the ad history is sent to the server and used for performance measurement.

[0781] Step 8: Watch an ad

[0782] The user watches the advertisement through the device. As input, an advertisement played on the device's video player or an advertisement on the cover of an electronic magazine is given. The user watches the advertisement and performs a specific interaction (e.g., click, skip). As output, the advertisement is watched by the user.

[0783] Step 9: Provide feedback

[0784] Users provide feedback on ads. As input, a feedback form about the ads they watched is provided. Users enter their opinions and ratings in the form and click the submit button. As output, the feedback data is sent to the server and used to optimize future ad formats.

[0785] Step 10: Emotion Recognition

[0786] The device uses an emotion engine to recognize the user's emotions. The input is the user's facial expressions and vocal tone. The emotion engine analyzes this data and generates emotion data. Specifically, it uses the built-in camera and microphone, and employs FaceReader and IBM Watson Tone Analyzer. The output is the recognized emotion data.

[0787] Step 11: Sending Emotion Data

[0788] The device sends the recognized emotion data to the server. As input, the recognized emotion data is given. This data is sent to the server using an HTTP POST request. As output, the emotion data is stored on the server and used to optimize ad formats.

[0789] Through these processing steps, the system can consistently generate, optimize, distribute, and sell ad formats, as well as display ads using user emotion data, thereby realizing a valuable advertising ecosystem for both advertisers and users.

[0790] (Application example 2)

[0791] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0792] Conventional advertising systems do not take into account the user's emotional state, making it difficult to maximize advertising effectiveness. Furthermore, ads are not displayed based on the user's interests or emotions, resulting in a poor user experience. Furthermore, real-time data collection and ad display optimization are insufficient, making it difficult for advertisers to measure advertising effectiveness.

[0793] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating an advertisement format, means for distributing the generated advertisement format to an external business, means for selling the advertisement format, means for converting the advertisement format into another format, emotion recognition means for recognizing a user's emotion in real time and transmitting the data to the server, and means for optimizing the display method of the advertisement format based on the user's emotion data. This makes it possible to display advertisements taking into account the user's emotion and maximize the effectiveness of the advertisements. In addition, real-time data collection and optimization of advertisement display are realized, making it easier for advertisers to measure effectiveness.

[0794] An "advertising format" is a standard format that defines the layout and design of advertising content.

[0795] "Third Party" means any other company or individual that creates and distributes advertising content using the Ad Format.

[0796] An "emotion recognition means" is a device or software that recognizes a user's emotional state in real time by analyzing the user's facial expressions, vocal tone, and other bodily responses.

[0797] "Server" means the computer system that generates, distributes, sells, and converts advertising formats, and receives and analyzes emotional data.

[0798] "User terminal" refers to a device on which a user views advertisements, such as a smartphone, smart glasses, or head-mounted display.

[0799] "Feedback" refers to opinions and impressions that users provide about advertisements.

[0800] "Ad display optimization" refers to the process of adjusting how ads are displayed to maximize their effectiveness based on collected data and feedback.

[0801] "Real-time" means near-immediate processing and response with little delay.

[0802] "Ad display history" refers to a record of the type, time, and interaction information of the ads displayed by the user.

[0803] The present invention is a system for generating, distributing, selling, converting, and optimizing advertising formats based on user emotion data. This system mainly operates in cooperation with a server, a terminal, a user, and an emotion recognition means.

[0804] Server Processing

[0805] The server has the ability to generate ad formats and distribute them to external parties. The server provides a dashboard for advertisers to purchase, apply, and change ad formats. The server also has the ability to convert ad formats into other formats, for example, converting a 15-second ad into a 30-second ad.

[0806] Furthermore, the emotional data of the user is received through the emotion recognition means, and the display method of the advertisement format is optimized based on the received data. The emotional data when the user watches the advertisement is analyzed in real time, and the content and display method of the advertisement are adjusted based on the specific emotional response.

[0807] Terminal handling

[0808] The device receives the advertisement data sent from the server and displays it to the user at the appropriate time. For example, if the user is wearing smart glasses, the device displays the advertisement through the glasses. The device also records the history of the displayed advertisements and sends it to the server.

[0809] The device uses an emotion recognition means to recognize the user's emotions in real time. This emotion data is generated by analyzing facial expressions, vocal tone, and other physical reactions, and is transmitted to a server for use in optimizing advertisement display.

[0810] User Action

[0811] Users can view advertisements provided through their devices and provide feedback on the advertisements, including opinions on whether the advertisements were interesting, relevant, annoying, etc. This feedback is collected through the devices and transmitted to a server.

[0812] Hardware and software used

[0813] Hardware:

[0814] Smart glasses (with camera)

[0815] Regular smartphones and tablets

[0816] software:

[0817] OpenCV: Used for emotion recognition.

[0818] requests: Used to communicate with the server.

[0819] Pre-trained Emotion Recognition Model.

[0820] Specific examples

[0821] While wearing the smart glasses, a user walks around town and a new advertisement is displayed every five minutes. While the user is looking at the advertisement, a camera analyzes their facial expressions and sends them to a server. The server uses this data to optimize the next advertisement to be displayed.

[0822] For example, if the smart glasses screen displays a message saying, "New drink campaign! Click here!" and the user responds, this information will also be sent to the server. Based on the emotional data, the next advertisement display will be further optimized.

[0823] Example of input prompt for generative AI model

[0824] I would like to develop an application for smart glasses that recognizes the user's facial expressions in real time while watching advertisements and sends the emotion data to a server. The content of the advertisements will be optimized based on the user's emotion data and displayed at regular intervals. Please explain in detail how to build this application.

[0825] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0826] Step 1:

[0827] The server generates ad formats. The input is basic ad materials (images, videos, text) provided by the advertiser, and the output is a formatted ad format. Specifically, the server uses template design and layout generation algorithms to generate 15-second or 30-second video ads, magazine cover ads, online rectangular ads, etc.

[0828] Step 2:

[0829] The server distributes the generated ad format. The input is the ad format generated in step 1, and the output is the format data provided to external businesses. Specifically, the server provides the ad format to external businesses via an API or download link.

[0830] Step 3:

[0831] The server sells ad formats. The input is the advertiser's purchase request, and the output is the sold ad format. Specifically, the server allows advertisers to purchase the ad formats they need through a dashboard, and advertisers use online tools to apply their ad content to the formats.

[0832] Step 4:

[0833] The server converts the ad format. The input is the advertiser's conversion request and emotional data, and the output is the converted ad format. Specifically, the server uses AI to change the ad format according to the advertiser's request, for example, changing a 15-second ad to 30 seconds. The server also optimizes the ad format based on the user's emotional data.

[0834] Step 5:

[0835] The device receives advertising data from the server. The input is the advertising data sent from the server, and the output is the locally stored advertising data. Specifically, the device periodically communicates with the server and downloads new advertising data.

[0836] Step 6:

[0837] The device displays advertisements to the user. The input is the saved advertisement data, and the output is the displayed advertisement. Specifically, when a user uses a video service, the device plays a 15- or 30-second advertisement before the main content, and inserts a cover advertisement while reading an e-magazine.

[0838] Step 7:

[0839] The device records the history of ads displayed and sends it to the server. The input is the user's ad viewing data, and the output is the history data sent to the server. Specifically, the device records information such as the time the ad was displayed and the user's interactions, and periodically sends this information to the server.

[0840] Step 8:

[0841] The device uses emotion recognition means to recognize the user's emotions in real time. The input is the user's facial expressions and voice, and the output is emotion data. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes them using an emotion recognition algorithm.

[0842] Step 9:

[0843] The device sends the recognized emotion data to the server. The input is the emotion data analyzed in real time, and the output is the emotion data sent to the server. Specifically, the device sends the recognized emotion data to the server via the network.

[0844] Step 10:

[0845] The user provides feedback on the advertisement. The input is the feedback information provided by the user, and the output is the feedback data recorded on the terminal. Specifically, the user uses the feedback function of the terminal to input their opinion on whether the advertisement was interesting, useful, or unpleasant.

[0846] Step 11:

[0847] The terminal sends the collected feedback to the server. The input is the user's feedback data, and the output is the feedback data sent to the server. Specifically, the terminal periodically sends the collected feedback to the server.

[0848] Step 12:

[0849] The server optimizes the display method of the ad format based on the collected emotional data and feedback. The input is the emotional data and feedback, and the output is the optimized ad display method. Specifically, the server analyzes the emotional data and feedback, evaluates which format and content of the ad is effective, and reflects this in the next ad display.

[0850] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0851] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0852] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0853] [Third embodiment]

[0854] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0855] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0856] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0857] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0858] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0859] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0860] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0861] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0862] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0863] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0864] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0865] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0866] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, and a user. The roles and processes of each are described in detail below.

[0867] 1. Server Processing

[0868] Ad format generation

[0869] The server generates various ad formats, including 15- and 30-second video ads, magazine cover ads, online rectangle ads, and in-stream ads, using template design and layout generation algorithms.

[0870] Ad format distribution

[0871] The server provides the format data via API or download link to distribute the generated ad formats to AI operators, allowing them to create ads using standardized ad formats.

[0872] Advertising sales

[0873] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[0874] Format Conversion

[0875] The server uses AI to change the size and format of ad formats in response to advertiser requests. For example, it may change a 15-second ad to a 30-second one, or convert a rectangular ad into an in-stream ad. These converted formats are then redistributed to AI providers and advertisers.

[0876] 2. Terminal Processing

[0877] Receiving and storing advertising data

[0878] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, which includes the advertisement format generated and converted by the server.

[0879] Displaying ads

[0880] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[0881] Recording and sending advertising history

[0882] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[0883] 3. User Actions

[0884] Watching an ad

[0885] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[0886] Providing feedback

[0887] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, by sending it to the server via their device. This feedback is used to optimize future ad formats.

[0888] Specific examples

[0889] Server processing example

[0890] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[0891] Terminal processing example

[0892] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0893] User processing example

[0894] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0895] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0896] The processing flow will be explained below.

[0897] 1. Server Processing

[0898] Step 1:

[0899] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats.

[0900] Step 2:

[0901] The server distributes the generated ad format to external businesses and transfers the format data to the external businesses' systems via API.

[0902] Step 3:

[0903] The server sells the generated ad formats to advertisers, providing a dashboard for advertisers to select and purchase the formats they need.

[0904] Step 4:

[0905] The server uses AI to convert ad formats based on requests from advertisers, such as changing a video ad from 15 seconds to 30 seconds.

[0906] Step 5:

[0907] The server then distributes the converted ad format back to external businesses and advertisers, providing format data via API or download link.

[0908] 2. Terminal Processing

[0909] Step 1:

[0910] The terminal receives the advertisement data from the server, the advertisement data including the generated and converted format.

[0911] Step 2:

[0912] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[0913] Step 3:

[0914] When a user uses a video service, the device displays an advertisement before the main content. The advertisement data is called from within the video service application and displayed in the video player.

[0915] Step 4:

[0916] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[0917] Step 5:

[0918] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[0919] 3. User Actions

[0920] Step 1:

[0921] The user watches advertisements provided through the terminal, such as advertisements played before the main video of a video service or advertisements displayed around the cover of an electronic magazine.

[0922] Step 2:

[0923] The user provides feedback on the advertisement, for example, whether the advertisement is interesting, relevant, annoying, etc., by using the feedback function of the device.

[0924] Step 3:

[0925] User feedback is sent from the device to the server, which uses the collected feedback to optimize future ads.

[0926] Specific examples

[0927] Server processing example

[0928] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells the ads to advertisers, who then use the purchased format.

[0929] Terminal processing example

[0930] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[0931] User processing example

[0932] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[0933] Example 1

[0934] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0935] In conventional ad delivery systems, the creation, distribution, sales, and optimization of ad formats are performed piecemeal, resulting in inefficiencies for both advertisers and users. Conversion between different ad formats is also performed manually, which is time-consuming and costly. Furthermore, even if user feedback is collected, there is a lack of a way to effectively incorporate it, making ad optimization difficult.

[0936] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0937] In this invention, the server includes a means for generating ad formats, a means for distributing the generated ad formats to external businesses via an API or a download link, a means for providing a dashboard for advertisers to sell the ad formats, and a means for converting the size and format of the ad formats using a machine learning algorithm in response to requests from advertisers. This makes it possible to consistently and efficiently perform processes from generating, distributing, selling, and optimizing ad formats.

[0938] "Ad format" refers to the template or design layout used to display advertising content.

[0939] "External businesses" are companies or organizations that use advertising formats to create the actual advertising content.

[0940] "API" stands for Application Programming Interface, an interface for exchanging functions and data between software programs.

[0941] A "download link" is a URL that allows a user to save a specific file or data from the Internet to their device.

[0942] A "dashboard" is a user interface that allows a user to visually display and manipulate the information and data they need.

[0943] A "machine learning algorithm" is a program or method that allows a computer to automatically learn using large amounts of data to perform specific tasks.

[0944] "Local storage" refers to a data storage device or storage medium installed within a terminal.

[0945] A "user" is a consumer who uses a video service or an electronic magazine.

[0946] "Feedback" refers to the evaluations and opinions of advertisements provided by users who have viewed them.

[0947] The present invention provides an advertisement distribution system that is efficient and effective for both advertisers and users by generating, distributing, selling, and converting advertisement formats. Detailed embodiments of the system are described below.

[0948] Server embodiment

[0949] Ad format generation

[0950] The server uses template design and layout generation algorithms to generate various ad formats, such as 15-second and 30-second video ads, magazine cover ads, online rectangle ads, in-stream ads, etc. This generation is done using image processing libraries such as Python's OpenCV and Pillow.

[0951] Ad format distribution

[0952] The server distributes the generated ad formats to external businesses via API or download links. This distribution utilizes a RESTful API, allowing external businesses to efficiently provide the ad format data they require.

[0953] Advertising sales

[0954] The server provides a dashboard for advertisers, allowing them to select and purchase the ad formats they need. This dashboard is built using React and Vue.js, making it intuitive and easy for users to use.

[0955] Ad format conversion

[0956] The server uses machine learning algorithms (e.g., TensorFlow, PyTorch) to convert the size and format of the ad format according to the advertiser's request. For example, if a 15-second video ad is extended to 30 seconds, additional content is generated based on the original ad content. This converted ad format is also distributed via API.

[0957] Terminal embodiment

[0958] Receiving and storing advertising data

[0959] The device receives the advertising data sent from the server and saves it in local storage (e.g., SQLite database). The receiving process is performed periodically, and new advertising data is downloaded and saved as needed.

[0960] Displaying ads

[0961] The device displays the saved advertisements when the user uses video services or digital magazines. This display uses HTML5 and JavaScript, and in video services, the advertisements are displayed before the main video. In digital magazines, advertisements are inserted around the cover while the user is reading.

[0962] Recording and sending advertising history

[0963] The device records the history of the displayed ads (playback time, user interaction information, etc.) and periodically sends it to the server. This history information is collected in JSON format and sent to the server via a REST API.

[0964] User's embodiment

[0965] Watching an ad

[0966] Users watch ads delivered through their devices. The UI is intuitive and designed to allow users to watch ads naturally. For example, users are presented with a 15-second full-screen ad before the main content of a video service.

[0967] Providing feedback

[0968] Users send feedback about ads to the server via their devices. For example, they can send their opinions on whether the ad was interesting, relevant, or annoying through a simple UI within the app. The feedback information is stored in a database and used to optimize future ads.

[0969] Examples and prompts

[0970] Server processing example

[0971] The server generates 15-second video ad formats and distributes them to third parties, who then use them to create advertising content. The server then sells the ads to advertisers, who then use the purchased formats in their own campaigns.

[0972] Terminal processing example

[0973] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[0974] User processing example

[0975] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[0976] Prompt sentence for generative AI model

[0977] "Generate a 15-second video ad format and distribute it to third parties."

[0978] "Collect user viewing history and feedback information to optimize advertising effectiveness."

[0979] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[0980] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0981] Step 1:

[0982] The server loads the template design data and starts generating ad formats. Specifically, it uses Python's Pillow library to generate image-based templates and OpenCV to assemble video frames to create 15-second and 30-second video ads. It uses the template design data as input and obtains the generated ad format files as output.

[0983] Step 2:

[0984] The server distributes the generated ad formats to external businesses via APIs and download links. Specifically, a RESTful API is built so that external businesses can download format data by sending an HTTP request. The generated ad format file is used as input, and a download link is obtained as output, which is shared with external businesses.

[0985] Step 3:

[0986] The server sells ad formats through a dashboard for advertisers. Specifically, we designed a user interface using React and Vue.js to allow advertisers to easily select and purchase the format they need. The server receives an advertiser's purchase request as input and obtains the purchased format data and a receipt as output.

[0987] Step 4:

[0988] The server uses machine learning algorithms to convert the size and format of ad formats based on the advertiser's requests. Specifically, it uses TensorFlow and PyTorch to perform processes such as extending a 15-second video ad to 30 seconds. The server uses the advertiser's conversion request and the original ad format as input, and obtains the converted ad format as output.

[0989] Step 5:

[0990] The device receives the advertising data sent from the server and stores it in local storage (e.g., SQLite database). Specifically, it periodically downloads new advertising data from the server and stores it appropriately in local storage. It uses the advertising data from the server as input and obtains the stored advertising data as output.

[0991] Step 6:

[0992] The device displays the stored advertising data when the user uses video services or digital magazines. Specifically, it uses HTML5 and JavaScript to play and display advertisements to the user. It uses the advertising data from local storage as input and obtains the displayed advertisement as output.

[0993] Step 7:

[0994] The device records the history of displayed ads and periodically sends it to the server. Specifically, it collects display time and user interaction information in JSON format and sends it to the server via REST API. It uses user interaction information and display history as input and obtains the ad history data sent to the server as output.

[0995] Step 8:

[0996] Users view advertisements provided through their devices. The UI is intuitive and designed to allow users to view advertisements naturally. The input is the advertisement content provided by the device, and the output is the information about the advertisements viewed.

[0997] Step 9:

[0998] Users send feedback about ads to the server via their devices. Specifically, they use a simple UI within the app to input their opinions, such as "interesting," "useful," or "unpleasant." The system uses user feedback as input and obtains the feedback information sent to the server as output.

[0999] (Application example 1)

[1000] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] Conventional ad delivery systems lack sufficient personalization based on user interests and behavior, limiting the effectiveness of ads. Furthermore, if the timing or content of ads is inappropriate for the user, they can be annoying. Therefore, there is a need for a system that efficiently utilizes user behavior data and feedback to deliver ads with optimal timing and content.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1003] In this invention, the server includes means for generating ad formats, means for distributing the generated ad formats to external businesses, means for selling ad formats, means for converting ad formats into other formats, means for personalizing ad formats based on user behavior data and feedback, and means for selecting an optimal ad format based on user behavior analysis using an AI model, thereby making it possible to provide users with personalized ads that are optimized for them.

[1004] An "ad format" is a template or design layout that defines the form and structure of advertising content.

[1005] An "external business" is a third-party business that creates advertising content using the advertising format.

[1006] "Personalization" refers to individually optimizing advertising content based on user behavioral data and feedback.

[1007] An "AI model" is a computational model that uses machine learning technology to analyze and learn from data, and make predictions and optimizations.

[1008] A "prompt sentence" is input text that provides specific instructions or explanations to a generative AI model.

[1009] "Server" means a centralized management system that generates, distributes, sells, and converts advertising formats.

[1010] A "user terminal" is a device (e.g., a smartphone, a tablet, etc.) that a user uses to view advertising content.

[1011] "Behavioral data" refers to records of the actions and reactions users take on advertisements and content.

[1012] "Feedback" refers to ratings and opinions provided by users who have viewed an advertisement.

[1013] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system of the present invention operates based on the roles of a server, a user terminal, and a user.

[1014] Server Processing

[1015] Ad format generation

[1016] The server generates various ad formats using template designs and layout generation algorithms, resulting in ad content in a variety of formats, including video ads and banner ads.

[1017] Ad format distribution

[1018] The generated ad formats are distributed to external businesses via APIs or download links via the server, allowing external businesses to create effective ad content using standardized ad formats.

[1019] Collecting user behavior data and feedback

[1020] The server collects and analyzes the behavioral data and feedback sent by users using an AI model (e.g., TensorFlow). This analysis extracts the user's behavioral patterns and areas of interest.

[1021] Ad personalization

[1022] Based on the analyzed data, the server provides personalized advertisements to users, including the ability to dynamically generate input prompts for the generative AI model.

[1023] User terminal processing

[1024] Receiving and displaying advertising data

[1025] The user device receives advertising data from the server and displays the advertisements at the appropriate time. For example, when a user uses a video service, a 15- or 30-second video advertisement is played before the main content, and when a user is reading an e-magazine, an advertisement is displayed around the cover.

[1026] Recording and sending advertising history and responses

[1027] The user device records the history of the ads displayed (such as the display time and user interaction information) and periodically sends this information to the server. If the user reacts to the ad, this information is also recorded and reflected in the next ad display.

[1028] User Action

[1029] Ad viewing and feedback

[1030] Users can view ads delivered through their devices and provide feedback on the ads. The feedback includes opinions on whether the ad was interesting, relevant, annoying, etc., and is sent from the device to the server. This feedback is used to optimize future ad formats.

[1031] Specific examples

[1032] Ad delivery process

[1033] 1. Ad generation and distribution: The 15-second video ad format generated by the server is then used by an external company to create a video for a new product introduction campaign. The generated ad format is then delivered to smartphones and saved in the "AdSmart" app.

[1034] 2. User behavior analysis: A user's browsing history of products in a specific category on an online shopping site is sent to the server, where a machine learning model analyzes this data and predicts the user's interests.

[1035] 3. Display Ads: When a user opens a video viewing app, AdSmart plays an appropriate 15-second video ad. After the ad ends, if the user shows interest, they will be provided with a link to a page with more details.

[1036] Prompt Sentence Examples

[1037] "Create a 15-second video ad based on a template design and layout generation. This ad is for a new product promotion campaign. Make sure it's visually appealing and compels users to click."

[1038] The above is a specific embodiment for carrying out the invention, which makes it possible to provide a personalized advertisement optimized for a user.

[1039] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1040] Step 1:

[1041] The server uses the template design and layout generation algorithm to generate ad formats, including various video and banner ads. The input is the template design configuration information, and the output is the generated ad format.

[1042] Step 2:

[1043] The server distributes the generated ad formats to external businesses via APIs or download links. The input is ad format data, and the output is an ad format that can be used by external businesses. Specifically, the ad format is retrieved from the server's storage and sent to the external business.

[1044] Step 3:

[1045] The user device receives a request from the server, retrieves the advertising data, and stores it locally. The input is the advertising data sent from the server, and the output is the advertising data stored on the user device. Specifically, the device receives the data using its receiving function and stores it in its internal storage.

[1046] Step 4:

[1047] The server collects user behavioral data and feedback and analyzes it using an AI model (e.g., TensorFlow). The input is user behavioral data and feedback information, and the output is the user's behavioral patterns and areas of interest. Specifically, the collected data is input into the AI ​​model and the analysis results are obtained.

[1048] Step 5:

[1049] The server provides the generative AI model with prompt sentences to generate personalized ads for users based on the analysis results. The inputs are the analysis results and a basic prompt sentence template, and the output is a personalized prompt sentence. Specifically, the prompt sentence template is dynamically edited based on the analysis results.

[1050] Step 6:

[1051] The user device displays the generated personalized advertisement to the user at an appropriate time. The input is the personalized advertisement data and the user's current operation status, and the output is the advertisement displayed to the user. Specifically, the advertisement is displayed before the main content while watching a video or when changing pages in an electronic magazine.

[1052] Step 7:

[1053] The user terminal records the history of displayed advertisements and the user's reactions, and periodically transmits them to the server. The input is the advertisement viewing history and reaction data, and the output is the history data sent to the server. Specifically, interaction information is collected when the advertisement ends and uploaded to the server.

[1054] Step 8:

[1055] The server optimizes the next ad display based on the collected feedback and ad viewing history. The input is the collected viewing history and feedback data, and the output is an improved setting for the next ad display. Specifically, it analyzes past data and optimizes the timing and content of the next ad to be displayed.

[1056] The above are the specific processing steps for realizing an advertising ecosystem.

[1057] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1058] The present invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, a user, and an emotion engine that recognizes the user's emotions. Each role and process will be explained in detail below.

[1059] 1. Server Processing

[1060] Ad format generation

[1061] The server generates various ad formats, including 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template design and layout generation algorithms.

[1062] Ad format distribution

[1063] The server provides format data via API or download link to distribute the generated ad formats to external businesses, allowing AI businesses to create ads using standardized ad formats.

[1064] Advertising sales

[1065] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[1066] Format Conversion

[1067] The server uses AI to convert ad formats in response to advertiser requests, such as changing a 15-second ad to a 30-second one. It can also optimize ad formats based on user sentiment data.

[1068] 2. Terminal Processing

[1069] Receiving and storing advertising data

[1070] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, including the generated and converted formats.

[1071] Displaying ads

[1072] When a user watches a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[1073] Recording and sending advertising history

[1074] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[1075] 3. User Actions

[1076] Watching an ad

[1077] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[1078] Providing feedback

[1079] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, using the device's feedback function. This feedback is used to optimize future ad formats.

[1080] 4. Emotion Engine Processing

[1081] User Emotion Recognition

[1082] The device uses an emotion engine to recognize the user's emotions in real time, which analyzes the user's facial expressions, vocal tone, and other physical responses to generate emotion data.

[1083] Sending emotional data

[1084] The device sends the recognized emotion data to a server, which uses this data to optimize ad formats. For example, ad formats that the user finds interesting can be displayed preferentially, or ad formats that the user finds annoying can be eliminated.

[1085] Specific examples

[1086] Server processing example

[1087] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[1088] Terminal processing example

[1089] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1090] User processing example

[1091] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1092] Emotion engine processing example

[1093] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1094] In this way, the present invention provides a system that integrates the creation, distribution, sale, and optimization of advertisements, and realizes advertisement delivery that takes user emotions into consideration, thereby realizing an advertisement delivery environment that is valuable to both advertisers and users.

[1095] The processing flow will be explained below.

[1096] 1. Server Processing

[1097] Step 1:

[1098] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template designs and layout generation algorithms.

[1099] Step 2:

[1100] The server distributes the generated ad format to external businesses, and transfers the format data to the external businesses' systems via API, allowing them to use it.

[1101] Step 3:

[1102] The server sells the generated ad formats to advertisers, who can select and purchase the formats they need using a dashboard provided by the server.

[1103] Step 4:

[1104] The server uses AI to convert ad formats based on requests from advertisers, for example, changing a 15-second ad format to a 30-second one.

[1105] Step 5:

[1106] The server then distributes the converted ad format back to external businesses and advertisers, providing the converted format data using an API.

[1107] Step 6:

[1108] The server receives the user's emotion data sent from the emotion engine and optimizes the ad format based on this data, for example by setting it to preferentially display ad formats that the user is interested in.

[1109] 2. Terminal Processing

[1110] Step 1:

[1111] The terminal receives the advertisement data sent from the server, including the generated and converted format.

[1112] Step 2:

[1113] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[1114] Step 3:

[1115] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. The ad data is called up and displayed in the video player.

[1116] Step 4:

[1117] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[1118] Step 5:

[1119] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[1120] 3. User Actions

[1121] Step 1:

[1122] Users view advertisements provided through their terminals, such as advertisements played before the main content of a video service or advertisements displayed around the cover of an electronic magazine.

[1123] Step 2:

[1124] The user can provide feedback on the advertisement, for example, by inputting into the terminal whether the advertisement is interesting, relevant, annoying, etc.

[1125] 4. Emotion Engine Processing

[1126] Step 1:

[1127] The device uses an emotion engine to recognize the user's emotions in real time, for example by analyzing the user's facial expressions with a camera and using voice tones and physical reactions to generate emotion data.

[1128] Step 2:

[1129] The emotion data recognized by the emotion engine is sent from the device to the server. The emotion data includes whether the user was interested, enjoyed, or displeased.

[1130] Step 3:

[1131] The server optimizes the display of ad formats based on the emotion data. For example, it prioritizes displaying formats that the user has previously shown interest in, and avoids displaying formats that the user has previously shown discomfort with.

[1132] Specific examples

[1133] Server processing example

[1134] The server generates a 15-second video ad format and distributes it to external businesses via API. The external businesses use this format to create ad content and provide it to the server. The server sells the ads to advertisers, who then use the purchased format.

[1135] Terminal processing example

[1136] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1137] User processing example

[1138] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1139] Emotion engine processing example

[1140] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1141] Example 2

[1142] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1143] In the modern advertising ecosystem, there is a need to maximize convenience and effectiveness for both advertisers and users. However, in existing systems, ad format generation, distribution, sales, and conversion are carried out independently, resulting in a lack of consistency. Furthermore, technology to utilize user sentiment data to achieve more effective ad delivery is not yet fully developed. Furthermore, there is a lack of means to properly manage and utilize ad display timing and feedback, making it difficult to optimize ad delivery.

[1144] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating advertisement formats, a means for distributing the generated advertisement formats to external businesses, a means for selling advertisement formats, a means for converting advertisement formats into other formats, and a means for collecting user emotion data and optimizing advertisement formats based on the data. This enables the consistent generation, distribution, sale, conversion, and optimization of advertisement formats, realizing an effective advertising ecosystem for both advertisers and users. Furthermore, by optimizing advertisements based on user emotion data collected by the server, advertisement delivery that is less irritating to users can be achieved. Furthermore, a system is provided that can measure and optimize advertisement effectiveness in an integrated manner, by appropriately receiving advertisement data, managing display timing, recording and transmitting advertisement history to the server, and collecting and optimizing user feedback.

[1145] "Ad Format" means the prescribed format and structure for delivering advertising content.

[1146] An "external business operator" is a third party that uses the advertising format provided by the server, and refers to an organization or individual that distributes advertisements, such as advertisers.

[1147] An "advertiser" is a company or individual that posts advertisements for its products or services and provides advertising content to appeal to users.

[1148] "Emotional data" refers to data that reflects a user's emotions, and refers to information obtained by analyzing facial expressions, vocal tone, and other physical reactions.

[1149] An "emotion engine" is a device or software that recognizes a user's emotions in real time and generates emotion data.

[1150] "Device" refers to the electronic device (e.g., smartphone, tablet, or PC) used by a user to view advertising content.

[1151] "Advertisement history" is a record of information about advertisements viewed by a user, including display time and interaction information.

[1152] "Feedback" refers to opinions and ratings provided by users regarding advertisements.

[1153] MODE FOR CARRYING OUT THE INVENTION

[1154] This invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system operates based on the roles of the server, terminal, user, and emotion engine that recognizes the user's emotions. Specific ways in which the present invention is implemented are described below.

[1155] Server Processing

[1156] The server generates ad formats using template designs and layout generation algorithms, typically using software like Adobe Illustrator or Canva. For example, the server creates 15-second and 30-second video ad formats, magazine cover ad formats, and web rectangle ad formats.

[1157] The generated ad formats are distributed to external businesses via the server using APIs and download links, allowing AI businesses to create ad content using standardized ad formats.

[1158] The server also provides a dashboard for advertisers, allowing them to purchase the ad formats they need. Online payments are made using payment gateways such as Stripe or PayPal. After purchase, advertisers apply their ad content to the formats using the provided online tools (e.g., Google Ads, Facebook Ads Manager).

[1159] The server then uses AI models (e.g., TensorFlow, PyTorch) to convert ad formats, such as changing a 15-second ad to a 30-second one, and optimizes ad formats based on user sentiment data using IBM Watson's sentiment analysis API.

[1160] Terminal handling

[1161] The device receives the advertising data sent from the server and saves it in local storage (e.g., an SQLite database). The received advertising data is displayed at the appropriate time while the user is using a video service or reading an e-magazine. Specifically, a 15-second video advertisement is played using the device's video player (e.g., an HTML5 video tag). In addition, e-magazine apps use a PDF rendering library to display advertisements around the cover.

[1162] The device records the history of the ads displayed and uploads it to a server at regular intervals. Information such as the display time and user clicks is saved and used to measure the effectiveness of the ads.

[1163] User Action

[1164] Users view advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while viewing. For example, advertisements displayed before the main content while using a video service are provided in a size and quality that is appropriate for the user.

[1165] Users can provide feedback on ads, for example, by entering comments about whether they found the ad interesting or annoying using the device's feedback function. This feedback is sent to the server in real time and used to optimize future ad formats.

[1166] Emotion engine processing

[1167] The device uses an emotion engine to recognize the user's emotions in real time, analyzing facial expressions, vocal tone, and other bodily responses to generate emotion data, using technologies such as FaceReader and IBM Watson Tone Analyzer.

[1168] The recognized emotion data is sent from the device to the server. This data is used to optimize ad formats, resulting in more effective ad display. For example, ad formats that the user finds interesting can be prioritized, or ad formats that the user finds annoying can be eliminated.

[1169] Examples of specific examples and prompts

[1170] Server processing example

[1171] The server generates 15-second video ad formats and distributes them to external businesses. The external businesses use these formats to create ad content and provide it to the server. The server sells these ads to advertisers, who then use the purchased formats in their own campaigns.

[1172] Terminal processing example

[1173] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1174] User processing example

[1175] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1176] Emotion engine processing example

[1177] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1178] Example prompt sentence:

[1179] "Optimize your ad formats based on users' emotional data. Emotional data includes facial expressions, vocal tone, and other bodily responses."

[1180] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1181] Step 1: Generate Ad Formats

[1182] The server starts generating ad formats. As input, it is given a template design and a layout generation algorithm. Specific software used is Adobe Illustrator or Canva. The server uses these tools to create 15-second and 30-second video ad formats and magazine cover ad formats. As output, it generates the generated ad formats.

[1183] Step 2: Distributing Ad Formats

[1184] The server distributes the generated ad formats to external providers. The generated ad formats and a list of external providers are given as input. In this step, the format data is sent via API. Alternatively, the server generates a file download link and notifies the provider. Specifically, the data is sent using a RESTful API. As output, the external provider receives the ad formats.

[1185] Step 3: Selling ads

[1186] The server provides a dashboard for advertisers. As input, it receives the ad format, advertiser information, and payment information. The advertiser selects the required ad format on the dashboard and makes an online payment (e.g., Stripe, PayPal). After completing the purchase, the advertiser applies their ad content to the format using the provided online tools (e.g., Google Ads, Facebook Ads Manager). As output, the advertiser receives the ad format and uses it in their campaign.

[1187] Step 4: Convert the format

[1188] The server uses an AI model to convert ad formats. It receives advertiser requests and existing ad formats as input. It uses TensorFlow or PyTorch to convert, for example, a 15-second ad to a 30-second one. It also uses IBM Watson's sentiment analysis API to optimize the ad format based on user sentiment data. The converted ad format is generated as output.

[1189] Step 5: Receiving and storing advertising data

[1190] The device receives advertising data sent from the server. As input, the advertising data sent from the server is given. The device saves this in local storage (e.g., SQLite database). Specifically, the device downloads the data using an HTTP request and saves it in SQLite. As output, the advertising data is saved on the device.

[1191] Step 6: Displaying the Ad

[1192] The device displays the advertisement. As input, it receives the locally stored advertisement data. When the user uses a video service or digital magazine, it plays a 15-second video advertisement using a video player (e.g., HTML5 video tag). It also uses a PDF rendering library to display an advertisement around the cover of the digital magazine. As output, the advertisement is displayed to the user.

[1193] Step 7: Record and submit your ad history

[1194] The device records the history of displayed ads. As input, it receives information about the displayed ads (e.g., display time, user click information). The device stores this information in an SQLite database. Periodically, it uploads the history data to the server using HTTP POST. As output, the ad history is sent to the server and used for performance measurement.

[1195] Step 8: Watch an ad

[1196] The user watches the advertisement through the device. As input, an advertisement played on the device's video player or an advertisement on the cover of an electronic magazine is given. The user watches the advertisement and performs a specific interaction (e.g., click, skip). As output, the advertisement is watched by the user.

[1197] Step 9: Provide feedback

[1198] Users provide feedback on ads. As input, a feedback form about the ads they watched is provided. Users enter their opinions and ratings in the form and click the submit button. As output, the feedback data is sent to the server and used to optimize future ad formats.

[1199] Step 10: Emotion Recognition

[1200] The device uses an emotion engine to recognize the user's emotions. The input is the user's facial expressions and vocal tone. The emotion engine analyzes this data and generates emotion data. Specifically, it uses the built-in camera and microphone, and employs FaceReader and IBM Watson Tone Analyzer. The output is the recognized emotion data.

[1201] Step 11: Sending Emotion Data

[1202] The device sends the recognized emotion data to the server. As input, the recognized emotion data is given. This data is sent to the server using an HTTP POST request. As output, the emotion data is stored on the server and used to optimize ad formats.

[1203] Through these processing steps, the system can consistently generate, optimize, distribute, and sell ad formats, as well as display ads using user emotion data, thereby realizing a valuable advertising ecosystem for both advertisers and users.

[1204] (Application example 2)

[1205] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1206] Conventional advertising systems do not take into account the user's emotional state, making it difficult to maximize advertising effectiveness. Furthermore, ads are not displayed based on the user's interests or emotions, resulting in a poor user experience. Furthermore, real-time data collection and ad display optimization are insufficient, making it difficult for advertisers to measure advertising effectiveness.

[1207] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating an advertisement format, means for distributing the generated advertisement format to an external business, means for selling the advertisement format, means for converting the advertisement format into another format, emotion recognition means for recognizing a user's emotion in real time and transmitting the data to the server, and means for optimizing the display method of the advertisement format based on the user's emotion data. This makes it possible to display advertisements taking into account the user's emotion and maximize the effectiveness of the advertisements. In addition, real-time data collection and optimization of advertisement display are realized, making it easier for advertisers to measure effectiveness.

[1208] An "advertising format" is a standard format that defines the layout and design of advertising content.

[1209] "Third Party" means any other company or individual that creates and distributes advertising content using the Ad Format.

[1210] An "emotion recognition means" is a device or software that recognizes a user's emotional state in real time by analyzing the user's facial expressions, vocal tone, and other bodily responses.

[1211] "Server" means the computer system that generates, distributes, sells, and converts advertising formats, and receives and analyzes emotional data.

[1212] "User terminal" refers to a device on which a user views advertisements, such as a smartphone, smart glasses, or head-mounted display.

[1213] "Feedback" refers to opinions and impressions that users provide about advertisements.

[1214] "Ad display optimization" refers to the process of adjusting how ads are displayed to maximize their effectiveness based on collected data and feedback.

[1215] "Real-time" means near-immediate processing and response with little delay.

[1216] "Ad display history" refers to a record of the type, time, and interaction information of the ads displayed by the user.

[1217] The present invention is a system for generating, distributing, selling, converting, and optimizing advertising formats based on user emotion data. This system mainly operates in cooperation with a server, a terminal, a user, and an emotion recognition means.

[1218] Server Processing

[1219] The server has the ability to generate ad formats and distribute them to external parties. The server provides a dashboard for advertisers to purchase, apply, and change ad formats. The server also has the ability to convert ad formats into other formats, for example, converting a 15-second ad into a 30-second ad.

[1220] Furthermore, the emotional data of the user is received through the emotion recognition means, and the display method of the advertisement format is optimized based on the received data. The emotional data when the user watches the advertisement is analyzed in real time, and the content and display method of the advertisement are adjusted based on the specific emotional response.

[1221] Terminal handling

[1222] The device receives the advertisement data sent from the server and displays it to the user at the appropriate time. For example, if the user is wearing smart glasses, the device displays the advertisement through the glasses. The device also records the history of the displayed advertisements and sends it to the server.

[1223] The device uses an emotion recognition means to recognize the user's emotions in real time. This emotion data is generated by analyzing facial expressions, vocal tone, and other physical reactions, and is transmitted to a server for use in optimizing advertisement display.

[1224] User Action

[1225] Users can view advertisements provided through their devices and provide feedback on the advertisements, including opinions on whether the advertisements were interesting, relevant, annoying, etc. This feedback is collected through the devices and transmitted to a server.

[1226] Hardware and software used

[1227] Hardware:

[1228] Smart glasses (with camera)

[1229] Regular smartphones and tablets

[1230] software:

[1231] OpenCV: Used for emotion recognition.

[1232] requests: Used to communicate with the server.

[1233] Pre-trained Emotion Recognition Model.

[1234] Specific examples

[1235] While wearing the smart glasses, a user walks around town and a new advertisement is displayed every five minutes. While the user is looking at the advertisement, a camera analyzes their facial expressions and sends them to a server. The server uses this data to optimize the next advertisement to be displayed.

[1236] For example, if the smart glasses screen displays a message saying, "New drink campaign! Click here!" and the user responds, this information will also be sent to the server. Based on the emotional data, the next advertisement display will be further optimized.

[1237] Example of input prompt for generative AI model

[1238] I would like to develop an application for smart glasses that recognizes the user's facial expressions in real time while watching advertisements and sends the emotion data to a server. The content of the advertisements will be optimized based on the user's emotion data and displayed at regular intervals. Please explain in detail how to build this application.

[1239] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1240] Step 1:

[1241] The server generates ad formats. The input is basic ad materials (images, videos, text) provided by the advertiser, and the output is a formatted ad format. Specifically, the server uses template design and layout generation algorithms to generate 15-second or 30-second video ads, magazine cover ads, online rectangular ads, etc.

[1242] Step 2:

[1243] The server distributes the generated ad format. The input is the ad format generated in step 1, and the output is the format data provided to external businesses. Specifically, the server provides the ad format to external businesses via an API or download link.

[1244] Step 3:

[1245] The server sells ad formats. The input is the advertiser's purchase request, and the output is the sold ad format. Specifically, the server allows advertisers to purchase the ad formats they need through a dashboard, and advertisers use online tools to apply their ad content to the formats.

[1246] Step 4:

[1247] The server converts the ad format. The input is the advertiser's conversion request and emotional data, and the output is the converted ad format. Specifically, the server uses AI to change the ad format according to the advertiser's request, for example, changing a 15-second ad to 30 seconds. The server also optimizes the ad format based on the user's emotional data.

[1248] Step 5:

[1249] The device receives advertising data from the server. The input is the advertising data sent from the server, and the output is the locally stored advertising data. Specifically, the device periodically communicates with the server and downloads new advertising data.

[1250] Step 6:

[1251] The device displays advertisements to the user. The input is the saved advertisement data, and the output is the displayed advertisement. Specifically, when a user uses a video service, the device plays a 15- or 30-second advertisement before the main content, and inserts a cover advertisement while reading an e-magazine.

[1252] Step 7:

[1253] The device records the history of ads displayed and sends it to the server. The input is the user's ad viewing data, and the output is the history data sent to the server. Specifically, the device records information such as the time the ad was displayed and the user's interactions, and periodically sends this information to the server.

[1254] Step 8:

[1255] The device uses emotion recognition means to recognize the user's emotions in real time. The input is the user's facial expressions and voice, and the output is emotion data. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes them using an emotion recognition algorithm.

[1256] Step 9:

[1257] The device sends the recognized emotion data to the server. The input is the emotion data analyzed in real time, and the output is the emotion data sent to the server. Specifically, the device sends the recognized emotion data to the server via the network.

[1258] Step 10:

[1259] The user provides feedback on the advertisement. The input is the feedback information provided by the user, and the output is the feedback data recorded on the terminal. Specifically, the user uses the feedback function of the terminal to input their opinion on whether the advertisement was interesting, useful, or unpleasant.

[1260] Step 11:

[1261] The terminal sends the collected feedback to the server. The input is the user's feedback data, and the output is the feedback data sent to the server. Specifically, the terminal periodically sends the collected feedback to the server.

[1262] Step 12:

[1263] The server optimizes the display method of the ad format based on the collected emotional data and feedback. The input is the emotional data and feedback, and the output is the optimized ad display method. Specifically, the server analyzes the emotional data and feedback, evaluates which format and content of the ad is effective, and reflects this in the next ad display.

[1264] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1265] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1266] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1267] [Fourth embodiment]

[1268] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1269] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1270] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1271] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1272] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1273] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1274] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1275] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1276] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1277] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1278] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1279] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1280] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1281] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, and a user. The roles and processes of each are described in detail below.

[1282] 1. Server Processing

[1283] Ad format generation

[1284] The server generates various ad formats, including 15- and 30-second video ads, magazine cover ads, online rectangle ads, and in-stream ads, using template design and layout generation algorithms.

[1285] Ad format distribution

[1286] The server provides the format data via API or download link to distribute the generated ad formats to AI operators, allowing them to create ads using standardized ad formats.

[1287] Advertising sales

[1288] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[1289] Format Conversion

[1290] The server uses AI to change the size and format of ad formats in response to advertiser requests. For example, it may change a 15-second ad to a 30-second one, or convert a rectangular ad into an in-stream ad. These converted formats are then redistributed to AI providers and advertisers.

[1291] 2. Terminal Processing

[1292] Receiving and storing advertising data

[1293] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, which includes the advertisement format generated and converted by the server.

[1294] Displaying ads

[1295] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[1296] Recording and sending advertising history

[1297] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[1298] 3. User Actions

[1299] Watching an ad

[1300] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[1301] Providing feedback

[1302] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, by sending it to the server via their device. This feedback is used to optimize future ad formats.

[1303] Specific examples

[1304] Server processing example

[1305] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[1306] Terminal processing example

[1307] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[1308] User processing example

[1309] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[1310] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[1311] The processing flow will be explained below.

[1312] 1. Server Processing

[1313] Step 1:

[1314] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats.

[1315] Step 2:

[1316] The server distributes the generated ad format to external businesses and transfers the format data to the external businesses' systems via API.

[1317] Step 3:

[1318] The server sells the generated ad formats to advertisers, providing a dashboard for advertisers to select and purchase the formats they need.

[1319] Step 4:

[1320] The server uses AI to convert ad formats based on requests from advertisers, such as changing a video ad from 15 seconds to 30 seconds.

[1321] Step 5:

[1322] The server then distributes the converted ad format back to external businesses and advertisers, providing format data via API or download link.

[1323] 2. Terminal Processing

[1324] Step 1:

[1325] The terminal receives the advertisement data from the server, the advertisement data including the generated and converted format.

[1326] Step 2:

[1327] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[1328] Step 3:

[1329] When a user uses a video service, the device displays an advertisement before the main content. The advertisement data is called from within the video service application and displayed in the video player.

[1330] Step 4:

[1331] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[1332] Step 5:

[1333] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[1334] 3. User Actions

[1335] Step 1:

[1336] The user watches advertisements provided through the terminal, such as advertisements played before the main video of a video service or advertisements displayed around the cover of an electronic magazine.

[1337] Step 2:

[1338] The user provides feedback on the advertisement, for example, whether the advertisement is interesting, relevant, annoying, etc., by using the feedback function of the device.

[1339] Step 3:

[1340] User feedback is sent from the device to the server, which uses the collected feedback to optimize future ads.

[1341] Specific examples

[1342] Server processing example

[1343] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells the ads to advertisers, who then use the purchased format.

[1344] Terminal processing example

[1345] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1346] User processing example

[1347] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1348] Example 1

[1349] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1350] In conventional ad delivery systems, the creation, distribution, sales, and optimization of ad formats are performed piecemeal, resulting in inefficiencies for both advertisers and users. Conversion between different ad formats is also performed manually, which is time-consuming and costly. Furthermore, even if user feedback is collected, there is a lack of a way to effectively incorporate it, making ad optimization difficult.

[1351] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1352] In this invention, the server includes a means for generating ad formats, a means for distributing the generated ad formats to external businesses via an API or a download link, a means for providing a dashboard for advertisers to sell the ad formats, and a means for converting the size and format of the ad formats using a machine learning algorithm in response to requests from advertisers. This makes it possible to consistently and efficiently perform processes from generating, distributing, selling, and optimizing ad formats.

[1353] "Ad format" refers to the template or design layout used to display advertising content.

[1354] "External businesses" are companies or organizations that use advertising formats to create the actual advertising content.

[1355] "API" stands for Application Programming Interface, an interface for exchanging functions and data between software programs.

[1356] A "download link" is a URL that allows a user to save a specific file or data from the Internet to their device.

[1357] A "dashboard" is a user interface that allows a user to visually display and manipulate the information and data they need.

[1358] A "machine learning algorithm" is a program or method that allows a computer to automatically learn using large amounts of data to perform specific tasks.

[1359] "Local storage" refers to a data storage device or storage medium installed within a terminal.

[1360] A "user" is a consumer who uses a video service or an electronic magazine.

[1361] "Feedback" refers to the evaluations and opinions of advertisements provided by users who have viewed them.

[1362] The present invention provides an advertisement distribution system that is efficient and effective for both advertisers and users by generating, distributing, selling, and converting advertisement formats. Detailed embodiments of the system are described below.

[1363] Server embodiment

[1364] Ad format generation

[1365] The server uses template design and layout generation algorithms to generate various ad formats, such as 15-second and 30-second video ads, magazine cover ads, online rectangle ads, in-stream ads, etc. This generation is done using image processing libraries such as Python's OpenCV and Pillow.

[1366] Ad format distribution

[1367] The server distributes the generated ad formats to external businesses via API or download links. This distribution utilizes a RESTful API, allowing external businesses to efficiently provide the ad format data they require.

[1368] Advertising sales

[1369] The server provides a dashboard for advertisers, allowing them to select and purchase the ad formats they need. This dashboard is built using React and Vue.js, making it intuitive and easy for users to use.

[1370] Ad format conversion

[1371] The server uses machine learning algorithms (e.g., TensorFlow, PyTorch) to convert the size and format of the ad format according to the advertiser's request. For example, if a 15-second video ad is extended to 30 seconds, additional content is generated based on the original ad content. This converted ad format is also distributed via API.

[1372] Terminal embodiment

[1373] Receiving and storing advertising data

[1374] The device receives the advertising data sent from the server and saves it in local storage (e.g., SQLite database). The receiving process is performed periodically, and new advertising data is downloaded and saved as needed.

[1375] Displaying ads

[1376] The device displays the saved advertisements when the user uses video services or digital magazines. This display uses HTML5 and JavaScript, and in video services, the advertisements are displayed before the main video. In digital magazines, advertisements are inserted around the cover while the user is reading.

[1377] Recording and sending advertising history

[1378] The device records the history of the displayed ads (playback time, user interaction information, etc.) and periodically sends it to the server. This history information is collected in JSON format and sent to the server via a REST API.

[1379] User's embodiment

[1380] Watching an ad

[1381] Users watch ads delivered through their devices. The UI is intuitive and designed to allow users to watch ads naturally. For example, users are presented with a 15-second full-screen ad before the main content of a video service.

[1382] Providing feedback

[1383] Users send feedback about ads to the server via their devices. For example, they can send their opinions on whether the ad was interesting, relevant, or annoying through a simple UI within the app. The feedback information is stored in a database and used to optimize future ads.

[1384] Examples and prompts

[1385] Server processing example

[1386] The server generates 15-second video ad formats and distributes them to third parties, who then use them to create advertising content. The server then sells the ads to advertisers, who then use the purchased formats in their own campaigns.

[1387] Terminal processing example

[1388] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main video. The device also records the user's viewing history and reactions to the advertisement and sends them to the server.

[1389] User processing example

[1390] When a user reads an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user can then provide feedback on whether the advertisement was useful or not.

[1391] Prompt sentence for generative AI model

[1392] "Generate a 15-second video ad format and distribute it to third parties."

[1393] "Collect user viewing history and feedback information to optimize advertising effectiveness."

[1394] In this way, the present invention provides a system that performs the creation, distribution, sale, and optimization of advertisements in an integrated manner, thereby realizing an advertisement distribution environment that is of high value to both advertisers and users.

[1395] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1396] Step 1:

[1397] The server loads the template design data and starts generating ad formats. Specifically, it uses Python's Pillow library to generate image-based templates and OpenCV to assemble video frames to create 15-second and 30-second video ads. It uses the template design data as input and obtains the generated ad format files as output.

[1398] Step 2:

[1399] The server distributes the generated ad formats to external businesses via APIs and download links. Specifically, a RESTful API is built so that external businesses can download format data by sending an HTTP request. The generated ad format file is used as input, and a download link is obtained as output, which is shared with external businesses.

[1400] Step 3:

[1401] The server sells ad formats through a dashboard for advertisers. Specifically, we designed a user interface using React and Vue.js to allow advertisers to easily select and purchase the format they need. The server receives an advertiser's purchase request as input and obtains the purchased format data and a receipt as output.

[1402] Step 4:

[1403] The server uses machine learning algorithms to convert the size and format of ad formats based on the advertiser's requests. Specifically, it uses TensorFlow and PyTorch to perform processes such as extending a 15-second video ad to 30 seconds. The server uses the advertiser's conversion request and the original ad format as input, and obtains the converted ad format as output.

[1404] Step 5:

[1405] The device receives the advertising data sent from the server and stores it in local storage (e.g., SQLite database). Specifically, it periodically downloads new advertising data from the server and stores it appropriately in local storage. It uses the advertising data from the server as input and obtains the stored advertising data as output.

[1406] Step 6:

[1407] The device displays the stored advertising data when the user uses video services or digital magazines. Specifically, it uses HTML5 and JavaScript to play and display advertisements to the user. It uses the advertising data from local storage as input and obtains the displayed advertisement as output.

[1408] Step 7:

[1409] The device records the history of displayed ads and periodically sends it to the server. Specifically, it collects display time and user interaction information in JSON format and sends it to the server via REST API. It uses user interaction information and display history as input and obtains the ad history data sent to the server as output.

[1410] Step 8:

[1411] Users view advertisements provided through their devices. The UI is intuitive and designed to allow users to view advertisements naturally. The input is the advertisement content provided by the device, and the output is the information about the advertisements viewed.

[1412] Step 9:

[1413] Users send feedback about ads to the server via their devices. Specifically, they use a simple UI within the app to input their opinions, such as "interesting," "useful," or "unpleasant." The system uses user feedback as input and obtains the feedback information sent to the server as output.

[1414] (Application example 1)

[1415] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1416] Conventional ad delivery systems lack sufficient personalization based on user interests and behavior, limiting the effectiveness of ads. Furthermore, if the timing or content of ads is inappropriate for the user, they can be annoying. Therefore, there is a need for a system that efficiently utilizes user behavior data and feedback to deliver ads with optimal timing and content.

[1417] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1418] In this invention, the server includes means for generating ad formats, means for distributing the generated ad formats to external businesses, means for selling ad formats, means for converting ad formats into other formats, means for personalizing ad formats based on user behavior data and feedback, and means for selecting an optimal ad format based on user behavior analysis using an AI model, thereby making it possible to provide users with personalized ads that are optimized for them.

[1419] An "ad format" is a template or design layout that defines the form and structure of advertising content.

[1420] An "external business" is a third-party business that creates advertising content using the advertising format.

[1421] "Personalization" refers to individually optimizing advertising content based on user behavioral data and feedback.

[1422] An "AI model" is a computational model that uses machine learning technology to analyze and learn from data, and make predictions and optimizations.

[1423] A "prompt sentence" is input text that provides specific instructions or explanations to a generative AI model.

[1424] "Server" means a centralized management system that generates, distributes, sells, and converts advertising formats.

[1425] A "user terminal" is a device (e.g., a smartphone, a tablet, etc.) that a user uses to view advertising content.

[1426] "Behavioral data" refers to records of the actions and reactions users take on advertisements and content.

[1427] "Feedback" refers to ratings and opinions provided by users who have viewed an advertisement.

[1428] The present invention provides a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system of the present invention operates based on the roles of a server, a user terminal, and a user.

[1429] Server Processing

[1430] Ad format generation

[1431] The server generates various ad formats using template designs and layout generation algorithms, resulting in ad content in a variety of formats, including video ads and banner ads.

[1432] Ad format distribution

[1433] The generated ad formats are distributed to external businesses via APIs or download links via the server, allowing external businesses to create effective ad content using standardized ad formats.

[1434] Collecting user behavior data and feedback

[1435] The server collects and analyzes the behavioral data and feedback sent by users using an AI model (e.g., TensorFlow). This analysis extracts the user's behavioral patterns and areas of interest.

[1436] Ad personalization

[1437] Based on the analyzed data, the server provides personalized advertisements to users, including the ability to dynamically generate input prompts for the generative AI model.

[1438] User terminal processing

[1439] Receiving and displaying advertising data

[1440] The user device receives advertising data from the server and displays the advertisements at the appropriate time. For example, when a user uses a video service, a 15- or 30-second video advertisement is played before the main content, and when a user is reading an e-magazine, an advertisement is displayed around the cover.

[1441] Recording and sending advertising history and responses

[1442] The user device records the history of the ads displayed (such as the display time and user interaction information) and periodically sends this information to the server. If the user reacts to the ad, this information is also recorded and reflected in the next ad display.

[1443] User Action

[1444] Ad viewing and feedback

[1445] Users can view ads delivered through their devices and provide feedback on the ads. The feedback includes opinions on whether the ad was interesting, relevant, annoying, etc., and is sent from the device to the server. This feedback is used to optimize future ad formats.

[1446] Specific examples

[1447] Ad delivery process

[1448] 1. Ad generation and distribution: The 15-second video ad format generated by the server is then used by an external company to create a video for a new product introduction campaign. The generated ad format is then delivered to smartphones and saved in the "AdSmart" app.

[1449] 2. User behavior analysis: A user's browsing history of products in a specific category on an online shopping site is sent to the server, where a machine learning model analyzes this data and predicts the user's interests.

[1450] 3. Display Ads: When a user opens a video viewing app, AdSmart plays an appropriate 15-second video ad. After the ad ends, if the user shows interest, they will be provided with a link to a page with more details.

[1451] Prompt Sentence Examples

[1452] "Create a 15-second video ad based on a template design and layout generation. This ad is for a new product promotion campaign. Make sure it's visually appealing and compels users to click."

[1453] The above is a specific embodiment for carrying out the invention, which makes it possible to provide a personalized advertisement optimized for a user.

[1454] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1455] Step 1:

[1456] The server uses the template design and layout generation algorithm to generate ad formats, including various video and banner ads. The input is the template design configuration information, and the output is the generated ad format.

[1457] Step 2:

[1458] The server distributes the generated ad formats to external businesses via APIs or download links. The input is ad format data, and the output is an ad format that can be used by external businesses. Specifically, the ad format is retrieved from the server's storage and sent to the external business.

[1459] Step 3:

[1460] The user device receives a request from the server, retrieves the advertising data, and stores it locally. The input is the advertising data sent from the server, and the output is the advertising data stored on the user device. Specifically, the device receives the data using its receiving function and stores it in its internal storage.

[1461] Step 4:

[1462] The server collects user behavioral data and feedback and analyzes it using an AI model (e.g., TensorFlow). The input is user behavioral data and feedback information, and the output is the user's behavioral patterns and areas of interest. Specifically, the collected data is input into the AI ​​model and the analysis results are obtained.

[1463] Step 5:

[1464] The server provides the generative AI model with prompt sentences to generate personalized ads for users based on the analysis results. The inputs are the analysis results and a basic prompt sentence template, and the output is a personalized prompt sentence. Specifically, the prompt sentence template is dynamically edited based on the analysis results.

[1465] Step 6:

[1466] The user device displays the generated personalized advertisement to the user at an appropriate time. The input is the personalized advertisement data and the user's current operation status, and the output is the advertisement displayed to the user. Specifically, the advertisement is displayed before the main content while watching a video or when changing pages in an electronic magazine.

[1467] Step 7:

[1468] The user terminal records the history of displayed advertisements and the user's reactions, and periodically transmits them to the server. The input is the advertisement viewing history and reaction data, and the output is the history data sent to the server. Specifically, interaction information is collected when the advertisement ends and uploaded to the server.

[1469] Step 8:

[1470] The server optimizes the next ad display based on the collected feedback and ad viewing history. The input is the collected viewing history and feedback data, and the output is an improved setting for the next ad display. Specifically, it analyzes past data and optimizes the timing and content of the next ad to be displayed.

[1471] The above are the specific processing steps for realizing an advertising ecosystem.

[1472] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1473] The present invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. This system operates based on the roles of a server, a terminal, a user, and an emotion engine that recognizes the user's emotions. Each role and process will be explained in detail below.

[1474] 1. Server Processing

[1475] Ad format generation

[1476] The server generates various ad formats, including 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template design and layout generation algorithms.

[1477] Ad format distribution

[1478] The server provides format data via API or download link to distribute the generated ad formats to external businesses, allowing AI businesses to create ads using standardized ad formats.

[1479] Advertising sales

[1480] The server provides advertisers with a dashboard where they can purchase the ad formats they need, after which they have access to online tools to apply their ad content to these formats.

[1481] Format Conversion

[1482] The server uses AI to convert ad formats in response to advertiser requests, such as changing a 15-second ad to a 30-second one. It can also optimize ad formats based on user sentiment data.

[1483] 2. Terminal Processing

[1484] Receiving and storing advertising data

[1485] The terminal receives the advertisement data sent from the server and locally stores the advertisement data, including the generated and converted formats.

[1486] Displaying ads

[1487] When a user watches a video service, the device displays a 15- or 30-second ad before the main content. Also, when a user is reading an e-magazine, an ad is inserted around the cover. This allows users to watch the ad in a natural way.

[1488] Recording and sending advertising history

[1489] The device records the history of the displayed advertisements (such as the display time and user interaction information) and periodically sends this information to the server, making it easier to measure the effectiveness of the advertisements.

[1490] 3. User Actions

[1491] Watching an ad

[1492] Users watch advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while watching. For example, advertisements displayed before the main content of video services are provided in a size and quality that is appropriate for the user.

[1493] Providing feedback

[1494] Users can provide feedback on ads, such as whether they found the ad interesting, relevant, or annoying, using the device's feedback function. This feedback is used to optimize future ad formats.

[1495] 4. Emotion Engine Processing

[1496] User Emotion Recognition

[1497] The device uses an emotion engine to recognize the user's emotions in real time, which analyzes the user's facial expressions, vocal tone, and other physical responses to generate emotion data.

[1498] Sending emotional data

[1499] The device sends the recognized emotion data to a server, which uses this data to optimize ad formats. For example, ad formats that the user finds interesting can be displayed preferentially, or ad formats that the user finds annoying can be eliminated.

[1500] Specific examples

[1501] Server processing example

[1502] The server generates a 15-second video ad format and distributes it to the AI ​​service provider. The AI ​​service provider uses this format to create ad content and provides it to the server. The server sells this ad to advertisers, who then use the purchased format in their own campaigns.

[1503] Terminal processing example

[1504] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1505] User processing example

[1506] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1507] Emotion engine processing example

[1508] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1509] In this way, the present invention provides a system that integrates the creation, distribution, sale, and optimization of advertisements, and realizes advertisement delivery that takes user emotions into consideration, thereby realizing an advertisement delivery environment that is valuable to both advertisers and users.

[1510] The processing flow will be explained below.

[1511] 1. Server Processing

[1512] Step 1:

[1513] The server generates ad formats, such as 15-second and 30-second video ad formats, magazine cover ad formats, online rectangle ad formats, and in-stream ad formats, using template designs and layout generation algorithms.

[1514] Step 2:

[1515] The server distributes the generated ad format to external businesses, and transfers the format data to the external businesses' systems via API, allowing them to use it.

[1516] Step 3:

[1517] The server sells the generated ad formats to advertisers, who can select and purchase the formats they need using a dashboard provided by the server.

[1518] Step 4:

[1519] The server uses AI to convert ad formats based on requests from advertisers, for example, changing a 15-second ad format to a 30-second one.

[1520] Step 5:

[1521] The server then distributes the converted ad format back to external businesses and advertisers, providing the converted format data using an API.

[1522] Step 6:

[1523] The server receives the user's emotion data sent from the emotion engine and optimizes the ad format based on this data, for example by setting it to preferentially display ad formats that the user is interested in.

[1524] 2. Terminal Processing

[1525] Step 1:

[1526] The terminal receives the advertisement data sent from the server, including the generated and converted format.

[1527] Step 2:

[1528] The device stores the received advertising data locally, in the appropriate folder or database, and prepares it for later display.

[1529] Step 3:

[1530] When a user uses a video service, the device displays a 15- or 30-second ad before the main content. The ad data is called up and displayed in the video player.

[1531] Step 4:

[1532] The device records the user's viewing history of advertisements, saving viewing time, click information, viewing completion rate, and other information as logs.

[1533] Step 5:

[1534] The device periodically sends the recorded advertising history to the server, allowing the effectiveness of advertising to be evaluated.

[1535] 3. User Actions

[1536] Step 1:

[1537] Users view advertisements provided through their terminals, such as advertisements played before the main content of a video service or advertisements displayed around the cover of an electronic magazine.

[1538] Step 2:

[1539] The user can provide feedback on the advertisement, for example, by inputting into the terminal whether the advertisement is interesting, relevant, annoying, etc.

[1540] 4. Emotion Engine Processing

[1541] Step 1:

[1542] The device uses an emotion engine to recognize the user's emotions in real time, for example by analyzing the user's facial expressions with a camera and using voice tones and physical reactions to generate emotion data.

[1543] Step 2:

[1544] The emotion data recognized by the emotion engine is sent from the device to the server. The emotion data includes whether the user was interested, enjoyed, or displeased.

[1545] Step 3:

[1546] The server optimizes the display of ad formats based on the emotion data. For example, it prioritizes displaying formats that the user has previously shown interest in, and avoids displaying formats that the user has previously shown discomfort with.

[1547] Specific examples

[1548] Server processing example

[1549] The server generates a 15-second video ad format and distributes it to external businesses via API. The external businesses use this format to create ad content and provide it to the server. The server sells the ads to advertisers, who then use the purchased format.

[1550] Terminal processing example

[1551] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1552] User processing example

[1553] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1554] Emotion engine processing example

[1555] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1556] Example 2

[1557] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] In the modern advertising ecosystem, there is a need to maximize convenience and effectiveness for both advertisers and users. However, in existing systems, ad format generation, distribution, sales, and conversion are carried out independently, resulting in a lack of consistency. Furthermore, technology to utilize user sentiment data to achieve more effective ad delivery is not yet fully developed. Furthermore, there is a lack of means to properly manage and utilize ad display timing and feedback, making it difficult to optimize ad delivery.

[1559] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for generating advertisement formats, a means for distributing the generated advertisement formats to external businesses, a means for selling advertisement formats, a means for converting advertisement formats into other formats, and a means for collecting user emotion data and optimizing advertisement formats based on the data. This enables the consistent generation, distribution, sale, conversion, and optimization of advertisement formats, realizing an effective advertising ecosystem for both advertisers and users. Furthermore, by optimizing advertisements based on user emotion data collected by the server, advertisement delivery that is less irritating to users can be achieved. Furthermore, a system is provided that can measure and optimize advertisement effectiveness in an integrated manner, by appropriately receiving advertisement data, managing display timing, recording and transmitting advertisement history to the server, and collecting and optimizing user feedback.

[1560] "Ad Format" means the prescribed format and structure for delivering advertising content.

[1561] An "external business operator" is a third party that uses the advertising format provided by the server, and refers to an organization or individual that distributes advertisements, such as advertisers.

[1562] An "advertiser" is a company or individual that posts advertisements for its products or services and provides advertising content to appeal to users.

[1563] "Emotional data" refers to data that reflects a user's emotions, and refers to information obtained by analyzing facial expressions, vocal tone, and other physical reactions.

[1564] An "emotion engine" is a device or software that recognizes a user's emotions in real time and generates emotion data.

[1565] "Device" refers to the electronic device (e.g., smartphone, tablet, or PC) used by a user to view advertising content.

[1566] "Advertisement history" is a record of information about advertisements viewed by a user, including display time and interaction information.

[1567] "Feedback" refers to opinions and ratings provided by users regarding advertisements.

[1568] MODE FOR CARRYING OUT THE INVENTION

[1569] This invention is a system for generating, distributing, selling, and converting advertising formats to provide a highly convenient and effective advertising ecosystem for both advertisers and users. The system operates based on the roles of the server, terminal, user, and emotion engine that recognizes the user's emotions. Specific ways in which the present invention is implemented are described below.

[1570] Server Processing

[1571] The server generates ad formats using template designs and layout generation algorithms, typically using software like Adobe Illustrator or Canva. For example, the server creates 15-second and 30-second video ad formats, magazine cover ad formats, and web rectangle ad formats.

[1572] The generated ad formats are distributed to external businesses via the server using APIs and download links, allowing AI businesses to create ad content using standardized ad formats.

[1573] The server also provides a dashboard for advertisers, allowing them to purchase the ad formats they need. Online payments are made using payment gateways such as Stripe or PayPal. After purchase, advertisers apply their ad content to the formats using the provided online tools (e.g., Google Ads, Facebook Ads Manager).

[1574] The server then uses AI models (e.g., TensorFlow, PyTorch) to convert ad formats, such as changing a 15-second ad to a 30-second one, and optimizes ad formats based on user sentiment data using IBM Watson's sentiment analysis API.

[1575] Terminal handling

[1576] The device receives the advertising data sent from the server and saves it in local storage (e.g., an SQLite database). The received advertising data is displayed at the appropriate time while the user is using a video service or reading an e-magazine. Specifically, a 15-second video advertisement is played using the device's video player (e.g., an HTML5 video tag). In addition, e-magazine apps use a PDF rendering library to display advertisements around the cover.

[1577] The device records the history of the ads displayed and uploads it to a server at regular intervals. Information such as the display time and user clicks is saved and used to measure the effectiveness of the ads.

[1578] User Action

[1579] Users view advertisements provided through their devices. The UI is designed to be intuitive and not cause discomfort to users while viewing. For example, advertisements displayed before the main content while using a video service are provided in a size and quality that is appropriate for the user.

[1580] Users can provide feedback on ads, for example, by entering comments about whether they found the ad interesting or annoying using the device's feedback function. This feedback is sent to the server in real time and used to optimize future ad formats.

[1581] Emotion engine processing

[1582] The device uses an emotion engine to recognize the user's emotions in real time, analyzing facial expressions, vocal tone, and other bodily responses to generate emotion data, using technologies such as FaceReader and IBM Watson Tone Analyzer.

[1583] The recognized emotion data is sent from the device to the server. This data is used to optimize ad formats, resulting in more effective ad display. For example, ad formats that the user finds interesting can be prioritized, or ad formats that the user finds annoying can be eliminated.

[1584] Examples of specific examples and prompts

[1585] Server processing example

[1586] The server generates 15-second video ad formats and distributes them to external businesses. The external businesses use these formats to create ad content and provide it to the server. The server sells these ads to advertisers, who then use the purchased formats in their own campaigns.

[1587] Terminal processing example

[1588] When a user uses a video service, the device displays a 15-second video advertisement received from the server before the main content. Once the advertisement is displayed, the device records the history and periodically sends it to the server.

[1589] User processing example

[1590] While a user is reading an e-magazine, the device displays an advertisement around the cover. If the user is interested in the advertisement and clicks on it, they can view more information. The user then provides feedback on whether the advertisement was useful or not.

[1591] Emotion engine processing example

[1592] When a user watches an ad, the emotion engine recognizes the user's emotions from their facial expressions and voice. For example, if the user finds the ad entertaining, that information is sent from the device to the server and reflected in future ad displays. Conversely, if the user finds the ad annoying, that ad format can be prevented from being displayed in the future.

[1593] Example prompt sentence:

[1594] "Optimize your ad formats based on users' emotional data. Emotional data includes facial expressions, vocal tone, and other bodily responses."

[1595] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1596] Step 1: Generate Ad Formats

[1597] The server starts generating ad formats. As input, it is given a template design and a layout generation algorithm. Specific software used is Adobe Illustrator or Canva. The server uses these tools to create 15-second and 30-second video ad formats and magazine cover ad formats. As output, it generates the generated ad formats.

[1598] Step 2: Distributing Ad Formats

[1599] The server distributes the generated ad formats to external providers. The generated ad formats and a list of external providers are given as input. In this step, the format data is sent via API. Alternatively, the server generates a file download link and notifies the provider. Specifically, the data is sent using a RESTful API. As output, the external provider receives the ad formats.

[1600] Step 3: Selling ads

[1601] The server provides a dashboard for advertisers. As input, it receives the ad format, advertiser information, and payment information. The advertiser selects the required ad format on the dashboard and makes an online payment (e.g., Stripe, PayPal). After completing the purchase, the advertiser applies their ad content to the format using the provided online tools (e.g., Google Ads, Facebook Ads Manager). As output, the advertiser receives the ad format and uses it in their campaign.

[1602] Step 4: Convert the format

[1603] The server uses an AI model to convert ad formats. It receives advertiser requests and existing ad formats as input. It uses TensorFlow or PyTorch to convert, for example, a 15-second ad to a 30-second one. It also uses IBM Watson's sentiment analysis API to optimize the ad format based on user sentiment data. The converted ad format is generated as output.

[1604] Step 5: Receiving and storing advertising data

[1605] The device receives advertising data sent from the server. As input, the advertising data sent from the server is given. The device saves this in local storage (e.g., SQLite database). Specifically, the device downloads the data using an HTTP request and saves it in SQLite. As output, the advertising data is saved on the device.

[1606] Step 6: Displaying the Ad

[1607] The device displays the advertisement. As input, it receives the locally stored advertisement data. When the user uses a video service or digital magazine, it plays a 15-second video advertisement using a video player (e.g., HTML5 video tag). It also uses a PDF rendering library to display an advertisement around the cover of the digital magazine. As output, the advertisement is displayed to the user.

[1608] Step 7: Record and submit your ad history

[1609] The device records the history of displayed ads. As input, it receives information about the displayed ads (e.g., display time, user click information). The device stores this information in an SQLite database. Periodically, it uploads the history data to the server using HTTP POST. As output, the ad history is sent to the server and used for performance measurement.

[1610] Step 8: Watch an ad

[1611] The user watches the advertisement through the device. As input, an advertisement played on the device's video player or an advertisement on the cover of an electronic magazine is given. The user watches the advertisement and performs a specific interaction (e.g., click, skip). As output, the advertisement is watched by the user.

[1612] Step 9: Provide feedback

[1613] Users provide feedback on ads. As input, a feedback form about the ads they watched is provided. Users enter their opinions and ratings in the form and click the submit button. As output, the feedback data is sent to the server and used to optimize future ad formats.

[1614] Step 10: Emotion Recognition

[1615] The device uses an emotion engine to recognize the user's emotions. The input is the user's facial expressions and vocal tone. The emotion engine analyzes this data and generates emotion data. Specifically, it uses the built-in camera and microphone, and employs FaceReader and IBM Watson Tone Analyzer. The output is the recognized emotion data.

[1616] Step 11: Sending Emotion Data

[1617] The device sends the recognized emotion data to the server. As input, the recognized emotion data is given. This data is sent to the server using an HTTP POST request. As output, the emotion data is stored on the server and used to optimize ad formats.

[1618] Through these processing steps, the system can consistently generate, optimize, distribute, and sell ad formats, as well as display ads using user emotion data, thereby realizing a valuable advertising ecosystem for both advertisers and users.

[1619] (Application example 2)

[1620] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1621] Conventional advertising systems do not take into account the user's emotional state, making it difficult to maximize advertising effectiveness. Furthermore, ads are not displayed based on the user's interests or emotions, resulting in a poor user experience. Furthermore, real-time data collection and ad display optimization are insufficient, making it difficult for advertisers to measure advertising effectiveness.

[1622] The specification process by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating an advertisement format, means for distributing the generated advertisement format to an external business, means for selling the advertisement format, means for converting the advertisement format into another format, emotion recognition means for recognizing a user's emotion in real time and transmitting the data to the server, and means for optimizing the display method of the advertisement format based on the user's emotion data. This makes it possible to display advertisements taking into account the user's emotion and maximize the effectiveness of the advertisements. In addition, real-time data collection and optimization of advertisement display are realized, making it easier for advertisers to measure effectiveness.

[1623] An "advertising format" is a standard format that defines the layout and design of advertising content.

[1624] "Third Party" means any other company or individual that creates and distributes advertising content using the Ad Format.

[1625] An "emotion recognition means" is a device or software that recognizes a user's emotional state in real time by analyzing the user's facial expressions, vocal tone, and other bodily responses.

[1626] "Server" means the computer system that generates, distributes, sells, and converts advertising formats, and receives and analyzes emotional data.

[1627] "User terminal" refers to a device on which a user views advertisements, such as a smartphone, smart glasses, or head-mounted display.

[1628] "Feedback" refers to opinions and impressions that users provide about advertisements.

[1629] "Ad display optimization" refers to the process of adjusting how ads are displayed to maximize their effectiveness based on collected data and feedback.

[1630] "Real-time" means near-immediate processing and response with little delay.

[1631] "Ad display history" refers to a record of the type, time, and interaction information of the ads displayed by the user.

[1632] The present invention is a system for generating, distributing, selling, converting, and optimizing advertising formats based on user emotion data. This system mainly operates in cooperation with a server, a terminal, a user, and an emotion recognition means.

[1633] Server Processing

[1634] The server has the ability to generate ad formats and distribute them to external parties. The server provides a dashboard for advertisers to purchase, apply, and change ad formats. The server also has the ability to convert ad formats into other formats, for example, converting a 15-second ad into a 30-second ad.

[1635] Furthermore, the emotional data of the user is received through the emotion recognition means, and the display method of the advertisement format is optimized based on the received data. The emotional data when the user watches the advertisement is analyzed in real time, and the content and display method of the advertisement are adjusted based on the specific emotional response.

[1636] Terminal handling

[1637] The device receives the advertisement data sent from the server and displays it to the user at the appropriate time. For example, if the user is wearing smart glasses, the device displays the advertisement through the glasses. The device also records the history of the displayed advertisements and sends it to the server.

[1638] The device uses an emotion recognition means to recognize the user's emotions in real time. This emotion data is generated by analyzing facial expressions, vocal tone, and other physical reactions, and is transmitted to a server for use in optimizing advertisement display.

[1639] User Action

[1640] Users can view advertisements provided through their devices and provide feedback on the advertisements, including opinions on whether the advertisements were interesting, relevant, annoying, etc. This feedback is collected through the devices and transmitted to a server.

[1641] Hardware and software used

[1642] Hardware:

[1643] Smart glasses (with camera)

[1644] Regular smartphones and tablets

[1645] software:

[1646] OpenCV: Used for emotion recognition.

[1647] requests: Used to communicate with the server.

[1648] Pre-trained Emotion Recognition Model.

[1649] Specific examples

[1650] While wearing the smart glasses, a user walks around town and a new advertisement is displayed every five minutes. While the user is looking at the advertisement, a camera analyzes their facial expressions and sends them to a server. The server uses this data to optimize the next advertisement to be displayed.

[1651] For example, if the smart glasses screen displays a message saying, "New drink campaign! Click here!" and the user responds, this information will also be sent to the server. Based on the emotional data, the next advertisement display will be further optimized.

[1652] Example of input prompt for generative AI model

[1653] I would like to develop an application for smart glasses that recognizes the user's facial expressions in real time while watching advertisements and sends the emotion data to a server. The content of the advertisements will be optimized based on the user's emotion data and displayed at regular intervals. Please explain in detail how to build this application.

[1654] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1655] Step 1:

[1656] The server generates ad formats. The input is basic ad materials (images, videos, text) provided by the advertiser, and the output is a formatted ad format. Specifically, the server uses template design and layout generation algorithms to generate 15-second or 30-second video ads, magazine cover ads, online rectangular ads, etc.

[1657] Step 2:

[1658] The server distributes the generated ad format. The input is the ad format generated in step 1, and the output is the format data provided to external businesses. Specifically, the server provides the ad format to external businesses via an API or download link.

[1659] Step 3:

[1660] The server sells ad formats. The input is the advertiser's purchase request, and the output is the sold ad format. Specifically, the server allows advertisers to purchase the ad formats they need through a dashboard, and advertisers use online tools to apply their ad content to the formats.

[1661] Step 4:

[1662] The server converts the ad format. The input is the advertiser's conversion request and emotional data, and the output is the converted ad format. Specifically, the server uses AI to change the ad format according to the advertiser's request, for example, changing a 15-second ad to 30 seconds. The server also optimizes the ad format based on the user's emotional data.

[1663] Step 5:

[1664] The device receives advertising data from the server. The input is the advertising data sent from the server, and the output is the locally stored advertising data. Specifically, the device periodically communicates with the server and downloads new advertising data.

[1665] Step 6:

[1666] The device displays advertisements to the user. The input is the saved advertisement data, and the output is the displayed advertisement. Specifically, when a user uses a video service, the device plays a 15- or 30-second advertisement before the main content, and inserts a cover advertisement while reading an e-magazine.

[1667] Step 7:

[1668] The device records the history of ads displayed and sends it to the server. The input is the user's ad viewing data, and the output is the history data sent to the server. Specifically, the device records information such as the time the ad was displayed and the user's interactions, and periodically sends this information to the server.

[1669] Step 8:

[1670] The device uses emotion recognition means to recognize the user's emotions in real time. The input is the user's facial expressions and voice, and the output is emotion data. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice, and analyzes them using an emotion recognition algorithm.

[1671] Step 9:

[1672] The device sends the recognized emotion data to the server. The input is the emotion data analyzed in real time, and the output is the emotion data sent to the server. Specifically, the device sends the recognized emotion data to the server via the network.

[1673] Step 10:

[1674] The user provides feedback on the advertisement. The input is the feedback information provided by the user, and the output is the feedback data recorded on the terminal. Specifically, the user uses the feedback function of the terminal to input their opinion on whether the advertisement was interesting, useful, or unpleasant.

[1675] Step 11:

[1676] The terminal sends the collected feedback to the server. The input is the user's feedback data, and the output is the feedback data sent to the server. Specifically, the terminal periodically sends the collected feedback to the server.

[1677] Step 12:

[1678] The server optimizes the display method of the ad format based on the collected emotional data and feedback. The input is the emotional data and feedback, and the output is the optimized ad display method. Specifically, the server analyzes the emotional data and feedback, evaluates which format and content of the ad is effective, and reflects this in the next ad display.

[1679] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1680] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1681] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1682] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1683] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1684] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1685] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1686] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1687] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1688] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1689] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1690] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1691] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1692] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1693] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1694] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1695] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1696] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1697] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1698] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1699] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1700] The following is further disclosed regarding the above embodiment.

[1701] (Claim 1)

[1702] means for generating an ad format;

[1703] A means for distributing the generated advertising format to an external business;

[1704] a means of selling advertising formats;

[1705] A means of converting advertising formats into other formats;

[1706] A system including:

[1707] (Claim 2)

[1708] means for receiving advertising data;

[1709] means for displaying the received advertising data on a user terminal at an appropriate timing;

[1710] a means for recording a history of displayed advertisements and transmitting the recorded history to a server;

[1711] The system of claim 1 further comprising:

[1712] (Claim 3)

[1713] means for collecting feedback from users about the advertisements and transmitting the feedback to a server;

[1714] A means to optimize the display of ad formats based on collected feedback; and

[1715] The system of claim 1 further comprising:

[1716] "Example 1"

[1717] (Claim 1)

[1718] means for generating an ad format;

[1719] A means of distributing the generated ad formats to external businesses via API or download link;

[1720] a means for providing a dashboard for advertisers to sell advertising formats;

[1721] A means to use machine learning algorithms to convert the size and format of ad formats according to advertiser requests;

[1722] A system including:

[1723] (Claim 2)

[1724] means for receiving and storing advertising data in local storage;

[1725] A means for displaying the stored advertisement data at an appropriate time when the user uses a video service or an electronic magazine;

[1726] a means for recording the history of displayed advertisements and periodically transmitting the history to a server;

[1727] The system of claim 1 further comprising:

[1728] (Claim 3)

[1729] means for collecting feedback from users about the advertisements and transmitting the feedback to a server;

[1730] a means of optimizing the display of ad formats using machine learning algorithms based on collected feedback; and

[1731] The system of claim 1 further comprising:

[1732] "Application Example 1"

[1733] (Claim 1)

[1734] means for generating an ad format;

[1735] A means for distributing the generated advertising format to an external business;

[1736] a means of selling advertising formats;

[1737] A means of converting advertising formats into other formats;

[1738] a means for personalizing ad formats based on user behavioral data and feedback;

[1739] A means for selecting the optimal advertising format based on user behavior analysis using an AI model;

[1740] A system including:

[1741] (Claim 2)

[1742] means for receiving advertising data;

[1743] means for displaying the received advertising data on a user terminal at an appropriate timing;

[1744] a means for recording a history of displayed advertisements and transmitting the recorded history to a server;

[1745] A means for recording a user's response to the displayed advertisement and reflecting the response in the next advertisement display;

[1746] The system of claim 1 further comprising:

[1747] (Claim 3)

[1748] means for collecting feedback from users about the advertisements and transmitting the feedback to a server;

[1749] A means to optimize the display of ad formats based on collected feedback; and

[1750] A means for dynamically generating input prompts to a generative AI model to provide personalized advertisements to a user;

[1751] The system of claim 1 further comprising:

[1752] "Example 2: Combining Emotion Engines"

[1753] (Claim 1)

[1754] means for generating an ad format;

[1755] A means for distributing the generated advertising format to an external business;

[1756] a means of selling advertising formats;

[1757] A means of converting advertising formats into other formats;

[1758] a means for collecting user sentiment data and optimizing ad formats based thereon;

[1759] A system including:

[1760] (Claim 2)

[1761] means for receiving advertising data;

[1762] means for displaying the received advertising data on a user terminal at an appropriate timing;

[1763] a means for recording a history of displayed advertisements and transmitting the recorded history to a server;

[1764] means for recognizing a user's emotion using an emotion engine;

[1765] The system of claim 1 further comprising:

[1766] (Claim 3)

[1767] means for collecting feedback from users about the advertisements and transmitting the feedback to a server;

[1768] A means to optimize the display of ad formats based on collected feedback; and

[1769] The system of claim 1 further comprising:

[1770] "Application example 2 when combining emotion engines"

[1771] (Claim 1)

[1772] means for generating an ad format;

[1773] A means for distributing the generated advertising format to an external business;

[1774] a means of selling advertising formats;

[1775] A means of converting advertising formats into other formats;

[1776] emotion recognition means for recognizing the user's emotions in real time and transmitting the data to a server;

[1777] A means for optimizing the display method of an ad format based on user emotion data;

[1778] A system including:

[1779] (Claim 2)

[1780] means for receiving advertising data;

[1781] means for displaying the received advertising data on a user terminal at an appropriate timing;

[1782] a means for recording a history of displayed advertisements and transmitting the recorded history to a server;

[1783] A means for acquiring user emotion data and transmitting it to a server;

[1784] The system of claim 1 further comprising:

[1785] (Claim 3)

[1786] means for collecting feedback from users about the advertisements and transmitting the feedback to a server;

[1787] A means to optimize the display of ad formats based on collected feedback and sentiment data; and

[1788] The system of claim 1 further comprising: [Explanation of symbols]

[1789] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for generating an ad format; A means for distributing the generated advertising format to an external business; a means of selling advertising formats; A means of converting advertising formats into other formats; A system including:

2. means for receiving advertising data; means for displaying the received advertising data on a user terminal at an appropriate timing; a means for recording a history of displayed advertisements and transmitting the recorded history to a server; The system of claim 1 further comprising:

3. means for collecting feedback from users about the advertisements and transmitting the feedback to a server; A means to optimize the display of ad formats based on collected feedback; and The system of claim 1 further comprising:

Images