System

Abstract

A system is provided.SOLUTION: A system comprising: means for generating new content based on material provided by generative artificial intelligence; means for creating a viewer profile based on a viewer's behavioral history and preferences; means for generating customized content based on the viewer profile; and means for delivering the customized content to the viewer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional video distribution platforms, it was difficult to provide individually customized content tailored to viewer preferences. This was because content creation by creators and the creation of profiles reflecting each viewer's preferences were done manually, requiring a great deal of time and effort. It was also not easy to optimally insert advertisements and match their content to viewer interests. These issues led to lower viewer satisfaction and reduced platform usability. [Means for solving the problem]

[0005] The present invention is a system that includes a means for generating new content based on materials provided by generative artificial intelligence, a means for creating a viewer profile based on the viewer's behavioral history and preferences, a means for generating customized content based on the viewer profile, and a means for delivering the customized content to the viewer. The system further includes a means for inserting advertisements into the generated customized content, a means for tracking the performance of the advertisements, and a means for collecting feedback from the viewer and updating the viewer profile. This enables the provision of high-quality video content that is fully tailored to the viewer's preferences and maximizes the effectiveness of advertising. As a result, viewer engagement and satisfaction are improved, and the value of the platform is enhanced.

[0006] "Generative AI" is AI that has the ability to automatically generate new content based on provided materials.

[0007] "Materials" are the data that form the basis of the content to be generated, such as video, audio, and text.

[0008] "Content" refers to digital information such as video and audio provided to viewers.

[0009] A "viewer profile" is a collection of information about a viewer that is created based on the viewer's behavioral history, preferences, etc.

[0010] "Customized content" is content that is tailored to a viewer's preferences based on a viewer profile.

[0011] "Distribution" refers to the act of transmitting the generated content to the viewer's terminal.

[0012] "Advertisement" refers to video and audio content that conveys promotional content to viewers.

[0013] "Ad performance" is an indicator of how effectively an advertisement reaches an audience.

[0014] "Feedback" refers to opinions and reactions collected from viewers.

[0015] "Viewer behavior history" refers to data about the content a viewer has viewed in the past and that viewing. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] Specific embodiments of the system related to the present invention will be described below.

[0038] System Overview

[0039] The system consists of three main components: a server, a terminal, and a user. It generates new content based on the provided material using generative AI, delivers customized content based on the viewer profile, optimizes advertising, and collects and updates feedback.

[0040] Program processing flow

[0041] Uploading and Preprocessing Materials

[0042] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality, and performs pre-processing such as encoding and trimming.

[0043] Examples:

[0044] Creators upload footage of natural scenery they have taken and the corresponding music files.

[0045] The server checks the video resolution and converts it into a unified format.

[0046] Generate audience profiles

[0047] The device collects the viewer's past viewing history, preferences, device information, etc. This information is sent to a server to create a viewer profile.

[0048] Examples:

[0049] The genre that users often watch is cooking-related videos.

[0050] The device records viewing history, sends it to the server, and creates a profile called "I love cooking."

[0051] Customized Content Generation

[0052] The server instructs the generative AI based on the viewer profile to generate new customized content.

[0053] Examples:

[0054] The server instructs the generation AI to add cooking scenes to travel footage for users who love cooking.

[0055] The generative AI generates scenes introducing famous local dishes to accompany travel footage.

[0056] Content distribution and engagement tracking

[0057] The server delivers the generated customized content to the device, and collects viewer engagement data (watch time, number of plays, reactions, etc.) to use for the next profile update.

[0058] Examples:

[0059] A server distributes travel videos including cooking scenes to users who love cooking.

[0060] The terminal collects the user's viewing data and sends it to the server.

[0061] Ad insertion and optimization

[0062] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the ad performance.

[0063] Examples:

[0064] The server selects organic food advertisements and inserts them into videos targeted at health-conscious viewers.

[0065] The server monitors the click-through rate and completion rate of the ads and makes appropriate adjustments.

[0066] Collecting feedback and updating your profile

[0067] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next video generation.

[0068] Examples:

[0069] The user provides feedback regarding preferences for particular effects and scenes.

[0070] The server adds this information to a profile and uses it the next time a video is generated.

[0071] conclusion

[0072] The system generates and delivers high-quality video content customized based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[0076] Specific behavior:

[0077] The server checks the format of the uploaded file (JPEG, MP4, etc.).

[0078] The server verifies the video resolution and audio bit rate.

[0079] Step 2:

[0080] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[0081] Specific behavior:

[0082] The server converts all video to a uniform resolution.

[0083] The server processes the audio track for noise reduction.

[0084] Step 3:

[0085] The device collects the viewer's behavioral history, preferences, device information, etc. and sends it to the server, which then creates a viewer profile based on this information.

[0086] Specific behavior:

[0087] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[0088] The terminal transmits the collected data to the server.

[0089] The server generates a viewer profile based on the received data.

[0090] Step 4:

[0091] The server instructs the generative AI based on the viewer profile to generate new customized content.

[0092] Specific behavior:

[0093] The server uses the viewer profile to pass appropriate content generation parameters to the generation AI.

[0094] Generative AI generates new content by adding elements that viewers like to specific scenes in a video.

[0095] Step 5:

[0096] The server delivers the generated customized content to the device, which collects viewer engagement data (watch time, number of plays, reactions, etc.) and sends it back to the server.

[0097] Specific behavior:

[0098] The server transmits the customized content to the terminal in a streaming format.

[0099] The device collects viewer engagement data in real time.

[0100] The engagement data collected by the device is periodically sent to the server.

[0101] Step 6:

[0102] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the performance of the ads.

[0103] Specific behavior:

[0104] The server analyzes the viewer profile and selects appropriate advertisements.

[0105] The server decides when to insert the advertisement during video playback.

[0106] The server tracks ad click rates and completion rates and uses that data to optimize ads.

[0107] Step 7:

[0108] The device collects feedback from viewers and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next content generation.

[0109] Specific behavior:

[0110] The device collects feedback provided by the viewer (e.g., "I like this scene").

[0111] The device sends the collected feedback to the server.

[0112] The server updates the viewer profile based on the feedback and reflects it in the parameters of the next generation AI.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] Video viewers often demand content that is individually customized based on their diverse preferences and viewing history. Streaming platforms are also required to insert appropriate advertisements and optimize their performance. However, conventional systems face challenges in automating these requirements, making it difficult and time-consuming.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a user profile based on the user's behavioral history and preferences, means for generating customized content based on the user profile, means for checking the format and quality of the material and encoding and trimming it, means for collecting user engagement data and updating the user profile, and means for delivering customized content to users. This enables the automatic generation and delivery of high-quality content customized based on user preferences.

[0118] (The system of claim 2)

[0119] The system further includes a means for inserting advertisements into the generated customized content and a means for tracking and optimizing the performance of the advertisements, thereby providing the most suitable advertisements to the viewer and maximizing their effectiveness.

[0120] "Generative AI" refers to AI that has the ability to generate new information and content based on diverse data.

[0121] A "user profile" is a database or collection of information constructed based on a user's behavioral history and preferences.

[0122] "Customized Content" means content that has been adapted or edited based on a specific user profile.

[0123] "Encoding" is the process of converting data into a particular format.

[0124] "Trimming" is the process of cutting out unnecessary parts of video or audio data.

[0125] "Engagement data" refers to behavioral data such as viewer viewing time, number of views, and reactions.

[0126] "Feedback" refers to opinions and ratings provided by users.

[0127] "Ad insertion" refers to the process of placing advertisements within video or audio content.

[0128] "Performance tracking" refers to the collection and analysis of data to measure and analyze the effectiveness of advertising and other activities.

[0129] "Optimization" is the process of adjusting or improving a system or process to maximize its efficiency or effectiveness.

[0130] MODE FOR CARRYING OUT THE INVENTION

[0131] The system related to this invention mainly consists of three elements: a server, a terminal, and a user. The system generates new content based on provided materials using a generative AI model, delivers customized content based on viewer profiles, and optimizes advertisements and collects and updates feedback.

[0132] server

[0133] The server manages everything from uploading materials to generating and distributing customized content. Specific processes include the following:

[0134] 1. Upload and pre-process materials

[0135] The server receives video, audio, and text data from creators. It checks the file format and quality, and encodes and trims it. For example, it uses the FFmpeg tool to convert and unify the format and quality of the material.

[0136] 2. Generate audience profiles

[0137] The server receives data on the viewer's behavior and preferences sent from the device, creates a viewer profile, and stores this data in a database (e.g., MySQL).

[0138] 3. Creating customized content

[0139] Based on the viewer profile, the server sends prompts to the AI ​​to generate customized content, such as "add cooking scenes to travel videos."

[0140] 4. Content distribution and engagement tracking

[0141] The server delivers the generated customized content to the device and collects viewer engagement data using data streaming technologies (e.g., RTMP, HLS).

[0142] 5. Ad insertion and optimization

[0143] The server selects and inserts appropriate ads based on viewer profile and video context, and also tracks and optimizes ad performance.

[0144] 6. Collect feedback and update your profile

[0145] The server collects feedback sent by the terminals and updates the viewer profile.

[0146] Terminal

[0147] The device collects the user's behavior history and sends it to the server. It also displays customized content and advertisements received from the server, collects viewing data and feedback, and sends them to the server.

[0148] User

[0149] Users can view content via their terminals and provide feedback, for example by entering their preferences and opinions about a particular scene in a feedback form.

[0150] Specific examples

[0151] For example, if a user frequently watches cooking-related videos, the device will collect that viewing history and send it to the server, which will then use that data to create a "cooking lover" profile and instruct the AI ​​to add cooking scenes to travel videos in the next customized content.

[0152] Prompt Sentence Examples

[0153] "For food-loving viewers, add a segment to your travel footage showcasing famous local dishes."

[0154] This system generates and distributes high-quality video content based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Program processing flow

[0157] Step 1. Upload and preprocess your materials

[0158] 1.1 Uploading Materials

[0159] Creators upload video footage, audio, and text data to the server, which triggers the action of selecting files through a web interface and pressing the upload button.

[0160] Input: Video material files, audio files, text files

[0161] Output: Original material data stored on the server

[0162] 1.2 File format and quality check

[0163] The server checks the format and quality of the uploaded file by using tools like FFmpeg to extract file information and verify things like format, resolution, bitrate, etc.

[0164] Input: Original material data stored on the server

[0165] Output: Format and quality verified material data

[0166] 1.3 Encoding and Trimming

[0167] The server encodes and trims the file as needed, for example converting the video file to the specified resolution and trimming unwanted parts.

[0168] Input: Format and quality verified material data

[0169] Output: Encoded and trimmed material data

[0170] Step 2. Generate your audience profile

[0171] 2.1 Collecting viewing history and preferences

[0172] The device collects information about the viewer's viewing history, preferred genres, and playback devices, including using browser and app cookies to record viewing data.

[0173] Input: Viewer viewing activity

[0174] Output: Collected viewing history data

[0175] 2.2 Sending profile data

[0176] The device sends the collected data to the server. The data is packetized in JSON format and sent via an HTTP POST request.

[0177] Input: Collected viewing history data

[0178] Output: Profile data sent to the server

[0179] 2.3 Creating an audience profile

[0180] The server creates a viewer profile based on the data it receives, and records the viewer's preferences and history in a database.

[0181] Input: Profile data sent to the server

[0182] Output: Viewer profile stored in a database

[0183] Step 3. Generate customized content

[0184] 3.1 Creating prompts for generative AI

[0185] The server creates a prompt to send to the AI ​​based on the viewer profile, such as "Add cooking scenes to travel videos for viewers who love cooking."

[0186] Input: Viewer profile stored in database

[0187] Output: Generated prompt statement

[0188] 3.2 Content Generation Instructions

[0189] The server sends a prompt to the generative AI model to instruct it to generate content. The prompt is sent to the generative AI using an API request.

[0190] Input: Generated prompt statement

[0191] Output: The prompt sent to the generative AI model

[0192] 3.3 Creating customized content

[0193] The generative AI generates new content based on the prompt and sends it back to the server, for example adding a cooking scene to a video and generating a text description.

[0194] Input: The prompt sent to the generative AI model

[0195] Output: Generated customization content

[0196] Step 4. Content distribution and engagement tracking

[0197] 4.1 Delivery of customized content

[0198] The server delivers the generated customized content to the terminal. The content is delivered using data streaming technology.

[0199] Input: Generated customization content

[0200] Output: Content delivered to the device

[0201] 4.2 Collection of viewing data

[0202] The device collects viewer engagement data (watch time, plays, reactions, etc.) and records this data using JavaScript code.

[0203] Input: Content delivered to the device

[0204] Output: Collected engagement data

[0205] 4.3 Transmission of Collected Data

[0206] The device sends engagement data to the server via an HTTP request for analysis.

[0207] Input: Collected engagement data

[0208] Output: Engagement data sent to the server

[0209] Step 5. Ad insertion and optimization

[0210] 5.1 Ad Selection

[0211] The server chooses the best ad based on the viewer profile and video context, selecting the appropriate ad from an ad database.

[0212] Input: Viewer profile stored in database

[0213] Output: Selected ads

[0214] 5.2 Ad Insertion

[0215] The server inserts the selected advertisement into the video at the specified position. The advertisement is then integrated into the video using a video editing tool.

[0216] Input: Selected ads, customized content

[0217] Output: Customized content with ad insertion

[0218] 5.3 Tracking Ad Performance

[0219] The server monitors the click-through rate and completion rate of ads and automatically adjusts the optimal ads. Performance data is collected using analytical tools and optimized by algorithms.

[0220] Input: Ad-inserted personalized content, viewing data

[0221] Output: Optimized ad placement

[0222] Step 6. Collect feedback and update your profile

[0223] 6.1 Feedback Collection

[0224] The device collects feedback from viewers, which involves viewers entering their opinions and thoughts through a form in their browser or app.

[0225] Input: Viewer feedback

[0226] Output: Collected feedback data

[0227] 6.2 Sending Feedback

[0228] The device sends the collected feedback to the server via an HTTP POST request.

[0229] Input: Collected feedback data

[0230] Output: Feedback data sent to the server

[0231] 6.3 Updating your profile

[0232] The server updates the viewer profile based on the feedback, updating the profile information in the database and reflecting it in the next content generation.

[0233] Input: Feedback data sent to the server

[0234] Output: Updated viewer profile

[0235] (Application example 1)

[0236] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0237] Conventional content distribution systems lack sufficient customization based on individual viewer preferences and behavioral history, and lack complete feedback and engagement tracking to improve the quality of the viewing experience. Furthermore, they are unable to select optimal ads based on viewer profiles and track their effectiveness, making it difficult to maximize advertising effectiveness. This results in insufficient improvement in viewer satisfaction and the value of the platform.

[0238] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0239] In this invention, the server includes means for generating new content based on materials provided by the generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for delivering the customized content to the viewer, means for collecting viewer engagement data and using it for the next profile update, means for inserting advertisements into the generated customized content, means for tracking the performance of the advertisements, means for selecting optimal advertisements based on the viewer profile and video context, means for collecting feedback from the viewer and updating the viewer profile, and means for reflecting the collected feedback in the profile and using it for generating the next video. This enables the generation and delivery of high-quality customized content based on the viewer's preferences and behavioral history, maximizing the effectiveness of advertisements and improving viewer satisfaction and the value of the platform.

[0240] "Generative AI" is an AI technology that generates new information and content based on input data.

[0241] "Materials" refers to the basic data for generating content, such as video, audio, and text data provided by creators.

[0242] "Customized content" is individual content that is generated based on a viewer profile and tailored to the viewer's preferences and behavioral history.

[0243] A "viewer profile" refers to a data structure that indicates the characteristics of a viewer, constructed by analyzing data related to the viewer's behavioral history and preferences.

[0244] "Viewer engagement data" refers to behavioral data of viewers when they watch content, including viewing time, number of plays, reactions, etc.

[0245] "Feedback" refers to information such as reactions, opinions, and ratings from viewers, which is used to generate next content and update profiles.

[0246] "Ad performance" refers to metrics that evaluate how effectively an ad works for viewers, including click-through rate and completion rate.

[0247] "Video context" refers to the background information surrounding the video, such as the content, theme, and scene situation of the video being generated.

[0248] As an embodiment of the present invention, a specific system configuration and its operation will be described below.

[0249] System configuration

[0250] Server: A central computer system that processes generative AI models, creates and updates audience profiles, collects and analyzes engagement data, and optimizes and tracks advertising effectiveness.

[0251] Terminal: A device used by viewers, such as a smartphone, tablet, or smart glasses, that collects viewers' behavioral history and transmits it to a server.

[0252] Users: Viewers and creators who use the system. Creators upload materials such as videos and audio, and viewers watch customized content.

[0253] Program processing

[0254] Uploading and Preprocessing Materials

[0255] Creators upload video footage, audio, text data, etc. to the server, which then uses OpenCV to divide the footage into frames, check the quality, encode, and trim as needed.

[0256] Generate audience profiles

[0257] The device collects information such as the viewer's past viewing history, preferences, and device information, and sends it to the server, which then uses Scikit-learn's KMeans clustering to generate a viewer profile.

[0258] Customized Content Generation

[0259] Based on the generated viewer profile, the server invokes a generative AI model using TensorFlow to generate customized content, adding elements that match the profile to the content in the process.

[0260] Content distribution and engagement tracking

[0261] The server delivers the generated customized content to the device, and the device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server.

[0262] Ad insertion and optimization

[0263] The server selects the best ads based on viewer profile and video context, inserts them within the customized content, and tracks and optimizes ad performance (e.g., click-through rate and completion rate).

[0264] Collecting feedback and updating your profile

[0265] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and generate customized content for the next viewing.

[0266] Specific examples

[0267] As a concrete example, consider the case where a user who loves cooking uses this system on a smartphone. The user uses the app just like a regular video streaming app. Based on the user's past viewing history, the server generates a profile for the user called "cooking lover" and provides new cooking-related content. For example, based on the user's profile, the server can generate and distribute new travel videos including Chinese food recipe videos.

[0268] Prompt Sentence Examples

[0269] "User profile: Loves cooking. Viewing genre: Chinese food. Content to generate: Generate travel footage that includes Chinese food recipes."

[0270] In this way, the system can efficiently generate and deliver high-quality customized content tailored to viewer preferences and behavioral history, while optimizing advertising effectiveness, thereby improving viewer satisfaction and enhancing the value of the platform.

[0271] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0272] Step 1:

[0273] Creators upload video footage, audio, text data, etc. to the server. The input is the content material created by the creator, and the output is when these materials are saved on the server and prepared for further processing. The server uses OpenCV to divide the video footage into frames, perform quality checks, encode, and trim as necessary.

[0274] Step 2:

[0275] The device collects the viewer's past viewing history, preferences, and device information, and sends it to the server. The input is the viewer's behavioral history and device information, and the output is this information sent to the server. The server generates a viewer profile using Scikit-learn's KMeans clustering. At this stage, the viewer's preferences and behavioral patterns are analyzed and a profile is constructed.

[0276] Step 3:

[0277] The server uses TensorFlow to call a generative AI model based on the generated viewer profile to generate customized content. The input is the viewer profile and uploaded materials, and the output is the customized content. The generative AI model adds elements that match the profile to the content.

[0278] Step 4:

[0279] The server delivers the generated customized content to the device. The device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server. The input is the customized content and the viewer's reactions, and the output is viewing data and engagement data. The collected engagement data is reflected in the next content generation.

[0280] Step 5:

[0281] The server selects the optimal ad based on the viewer profile and video context and inserts the ad into the customized content. The input is the viewer profile, content context, and candidate ads, and the output is the customized content with the ad inserted. In addition, the server tracks the ad performance (click-through rate, completion rate, etc.) and performs optimization to maximize the advertising effect.

[0282] Step 6:

[0283] The device collects feedback from viewers and sends it to the server. The server updates the viewer profile based on this feedback and uses it to generate the next customized content. The input is viewer feedback, and the output is an updated viewer profile. Based on the information gained from the feedback, the quality of the next content provided can be further improved.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] Specific embodiments of the system related to the present invention will be described below.

[0286] System Overview

[0287] This system combines three main elements - the server, the device, and the user - with an emotion engine that recognizes the user's emotions to provide a more personalized viewing experience. It generates new content based on the materials provided by the generative AI, delivers customized content based on the viewer profile, and optimizes advertising using emotional data.

[0288] Program processing flow

[0289] Uploading and Preprocessing Materials

[0290] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the material, and performs pre-processing such as encoding and trimming.

[0291] Examples:

[0292] Creators upload footage of natural scenery they have taken and the corresponding music files.

[0293] The server checks the video resolution and converts it into a unified format.

[0294] Collecting Emotional Data

[0295] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[0296] Examples:

[0297] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[0298] The device sends the analysis results to the server.

[0299] Generate audience profiles

[0300] The device collects the viewer's behavioral history, preferences, device information, as well as emotional data, and sends it to the server, which then creates a viewer profile based on this information.

[0301] Examples:

[0302] The device collects the viewer's past viewing data and emotional data and sends it to the server.

[0303] The server analyzes scenes that elicit a positive emotional response and adds the viewer's "preferences" to their profile.

[0304] Customized Content Generation

[0305] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account emotional data as well.

[0306] Examples:

[0307] The server gives instructions to the generation AI, focusing on scenes that make the server feel happy.

[0308] Generative AI generates new content that includes many scenes that evoke positive emotions.

[0309] Content distribution and engagement tracking

[0310] The server delivers the generated customized content to the device, which collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends it back to the server.

[0311] Examples:

[0312] The server transmits the customized content to the terminal in streaming format.

[0313] The device collects viewer engagement data and emotional reactions and sends them to a server.

[0314] Ad insertion and optimization

[0315] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[0316] Examples:

[0317] The server inserts relevant advertisements immediately after scenes that elicit positive emotions.

[0318] The server monitors the click-through rate and completion rate of the advertisements to optimize the effectiveness of the advertisements.

[0319] Collecting feedback and updating your profile

[0320] The device collects feedback from viewers and sends it to the server, which updates the viewer profile based on this feedback and emotional data and reflects it in the next content generation.

[0321] Examples:

[0322] Users provide feedback on their favorite scenes and effects.

[0323] The server uses the feedback and emotional data to update the viewer profile and use it to generate the next piece of content.

[0324] This enables the system to provide highly personalized video content and optimize advertising in response to viewer emotions, improving the viewing experience and increasing the value of the platform.

[0325] The processing flow will be explained below.

[0326] Step 1:

[0327] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[0328] Specific behavior:

[0329] Creators upload footage of natural scenery they have taken and the corresponding music files to a server.

[0330] The server checks the format of the uploaded file (JPEG, MP4, etc.) and verifies the video resolution and audio bitrate.

[0331] Step 2:

[0332] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[0333] Specific behavior:

[0334] The server converts all video files to a uniform resolution.

[0335] The server processes the audio track for noise reduction.

[0336] Step 3:

[0337] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[0338] Specific behavior:

[0339] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[0340] The device uses a microphone to analyze the tone and intensity of the viewer's voice.

[0341] The device sends the analysis results to the server.

[0342] Step 4:

[0343] The device also collects the viewer's behavioral history, preferences, and device information, and sends this data to the server, which then creates a viewer profile based on this information.

[0344] Specific behavior:

[0345] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[0346] Your device collects device information (e.g., the type of browser and device you are using).

[0347] The server generates a viewer profile based on the data received, which also includes emotional data.

[0348] Step 5:

[0349] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account the collected emotional data.

[0350] Specific behavior:

[0351] The server gives instructions to the generation AI, placing emphasis on scenes that elicit strong emotional responses of "surprise" and "joy."

[0352] Generative AI takes into account the viewer's emotions and generates new content that contains many elements that viewers will respond to positively.

[0353] Step 6:

[0354] The server delivers the generated customized content to the device, and the device collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends them to the server.

[0355] Specific behavior:

[0356] The server transmits the customized content to the terminal in streaming format.

[0357] The device continues to monitor the viewer's emotional response while the video is playing.

[0358] The device periodically transmits viewing data and emotion data to the server.

[0359] Step 7:

[0360] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[0361] Specific behavior:

[0362] The server selects relevant advertisements immediately following scenes that elicit positive emotions.

[0363] The server inserts the ads at the appropriate times within the video.

[0364] The server tracks ad click rates and completion rates, and uses that data to optimize the effectiveness of the ads.

[0365] Step 8:

[0366] The device collects feedback from viewers and sends it to the server, which uses this feedback and emotional data to update the viewer profile and reflect it in the next content generation.

[0367] Specific behavior:

[0368] Users provide feedback on their impressions of specific scenes and effects.

[0369] The device sends the collected feedback to the server.

[0370] The server updates the viewer profile based on the collected feedback and sentiment data and reflects it in the next content generation.

[0371] Example 2

[0372] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0373] Conventional content generation systems do not adequately personalize content based on viewer emotions and behavioral history, making it difficult to improve viewer satisfaction and deliver advertisements effectively. Furthermore, they lack a mechanism for effectively utilizing post-viewing feedback and reflecting it in the next content generation. A new system that can solve these issues and provide viewers with highly personalized content and an optimal advertising experience is needed.

[0374] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0375] In this invention, the server includes means for preprocessing materials uploaded by creators, means for collecting and analyzing viewer emotion data, means for generating new content based on generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and emotion data, means for generating customized content based on the viewer profile, and means for delivering the customized content to the viewer. This enables advanced personalization that utilizes the viewer's emotion and behavioral history, and optimizes the viewing experience and advertising experience.

[0376] "Creator" refers to a person who creates content and uploads it to the system.

[0377] "Materials" refers to the data that forms the basis of content, such as video, audio, and text data.

[0378] "Preprocessing" refers to the process of checking the format and quality of the uploaded material and performing encoding, trimming, etc.

[0379] "Emotional data" refers to data related to emotions analyzed from viewers' facial expressions and voices.

[0380] A "viewer profile" refers to information that reflects a viewer's preferences, etc., and is generated based on the viewer's behavioral history and emotional data.

[0381] "Generative AI" refers to AI technology that generates new content based on viewer profiles, emotional data, etc.

[0382] "Content" refers to information such as video, audio, and text that viewers can view.

[0383] "Customized Content" refers to content that is individually generated based on a viewer profile.

[0384] "Engagement data" refers to data about viewers' viewing behavior, such as viewing time, number of views, and reactions.

[0385] "Advertising" means any promotional message or advertising content displayed to a viewer.

[0386] "Ad performance" refers to data used to measure the effectiveness of an ad, such as click-through rate and completion rate.

[0387] "Feedback" refers to the impressions and opinions that viewers provide after watching a program.

[0388] The system of the present invention functions mainly through a server, a terminal, a creator, and a user. The roles of each are explained below.

[0389] Hardware and Software Configuration

[0390] 1. Server: This plays a central role in the system. It has high-performance processing capabilities and is ideally suited to using a cloud server for data storage, data analysis, and implementing generative AI models. Specifically, Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc. are suitable.

[0391] 2. Device: A device used by the user to view and provide feedback. This can be a smartphone, tablet, or PC. The device has a built-in camera and microphone, and an application to run the emotion engine is installed.

[0392] 3. Creator: An entity that creates content and uploads it to the system. They create content using image editing software (e.g., Adobe Premiere, Final Cut Pro) and audio editing software.

[0393] Content Creation and Customization Process

[0394] The system of the present invention operates in the following manner.

[0395] 1. Upload and pre-process materials:

[0396] Creators upload video footage, audio, and text data to the server using a web application interface.

[0397] The server checks the format and quality of the received material and performs pre-processing such as encoding and trimming. Specifically, it performs encoding to unify the resolution of the video files.

[0398] 2. Collecting Emotional Data:

[0399] The device uses a built-in camera and microphone to capture the viewer's facial expressions and voice while they are watching the video, and then analyzes them in real time using an emotion engine.

[0400] The device transmits the analysis result, emotional data, to the server. This data includes the viewer's emotional response (e.g., joy, surprise, sadness, etc.).

[0401] 3. Generate audience profiles:

[0402] The terminal records the viewer's behavioral history and device information and sends this to the server.

[0403] The server generates a viewer profile based on the received emotion data and behavioral history data, integrates this information into the database, and updates the profile information.

[0404] 4. Generate customized content:

[0405] The server issues instructions to the generation AI based on the viewer profile to generate new customized content.

[0406] An example of a specific prompt would be, "Generate a new video of a natural scene, focusing on a scene that the viewer found pleasurable."

[0407] The generative AI model generates new content based on this prompt.

[0408] 5. Content distribution and engagement tracking:

[0409] The server distributes the generated customized content to the terminal in streaming format.

[0410] The device collects engagement data such as viewer viewing time, number of plays, and reactions, and sends this back to the server.

[0411] 6. Ad Insertion and Optimization:

[0412] The server selects the most suitable advertisement based on the viewer profile and emotional data and inserts it into the video.

[0413] The server tracks and optimizes the performance of ads, specifically by monitoring click-through rates and completion rates.

[0414] 7. Collecting Feedback and Updating Your Profile:

[0415] The terminal collects feedback from the viewer and sends it to the server.

[0416] The server updates the viewer profile based on the feedback and emotional data and reflects it in the next content generation.

[0417] By implementing the above steps, the system of the present invention can achieve advanced personalization based on the viewer's emotions and behavioral history, optimizing the viewing experience and advertising experience. This system allows users to enjoy more attractive and personalized content, and maximizes the effectiveness of advertising.

[0418] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0419] Step 1:

[0420] Creators upload materials.

[0421] Input: Video material, audio, and text data created by creators.

[0422] Output: Material data uploaded and stored on the server.

[0423] How it works: When a creator accesses the upload page of the web application, selects a local file, and presses the "Upload" button, the material data is transferred to the server, which receives it and stores it in storage.

[0424] Step 2:

[0425] The server pre-processes the material.

[0426] Input: Video footage, audio, and text data uploaded by creators.

[0427] Output: Material data that has been encoded, trimmed, and converted into a unified format.

[0428] What it does: The server checks the format and quality of the received material, encodes the video files to a specific resolution, standardizes the format of the audio files, trims and removes unnecessary parts, and converts the material to a standardized format.

[0429] Step 3:

[0430] The device captures the viewer's emotional data.

[0431] Input: The viewer's facial expressions and voice, captured by the device's built-in camera and microphone.

[0432] Output: Viewer sentiment data analyzed by the sentiment engine.

[0433] How it works: While the viewer is watching the video, the device automatically activates the camera and microphone to capture the viewer's face and voice, and performs real-time facial analysis to generate emotional data such as joy or surprise.

[0434] Step 4:

[0435] The device transmits the emotion data to the server.

[0436] Input: Viewer sentiment data analyzed on the device.

[0437] Output: The emotion data is transferred to the server and stored in a database.

[0438] Specific operation: The device generates emotion data as data packets at regular intervals and sends them to the server via HTTP requests. The server then records the received data in a database.

[0439] Step 5:

[0440] The terminal transmits the viewing data to the server.

[0441] Input: Viewer behavior history, device information.

[0442] Output: Viewing data sent to the server and stored in a database.

[0443] Specific operation: The terminal records the viewer's behavior history (viewing start time, end time, scenes viewed) and device information, and sends this to the server in batch processing.

[0444] Step 6:

[0445] The server generates a viewer profile.

[0446] Input: Emotion data and viewing data sent from the device.

[0447] Output: The created audience profile.

[0448] Specific operation: Based on the received data, the server analyzes the viewer's preferences and behavioral patterns and stores the viewer profile in a database.

[0449] Step 7:

[0450] The server gives instructions to the generated AI.

[0451] Input: Audience profile.

[0452] Output: The prompt sent to the generation AI.

[0453] Specific operation: The server analyzes the viewer profile and generates a prompt to instruct the AI ​​to generate new content. An example of a prompt is "Generate a new nature video centered around scenes that the viewer found enjoyable."

[0454] Step 8:

[0455] Generative AI generates customized content.

[0456] Input: The prompt text received from the server.

[0457] Output: The generated customization content.

[0458] How it works: Based on the prompt, the generative AI model generates new content that matches the preferences of the viewer profile.

[0459] Step 9:

[0460] The server delivers the customized content.

[0461] Input: Generated customization content.

[0462] Output: Streamed content delivered to a device.

[0463] Specific operation: The server generates a content URL and sends it to the device to start viewing.

[0464] Step 10:

[0465] The device collects engagement data.

[0466] Input: Viewer viewing behavior data.

[0467] Output: Collected engagement data is sent to a server.

[0468] Specific operation: The device tracks viewing time, number of plays, reactions, etc. and sends the data to the server in batches.

[0469] Step 11:

[0470] The server selects and inserts the advertisements.

[0471] Input: Audience profile, sentiment data.

[0472] Output: Ads inserted into the generated content.

[0473] Specific operation: The server selects the most suitable advertisement based on the profile and emotional data, analyzes the appropriate insertion point in the video, and inserts the advertisement.

[0474] Step 12:

[0475] The server tracks and optimizes the performance of the ads.

[0476] Input: Ad click-through rate, view completion rate.

[0477] Output: Optimization of advertising effectiveness.

[0478] Specific operation: The server aggregates ad performance data and updates the ad selection logic using machine learning algorithms.

[0479] Step 13:

[0480] The device collects viewer feedback.

[0481] Input: Feedback provided by the audience.

[0482] Output: Feedback sent to server and stored in database.

[0483] Specific operation: The device collects feedback from the user through an in-app feedback form and sends it to the server.

[0484] Step 14:

[0485] The server updates the profile based on the feedback.

[0486] Input: Audience feedback, sentiment data.

[0487] Output: Updated audience profile.

[0488] What it does: The server analyzes the feedback and sentiment data, updates the viewer profile in a database, and uses it to generate the next piece of content.

[0489] (Application example 2)

[0490] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0491] Conventional content distribution services provide content customized based on a viewer's behavioral history and preferences, but do not take into account the viewer's real-time emotions. This makes it difficult to deliver content that optimally responds to the viewer's emotions, and the improvement of the viewing experience and optimization of advertising have not been fully achieved. Therefore, the objective of the present invention is to provide a system that recognizes viewer emotions in real time, generates customized content based on that data, and further optimizes the effectiveness of advertising.

[0492] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0493] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for collecting viewer emotion data using an emotion engine, means for generating customized content taking the emotion data into consideration, and means for delivering the customized content to the viewer. This makes it possible to provide highly personalized video content and optimize advertisements according to the viewer's emotions.

[0494] A "server" is a computer system that provides services to other computers (clients) on a network.

[0495] A "viewer profile" is personal information constructed based on a viewer's behavioral history, preferences, and emotional data.

[0496] "Generative AI" refers to AI that learns from large amounts of data and performs tasks such as natural language generation and image generation.

[0497] An "emotion engine" is software or hardware that analyzes emotions from viewers' facial expressions, voice, behavior, etc.

[0498] "Customized Content" means video and audio content that is generated and optimized for a viewer based on viewer profile and emotional data.

[0499] "Advertising optimization" refers to the process of delivering advertisements with optimal timing and content based on viewer profiles and emotional data, thereby maximizing their effectiveness.

[0500] "Feedback" means information such as preferences and opinions provided by viewers that is used to improve profiles and content offerings.

[0501] A specific embodiment of a system for carrying out the present invention will be described below. The system is composed of a server, terminals, and users, and is provided with various means for realizing the contents of the invention.

[0502] Hardware and software used

[0503] server:

[0504] Software: Generative AI (generative AI model), database, web server

[0505] Role: Pre-processing material, generating audience profiles, generating and delivering customized content, inserting and optimizing ads, collecting feedback and updating profiles

[0506] Device:

[0507] Hardware: Camera, microphone, display

[0508] Software: Emotion recognition engine (e.g., facial expression recognition model)

[0509] Role: Collect emotional data, watch customized content, send emotional data and feedback

[0510] User:

[0511] Interact: Watch content, provide feedback

[0512] System flow and processing

[0513] The server generates new content from the materials provided by the generative AI. Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the materials, and performs pre-processing such as encoding and trimming. At this stage, the content provided by creators might be, for example, footage of natural scenery and music files.

[0514] Next, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice while they are watching the content. This emotional data is analyzed using facial expression recognition technology to identify various emotions such as joy, surprise, and sadness. The collected emotional data is then sent to a server.

[0515] The server analyzes this emotional data along with the viewer's behavioral history, preferences, and device information to generate a viewer profile. For example, it can identify scenes from past viewing data that evoked positive emotional responses and add them to the viewer's profile as "preferences."

[0516] Based on the viewer profile, the server issues instructions to the AI ​​to generate new customized content, taking into account emotional data. For example, the server can instruct the AI ​​to prioritize scenes that make viewers feel happy, generating new content that includes many scenes that evoke positive emotions.

[0517] The generated customized content is delivered from the server to the device, where the viewer watches it. The device collects the viewer's engagement data (viewing time, number of plays, reactions) and emotional data and sends it back to the server.

[0518] The server selects the most suitable advertisements based on viewer profiles and emotional data and inserts them into the video. It also tracks and optimizes the advertisement performance (click-through rate, viewing completion rate). For example, it can insert a relevant advertisement immediately after a scene that evokes positive emotions, increasing the effectiveness of the advertisement.

[0519] It collects feedback from viewers and updates their profiles, including their preferences and opinions, which the server then incorporates into the next content generation.

[0520] Specific examples

[0521] This is an example of a generative AI model generating new content based on a provided prompt.

[0522] Example prompt sentence:

[0523] "The input video contains multiple scenes, each with elements that elicit different emotional responses. Please generate a new video by focusing on scenes in which viewers express the 'Happy' emotion. Also, please add upbeat music and fun characters as additional elements to elicit the 'Happy' emotion."

[0524] In this way, a highly personalized viewing experience based on the viewer's emotions can be provided.

[0525] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0526] Step 1:

[0527] The server generates new content based on the materials provided by artificial intelligence. Specifically, creators upload video material, audio, and text data to the server, which then receives it and performs pre-processing (format and quality checks, encoding, trimming).

[0528] Input: Video material, audio, text data

[0529] Output: Encoded and trimmed material

[0530] Step 2:

[0531] While the viewer is watching the content, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice. For example, the device's camera captures the viewer's face and uses facial recognition technology to analyze their emotions (happiness, surprise, sadness, etc.). The analysis results are sent to the server.

[0532] Input: Viewer's facial expressions and voice

[0533] Output: Parsed emotion data

[0534] Step 3:

[0535] The server analyzes the viewer's behavioral history, preferences, device information, and emotional data to generate a viewer profile. Based on the viewing and emotional data, scenes that evoked a positive emotional response are identified and added to the profile as the viewer's "preferences."

[0536] Input: Viewer behavior history, preferences, device information, emotional data

[0537] Output: Audience profile

[0538] Step 4:

[0539] The server issues prompts to the AI ​​based on the viewer profile to generate new customized content. It also takes into account emotional data, and instructs the AI ​​to prioritize scenes that elicit positive emotions, for example, generating content that includes many scenes that evoke positive emotions.

[0540] Input: Viewer profile, prompt

[0541] Output: New, customized content

[0542] Step 5:

[0543] The server delivers the generated customized content to the device, which then provides the content to the viewer, collecting viewer engagement data (watching time, number of plays, reactions) and emotional data and sending it back to the server.

[0544] Input: Customized Content

[0545] Output: Audience engagement data, sentiment data

[0546] Step 6:

[0547] The server selects the most suitable advertisement based on the viewer profile and emotional data, and inserts it into the middle of the video. It also tracks the performance of the advertisement (click-through rate, completion rate), and optimizes the effectiveness of the advertisement based on the results. For example, it inserts a relevant advertisement immediately after a scene that evokes positive emotions.

[0548] Input: Viewer profile, emotional data

[0549] Output: Optimized Ad

[0550] Step 7:

[0551] The server collects feedback from viewers and updates viewer profiles, including information such as viewer preferences and opinions, to reflect in the next content generation.

[0552] Input: Viewer feedback, sentiment data

[0553] Output: Updated viewer profile

[0554] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0555] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0556] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0557] [Second embodiment]

[0558] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0559] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0560] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0561] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0562] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0563] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0564] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0565] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0566] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0567] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0568] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0569] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0570] Specific embodiments of the system related to the present invention will be described below.

[0571] System Overview

[0572] The system consists of three main components: a server, a terminal, and a user. It generates new content based on the provided material using generative AI, delivers customized content based on the viewer profile, optimizes advertising, and collects and updates feedback.

[0573] Program processing flow

[0574] Uploading and Preprocessing Materials

[0575] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality, and performs pre-processing such as encoding and trimming.

[0576] Examples:

[0577] Creators upload footage of natural scenery they have taken and the corresponding music files.

[0578] The server checks the video resolution and converts it into a unified format.

[0579] Generate audience profiles

[0580] The device collects the viewer's past viewing history, preferences, device information, etc. This information is sent to a server to create a viewer profile.

[0581] Examples:

[0582] The genre that users often watch is cooking-related videos.

[0583] The device records viewing history, sends it to the server, and creates a profile called "I love cooking."

[0584] Customized Content Generation

[0585] The server instructs the generative AI based on the viewer profile to generate new customized content.

[0586] Examples:

[0587] The server instructs the generation AI to add cooking scenes to travel footage for users who love cooking.

[0588] The generative AI generates scenes introducing famous local dishes to accompany travel footage.

[0589] Content distribution and engagement tracking

[0590] The server delivers the generated customized content to the device, and collects viewer engagement data (watch time, number of plays, reactions, etc.) to use for the next profile update.

[0591] Examples:

[0592] A server distributes travel videos including cooking scenes to users who love cooking.

[0593] The terminal collects the user's viewing data and sends it to the server.

[0594] Ad insertion and optimization

[0595] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the ad performance.

[0596] Examples:

[0597] The server selects organic food advertisements and inserts them into videos targeted at health-conscious viewers.

[0598] The server monitors the click-through rate and completion rate of the ads and makes appropriate adjustments.

[0599] Collecting feedback and updating your profile

[0600] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next video generation.

[0601] Examples:

[0602] The user provides feedback regarding preferences for particular effects and scenes.

[0603] The server adds this information to a profile and uses it the next time a video is generated.

[0604] conclusion

[0605] The system generates and delivers high-quality video content customized based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[0606] The processing flow will be explained below.

[0607] Step 1:

[0608] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[0609] Specific behavior:

[0610] The server checks the format of the uploaded file (JPEG, MP4, etc.).

[0611] The server verifies the video resolution and audio bit rate.

[0612] Step 2:

[0613] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[0614] Specific behavior:

[0615] The server converts all video to a uniform resolution.

[0616] The server processes the audio track for noise reduction.

[0617] Step 3:

[0618] The device collects the viewer's behavioral history, preferences, device information, etc. and sends it to the server, which then creates a viewer profile based on this information.

[0619] Specific behavior:

[0620] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[0621] The terminal transmits the collected data to the server.

[0622] The server generates a viewer profile based on the received data.

[0623] Step 4:

[0624] The server instructs the generative AI based on the viewer profile to generate new customized content.

[0625] Specific behavior:

[0626] The server uses the viewer profile to pass appropriate content generation parameters to the generation AI.

[0627] Generative AI generates new content by adding elements that viewers like to specific scenes in a video.

[0628] Step 5:

[0629] The server delivers the generated customized content to the device, which collects viewer engagement data (watch time, number of plays, reactions, etc.) and sends it back to the server.

[0630] Specific behavior:

[0631] The server transmits the customized content to the terminal in a streaming format.

[0632] The device collects viewer engagement data in real time.

[0633] The engagement data collected by the device is periodically sent to the server.

[0634] Step 6:

[0635] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the performance of the ads.

[0636] Specific behavior:

[0637] The server analyzes the viewer profile and selects appropriate advertisements.

[0638] The server decides when to insert the advertisement during video playback.

[0639] The server tracks ad click rates and completion rates and uses that data to optimize ads.

[0640] Step 7:

[0641] The device collects feedback from viewers and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next content generation.

[0642] Specific behavior:

[0643] The device collects feedback provided by the viewer (e.g., "I like this scene").

[0644] The device sends the collected feedback to the server.

[0645] The server updates the viewer profile based on the feedback and reflects it in the parameters of the next generation AI.

[0646] Example 1

[0647] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0648] Video viewers often demand content that is individually customized based on their diverse preferences and viewing history. Streaming platforms are also required to insert appropriate advertisements and optimize their performance. However, conventional systems face challenges in automating these requirements, making it difficult and time-consuming.

[0649] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0650] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a user profile based on the user's behavioral history and preferences, means for generating customized content based on the user profile, means for checking the format and quality of the material and encoding and trimming it, means for collecting user engagement data and updating the user profile, and means for delivering customized content to users. This enables the automatic generation and delivery of high-quality content customized based on user preferences.

[0651] (The system of claim 2)

[0652] The system further includes a means for inserting advertisements into the generated customized content and a means for tracking and optimizing the performance of the advertisements, thereby providing the most suitable advertisements to the viewer and maximizing their effectiveness.

[0653] "Generative AI" refers to AI that has the ability to generate new information and content based on diverse data.

[0654] A "user profile" is a database or collection of information constructed based on a user's behavioral history and preferences.

[0655] "Customized Content" means content that has been adapted or edited based on a specific user profile.

[0656] "Encoding" is the process of converting data into a particular format.

[0657] "Trimming" is the process of cutting out unnecessary parts of video or audio data.

[0658] "Engagement data" refers to behavioral data such as viewer viewing time, number of views, and reactions.

[0659] "Feedback" refers to opinions and ratings provided by users.

[0660] "Ad insertion" refers to the process of placing advertisements within video or audio content.

[0661] "Performance tracking" refers to the collection and analysis of data to measure and analyze the effectiveness of advertising and other activities.

[0662] "Optimization" is the process of adjusting or improving a system or process to maximize its efficiency or effectiveness.

[0663] MODE FOR CARRYING OUT THE INVENTION

[0664] The system related to this invention mainly consists of three elements: a server, a terminal, and a user. The system generates new content based on provided materials using a generative AI model, delivers customized content based on viewer profiles, and optimizes advertisements and collects and updates feedback.

[0665] server

[0666] The server manages everything from uploading materials to generating and distributing customized content. Specific processes include the following:

[0667] 1. Upload and pre-process materials

[0668] The server receives video, audio, and text data from creators. It checks the file format and quality, and encodes and trims it. For example, it uses the FFmpeg tool to convert and unify the format and quality of the material.

[0669] 2. Generate audience profiles

[0670] The server receives data on the viewer's behavior and preferences sent from the device, creates a viewer profile, and stores this data in a database (e.g., MySQL).

[0671] 3. Creating customized content

[0672] Based on the viewer profile, the server sends prompts to the AI ​​to generate customized content, such as "add cooking scenes to travel videos."

[0673] 4. Content distribution and engagement tracking

[0674] The server delivers the generated customized content to the device and collects viewer engagement data using data streaming technologies (e.g., RTMP, HLS).

[0675] 5. Ad insertion and optimization

[0676] The server selects and inserts appropriate ads based on viewer profile and video context, and also tracks and optimizes ad performance.

[0677] 6. Collect feedback and update your profile

[0678] The server collects feedback sent by the terminals and updates the viewer profile.

[0679] Terminal

[0680] The device collects the user's behavior history and sends it to the server. It also displays customized content and advertisements received from the server, collects viewing data and feedback, and sends them to the server.

[0681] User

[0682] Users can view content via their terminals and provide feedback, for example by entering their preferences and opinions about a particular scene in a feedback form.

[0683] Specific examples

[0684] For example, if a user frequently watches cooking-related videos, the device will collect that viewing history and send it to the server, which will then use that data to create a "cooking lover" profile and instruct the AI ​​to add cooking scenes to travel videos in the next customized content.

[0685] Prompt Sentence Examples

[0686] "For food-loving viewers, add a segment to your travel footage showcasing famous local dishes."

[0687] This system generates and distributes high-quality video content based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[0688] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0689] Program processing flow

[0690] Step 1. Upload and preprocess your materials

[0691] 1.1 Uploading Materials

[0692] Creators upload video footage, audio, and text data to the server, which triggers the action of selecting files through a web interface and pressing the upload button.

[0693] Input: Video material files, audio files, text files

[0694] Output: Original material data stored on the server

[0695] 1.2 File format and quality check

[0696] The server checks the format and quality of the uploaded file by using tools like FFmpeg to extract file information and verify things like format, resolution, bitrate, etc.

[0697] Input: Original material data stored on the server

[0698] Output: Format and quality verified material data

[0699] 1.3 Encoding and Trimming

[0700] The server encodes and trims the file as needed, for example converting the video file to the specified resolution and trimming unwanted parts.

[0701] Input: Format and quality verified material data

[0702] Output: Encoded and trimmed material data

[0703] Step 2. Generate your audience profile

[0704] 2.1 Collecting viewing history and preferences

[0705] The device collects information about the viewer's viewing history, preferred genres, and playback devices, including using browser and app cookies to record viewing data.

[0706] Input: Viewer viewing activity

[0707] Output: Collected viewing history data

[0708] 2.2 Sending profile data

[0709] The device sends the collected data to the server. The data is packetized in JSON format and sent via an HTTP POST request.

[0710] Input: Collected viewing history data

[0711] Output: Profile data sent to the server

[0712] 2.3 Creating an audience profile

[0713] The server creates a viewer profile based on the data it receives, and records the viewer's preferences and history in a database.

[0714] Input: Profile data sent to the server

[0715] Output: Viewer profile stored in a database

[0716] Step 3. Generate customized content

[0717] 3.1 Creating prompts for generative AI

[0718] The server creates a prompt to send to the AI ​​based on the viewer profile, such as "Add cooking scenes to travel videos for viewers who love cooking."

[0719] Input: Viewer profile stored in database

[0720] Output: Generated prompt statement

[0721] 3.2 Content Generation Instructions

[0722] The server sends a prompt to the generative AI model to instruct it to generate content. The prompt is sent to the generative AI using an API request.

[0723] Input: Generated prompt statement

[0724] Output: The prompt sent to the generative AI model

[0725] 3.3 Creating customized content

[0726] The generative AI generates new content based on the prompt and sends it back to the server, for example adding a cooking scene to a video and generating a text description.

[0727] Input: The prompt sent to the generative AI model

[0728] Output: Generated customization content

[0729] Step 4. Content distribution and engagement tracking

[0730] 4.1 Delivery of customized content

[0731] The server delivers the generated customized content to the terminal. The content is delivered using data streaming technology.

[0732] Input: Generated customization content

[0733] Output: Content delivered to the device

[0734] 4.2 Collection of viewing data

[0735] The device collects viewer engagement data (watch time, plays, reactions, etc.) and records this data using JavaScript code.

[0736] Input: Content delivered to the device

[0737] Output: Collected engagement data

[0738] 4.3 Transmission of Collected Data

[0739] The device sends engagement data to the server via an HTTP request for analysis.

[0740] Input: Collected engagement data

[0741] Output: Engagement data sent to the server

[0742] Step 5. Ad insertion and optimization

[0743] 5.1 Ad Selection

[0744] The server chooses the best ad based on the viewer profile and video context, selecting the appropriate ad from an ad database.

[0745] Input: Viewer profile stored in database

[0746] Output: Selected ads

[0747] 5.2 Ad Insertion

[0748] The server inserts the selected advertisement into the video at the specified position. The advertisement is then integrated into the video using a video editing tool.

[0749] Input: Selected ads, customized content

[0750] Output: Customized content with ad insertion

[0751] 5.3 Tracking Ad Performance

[0752] The server monitors the click-through rate and completion rate of ads and automatically adjusts the optimal ads. Performance data is collected using analytical tools and optimized by algorithms.

[0753] Input: Ad-inserted personalized content, viewing data

[0754] Output: Optimized ad placement

[0755] Step 6. Collect feedback and update your profile

[0756] 6.1 Feedback Collection

[0757] The device collects feedback from viewers, which involves viewers entering their opinions and thoughts through a form in their browser or app.

[0758] Input: Viewer feedback

[0759] Output: Collected feedback data

[0760] 6.2 Sending Feedback

[0761] The device sends the collected feedback to the server via an HTTP POST request.

[0762] Input: Collected feedback data

[0763] Output: Feedback data sent to the server

[0764] 6.3 Updating your profile

[0765] The server updates the viewer profile based on the feedback, updating the profile information in the database and reflecting it in the next content generation.

[0766] Input: Feedback data sent to the server

[0767] Output: Updated viewer profile

[0768] (Application example 1)

[0769] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0770] Conventional content distribution systems lack sufficient customization based on individual viewer preferences and behavioral history, and lack complete feedback and engagement tracking to improve the quality of the viewing experience. Furthermore, they are unable to select optimal ads based on viewer profiles and track their effectiveness, making it difficult to maximize advertising effectiveness. This results in insufficient improvement in viewer satisfaction and the value of the platform.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0772] In this invention, the server includes means for generating new content based on materials provided by the generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for delivering the customized content to the viewer, means for collecting viewer engagement data and using it for the next profile update, means for inserting advertisements into the generated customized content, means for tracking the performance of the advertisements, means for selecting optimal advertisements based on the viewer profile and video context, means for collecting feedback from the viewer and updating the viewer profile, and means for reflecting the collected feedback in the profile and using it for generating the next video. This enables the generation and delivery of high-quality customized content based on the viewer's preferences and behavioral history, maximizing the effectiveness of advertisements and improving viewer satisfaction and the value of the platform.

[0773] "Generative AI" is an AI technology that generates new information and content based on input data.

[0774] "Materials" refers to the basic data for generating content, such as video, audio, and text data provided by creators.

[0775] "Customized content" is individual content that is generated based on a viewer profile and tailored to the viewer's preferences and behavioral history.

[0776] A "viewer profile" refers to a data structure that indicates the characteristics of a viewer, constructed by analyzing data related to the viewer's behavioral history and preferences.

[0777] "Viewer engagement data" refers to behavioral data of viewers when they watch content, including viewing time, number of plays, reactions, etc.

[0778] "Feedback" refers to information such as reactions, opinions, and ratings from viewers, which is used to generate next content and update profiles.

[0779] "Ad performance" refers to metrics that evaluate how effectively an ad works for viewers, including click-through rate and completion rate.

[0780] "Video context" refers to the background information surrounding the video, such as the content, theme, and scene situation of the video being generated.

[0781] As an embodiment of the present invention, a specific system configuration and its operation will be described below.

[0782] System configuration

[0783] Server: A central computer system that processes generative AI models, creates and updates audience profiles, collects and analyzes engagement data, and optimizes and tracks advertising effectiveness.

[0784] Terminal: A device used by viewers, such as a smartphone, tablet, or smart glasses, that collects viewers' behavioral history and transmits it to a server.

[0785] Users: Viewers and creators who use the system. Creators upload materials such as videos and audio, and viewers watch customized content.

[0786] Program processing

[0787] Uploading and Preprocessing Materials

[0788] Creators upload video footage, audio, text data, etc. to the server, which then uses OpenCV to divide the footage into frames, check the quality, encode, and trim as needed.

[0789] Generate audience profiles

[0790] The device collects information such as the viewer's past viewing history, preferences, and device information, and sends it to the server, which then uses Scikit-learn's KMeans clustering to generate a viewer profile.

[0791] Customized Content Generation

[0792] Based on the generated viewer profile, the server invokes a generative AI model using TensorFlow to generate customized content, adding elements that match the profile to the content in the process.

[0793] Content distribution and engagement tracking

[0794] The server delivers the generated customized content to the device, and the device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server.

[0795] Ad insertion and optimization

[0796] The server selects the best ads based on viewer profile and video context, inserts them within the customized content, and tracks and optimizes ad performance (e.g., click-through rate and completion rate).

[0797] Collecting feedback and updating your profile

[0798] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and generate customized content for the next viewing.

[0799] Specific examples

[0800] As a concrete example, consider the case where a user who loves cooking uses this system on a smartphone. The user uses the app just like a regular video streaming app. Based on the user's past viewing history, the server generates a profile for the user called "cooking lover" and provides new cooking-related content. For example, based on the user's profile, the server can generate and distribute new travel videos including Chinese food recipe videos.

[0801] Prompt Sentence Examples

[0802] "User profile: Loves cooking. Viewing genre: Chinese food. Content to generate: Generate travel footage that includes Chinese food recipes."

[0803] In this way, the system can efficiently generate and deliver high-quality customized content tailored to viewer preferences and behavioral history, while optimizing advertising effectiveness, thereby improving viewer satisfaction and enhancing the value of the platform.

[0804] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0805] Step 1:

[0806] Creators upload video footage, audio, text data, etc. to the server. The input is the content material created by the creator, and the output is when these materials are saved on the server and prepared for further processing. The server uses OpenCV to divide the video footage into frames, perform quality checks, encode, and trim as necessary.

[0807] Step 2:

[0808] The device collects the viewer's past viewing history, preferences, and device information, and sends it to the server. The input is the viewer's behavioral history and device information, and the output is this information sent to the server. The server generates a viewer profile using Scikit-learn's KMeans clustering. At this stage, the viewer's preferences and behavioral patterns are analyzed and a profile is constructed.

[0809] Step 3:

[0810] The server uses TensorFlow to call a generative AI model based on the generated viewer profile to generate customized content. The input is the viewer profile and uploaded materials, and the output is the customized content. The generative AI model adds elements that match the profile to the content.

[0811] Step 4:

[0812] The server delivers the generated customized content to the device. The device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server. The input is the customized content and the viewer's reactions, and the output is viewing data and engagement data. The collected engagement data is reflected in the next content generation.

[0813] Step 5:

[0814] The server selects the optimal ad based on the viewer profile and video context and inserts the ad into the customized content. The input is the viewer profile, content context, and candidate ads, and the output is the customized content with the ad inserted. In addition, the server tracks the ad performance (click-through rate, completion rate, etc.) and performs optimization to maximize the advertising effect.

[0815] Step 6:

[0816] The device collects feedback from viewers and sends it to the server. The server updates the viewer profile based on this feedback and uses it to generate the next customized content. The input is viewer feedback, and the output is an updated viewer profile. Based on the information gained from the feedback, the quality of the next content provided can be further improved.

[0817] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0818] Specific embodiments of the system related to the present invention will be described below.

[0819] System Overview

[0820] This system combines three main elements - the server, the device, and the user - with an emotion engine that recognizes the user's emotions to provide a more personalized viewing experience. It generates new content based on the materials provided by the generative AI, delivers customized content based on the viewer profile, and optimizes advertising using emotional data.

[0821] Program processing flow

[0822] Uploading and Preprocessing Materials

[0823] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the material, and performs pre-processing such as encoding and trimming.

[0824] Examples:

[0825] Creators upload footage of natural scenery they have taken and the corresponding music files.

[0826] The server checks the video resolution and converts it into a unified format.

[0827] Collecting Emotional Data

[0828] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[0829] Examples:

[0830] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[0831] The device sends the analysis results to the server.

[0832] Generate audience profiles

[0833] The device collects the viewer's behavioral history, preferences, device information, as well as emotional data, and sends it to the server, which then creates a viewer profile based on this information.

[0834] Examples:

[0835] The device collects the viewer's past viewing data and emotional data and sends it to the server.

[0836] The server analyzes scenes that elicit a positive emotional response and adds the viewer's "preferences" to their profile.

[0837] Customized Content Generation

[0838] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account emotional data as well.

[0839] Examples:

[0840] The server gives instructions to the generation AI, focusing on scenes that make the server feel happy.

[0841] Generative AI generates new content that includes many scenes that evoke positive emotions.

[0842] Content distribution and engagement tracking

[0843] The server delivers the generated customized content to the device, which collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends it back to the server.

[0844] Examples:

[0845] The server transmits the customized content to the terminal in streaming format.

[0846] The device collects viewer engagement data and emotional reactions and sends them to a server.

[0847] Ad insertion and optimization

[0848] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[0849] Examples:

[0850] The server inserts relevant advertisements immediately after scenes that elicit positive emotions.

[0851] The server monitors the click-through rate and completion rate of the advertisements to optimize the effectiveness of the advertisements.

[0852] Collecting feedback and updating your profile

[0853] The device collects feedback from viewers and sends it to the server, which updates the viewer profile based on this feedback and emotional data and reflects it in the next content generation.

[0854] Examples:

[0855] Users provide feedback on their favorite scenes and effects.

[0856] The server uses the feedback and emotional data to update the viewer profile and use it to generate the next piece of content.

[0857] This enables the system to provide highly personalized video content and optimize advertising in response to viewer emotions, improving the viewing experience and increasing the value of the platform.

[0858] The processing flow will be explained below.

[0859] Step 1:

[0860] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[0861] Specific behavior:

[0862] Creators upload footage of natural scenery they have taken and the corresponding music files to a server.

[0863] The server checks the format of the uploaded file (JPEG, MP4, etc.) and verifies the video resolution and audio bitrate.

[0864] Step 2:

[0865] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[0866] Specific behavior:

[0867] The server converts all video files to a uniform resolution.

[0868] The server processes the audio track for noise reduction.

[0869] Step 3:

[0870] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[0871] Specific behavior:

[0872] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[0873] The device uses a microphone to analyze the tone and intensity of the viewer's voice.

[0874] The device sends the analysis results to the server.

[0875] Step 4:

[0876] The device also collects the viewer's behavioral history, preferences, and device information, and sends this data to the server, which then creates a viewer profile based on this information.

[0877] Specific behavior:

[0878] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[0879] Your device collects device information (e.g., the type of browser and device you are using).

[0880] The server generates a viewer profile based on the data received, which also includes emotional data.

[0881] Step 5:

[0882] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account the collected emotional data.

[0883] Specific behavior:

[0884] The server gives instructions to the generation AI, placing emphasis on scenes that elicit strong emotional responses of "surprise" and "joy."

[0885] Generative AI takes into account the viewer's emotions and generates new content that contains many elements that viewers will respond to positively.

[0886] Step 6:

[0887] The server delivers the generated customized content to the device, and the device collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends them to the server.

[0888] Specific behavior:

[0889] The server transmits the customized content to the terminal in streaming format.

[0890] The device continues to monitor the viewer's emotional response while the video is playing.

[0891] The device periodically transmits viewing data and emotion data to the server.

[0892] Step 7:

[0893] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[0894] Specific behavior:

[0895] The server selects relevant advertisements immediately following scenes that elicit positive emotions.

[0896] The server inserts the ads at the appropriate times within the video.

[0897] The server tracks ad click rates and completion rates, and uses that data to optimize the effectiveness of the ads.

[0898] Step 8:

[0899] The device collects feedback from viewers and sends it to the server, which uses this feedback and emotional data to update the viewer profile and reflect it in the next content generation.

[0900] Specific behavior:

[0901] Users provide feedback on their impressions of specific scenes and effects.

[0902] The device sends the collected feedback to the server.

[0903] The server updates the viewer profile based on the collected feedback and sentiment data and reflects it in the next content generation.

[0904] Example 2

[0905] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0906] Conventional content generation systems do not adequately personalize content based on viewer emotions and behavioral history, making it difficult to improve viewer satisfaction and deliver advertisements effectively. Furthermore, they lack a mechanism for effectively utilizing post-viewing feedback and reflecting it in the next content generation. A new system that can solve these issues and provide viewers with highly personalized content and an optimal advertising experience is needed.

[0907] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0908] In this invention, the server includes means for preprocessing materials uploaded by creators, means for collecting and analyzing viewer emotion data, means for generating new content based on generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and emotion data, means for generating customized content based on the viewer profile, and means for delivering the customized content to the viewer. This enables advanced personalization that utilizes the viewer's emotion and behavioral history, and optimizes the viewing experience and advertising experience.

[0909] "Creator" refers to a person who creates content and uploads it to the system.

[0910] "Materials" refers to the data that forms the basis of content, such as video, audio, and text data.

[0911] "Preprocessing" refers to the process of checking the format and quality of the uploaded material and performing encoding, trimming, etc.

[0912] "Emotional data" refers to data related to emotions analyzed from viewers' facial expressions and voices.

[0913] A "viewer profile" refers to information that reflects a viewer's preferences, etc., and is generated based on the viewer's behavioral history and emotional data.

[0914] "Generative AI" refers to AI technology that generates new content based on viewer profiles, emotional data, etc.

[0915] "Content" refers to information such as video, audio, and text that viewers can view.

[0916] "Customized Content" refers to content that is individually generated based on a viewer profile.

[0917] "Engagement data" refers to data about viewers' viewing behavior, such as viewing time, number of views, and reactions.

[0918] "Advertising" means any promotional message or advertising content displayed to a viewer.

[0919] "Ad performance" refers to data used to measure the effectiveness of an ad, such as click-through rate and completion rate.

[0920] "Feedback" refers to the impressions and opinions that viewers provide after watching a program.

[0921] The system of the present invention functions mainly through a server, a terminal, a creator, and a user. The roles of each are explained below.

[0922] Hardware and Software Configuration

[0923] 1. Server: This plays a central role in the system. It has high-performance processing capabilities and is ideally suited to using a cloud server for data storage, data analysis, and implementing generative AI models. Specifically, Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc. are suitable.

[0924] 2. Device: A device used by the user to view and provide feedback. This can be a smartphone, tablet, or PC. The device has a built-in camera and microphone, and an application to run the emotion engine is installed.

[0925] 3. Creator: An entity that creates content and uploads it to the system. They create content using image editing software (e.g., Adobe Premiere, Final Cut Pro) and audio editing software.

[0926] Content Creation and Customization Process

[0927] The system of the present invention operates in the following manner.

[0928] 1. Upload and pre-process materials:

[0929] Creators upload video footage, audio, and text data to the server using a web application interface.

[0930] The server checks the format and quality of the received material and performs pre-processing such as encoding and trimming. Specifically, it performs encoding to unify the resolution of the video files.

[0931] 2. Collecting Emotional Data:

[0932] The device uses a built-in camera and microphone to capture the viewer's facial expressions and voice while they are watching the video, and then analyzes them in real time using an emotion engine.

[0933] The device transmits the analysis result, emotional data, to the server. This data includes the viewer's emotional response (e.g., joy, surprise, sadness, etc.).

[0934] 3. Generate audience profiles:

[0935] The terminal records the viewer's behavioral history and device information and sends this to the server.

[0936] The server generates a viewer profile based on the received emotion data and behavioral history data, integrates this information into the database, and updates the profile information.

[0937] 4. Generate customized content:

[0938] The server issues instructions to the generation AI based on the viewer profile to generate new customized content.

[0939] An example of a specific prompt would be, "Generate a new video of a natural scene, focusing on a scene that the viewer found pleasurable."

[0940] The generative AI model generates new content based on this prompt.

[0941] 5. Content distribution and engagement tracking:

[0942] The server distributes the generated customized content to the terminal in streaming format.

[0943] The device collects engagement data such as viewer viewing time, number of plays, and reactions, and sends this back to the server.

[0944] 6. Ad Insertion and Optimization:

[0945] The server selects the most suitable advertisement based on the viewer profile and emotional data and inserts it into the video.

[0946] The server tracks and optimizes the performance of ads, specifically by monitoring click-through rates and completion rates.

[0947] 7. Collecting Feedback and Updating Your Profile:

[0948] The terminal collects feedback from the viewer and sends it to the server.

[0949] The server updates the viewer profile based on the feedback and emotional data and reflects it in the next content generation.

[0950] By implementing the above steps, the system of the present invention can achieve advanced personalization based on the viewer's emotions and behavioral history, optimizing the viewing experience and advertising experience. This system allows users to enjoy more attractive and personalized content, and maximizes the effectiveness of advertising.

[0951] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0952] Step 1:

[0953] Creators upload materials.

[0954] Input: Video material, audio, and text data created by creators.

[0955] Output: Material data uploaded and stored on the server.

[0956] How it works: When a creator accesses the upload page of the web application, selects a local file, and presses the "Upload" button, the material data is transferred to the server, which receives it and stores it in storage.

[0957] Step 2:

[0958] The server pre-processes the material.

[0959] Input: Video footage, audio, and text data uploaded by creators.

[0960] Output: Material data that has been encoded, trimmed, and converted into a unified format.

[0961] What it does: The server checks the format and quality of the received material, encodes the video files to a specific resolution, standardizes the format of the audio files, trims and removes unnecessary parts, and converts the material to a standardized format.

[0962] Step 3:

[0963] The device captures the viewer's emotional data.

[0964] Input: The viewer's facial expressions and voice, captured by the device's built-in camera and microphone.

[0965] Output: Viewer sentiment data analyzed by the sentiment engine.

[0966] How it works: While the viewer is watching the video, the device automatically activates the camera and microphone to capture the viewer's face and voice, and performs real-time facial analysis to generate emotional data such as joy or surprise.

[0967] Step 4:

[0968] The device transmits the emotion data to the server.

[0969] Input: Viewer sentiment data analyzed on the device.

[0970] Output: The emotion data is transferred to the server and stored in a database.

[0971] Specific operation: The device generates emotion data as data packets at regular intervals and sends them to the server via HTTP requests. The server then records the received data in a database.

[0972] Step 5:

[0973] The terminal transmits the viewing data to the server.

[0974] Input: Viewer behavior history, device information.

[0975] Output: Viewing data sent to the server and stored in a database.

[0976] Specific operation: The terminal records the viewer's behavior history (viewing start time, end time, scenes viewed) and device information, and sends this to the server in batch processing.

[0977] Step 6:

[0978] The server generates a viewer profile.

[0979] Input: Emotion data and viewing data sent from the device.

[0980] Output: The created audience profile.

[0981] Specific operation: Based on the received data, the server analyzes the viewer's preferences and behavioral patterns and stores the viewer profile in a database.

[0982] Step 7:

[0983] The server gives instructions to the generated AI.

[0984] Input: Audience profile.

[0985] Output: The prompt sent to the generation AI.

[0986] Specific operation: The server analyzes the viewer profile and generates a prompt to instruct the AI ​​to generate new content. An example of a prompt is "Generate a new nature video centered around scenes that the viewer found enjoyable."

[0987] Step 8:

[0988] Generative AI generates customized content.

[0989] Input: The prompt text received from the server.

[0990] Output: The generated customization content.

[0991] How it works: Based on the prompt, the generative AI model generates new content that matches the preferences of the viewer profile.

[0992] Step 9:

[0993] The server delivers the customized content.

[0994] Input: Generated customization content.

[0995] Output: Streamed content delivered to a device.

[0996] Specific operation: The server generates a content URL and sends it to the device to start viewing.

[0997] Step 10:

[0998] The device collects engagement data.

[0999] Input: Viewer viewing behavior data.

[1000] Output: Collected engagement data is sent to a server.

[1001] Specific operation: The device tracks viewing time, number of plays, reactions, etc. and sends the data to the server in batches.

[1002] Step 11:

[1003] The server selects and inserts the advertisements.

[1004] Input: Audience profile, sentiment data.

[1005] Output: Ads inserted into the generated content.

[1006] Specific operation: The server selects the most suitable advertisement based on the profile and emotional data, analyzes the appropriate insertion point in the video, and inserts the advertisement.

[1007] Step 12:

[1008] The server tracks and optimizes the performance of the ads.

[1009] Input: Ad click-through rate, view completion rate.

[1010] Output: Optimization of advertising effectiveness.

[1011] Specific operation: The server aggregates ad performance data and updates the ad selection logic using machine learning algorithms.

[1012] Step 13:

[1013] The device collects viewer feedback.

[1014] Input: Feedback provided by the audience.

[1015] Output: Feedback sent to server and stored in database.

[1016] Specific operation: The device collects feedback from the user through an in-app feedback form and sends it to the server.

[1017] Step 14:

[1018] The server updates the profile based on the feedback.

[1019] Input: Audience feedback, sentiment data.

[1020] Output: Updated audience profile.

[1021] What it does: The server analyzes the feedback and sentiment data, updates the viewer profile in a database, and uses it to generate the next piece of content.

[1022] (Application example 2)

[1023] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1024] Conventional content distribution services provide content customized based on a viewer's behavioral history and preferences, but do not take into account the viewer's real-time emotions. This makes it difficult to deliver content that optimally responds to the viewer's emotions, and the improvement of the viewing experience and optimization of advertising have not been fully achieved. Therefore, the objective of the present invention is to provide a system that recognizes viewer emotions in real time, generates customized content based on that data, and further optimizes the effectiveness of advertising.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1026] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for collecting viewer emotion data using an emotion engine, means for generating customized content taking the emotion data into consideration, and means for delivering the customized content to the viewer. This makes it possible to provide highly personalized video content and optimize advertisements according to the viewer's emotions.

[1027] A "server" is a computer system that provides services to other computers (clients) on a network.

[1028] A "viewer profile" is personal information constructed based on a viewer's behavioral history, preferences, and emotional data.

[1029] "Generative AI" refers to AI that learns from large amounts of data and performs tasks such as natural language generation and image generation.

[1030] An "emotion engine" is software or hardware that analyzes emotions from viewers' facial expressions, voice, behavior, etc.

[1031] "Customized Content" means video and audio content that is generated and optimized for a viewer based on viewer profile and emotional data.

[1032] "Advertising optimization" refers to the process of delivering advertisements with optimal timing and content based on viewer profiles and emotional data, thereby maximizing their effectiveness.

[1033] "Feedback" means information such as preferences and opinions provided by viewers that is used to improve profiles and content offerings.

[1034] A specific embodiment of a system for carrying out the present invention will be described below. The system is composed of a server, terminals, and users, and is provided with various means for realizing the contents of the invention.

[1035] Hardware and software used

[1036] server:

[1037] Software: Generative AI (generative AI model), database, web server

[1038] Role: Pre-processing material, generating audience profiles, generating and delivering customized content, inserting and optimizing ads, collecting feedback and updating profiles

[1039] Device:

[1040] Hardware: Camera, microphone, display

[1041] Software: Emotion recognition engine (e.g., facial expression recognition model)

[1042] Role: Collect emotional data, watch customized content, send emotional data and feedback

[1043] User:

[1044] Interact: Watch content, provide feedback

[1045] System flow and processing

[1046] The server generates new content from the materials provided by the generative AI. Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the materials, and performs pre-processing such as encoding and trimming. At this stage, the content provided by creators might be, for example, footage of natural scenery and music files.

[1047] Next, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice while they are watching the content. This emotional data is analyzed using facial expression recognition technology to identify various emotions such as joy, surprise, and sadness. The collected emotional data is then sent to a server.

[1048] The server analyzes this emotional data along with the viewer's behavioral history, preferences, and device information to generate a viewer profile. For example, it can identify scenes from past viewing data that evoked positive emotional responses and add them to the viewer's profile as "preferences."

[1049] Based on the viewer profile, the server issues instructions to the AI ​​to generate new customized content, taking into account emotional data. For example, the server can instruct the AI ​​to prioritize scenes that make viewers feel happy, generating new content that includes many scenes that evoke positive emotions.

[1050] The generated customized content is delivered from the server to the device, where the viewer watches it. The device collects the viewer's engagement data (viewing time, number of plays, reactions) and emotional data and sends it back to the server.

[1051] The server selects the most suitable advertisements based on viewer profiles and emotional data and inserts them into the video. It also tracks and optimizes the advertisement performance (click-through rate, viewing completion rate). For example, it can insert a relevant advertisement immediately after a scene that evokes positive emotions, increasing the effectiveness of the advertisement.

[1052] It collects feedback from viewers and updates their profiles, including their preferences and opinions, which the server then incorporates into the next content generation.

[1053] Specific examples

[1054] This is an example of a generative AI model generating new content based on a provided prompt.

[1055] Example prompt sentence:

[1056] "The input video contains multiple scenes, each with elements that elicit different emotional responses. Please generate a new video by focusing on scenes in which viewers express the 'Happy' emotion. Also, please add upbeat music and fun characters as additional elements to elicit the 'Happy' emotion."

[1057] In this way, a highly personalized viewing experience based on the viewer's emotions can be provided.

[1058] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1059] Step 1:

[1060] The server generates new content based on the materials provided by artificial intelligence. Specifically, creators upload video material, audio, and text data to the server, which then receives it and performs pre-processing (format and quality checks, encoding, trimming).

[1061] Input: Video material, audio, text data

[1062] Output: Encoded and trimmed material

[1063] Step 2:

[1064] While the viewer is watching the content, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice. For example, the device's camera captures the viewer's face and uses facial recognition technology to analyze their emotions (happiness, surprise, sadness, etc.). The analysis results are sent to the server.

[1065] Input: Viewer's facial expressions and voice

[1066] Output: Parsed emotion data

[1067] Step 3:

[1068] The server analyzes the viewer's behavioral history, preferences, device information, and emotional data to generate a viewer profile. Based on the viewing and emotional data, scenes that evoked a positive emotional response are identified and added to the profile as the viewer's "preferences."

[1069] Input: Viewer behavior history, preferences, device information, emotional data

[1070] Output: Audience profile

[1071] Step 4:

[1072] The server issues prompts to the AI ​​based on the viewer profile to generate new customized content. It also takes into account emotional data, and instructs the AI ​​to prioritize scenes that elicit positive emotions, for example, generating content that includes many scenes that evoke positive emotions.

[1073] Input: Viewer profile, prompt

[1074] Output: New, customized content

[1075] Step 5:

[1076] The server delivers the generated customized content to the device, which then provides the content to the viewer, collecting viewer engagement data (watching time, number of plays, reactions) and emotional data and sending it back to the server.

[1077] Input: Customized Content

[1078] Output: Audience engagement data, sentiment data

[1079] Step 6:

[1080] The server selects the most suitable advertisement based on the viewer profile and emotional data, and inserts it into the middle of the video. It also tracks the performance of the advertisement (click-through rate, completion rate), and optimizes the effectiveness of the advertisement based on the results. For example, it inserts a relevant advertisement immediately after a scene that evokes positive emotions.

[1081] Input: Viewer profile, emotional data

[1082] Output: Optimized Ad

[1083] Step 7:

[1084] The server collects feedback from viewers and updates viewer profiles, including information such as viewer preferences and opinions, to reflect in the next content generation.

[1085] Input: Viewer feedback, sentiment data

[1086] Output: Updated viewer profile

[1087] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1088] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1089] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1090] [Third embodiment]

[1091] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1092] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1093] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1094] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1095] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1096] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1097] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1098] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1099] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1100] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1101] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1102] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1103] Specific embodiments of the system related to the present invention will be described below.

[1104] System Overview

[1105] The system consists of three main components: a server, a terminal, and a user. It generates new content based on the provided material using generative AI, delivers customized content based on the viewer profile, optimizes advertising, and collects and updates feedback.

[1106] Program processing flow

[1107] Uploading and Preprocessing Materials

[1108] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality, and performs pre-processing such as encoding and trimming.

[1109] Examples:

[1110] Creators upload footage of natural scenery they have taken and the corresponding music files.

[1111] The server checks the video resolution and converts it into a unified format.

[1112] Generate audience profiles

[1113] The device collects the viewer's past viewing history, preferences, device information, etc. This information is sent to a server to create a viewer profile.

[1114] Examples:

[1115] The genre that users often watch is cooking-related videos.

[1116] The device records viewing history, sends it to the server, and creates a profile called "I love cooking."

[1117] Customized Content Generation

[1118] The server instructs the generative AI based on the viewer profile to generate new customized content.

[1119] Examples:

[1120] The server instructs the generation AI to add cooking scenes to travel footage for users who love cooking.

[1121] The generative AI generates scenes introducing famous local dishes to accompany travel footage.

[1122] Content distribution and engagement tracking

[1123] The server delivers the generated customized content to the device, and collects viewer engagement data (watch time, number of plays, reactions, etc.) to use for the next profile update.

[1124] Examples:

[1125] A server distributes travel videos including cooking scenes to users who love cooking.

[1126] The terminal collects the user's viewing data and sends it to the server.

[1127] Ad insertion and optimization

[1128] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the ad performance.

[1129] Examples:

[1130] The server selects organic food advertisements and inserts them into videos targeted at health-conscious viewers.

[1131] The server monitors the click-through rate and completion rate of the ads and makes appropriate adjustments.

[1132] Collecting feedback and updating your profile

[1133] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next video generation.

[1134] Examples:

[1135] The user provides feedback regarding preferences for particular effects and scenes.

[1136] The server adds this information to a profile and uses it the next time a video is generated.

[1137] conclusion

[1138] The system generates and delivers high-quality video content customized based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[1139] The processing flow will be explained below.

[1140] Step 1:

[1141] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[1142] Specific behavior:

[1143] The server checks the format of the uploaded file (JPEG, MP4, etc.).

[1144] The server verifies the video resolution and audio bit rate.

[1145] Step 2:

[1146] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[1147] Specific behavior:

[1148] The server converts all video to a uniform resolution.

[1149] The server processes the audio track for noise reduction.

[1150] Step 3:

[1151] The device collects the viewer's behavioral history, preferences, device information, etc. and sends it to the server, which then creates a viewer profile based on this information.

[1152] Specific behavior:

[1153] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[1154] The terminal transmits the collected data to the server.

[1155] The server generates a viewer profile based on the received data.

[1156] Step 4:

[1157] The server instructs the generative AI based on the viewer profile to generate new customized content.

[1158] Specific behavior:

[1159] The server uses the viewer profile to pass appropriate content generation parameters to the generation AI.

[1160] Generative AI generates new content by adding elements that viewers like to specific scenes in a video.

[1161] Step 5:

[1162] The server delivers the generated customized content to the device, which collects viewer engagement data (watch time, number of plays, reactions, etc.) and sends it back to the server.

[1163] Specific behavior:

[1164] The server transmits the customized content to the terminal in a streaming format.

[1165] The device collects viewer engagement data in real time.

[1166] The engagement data collected by the device is periodically sent to the server.

[1167] Step 6:

[1168] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the performance of the ads.

[1169] Specific behavior:

[1170] The server analyzes the viewer profile and selects appropriate advertisements.

[1171] The server decides when to insert the advertisement during video playback.

[1172] The server tracks ad click rates and completion rates and uses that data to optimize ads.

[1173] Step 7:

[1174] The device collects feedback from viewers and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next content generation.

[1175] Specific behavior:

[1176] The device collects feedback provided by the viewer (e.g., "I like this scene").

[1177] The device sends the collected feedback to the server.

[1178] The server updates the viewer profile based on the feedback and reflects it in the parameters of the next generation AI.

[1179] Example 1

[1180] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1181] Video viewers often demand content that is individually customized based on their diverse preferences and viewing history. Streaming platforms are also required to insert appropriate advertisements and optimize their performance. However, conventional systems face challenges in automating these requirements, making it difficult and time-consuming.

[1182] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1183] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a user profile based on the user's behavioral history and preferences, means for generating customized content based on the user profile, means for checking the format and quality of the material and encoding and trimming it, means for collecting user engagement data and updating the user profile, and means for delivering customized content to users. This enables the automatic generation and delivery of high-quality content customized based on user preferences.

[1184] (The system of claim 2)

[1185] The system further includes a means for inserting advertisements into the generated customized content and a means for tracking and optimizing the performance of the advertisements, thereby providing the most suitable advertisements to the viewer and maximizing their effectiveness.

[1186] "Generative AI" refers to AI that has the ability to generate new information and content based on diverse data.

[1187] A "user profile" is a database or collection of information constructed based on a user's behavioral history and preferences.

[1188] "Customized Content" means content that has been adapted or edited based on a specific user profile.

[1189] "Encoding" is the process of converting data into a particular format.

[1190] "Trimming" is the process of cutting out unnecessary parts of video or audio data.

[1191] "Engagement data" refers to behavioral data such as viewer viewing time, number of views, and reactions.

[1192] "Feedback" refers to opinions and ratings provided by users.

[1193] "Ad insertion" refers to the process of placing advertisements within video or audio content.

[1194] "Performance tracking" refers to the collection and analysis of data to measure and analyze the effectiveness of advertising and other activities.

[1195] "Optimization" is the process of adjusting or improving a system or process to maximize its efficiency or effectiveness.

[1196] MODE FOR CARRYING OUT THE INVENTION

[1197] The system related to this invention mainly consists of three elements: a server, a terminal, and a user. The system generates new content based on provided materials using a generative AI model, delivers customized content based on viewer profiles, and optimizes advertisements and collects and updates feedback.

[1198] server

[1199] The server manages everything from uploading materials to generating and distributing customized content. Specific processes include the following:

[1200] 1. Upload and pre-process materials

[1201] The server receives video, audio, and text data from creators. It checks the file format and quality, and encodes and trims it. For example, it uses the FFmpeg tool to convert and unify the format and quality of the material.

[1202] 2. Generate audience profiles

[1203] The server receives data on the viewer's behavior and preferences sent from the device, creates a viewer profile, and stores this data in a database (e.g., MySQL).

[1204] 3. Creating customized content

[1205] Based on the viewer profile, the server sends prompts to the AI ​​to generate customized content, such as "add cooking scenes to travel videos."

[1206] 4. Content distribution and engagement tracking

[1207] The server delivers the generated customized content to the device and collects viewer engagement data using data streaming technologies (e.g., RTMP, HLS).

[1208] 5. Ad insertion and optimization

[1209] The server selects and inserts appropriate ads based on viewer profile and video context, and also tracks and optimizes ad performance.

[1210] 6. Collect feedback and update your profile

[1211] The server collects feedback sent by the terminals and updates the viewer profile.

[1212] Terminal

[1213] The device collects the user's behavior history and sends it to the server. It also displays customized content and advertisements received from the server, collects viewing data and feedback, and sends them to the server.

[1214] User

[1215] Users can view content via their terminals and provide feedback, for example by entering their preferences and opinions about a particular scene in a feedback form.

[1216] Specific examples

[1217] For example, if a user frequently watches cooking-related videos, the device will collect that viewing history and send it to the server, which will then use that data to create a "cooking lover" profile and instruct the AI ​​to add cooking scenes to travel videos in the next customized content.

[1218] Prompt Sentence Examples

[1219] "For food-loving viewers, add a segment to your travel footage showcasing famous local dishes."

[1220] This system generates and distributes high-quality video content based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[1221] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1222] Program processing flow

[1223] Step 1. Upload and preprocess your materials

[1224] 1.1 Uploading Materials

[1225] Creators upload video footage, audio, and text data to the server, which triggers the action of selecting files through a web interface and pressing the upload button.

[1226] Input: Video material files, audio files, text files

[1227] Output: Original material data stored on the server

[1228] 1.2 File format and quality check

[1229] The server checks the format and quality of the uploaded file by using tools like FFmpeg to extract file information and verify things like format, resolution, bitrate, etc.

[1230] Input: Original material data stored on the server

[1231] Output: Format and quality verified material data

[1232] 1.3 Encoding and Trimming

[1233] The server encodes and trims the file as needed, for example converting the video file to the specified resolution and trimming unwanted parts.

[1234] Input: Format and quality verified material data

[1235] Output: Encoded and trimmed material data

[1236] Step 2. Generate your audience profile

[1237] 2.1 Collecting viewing history and preferences

[1238] The device collects information about the viewer's viewing history, preferred genres, and playback devices, including using browser and app cookies to record viewing data.

[1239] Input: Viewer viewing activity

[1240] Output: Collected viewing history data

[1241] 2.2 Sending profile data

[1242] The device sends the collected data to the server. The data is packetized in JSON format and sent via an HTTP POST request.

[1243] Input: Collected viewing history data

[1244] Output: Profile data sent to the server

[1245] 2.3 Creating an audience profile

[1246] The server creates a viewer profile based on the data it receives, and records the viewer's preferences and history in a database.

[1247] Input: Profile data sent to the server

[1248] Output: Viewer profile stored in a database

[1249] Step 3. Generate customized content

[1250] 3.1 Creating prompts for generative AI

[1251] The server creates a prompt to send to the AI ​​based on the viewer profile, such as "Add cooking scenes to travel videos for viewers who love cooking."

[1252] Input: Viewer profile stored in database

[1253] Output: Generated prompt statement

[1254] 3.2 Content Generation Instructions

[1255] The server sends a prompt to the generative AI model to instruct it to generate content. The prompt is sent to the generative AI using an API request.

[1256] Input: Generated prompt statement

[1257] Output: The prompt sent to the generative AI model

[1258] 3.3 Creating customized content

[1259] The generative AI generates new content based on the prompt and sends it back to the server, for example adding a cooking scene to a video and generating a text description.

[1260] Input: The prompt sent to the generative AI model

[1261] Output: Generated customization content

[1262] Step 4. Content distribution and engagement tracking

[1263] 4.1 Delivery of customized content

[1264] The server delivers the generated customized content to the terminal. The content is delivered using data streaming technology.

[1265] Input: Generated customization content

[1266] Output: Content delivered to the device

[1267] 4.2 Collection of viewing data

[1268] The device collects viewer engagement data (watch time, plays, reactions, etc.) and records this data using JavaScript code.

[1269] Input: Content delivered to the device

[1270] Output: Collected engagement data

[1271] 4.3 Transmission of Collected Data

[1272] The device sends engagement data to the server via an HTTP request for analysis.

[1273] Input: Collected engagement data

[1274] Output: Engagement data sent to the server

[1275] Step 5. Ad insertion and optimization

[1276] 5.1 Ad Selection

[1277] The server chooses the best ad based on the viewer profile and video context, selecting the appropriate ad from an ad database.

[1278] Input: Viewer profile stored in database

[1279] Output: Selected ads

[1280] 5.2 Ad Insertion

[1281] The server inserts the selected advertisement into the video at the specified position. The advertisement is then integrated into the video using a video editing tool.

[1282] Input: Selected ads, customized content

[1283] Output: Customized content with ad insertion

[1284] 5.3 Tracking Ad Performance

[1285] The server monitors the click-through rate and completion rate of ads and automatically adjusts the optimal ads. Performance data is collected using analytical tools and optimized by algorithms.

[1286] Input: Ad-inserted personalized content, viewing data

[1287] Output: Optimized ad placement

[1288] Step 6. Collect feedback and update your profile

[1289] 6.1 Feedback Collection

[1290] The device collects feedback from viewers, which involves viewers entering their opinions and thoughts through a form in their browser or app.

[1291] Input: Viewer feedback

[1292] Output: Collected feedback data

[1293] 6.2 Sending Feedback

[1294] The device sends the collected feedback to the server via an HTTP POST request.

[1295] Input: Collected feedback data

[1296] Output: Feedback data sent to the server

[1297] 6.3 Updating your profile

[1298] The server updates the viewer profile based on the feedback, updating the profile information in the database and reflecting it in the next content generation.

[1299] Input: Feedback data sent to the server

[1300] Output: Updated viewer profile

[1301] (Application example 1)

[1302] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1303] Conventional content distribution systems lack sufficient customization based on individual viewer preferences and behavioral history, and lack complete feedback and engagement tracking to improve the quality of the viewing experience. Furthermore, they are unable to select optimal ads based on viewer profiles and track their effectiveness, making it difficult to maximize advertising effectiveness. This results in insufficient improvement in viewer satisfaction and the value of the platform.

[1304] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1305] In this invention, the server includes means for generating new content based on materials provided by the generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for delivering the customized content to the viewer, means for collecting viewer engagement data and using it for the next profile update, means for inserting advertisements into the generated customized content, means for tracking the performance of the advertisements, means for selecting optimal advertisements based on the viewer profile and video context, means for collecting feedback from the viewer and updating the viewer profile, and means for reflecting the collected feedback in the profile and using it for generating the next video. This enables the generation and delivery of high-quality customized content based on the viewer's preferences and behavioral history, maximizing the effectiveness of advertisements and improving viewer satisfaction and the value of the platform.

[1306] "Generative AI" is an AI technology that generates new information and content based on input data.

[1307] "Materials" refers to the basic data for generating content, such as video, audio, and text data provided by creators.

[1308] "Customized content" is individual content that is generated based on a viewer profile and tailored to the viewer's preferences and behavioral history.

[1309] A "viewer profile" refers to a data structure that indicates the characteristics of a viewer, constructed by analyzing data related to the viewer's behavioral history and preferences.

[1310] "Viewer engagement data" refers to behavioral data of viewers when they watch content, including viewing time, number of plays, reactions, etc.

[1311] "Feedback" refers to information such as reactions, opinions, and ratings from viewers, which is used to generate next content and update profiles.

[1312] "Ad performance" refers to metrics that evaluate how effectively an ad works for viewers, including click-through rate and completion rate.

[1313] "Video context" refers to the background information surrounding the video, such as the content, theme, and scene situation of the video being generated.

[1314] As an embodiment of the present invention, a specific system configuration and its operation will be described below.

[1315] System configuration

[1316] Server: A central computer system that processes generative AI models, creates and updates audience profiles, collects and analyzes engagement data, and optimizes and tracks advertising effectiveness.

[1317] Terminal: A device used by viewers, such as a smartphone, tablet, or smart glasses, that collects viewers' behavioral history and transmits it to a server.

[1318] Users: Viewers and creators who use the system. Creators upload materials such as videos and audio, and viewers watch customized content.

[1319] Program processing

[1320] Uploading and Preprocessing Materials

[1321] Creators upload video footage, audio, text data, etc. to the server, which then uses OpenCV to divide the footage into frames, check the quality, encode, and trim as needed.

[1322] Generate audience profiles

[1323] The device collects information such as the viewer's past viewing history, preferences, and device information, and sends it to the server, which then uses Scikit-learn's KMeans clustering to generate a viewer profile.

[1324] Customized Content Generation

[1325] Based on the generated viewer profile, the server invokes a generative AI model using TensorFlow to generate customized content, adding elements that match the profile to the content in the process.

[1326] Content distribution and engagement tracking

[1327] The server delivers the generated customized content to the device, and the device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server.

[1328] Ad insertion and optimization

[1329] The server selects the best ads based on viewer profile and video context, inserts them within the customized content, and tracks and optimizes ad performance (e.g., click-through rate and completion rate).

[1330] Collecting feedback and updating your profile

[1331] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and generate customized content for the next viewing.

[1332] Specific examples

[1333] As a concrete example, consider the case where a user who loves cooking uses this system on a smartphone. The user uses the app just like a regular video streaming app. Based on the user's past viewing history, the server generates a profile for the user called "cooking lover" and provides new cooking-related content. For example, based on the user's profile, the server can generate and distribute new travel videos including Chinese food recipe videos.

[1334] Prompt Sentence Examples

[1335] "User profile: Loves cooking. Viewing genre: Chinese food. Content to generate: Generate travel footage that includes Chinese food recipes."

[1336] In this way, the system can efficiently generate and deliver high-quality customized content tailored to viewer preferences and behavioral history, while optimizing advertising effectiveness, thereby improving viewer satisfaction and enhancing the value of the platform.

[1337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1338] Step 1:

[1339] Creators upload video footage, audio, text data, etc. to the server. The input is the content material created by the creator, and the output is when these materials are saved on the server and prepared for further processing. The server uses OpenCV to divide the video footage into frames, perform quality checks, encode, and trim as necessary.

[1340] Step 2:

[1341] The device collects the viewer's past viewing history, preferences, and device information, and sends it to the server. The input is the viewer's behavioral history and device information, and the output is this information sent to the server. The server generates a viewer profile using Scikit-learn's KMeans clustering. At this stage, the viewer's preferences and behavioral patterns are analyzed and a profile is constructed.

[1342] Step 3:

[1343] The server uses TensorFlow to call a generative AI model based on the generated viewer profile to generate customized content. The input is the viewer profile and uploaded materials, and the output is the customized content. The generative AI model adds elements that match the profile to the content.

[1344] Step 4:

[1345] The server delivers the generated customized content to the device. The device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server. The input is the customized content and the viewer's reactions, and the output is viewing data and engagement data. The collected engagement data is reflected in the next content generation.

[1346] Step 5:

[1347] The server selects the optimal ad based on the viewer profile and video context and inserts the ad into the customized content. The input is the viewer profile, content context, and candidate ads, and the output is the customized content with the ad inserted. In addition, the server tracks the ad performance (click-through rate, completion rate, etc.) and performs optimization to maximize the advertising effect.

[1348] Step 6:

[1349] The device collects feedback from viewers and sends it to the server. The server updates the viewer profile based on this feedback and uses it to generate the next customized content. The input is viewer feedback, and the output is an updated viewer profile. Based on the information gained from the feedback, the quality of the next content provided can be further improved.

[1350] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1351] Specific embodiments of the system related to the present invention will be described below.

[1352] System Overview

[1353] This system combines three main elements - the server, the device, and the user - with an emotion engine that recognizes the user's emotions to provide a more personalized viewing experience. It generates new content based on the materials provided by the generative AI, delivers customized content based on the viewer profile, and optimizes advertising using emotional data.

[1354] Program processing flow

[1355] Uploading and Preprocessing Materials

[1356] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the material, and performs pre-processing such as encoding and trimming.

[1357] Examples:

[1358] Creators upload footage of natural scenery they have taken and the corresponding music files.

[1359] The server checks the video resolution and converts it into a unified format.

[1360] Collecting Emotional Data

[1361] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[1362] Examples:

[1363] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[1364] The device sends the analysis results to the server.

[1365] Generate audience profiles

[1366] The device collects the viewer's behavioral history, preferences, device information, as well as emotional data, and sends it to the server, which then creates a viewer profile based on this information.

[1367] Examples:

[1368] The device collects the viewer's past viewing data and emotional data and sends it to the server.

[1369] The server analyzes scenes that elicit a positive emotional response and adds the viewer's "preferences" to their profile.

[1370] Customized Content Generation

[1371] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account emotional data as well.

[1372] Examples:

[1373] The server gives instructions to the generation AI, focusing on scenes that make the server feel happy.

[1374] Generative AI generates new content that includes many scenes that evoke positive emotions.

[1375] Content distribution and engagement tracking

[1376] The server delivers the generated customized content to the device, which collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends it back to the server.

[1377] Examples:

[1378] The server transmits the customized content to the terminal in streaming format.

[1379] The device collects viewer engagement data and emotional reactions and sends them to a server.

[1380] Ad insertion and optimization

[1381] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[1382] Examples:

[1383] The server inserts relevant advertisements immediately after scenes that elicit positive emotions.

[1384] The server monitors the click-through rate and completion rate of the advertisements to optimize the effectiveness of the advertisements.

[1385] Collecting feedback and updating your profile

[1386] The device collects feedback from viewers and sends it to the server, which updates the viewer profile based on this feedback and emotional data and reflects it in the next content generation.

[1387] Examples:

[1388] Users provide feedback on their favorite scenes and effects.

[1389] The server uses the feedback and emotional data to update the viewer profile and use it to generate the next piece of content.

[1390] This enables the system to provide highly personalized video content and optimize advertising in response to viewer emotions, improving the viewing experience and increasing the value of the platform.

[1391] The processing flow will be explained below.

[1392] Step 1:

[1393] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[1394] Specific behavior:

[1395] Creators upload footage of natural scenery they have taken and the corresponding music files to a server.

[1396] The server checks the format of the uploaded file (JPEG, MP4, etc.) and verifies the video resolution and audio bitrate.

[1397] Step 2:

[1398] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[1399] Specific behavior:

[1400] The server converts all video files to a uniform resolution.

[1401] The server processes the audio track for noise reduction.

[1402] Step 3:

[1403] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[1404] Specific behavior:

[1405] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[1406] The device uses a microphone to analyze the tone and intensity of the viewer's voice.

[1407] The device sends the analysis results to the server.

[1408] Step 4:

[1409] The device also collects the viewer's behavioral history, preferences, and device information, and sends this data to the server, which then creates a viewer profile based on this information.

[1410] Specific behavior:

[1411] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[1412] Your device collects device information (e.g., the type of browser and device you are using).

[1413] The server generates a viewer profile based on the data received, which also includes emotional data.

[1414] Step 5:

[1415] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account the collected emotional data.

[1416] Specific behavior:

[1417] The server gives instructions to the generation AI, placing emphasis on scenes that elicit strong emotional responses of "surprise" and "joy."

[1418] Generative AI takes into account the viewer's emotions and generates new content that contains many elements that viewers will respond to positively.

[1419] Step 6:

[1420] The server delivers the generated customized content to the device, and the device collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends them to the server.

[1421] Specific behavior:

[1422] The server transmits the customized content to the terminal in streaming format.

[1423] The device continues to monitor the viewer's emotional response while the video is playing.

[1424] The device periodically transmits viewing data and emotion data to the server.

[1425] Step 7:

[1426] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[1427] Specific behavior:

[1428] The server selects relevant advertisements immediately following scenes that elicit positive emotions.

[1429] The server inserts the ads at the appropriate times within the video.

[1430] The server tracks ad click rates and completion rates, and uses that data to optimize the effectiveness of the ads.

[1431] Step 8:

[1432] The device collects feedback from viewers and sends it to the server, which uses this feedback and emotional data to update the viewer profile and reflect it in the next content generation.

[1433] Specific behavior:

[1434] Users provide feedback on their impressions of specific scenes and effects.

[1435] The device sends the collected feedback to the server.

[1436] The server updates the viewer profile based on the collected feedback and sentiment data and reflects it in the next content generation.

[1437] Example 2

[1438] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1439] Conventional content generation systems do not adequately personalize content based on viewer emotions and behavioral history, making it difficult to improve viewer satisfaction and deliver advertisements effectively. Furthermore, they lack a mechanism for effectively utilizing post-viewing feedback and reflecting it in the next content generation. A new system that can solve these issues and provide viewers with highly personalized content and an optimal advertising experience is needed.

[1440] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1441] In this invention, the server includes means for preprocessing materials uploaded by creators, means for collecting and analyzing viewer emotion data, means for generating new content based on generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and emotion data, means for generating customized content based on the viewer profile, and means for delivering the customized content to the viewer. This enables advanced personalization that utilizes the viewer's emotion and behavioral history, and optimizes the viewing experience and advertising experience.

[1442] "Creator" refers to a person who creates content and uploads it to the system.

[1443] "Materials" refers to the data that forms the basis of content, such as video, audio, and text data.

[1444] "Preprocessing" refers to the process of checking the format and quality of the uploaded material and performing encoding, trimming, etc.

[1445] "Emotional data" refers to data related to emotions analyzed from viewers' facial expressions and voices.

[1446] A "viewer profile" refers to information that reflects a viewer's preferences, etc., and is generated based on the viewer's behavioral history and emotional data.

[1447] "Generative AI" refers to AI technology that generates new content based on viewer profiles, emotional data, etc.

[1448] "Content" refers to information such as video, audio, and text that viewers can view.

[1449] "Customized Content" refers to content that is individually generated based on a viewer profile.

[1450] "Engagement data" refers to data about viewers' viewing behavior, such as viewing time, number of views, and reactions.

[1451] "Advertising" means any promotional message or advertising content displayed to a viewer.

[1452] "Ad performance" refers to data used to measure the effectiveness of an ad, such as click-through rate and completion rate.

[1453] "Feedback" refers to the impressions and opinions that viewers provide after watching a program.

[1454] The system of the present invention functions mainly through a server, a terminal, a creator, and a user. The roles of each are explained below.

[1455] Hardware and Software Configuration

[1456] 1. Server: This plays a central role in the system. It has high-performance processing capabilities and is ideally suited to using a cloud server for data storage, data analysis, and implementing generative AI models. Specifically, Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc. are suitable.

[1457] 2. Device: A device used by the user to view and provide feedback. This can be a smartphone, tablet, or PC. The device has a built-in camera and microphone, and an application to run the emotion engine is installed.

[1458] 3. Creator: An entity that creates content and uploads it to the system. They create content using image editing software (e.g., Adobe Premiere, Final Cut Pro) and audio editing software.

[1459] Content Creation and Customization Process

[1460] The system of the present invention operates in the following manner.

[1461] 1. Upload and pre-process materials:

[1462] Creators upload video footage, audio, and text data to the server using a web application interface.

[1463] The server checks the format and quality of the received material and performs pre-processing such as encoding and trimming. Specifically, it performs encoding to unify the resolution of the video files.

[1464] 2. Collecting Emotional Data:

[1465] The device uses a built-in camera and microphone to capture the viewer's facial expressions and voice while they are watching the video, and then analyzes them in real time using an emotion engine.

[1466] The device transmits the analysis result, emotional data, to the server. This data includes the viewer's emotional response (e.g., joy, surprise, sadness, etc.).

[1467] 3. Generate audience profiles:

[1468] The terminal records the viewer's behavioral history and device information and sends this to the server.

[1469] The server generates a viewer profile based on the received emotion data and behavioral history data, integrates this information into the database, and updates the profile information.

[1470] 4. Generate customized content:

[1471] The server issues instructions to the generation AI based on the viewer profile to generate new customized content.

[1472] An example of a specific prompt would be, "Generate a new video of a natural scene, focusing on a scene that the viewer found pleasurable."

[1473] The generative AI model generates new content based on this prompt.

[1474] 5. Content distribution and engagement tracking:

[1475] The server distributes the generated customized content to the terminal in streaming format.

[1476] The device collects engagement data such as viewer viewing time, number of plays, and reactions, and sends this back to the server.

[1477] 6. Ad Insertion and Optimization:

[1478] The server selects the most suitable advertisement based on the viewer profile and emotional data and inserts it into the video.

[1479] The server tracks and optimizes the performance of ads, specifically by monitoring click-through rates and completion rates.

[1480] 7. Collecting Feedback and Updating Your Profile:

[1481] The terminal collects feedback from the viewer and sends it to the server.

[1482] The server updates the viewer profile based on the feedback and emotional data and reflects it in the next content generation.

[1483] By implementing the above steps, the system of the present invention can achieve advanced personalization based on the viewer's emotions and behavioral history, optimizing the viewing experience and advertising experience. This system allows users to enjoy more attractive and personalized content, and maximizes the effectiveness of advertising.

[1484] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1485] Step 1:

[1486] Creators upload materials.

[1487] Input: Video material, audio, and text data created by creators.

[1488] Output: Material data uploaded and stored on the server.

[1489] How it works: When a creator accesses the upload page of the web application, selects a local file, and presses the "Upload" button, the material data is transferred to the server, which receives it and stores it in storage.

[1490] Step 2:

[1491] The server pre-processes the material.

[1492] Input: Video footage, audio, and text data uploaded by creators.

[1493] Output: Material data that has been encoded, trimmed, and converted into a unified format.

[1494] What it does: The server checks the format and quality of the received material, encodes the video files to a specific resolution, standardizes the format of the audio files, trims and removes unnecessary parts, and converts the material to a standardized format.

[1495] Step 3:

[1496] The device captures the viewer's emotional data.

[1497] Input: The viewer's facial expressions and voice, captured by the device's built-in camera and microphone.

[1498] Output: Viewer sentiment data analyzed by the sentiment engine.

[1499] How it works: While the viewer is watching the video, the device automatically activates the camera and microphone to capture the viewer's face and voice, and performs real-time facial analysis to generate emotional data such as joy or surprise.

[1500] Step 4:

[1501] The device transmits the emotion data to the server.

[1502] Input: Viewer sentiment data analyzed on the device.

[1503] Output: The emotion data is transferred to the server and stored in a database.

[1504] Specific operation: The device generates emotion data as data packets at regular intervals and sends them to the server via HTTP requests. The server then records the received data in a database.

[1505] Step 5:

[1506] The terminal transmits the viewing data to the server.

[1507] Input: Viewer behavior history, device information.

[1508] Output: Viewing data sent to the server and stored in a database.

[1509] Specific operation: The terminal records the viewer's behavior history (viewing start time, end time, scenes viewed) and device information, and sends this to the server in batch processing.

[1510] Step 6:

[1511] The server generates a viewer profile.

[1512] Input: Emotion data and viewing data sent from the device.

[1513] Output: The created audience profile.

[1514] Specific operation: Based on the received data, the server analyzes the viewer's preferences and behavioral patterns and stores the viewer profile in a database.

[1515] Step 7:

[1516] The server gives instructions to the generated AI.

[1517] Input: Audience profile.

[1518] Output: The prompt sent to the generation AI.

[1519] Specific operation: The server analyzes the viewer profile and generates a prompt to instruct the AI ​​to generate new content. An example of a prompt is "Generate a new nature video centered around scenes that the viewer found enjoyable."

[1520] Step 8:

[1521] Generative AI generates customized content.

[1522] Input: The prompt text received from the server.

[1523] Output: The generated customization content.

[1524] How it works: Based on the prompt, the generative AI model generates new content that matches the preferences of the viewer profile.

[1525] Step 9:

[1526] The server delivers the customized content.

[1527] Input: Generated customization content.

[1528] Output: Streamed content delivered to a device.

[1529] Specific operation: The server generates a content URL and sends it to the device to start viewing.

[1530] Step 10:

[1531] The device collects engagement data.

[1532] Input: Viewer viewing behavior data.

[1533] Output: Collected engagement data is sent to a server.

[1534] Specific operation: The device tracks viewing time, number of plays, reactions, etc. and sends the data to the server in batches.

[1535] Step 11:

[1536] The server selects and inserts the advertisements.

[1537] Input: Audience profile, sentiment data.

[1538] Output: Ads inserted into the generated content.

[1539] Specific operation: The server selects the most suitable advertisement based on the profile and emotional data, analyzes the appropriate insertion point in the video, and inserts the advertisement.

[1540] Step 12:

[1541] The server tracks and optimizes the performance of the ads.

[1542] Input: Ad click-through rate, view completion rate.

[1543] Output: Optimization of advertising effectiveness.

[1544] Specific operation: The server aggregates ad performance data and updates the ad selection logic using machine learning algorithms.

[1545] Step 13:

[1546] The device collects viewer feedback.

[1547] Input: Feedback provided by the audience.

[1548] Output: Feedback sent to server and stored in database.

[1549] Specific operation: The device collects feedback from the user through an in-app feedback form and sends it to the server.

[1550] Step 14:

[1551] The server updates the profile based on the feedback.

[1552] Input: Audience feedback, sentiment data.

[1553] Output: Updated audience profile.

[1554] What it does: The server analyzes the feedback and sentiment data, updates the viewer profile in a database, and uses it to generate the next piece of content.

[1555] (Application example 2)

[1556] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1557] Conventional content distribution services provide content customized based on a viewer's behavioral history and preferences, but do not take into account the viewer's real-time emotions. This makes it difficult to deliver content that optimally responds to the viewer's emotions, and the improvement of the viewing experience and optimization of advertising have not been fully achieved. Therefore, the objective of the present invention is to provide a system that recognizes viewer emotions in real time, generates customized content based on that data, and further optimizes the effectiveness of advertising.

[1558] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1559] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for collecting viewer emotion data using an emotion engine, means for generating customized content taking the emotion data into consideration, and means for delivering the customized content to the viewer. This makes it possible to provide highly personalized video content and optimize advertisements according to the viewer's emotions.

[1560] A "server" is a computer system that provides services to other computers (clients) on a network.

[1561] A "viewer profile" is personal information constructed based on a viewer's behavioral history, preferences, and emotional data.

[1562] "Generative AI" refers to AI that learns from large amounts of data and performs tasks such as natural language generation and image generation.

[1563] An "emotion engine" is software or hardware that analyzes emotions from viewers' facial expressions, voice, behavior, etc.

[1564] "Customized Content" means video and audio content that is generated and optimized for a viewer based on viewer profile and emotional data.

[1565] "Advertising optimization" refers to the process of delivering advertisements with optimal timing and content based on viewer profiles and emotional data, thereby maximizing their effectiveness.

[1566] "Feedback" means information such as preferences and opinions provided by viewers that is used to improve profiles and content offerings.

[1567] A specific embodiment of a system for carrying out the present invention will be described below. The system is composed of a server, terminals, and users, and is provided with various means for realizing the contents of the invention.

[1568] Hardware and software used

[1569] server:

[1570] Software: Generative AI (generative AI model), database, web server

[1571] Role: Pre-processing material, generating audience profiles, generating and delivering customized content, inserting and optimizing ads, collecting feedback and updating profiles

[1572] Device:

[1573] Hardware: Camera, microphone, display

[1574] Software: Emotion recognition engine (e.g., facial expression recognition model)

[1575] Role: Collect emotional data, watch customized content, send emotional data and feedback

[1576] User:

[1577] Interact: Watch content, provide feedback

[1578] System flow and processing

[1579] The server generates new content from the materials provided by the generative AI. Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the materials, and performs pre-processing such as encoding and trimming. At this stage, the content provided by creators might be, for example, footage of natural scenery and music files.

[1580] Next, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice while they are watching the content. This emotional data is analyzed using facial expression recognition technology to identify various emotions such as joy, surprise, and sadness. The collected emotional data is then sent to a server.

[1581] The server analyzes this emotional data along with the viewer's behavioral history, preferences, and device information to generate a viewer profile. For example, it can identify scenes from past viewing data that evoked positive emotional responses and add them to the viewer's profile as "preferences."

[1582] Based on the viewer profile, the server issues instructions to the AI ​​to generate new customized content, taking into account emotional data. For example, the server can instruct the AI ​​to prioritize scenes that make viewers feel happy, generating new content that includes many scenes that evoke positive emotions.

[1583] The generated customized content is delivered from the server to the device, where the viewer watches it. The device collects the viewer's engagement data (viewing time, number of plays, reactions) and emotional data and sends it back to the server.

[1584] The server selects the most suitable advertisements based on viewer profiles and emotional data and inserts them into the video. It also tracks and optimizes the advertisement performance (click-through rate, viewing completion rate). For example, it can insert a relevant advertisement immediately after a scene that evokes positive emotions, increasing the effectiveness of the advertisement.

[1585] It collects feedback from viewers and updates their profiles, including their preferences and opinions, which the server then incorporates into the next content generation.

[1586] Specific examples

[1587] This is an example of a generative AI model generating new content based on a provided prompt.

[1588] Example prompt sentence:

[1589] "The input video contains multiple scenes, each with elements that elicit different emotional responses. Please generate a new video by focusing on scenes in which viewers express the 'Happy' emotion. Also, please add upbeat music and fun characters as additional elements to elicit the 'Happy' emotion."

[1590] In this way, a highly personalized viewing experience based on the viewer's emotions can be provided.

[1591] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1592] Step 1:

[1593] The server generates new content based on the materials provided by artificial intelligence. Specifically, creators upload video material, audio, and text data to the server, which then receives it and performs pre-processing (format and quality checks, encoding, trimming).

[1594] Input: Video material, audio, text data

[1595] Output: Encoded and trimmed material

[1596] Step 2:

[1597] While the viewer is watching the content, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice. For example, the device's camera captures the viewer's face and uses facial recognition technology to analyze their emotions (happiness, surprise, sadness, etc.). The analysis results are sent to the server.

[1598] Input: Viewer's facial expressions and voice

[1599] Output: Parsed emotion data

[1600] Step 3:

[1601] The server analyzes the viewer's behavioral history, preferences, device information, and emotional data to generate a viewer profile. Based on the viewing and emotional data, scenes that evoked a positive emotional response are identified and added to the profile as the viewer's "preferences."

[1602] Input: Viewer behavior history, preferences, device information, emotional data

[1603] Output: Audience profile

[1604] Step 4:

[1605] The server issues prompts to the AI ​​based on the viewer profile to generate new customized content. It also takes into account emotional data, and instructs the AI ​​to prioritize scenes that elicit positive emotions, for example, generating content that includes many scenes that evoke positive emotions.

[1606] Input: Viewer profile, prompt

[1607] Output: New, customized content

[1608] Step 5:

[1609] The server delivers the generated customized content to the device, which then provides the content to the viewer, collecting viewer engagement data (watching time, number of plays, reactions) and emotional data and sending it back to the server.

[1610] Input: Customized Content

[1611] Output: Audience engagement data, sentiment data

[1612] Step 6:

[1613] The server selects the most suitable advertisement based on the viewer profile and emotional data, and inserts it into the middle of the video. It also tracks the performance of the advertisement (click-through rate, completion rate), and optimizes the effectiveness of the advertisement based on the results. For example, it inserts a relevant advertisement immediately after a scene that evokes positive emotions.

[1614] Input: Viewer profile, emotional data

[1615] Output: Optimized Ad

[1616] Step 7:

[1617] The server collects feedback from viewers and updates viewer profiles, including information such as viewer preferences and opinions, to reflect in the next content generation.

[1618] Input: Viewer feedback, sentiment data

[1619] Output: Updated viewer profile

[1620] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1621] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1622] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1623] [Fourth embodiment]

[1624] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1625] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1626] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1627] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1628] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1629] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1630] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1631] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1632] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1633] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1634] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1635] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1636] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1637] Specific embodiments of the system related to the present invention will be described below.

[1638] System Overview

[1639] The system consists of three main components: a server, a terminal, and a user. It generates new content based on the provided material using generative AI, delivers customized content based on the viewer profile, optimizes advertising, and collects and updates feedback.

[1640] Program processing flow

[1641] Uploading and Preprocessing Materials

[1642] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality, and performs pre-processing such as encoding and trimming.

[1643] Examples:

[1644] Creators upload footage of natural scenery they have taken and the corresponding music files.

[1645] The server checks the video resolution and converts it into a unified format.

[1646] Generate audience profiles

[1647] The device collects the viewer's past viewing history, preferences, device information, etc. This information is sent to a server to create a viewer profile.

[1648] Examples:

[1649] The genre that users often watch is cooking-related videos.

[1650] The device records viewing history, sends it to the server, and creates a profile called "I love cooking."

[1651] Customized Content Generation

[1652] The server instructs the generative AI based on the viewer profile to generate new customized content.

[1653] Examples:

[1654] The server instructs the generation AI to add cooking scenes to travel footage for users who love cooking.

[1655] The generative AI generates scenes introducing famous local dishes to accompany travel footage.

[1656] Content distribution and engagement tracking

[1657] The server delivers the generated customized content to the device, and collects viewer engagement data (watch time, number of plays, reactions, etc.) to use for the next profile update.

[1658] Examples:

[1659] A server distributes travel videos including cooking scenes to users who love cooking.

[1660] The terminal collects the user's viewing data and sends it to the server.

[1661] Ad insertion and optimization

[1662] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the ad performance.

[1663] Examples:

[1664] The server selects organic food advertisements and inserts them into videos targeted at health-conscious viewers.

[1665] The server monitors the click-through rate and completion rate of the ads and makes appropriate adjustments.

[1666] Collecting feedback and updating your profile

[1667] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next video generation.

[1668] Examples:

[1669] The user provides feedback regarding preferences for particular effects and scenes.

[1670] The server adds this information to a profile and uses it the next time a video is generated.

[1671] conclusion

[1672] The system generates and delivers high-quality video content customized based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[1673] The processing flow will be explained below.

[1674] Step 1:

[1675] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[1676] Specific behavior:

[1677] The server checks the format of the uploaded file (JPEG, MP4, etc.).

[1678] The server verifies the video resolution and audio bit rate.

[1679] Step 2:

[1680] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[1681] Specific behavior:

[1682] The server converts all video to a uniform resolution.

[1683] The server processes the audio track for noise reduction.

[1684] Step 3:

[1685] The device collects the viewer's behavioral history, preferences, device information, etc. and sends it to the server, which then creates a viewer profile based on this information.

[1686] Specific behavior:

[1687] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[1688] The terminal transmits the collected data to the server.

[1689] The server generates a viewer profile based on the received data.

[1690] Step 4:

[1691] The server instructs the generative AI based on the viewer profile to generate new customized content.

[1692] Specific behavior:

[1693] The server uses the viewer profile to pass appropriate content generation parameters to the generation AI.

[1694] Generative AI generates new content by adding elements that viewers like to specific scenes in a video.

[1695] Step 5:

[1696] The server delivers the generated customized content to the device, which collects viewer engagement data (watch time, number of plays, reactions, etc.) and sends it back to the server.

[1697] Specific behavior:

[1698] The server transmits the customized content to the terminal in a streaming format.

[1699] The device collects viewer engagement data in real time.

[1700] The engagement data collected by the device is periodically sent to the server.

[1701] Step 6:

[1702] The server selects the best ads to insert into the video based on the viewer profile and video context, and also tracks and optimizes the performance of the ads.

[1703] Specific behavior:

[1704] The server analyzes the viewer profile and selects appropriate advertisements.

[1705] The server decides when to insert the advertisement during video playback.

[1706] The server tracks ad click rates and completion rates and uses that data to optimize ads.

[1707] Step 7:

[1708] The device collects feedback from viewers and sends it to the server, which uses this feedback to update the viewer profile and reflect it in the next content generation.

[1709] Specific behavior:

[1710] The device collects feedback provided by the viewer (e.g., "I like this scene").

[1711] The device sends the collected feedback to the server.

[1712] The server updates the viewer profile based on the feedback and reflects it in the parameters of the next generation AI.

[1713] Example 1

[1714] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1715] Video viewers often demand content that is individually customized based on their diverse preferences and viewing history. Streaming platforms are also required to insert appropriate advertisements and optimize their performance. However, conventional systems face challenges in automating these requirements, making it difficult and time-consuming.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1717] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a user profile based on the user's behavioral history and preferences, means for generating customized content based on the user profile, means for checking the format and quality of the material and encoding and trimming it, means for collecting user engagement data and updating the user profile, and means for delivering customized content to users. This enables the automatic generation and delivery of high-quality content customized based on user preferences.

[1718] (The system of claim 2)

[1719] The system further includes a means for inserting advertisements into the generated customized content and a means for tracking and optimizing the performance of the advertisements, thereby providing the most suitable advertisements to the viewer and maximizing their effectiveness.

[1720] "Generative AI" refers to AI that has the ability to generate new information and content based on diverse data.

[1721] A "user profile" is a database or collection of information constructed based on a user's behavioral history and preferences.

[1722] "Customized Content" means content that has been adapted or edited based on a specific user profile.

[1723] "Encoding" is the process of converting data into a particular format.

[1724] "Trimming" is the process of cutting out unnecessary parts of video or audio data.

[1725] "Engagement data" refers to behavioral data such as viewer viewing time, number of views, and reactions.

[1726] "Feedback" refers to opinions and ratings provided by users.

[1727] "Ad insertion" refers to the process of placing advertisements within video or audio content.

[1728] "Performance tracking" refers to the collection and analysis of data to measure and analyze the effectiveness of advertising and other activities.

[1729] "Optimization" is the process of adjusting or improving a system or process to maximize its efficiency or effectiveness.

[1730] MODE FOR CARRYING OUT THE INVENTION

[1731] The system related to this invention mainly consists of three elements: a server, a terminal, and a user. The system generates new content based on provided materials using a generative AI model, delivers customized content based on viewer profiles, and optimizes advertisements and collects and updates feedback.

[1732] server

[1733] The server manages everything from uploading materials to generating and distributing customized content. Specific processes include the following:

[1734] 1. Upload and pre-process materials

[1735] The server receives video, audio, and text data from creators. It checks the file format and quality, and encodes and trims it. For example, it uses the FFmpeg tool to convert and unify the format and quality of the material.

[1736] 2. Generate audience profiles

[1737] The server receives data on the viewer's behavior and preferences sent from the device, creates a viewer profile, and stores this data in a database (e.g., MySQL).

[1738] 3. Creating customized content

[1739] Based on the viewer profile, the server sends prompts to the AI ​​to generate customized content, such as "add cooking scenes to travel videos."

[1740] 4. Content distribution and engagement tracking

[1741] The server delivers the generated customized content to the device and collects viewer engagement data using data streaming technologies (e.g., RTMP, HLS).

[1742] 5. Ad insertion and optimization

[1743] The server selects and inserts appropriate ads based on viewer profile and video context, and also tracks and optimizes ad performance.

[1744] 6. Collect feedback and update your profile

[1745] The server collects feedback sent by the terminals and updates the viewer profile.

[1746] Terminal

[1747] The device collects the user's behavior history and sends it to the server. It also displays customized content and advertisements received from the server, collects viewing data and feedback, and sends them to the server.

[1748] User

[1749] Users can view content via their terminals and provide feedback, for example by entering their preferences and opinions about a particular scene in a feedback form.

[1750] Specific examples

[1751] For example, if a user frequently watches cooking-related videos, the device will collect that viewing history and send it to the server, which will then use that data to create a "cooking lover" profile and instruct the AI ​​to add cooking scenes to travel videos in the next customized content.

[1752] Prompt Sentence Examples

[1753] "For food-loving viewers, add a segment to your travel footage showcasing famous local dishes."

[1754] This system generates and distributes high-quality video content based on viewer profiles, optimizes advertising, and collects and updates feedback, thereby improving viewer satisfaction and increasing the value of the platform.

[1755] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1756] Program processing flow

[1757] Step 1. Upload and preprocess your materials

[1758] 1.1 Uploading Materials

[1759] Creators upload video footage, audio, and text data to the server, which triggers the action of selecting files through a web interface and pressing the upload button.

[1760] Input: Video material files, audio files, text files

[1761] Output: Original material data stored on the server

[1762] 1.2 File format and quality check

[1763] The server checks the format and quality of the uploaded file by using tools like FFmpeg to extract file information and verify things like format, resolution, bitrate, etc.

[1764] Input: Original material data stored on the server

[1765] Output: Format and quality verified material data

[1766] 1.3 Encoding and Trimming

[1767] The server encodes and trims the file as needed, for example converting the video file to the specified resolution and trimming unwanted parts.

[1768] Input: Format and quality verified material data

[1769] Output: Encoded and trimmed material data

[1770] Step 2. Generate your audience profile

[1771] 2.1 Collecting viewing history and preferences

[1772] The device collects information about the viewer's viewing history, preferred genres, and playback devices, including using browser and app cookies to record viewing data.

[1773] Input: Viewer viewing activity

[1774] Output: Collected viewing history data

[1775] 2.2 Sending profile data

[1776] The device sends the collected data to the server. The data is packetized in JSON format and sent via an HTTP POST request.

[1777] Input: Collected viewing history data

[1778] Output: Profile data sent to the server

[1779] 2.3 Creating an audience profile

[1780] The server creates a viewer profile based on the data it receives, and records the viewer's preferences and history in a database.

[1781] Input: Profile data sent to the server

[1782] Output: Viewer profile stored in a database

[1783] Step 3. Generate customized content

[1784] 3.1 Creating prompts for generative AI

[1785] The server creates a prompt to send to the AI ​​based on the viewer profile, such as "Add cooking scenes to travel videos for viewers who love cooking."

[1786] Input: Viewer profile stored in database

[1787] Output: Generated prompt statement

[1788] 3.2 Content Generation Instructions

[1789] The server sends a prompt to the generative AI model to instruct it to generate content. The prompt is sent to the generative AI using an API request.

[1790] Input: Generated prompt statement

[1791] Output: The prompt sent to the generative AI model

[1792] 3.3 Creating customized content

[1793] The generative AI generates new content based on the prompt and sends it back to the server, for example adding a cooking scene to a video and generating a text description.

[1794] Input: The prompt sent to the generative AI model

[1795] Output: Generated customization content

[1796] Step 4. Content distribution and engagement tracking

[1797] 4.1 Delivery of customized content

[1798] The server delivers the generated customized content to the terminal. The content is delivered using data streaming technology.

[1799] Input: Generated customization content

[1800] Output: Content delivered to the device

[1801] 4.2 Collection of viewing data

[1802] The device collects viewer engagement data (watch time, plays, reactions, etc.) and records this data using JavaScript code.

[1803] Input: Content delivered to the device

[1804] Output: Collected engagement data

[1805] 4.3 Transmission of Collected Data

[1806] The device sends engagement data to the server via an HTTP request for analysis.

[1807] Input: Collected engagement data

[1808] Output: Engagement data sent to the server

[1809] Step 5. Ad insertion and optimization

[1810] 5.1 Ad Selection

[1811] The server chooses the best ad based on the viewer profile and video context, selecting the appropriate ad from an ad database.

[1812] Input: Viewer profile stored in database

[1813] Output: Selected ads

[1814] 5.2 Ad Insertion

[1815] The server inserts the selected advertisement into the video at the specified position. The advertisement is then integrated into the video using a video editing tool.

[1816] Input: Selected ads, customized content

[1817] Output: Customized content with ad insertion

[1818] 5.3 Tracking Ad Performance

[1819] The server monitors the click-through rate and completion rate of ads and automatically adjusts the optimal ads. Performance data is collected using analytical tools and optimized by algorithms.

[1820] Input: Ad-inserted personalized content, viewing data

[1821] Output: Optimized ad placement

[1822] Step 6. Collect feedback and update your profile

[1823] 6.1 Feedback Collection

[1824] The device collects feedback from viewers, which involves viewers entering their opinions and thoughts through a form in their browser or app.

[1825] Input: Viewer feedback

[1826] Output: Collected feedback data

[1827] 6.2 Sending Feedback

[1828] The device sends the collected feedback to the server via an HTTP POST request.

[1829] Input: Collected feedback data

[1830] Output: Feedback data sent to the server

[1831] 6.3 Updating your profile

[1832] The server updates the viewer profile based on the feedback, updating the profile information in the database and reflecting it in the next content generation.

[1833] Input: Feedback data sent to the server

[1834] Output: Updated viewer profile

[1835] (Application example 1)

[1836] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1837] Conventional content distribution systems lack sufficient customization based on individual viewer preferences and behavioral history, and lack complete feedback and engagement tracking to improve the quality of the viewing experience. Furthermore, they are unable to select optimal ads based on viewer profiles and track their effectiveness, making it difficult to maximize advertising effectiveness. This results in insufficient improvement in viewer satisfaction and the value of the platform.

[1838] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1839] In this invention, the server includes means for generating new content based on materials provided by the generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for delivering the customized content to the viewer, means for collecting viewer engagement data and using it for the next profile update, means for inserting advertisements into the generated customized content, means for tracking the performance of the advertisements, means for selecting optimal advertisements based on the viewer profile and video context, means for collecting feedback from the viewer and updating the viewer profile, and means for reflecting the collected feedback in the profile and using it for generating the next video. This enables the generation and delivery of high-quality customized content based on the viewer's preferences and behavioral history, maximizing the effectiveness of advertisements and improving viewer satisfaction and the value of the platform.

[1840] "Generative AI" is an AI technology that generates new information and content based on input data.

[1841] "Materials" refers to the basic data for generating content, such as video, audio, and text data provided by creators.

[1842] "Customized content" is individual content that is generated based on a viewer profile and tailored to the viewer's preferences and behavioral history.

[1843] A "viewer profile" refers to a data structure that indicates the characteristics of a viewer, constructed by analyzing data related to the viewer's behavioral history and preferences.

[1844] "Viewer engagement data" refers to behavioral data of viewers when they watch content, including viewing time, number of plays, reactions, etc.

[1845] "Feedback" refers to information such as reactions, opinions, and ratings from viewers, which is used to generate next content and update profiles.

[1846] "Ad performance" refers to metrics that evaluate how effectively an ad works for viewers, including click-through rate and completion rate.

[1847] "Video context" refers to the background information surrounding the video, such as the content, theme, and scene situation of the video being generated.

[1848] As an embodiment of the present invention, a specific system configuration and its operation will be described below.

[1849] System configuration

[1850] Server: A central computer system that processes generative AI models, creates and updates audience profiles, collects and analyzes engagement data, and optimizes and tracks advertising effectiveness.

[1851] Terminal: A device used by viewers, such as a smartphone, tablet, or smart glasses, that collects viewers' behavioral history and transmits it to a server.

[1852] Users: Viewers and creators who use the system. Creators upload materials such as videos and audio, and viewers watch customized content.

[1853] Program processing

[1854] Uploading and Preprocessing Materials

[1855] Creators upload video footage, audio, text data, etc. to the server, which then uses OpenCV to divide the footage into frames, check the quality, encode, and trim as needed.

[1856] Generate audience profiles

[1857] The device collects information such as the viewer's past viewing history, preferences, and device information, and sends it to the server, which then uses Scikit-learn's KMeans clustering to generate a viewer profile.

[1858] Customized Content Generation

[1859] Based on the generated viewer profile, the server invokes a generative AI model using TensorFlow to generate customized content, adding elements that match the profile to the content in the process.

[1860] Content distribution and engagement tracking

[1861] The server delivers the generated customized content to the device, and the device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server.

[1862] Ad insertion and optimization

[1863] The server selects the best ads based on viewer profile and video context, inserts them within the customized content, and tracks and optimizes ad performance (e.g., click-through rate and completion rate).

[1864] Collecting feedback and updating your profile

[1865] The device collects feedback from the viewer and sends it to the server, which uses this feedback to update the viewer profile and generate customized content for the next viewing.

[1866] Specific examples

[1867] As a concrete example, consider the case where a user who loves cooking uses this system on a smartphone. The user uses the app just like a regular video streaming app. Based on the user's past viewing history, the server generates a profile for the user called "cooking lover" and provides new cooking-related content. For example, based on the user's profile, the server can generate and distribute new travel videos including Chinese food recipe videos.

[1868] Prompt Sentence Examples

[1869] "User profile: Loves cooking. Viewing genre: Chinese food. Content to generate: Generate travel footage that includes Chinese food recipes."

[1870] In this way, the system can efficiently generate and deliver high-quality customized content tailored to viewer preferences and behavioral history, while optimizing advertising effectiveness, thereby improving viewer satisfaction and enhancing the value of the platform.

[1871] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1872] Step 1:

[1873] Creators upload video footage, audio, text data, etc. to the server. The input is the content material created by the creator, and the output is when these materials are saved on the server and prepared for further processing. The server uses OpenCV to divide the video footage into frames, perform quality checks, encode, and trim as necessary.

[1874] Step 2:

[1875] The device collects the viewer's past viewing history, preferences, and device information, and sends it to the server. The input is the viewer's behavioral history and device information, and the output is this information sent to the server. The server generates a viewer profile using Scikit-learn's KMeans clustering. At this stage, the viewer's preferences and behavioral patterns are analyzed and a profile is constructed.

[1876] Step 3:

[1877] The server uses TensorFlow to call a generative AI model based on the generated viewer profile to generate customized content. The input is the viewer profile and uploaded materials, and the output is the customized content. The generative AI model adds elements that match the profile to the content.

[1878] Step 4:

[1879] The server delivers the generated customized content to the device. The device collects viewer engagement data (viewing time, number of plays, reactions, etc.) and sends it to the server. The input is the customized content and the viewer's reactions, and the output is viewing data and engagement data. The collected engagement data is reflected in the next content generation.

[1880] Step 5:

[1881] The server selects the optimal ad based on the viewer profile and video context and inserts the ad into the customized content. The input is the viewer profile, content context, and candidate ads, and the output is the customized content with the ad inserted. In addition, the server tracks the ad performance (click-through rate, completion rate, etc.) and performs optimization to maximize the advertising effect.

[1882] Step 6:

[1883] The device collects feedback from viewers and sends it to the server. The server updates the viewer profile based on this feedback and uses it to generate the next customized content. The input is viewer feedback, and the output is an updated viewer profile. Based on the information gained from the feedback, the quality of the next content provided can be further improved.

[1884] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1885] Specific embodiments of the system related to the present invention will be described below.

[1886] System Overview

[1887] This system combines three main elements - the server, the device, and the user - with an emotion engine that recognizes the user's emotions to provide a more personalized viewing experience. It generates new content based on the materials provided by the generative AI, delivers customized content based on the viewer profile, and optimizes advertising using emotional data.

[1888] Program processing flow

[1889] Uploading and Preprocessing Materials

[1890] Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the material, and performs pre-processing such as encoding and trimming.

[1891] Examples:

[1892] Creators upload footage of natural scenery they have taken and the corresponding music files.

[1893] The server checks the video resolution and converts it into a unified format.

[1894] Collecting Emotional Data

[1895] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[1896] Examples:

[1897] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[1898] The device sends the analysis results to the server.

[1899] Generate audience profiles

[1900] The device collects the viewer's behavioral history, preferences, device information, as well as emotional data, and sends it to the server, which then creates a viewer profile based on this information.

[1901] Examples:

[1902] The device collects the viewer's past viewing data and emotional data and sends it to the server.

[1903] The server analyzes scenes that elicit a positive emotional response and adds the viewer's "preferences" to their profile.

[1904] Customized Content Generation

[1905] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account emotional data as well.

[1906] Examples:

[1907] The server gives instructions to the generation AI, focusing on scenes that make the server feel happy.

[1908] Generative AI generates new content that includes many scenes that evoke positive emotions.

[1909] Content distribution and engagement tracking

[1910] The server delivers the generated customized content to the device, which collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends it back to the server.

[1911] Examples:

[1912] The server transmits the customized content to the terminal in streaming format.

[1913] The device collects viewer engagement data and emotional reactions and sends them to a server.

[1914] Ad insertion and optimization

[1915] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[1916] Examples:

[1917] The server inserts relevant advertisements immediately after scenes that elicit positive emotions.

[1918] The server monitors the click-through rate and completion rate of the advertisements to optimize the effectiveness of the advertisements.

[1919] Collecting feedback and updating your profile

[1920] The device collects feedback from viewers and sends it to the server, which updates the viewer profile based on this feedback and emotional data and reflects it in the next content generation.

[1921] Examples:

[1922] Users provide feedback on their favorite scenes and effects.

[1923] The server uses the feedback and emotional data to update the viewer profile and use it to generate the next piece of content.

[1924] This enables the system to provide highly personalized video content and optimize advertising in response to viewer emotions, improving the viewing experience and increasing the value of the platform.

[1925] The processing flow will be explained below.

[1926] Step 1:

[1927] Creators upload video footage, audio, and text data to the server, which receives it and checks the format and quality of the material.

[1928] Specific behavior:

[1929] Creators upload footage of natural scenery they have taken and the corresponding music files to a server.

[1930] The server checks the format of the uploaded file (JPEG, MP4, etc.) and verifies the video resolution and audio bitrate.

[1931] Step 2:

[1932] The server performs pre-processing such as encoding and trimming of the uploaded material as necessary.

[1933] Specific behavior:

[1934] The server converts all video files to a uniform resolution.

[1935] The server processes the audio track for noise reduction.

[1936] Step 3:

[1937] While the viewer is watching the video, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice and transmits it to the server.

[1938] Specific behavior:

[1939] The device captures the viewer's face with a camera and uses facial recognition technology to analyze emotions (e.g., joy, surprise, sadness, etc.).

[1940] The device uses a microphone to analyze the tone and intensity of the viewer's voice.

[1941] The device sends the analysis results to the server.

[1942] Step 4:

[1943] The device also collects the viewer's behavioral history, preferences, and device information, and sends this data to the server, which then creates a viewer profile based on this information.

[1944] Specific behavior:

[1945] The device collects the viewer's past viewing data (playback time, genre of viewed content, etc.).

[1946] Your device collects device information (e.g., the type of browser and device you are using).

[1947] The server generates a viewer profile based on the data received, which also includes emotional data.

[1948] Step 5:

[1949] The server then directs the generative AI to generate new, customized content based on the viewer profile, taking into account the collected emotional data.

[1950] Specific behavior:

[1951] The server gives instructions to the generation AI, placing emphasis on scenes that elicit strong emotional responses of "surprise" and "joy."

[1952] Generative AI takes into account the viewer's emotions and generates new content that contains many elements that viewers will respond to positively.

[1953] Step 6:

[1954] The server delivers the generated customized content to the device, and the device collects viewer engagement data (watching time, number of plays, reactions) and emotional data and sends them to the server.

[1955] Specific behavior:

[1956] The server transmits the customized content to the terminal in streaming format.

[1957] The device continues to monitor the viewer's emotional response while the video is playing.

[1958] The device periodically transmits viewing data and emotion data to the server.

[1959] Step 7:

[1960] The server selects the best ads to insert into the video based on viewer profile and emotional data, and also tracks and optimizes the performance of the ads.

[1961] Specific behavior:

[1962] The server selects relevant advertisements immediately following scenes that elicit positive emotions.

[1963] The server inserts the ads at the appropriate times within the video.

[1964] The server tracks ad click rates and completion rates, and uses that data to optimize the effectiveness of the ads.

[1965] Step 8:

[1966] The device collects feedback from viewers and sends it to the server, which uses this feedback and emotional data to update the viewer profile and reflect it in the next content generation.

[1967] Specific behavior:

[1968] Users provide feedback on their impressions of specific scenes and effects.

[1969] The device sends the collected feedback to the server.

[1970] The server updates the viewer profile based on the collected feedback and sentiment data and reflects it in the next content generation.

[1971] Example 2

[1972] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1973] Conventional content generation systems do not adequately personalize content based on viewer emotions and behavioral history, making it difficult to improve viewer satisfaction and deliver advertisements effectively. Furthermore, they lack a mechanism for effectively utilizing post-viewing feedback and reflecting it in the next content generation. A new system that can solve these issues and provide viewers with highly personalized content and an optimal advertising experience is needed.

[1974] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1975] In this invention, the server includes means for preprocessing materials uploaded by creators, means for collecting and analyzing viewer emotion data, means for generating new content based on generative artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and emotion data, means for generating customized content based on the viewer profile, and means for delivering the customized content to the viewer. This enables advanced personalization that utilizes the viewer's emotion and behavioral history, and optimizes the viewing experience and advertising experience.

[1976] "Creator" refers to a person who creates content and uploads it to the system.

[1977] "Materials" refers to the data that forms the basis of content, such as video, audio, and text data.

[1978] "Preprocessing" refers to the process of checking the format and quality of the uploaded material and performing encoding, trimming, etc.

[1979] "Emotional data" refers to data related to emotions analyzed from viewers' facial expressions and voices.

[1980] A "viewer profile" refers to information that reflects a viewer's preferences, etc., and is generated based on the viewer's behavioral history and emotional data.

[1981] "Generative AI" refers to AI technology that generates new content based on viewer profiles, emotional data, etc.

[1982] "Content" refers to information such as video, audio, and text that viewers can view.

[1983] "Customized Content" refers to content that is individually generated based on a viewer profile.

[1984] "Engagement data" refers to data about viewers' viewing behavior, such as viewing time, number of views, and reactions.

[1985] "Advertising" means any promotional message or advertising content displayed to a viewer.

[1986] "Ad performance" refers to data used to measure the effectiveness of an ad, such as click-through rate and completion rate.

[1987] "Feedback" refers to the impressions and opinions that viewers provide after watching a program.

[1988] The system of the present invention functions mainly through a server, a terminal, a creator, and a user. The roles of each are explained below.

[1989] Hardware and Software Configuration

[1990] 1. Server: This plays a central role in the system. It has high-performance processing capabilities and is ideally suited to using a cloud server for data storage, data analysis, and implementing generative AI models. Specifically, Amazon Web Services (AWS), Google Cloud Platform (GCP), Microsoft Azure, etc. are suitable.

[1991] 2. Device: A device used by the user to view and provide feedback. This can be a smartphone, tablet, or PC. The device has a built-in camera and microphone, and an application to run the emotion engine is installed.

[1992] 3. Creator: An entity that creates content and uploads it to the system. They create content using image editing software (e.g., Adobe Premiere, Final Cut Pro) and audio editing software.

[1993] Content Creation and Customization Process

[1994] The system of the present invention operates in the following manner.

[1995] 1. Upload and pre-process materials:

[1996] Creators upload video footage, audio, and text data to the server using a web application interface.

[1997] The server checks the format and quality of the received material and performs pre-processing such as encoding and trimming. Specifically, it performs encoding to unify the resolution of the video files.

[1998] 2. Collecting Emotional Data:

[1999] The device uses a built-in camera and microphone to capture the viewer's facial expressions and voice while they are watching the video, and then analyzes them in real time using an emotion engine.

[2000] The device transmits the analysis result, emotional data, to the server. This data includes the viewer's emotional response (e.g., joy, surprise, sadness, etc.).

[2001] 3. Generate audience profiles:

[2002] The terminal records the viewer's behavioral history and device information and sends this to the server.

[2003] The server generates a viewer profile based on the received emotion data and behavioral history data, integrates this information into the database, and updates the profile information.

[2004] 4. Generate customized content:

[2005] The server issues instructions to the generation AI based on the viewer profile to generate new customized content.

[2006] An example of a specific prompt would be, "Generate a new video of a natural scene, focusing on a scene that the viewer found pleasurable."

[2007] The generative AI model generates new content based on this prompt.

[2008] 5. Content distribution and engagement tracking:

[2009] The server distributes the generated customized content to the terminal in streaming format.

[2010] The device collects engagement data such as viewer viewing time, number of plays, and reactions, and sends this back to the server.

[2011] 6. Ad Insertion and Optimization:

[2012] The server selects the most suitable advertisement based on the viewer profile and emotional data and inserts it into the video.

[2013] The server tracks and optimizes the performance of ads, specifically by monitoring click-through rates and completion rates.

[2014] 7. Collecting Feedback and Updating Your Profile:

[2015] The terminal collects feedback from the viewer and sends it to the server.

[2016] The server updates the viewer profile based on the feedback and emotional data and reflects it in the next content generation.

[2017] By implementing the above steps, the system of the present invention can achieve advanced personalization based on the viewer's emotions and behavioral history, optimizing the viewing experience and advertising experience. This system allows users to enjoy more attractive and personalized content, and maximizes the effectiveness of advertising.

[2018] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2019] Step 1:

[2020] Creators upload materials.

[2021] Input: Video material, audio, and text data created by creators.

[2022] Output: Material data uploaded and stored on the server.

[2023] How it works: When a creator accesses the upload page of the web application, selects a local file, and presses the "Upload" button, the material data is transferred to the server, which receives it and stores it in storage.

[2024] Step 2:

[2025] The server pre-processes the material.

[2026] Input: Video footage, audio, and text data uploaded by creators.

[2027] Output: Material data that has been encoded, trimmed, and converted into a unified format.

[2028] What it does: The server checks the format and quality of the received material, encodes the video files to a specific resolution, standardizes the format of the audio files, trims and removes unnecessary parts, and converts the material to a standardized format.

[2029] Step 3:

[2030] The device captures the viewer's emotional data.

[2031] Input: The viewer's facial expressions and voice, captured by the device's built-in camera and microphone.

[2032] Output: Viewer sentiment data analyzed by the sentiment engine.

[2033] How it works: While the viewer is watching the video, the device automatically activates the camera and microphone to capture the viewer's face and voice, and performs real-time facial analysis to generate emotional data such as joy or surprise.

[2034] Step 4:

[2035] The device transmits the emotion data to the server.

[2036] Input: Viewer sentiment data analyzed on the device.

[2037] Output: The emotion data is transferred to the server and stored in a database.

[2038] Specific operation: The device generates emotion data as data packets at regular intervals and sends them to the server via HTTP requests. The server then records the received data in a database.

[2039] Step 5:

[2040] The terminal transmits the viewing data to the server.

[2041] Input: Viewer behavior history, device information.

[2042] Output: Viewing data sent to the server and stored in a database.

[2043] Specific operation: The terminal records the viewer's behavior history (viewing start time, end time, scenes viewed) and device information, and sends this to the server in batch processing.

[2044] Step 6:

[2045] The server generates a viewer profile.

[2046] Input: Emotion data and viewing data sent from the device.

[2047] Output: The created audience profile.

[2048] Specific operation: Based on the received data, the server analyzes the viewer's preferences and behavioral patterns and stores the viewer profile in a database.

[2049] Step 7:

[2050] The server gives instructions to the generated AI.

[2051] Input: Audience profile.

[2052] Output: The prompt sent to the generation AI.

[2053] Specific operation: The server analyzes the viewer profile and generates a prompt to instruct the AI ​​to generate new content. An example of a prompt is "Generate a new nature video centered around scenes that the viewer found enjoyable."

[2054] Step 8:

[2055] Generative AI generates customized content.

[2056] Input: The prompt text received from the server.

[2057] Output: The generated customization content.

[2058] How it works: Based on the prompt, the generative AI model generates new content that matches the preferences of the viewer profile.

[2059] Step 9:

[2060] The server delivers the customized content.

[2061] Input: Generated customization content.

[2062] Output: Streamed content delivered to a device.

[2063] Specific operation: The server generates a content URL and sends it to the device to start viewing.

[2064] Step 10:

[2065] The device collects engagement data.

[2066] Input: Viewer viewing behavior data.

[2067] Output: Collected engagement data is sent to a server.

[2068] Specific operation: The device tracks viewing time, number of plays, reactions, etc. and sends the data to the server in batches.

[2069] Step 11:

[2070] The server selects and inserts the advertisements.

[2071] Input: Audience profile, sentiment data.

[2072] Output: Ads inserted into the generated content.

[2073] Specific operation: The server selects the most suitable advertisement based on the profile and emotional data, analyzes the appropriate insertion point in the video, and inserts the advertisement.

[2074] Step 12:

[2075] The server tracks and optimizes the performance of the ads.

[2076] Input: Ad click-through rate, view completion rate.

[2077] Output: Optimization of advertising effectiveness.

[2078] Specific operation: The server aggregates ad performance data and updates the ad selection logic using machine learning algorithms.

[2079] Step 13:

[2080] The device collects viewer feedback.

[2081] Input: Feedback provided by the audience.

[2082] Output: Feedback sent to server and stored in database.

[2083] Specific operation: The device collects feedback from the user through an in-app feedback form and sends it to the server.

[2084] Step 14:

[2085] The server updates the profile based on the feedback.

[2086] Input: Audience feedback, sentiment data.

[2087] Output: Updated audience profile.

[2088] What it does: The server analyzes the feedback and sentiment data, updates the viewer profile in a database, and uses it to generate the next piece of content.

[2089] (Application example 2)

[2090] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2091] Conventional content distribution services provide content customized based on a viewer's behavioral history and preferences, but do not take into account the viewer's real-time emotions. This makes it difficult to deliver content that optimally responds to the viewer's emotions, and the improvement of the viewing experience and optimization of advertising have not been fully achieved. Therefore, the objective of the present invention is to provide a system that recognizes viewer emotions in real time, generates customized content based on that data, and further optimizes the effectiveness of advertising.

[2092] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2093] In this invention, the server includes means for generating new content based on materials provided by the generation artificial intelligence, means for creating a viewer profile based on the viewer's behavioral history and preferences, means for generating customized content based on the viewer profile, means for collecting viewer emotion data using an emotion engine, means for generating customized content taking the emotion data into consideration, and means for delivering the customized content to the viewer. This makes it possible to provide highly personalized video content and optimize advertisements according to the viewer's emotions.

[2094] A "server" is a computer system that provides services to other computers (clients) on a network.

[2095] A "viewer profile" is personal information constructed based on a viewer's behavioral history, preferences, and emotional data.

[2096] "Generative AI" refers to AI that learns from large amounts of data and performs tasks such as natural language generation and image generation.

[2097] An "emotion engine" is software or hardware that analyzes emotions from viewers' facial expressions, voice, behavior, etc.

[2098] "Customized Content" means video and audio content that is generated and optimized for a viewer based on viewer profile and emotional data.

[2099] "Advertising optimization" refers to the process of delivering advertisements with optimal timing and content based on viewer profiles and emotional data, thereby maximizing their effectiveness.

[2100] "Feedback" means information such as preferences and opinions provided by viewers that is used to improve profiles and content offerings.

[2101] A specific embodiment of a system for carrying out the present invention will be described below. The system is composed of a server, terminals, and users, and is provided with various means for realizing the contents of the invention.

[2102] Hardware and software used

[2103] server:

[2104] Software: Generative AI (generative AI model), database, web server

[2105] Role: Pre-processing material, generating audience profiles, generating and delivering customized content, inserting and optimizing ads, collecting feedback and updating profiles

[2106] Device:

[2107] Hardware: Camera, microphone, display

[2108] Software: Emotion recognition engine (e.g., facial expression recognition model)

[2109] Role: Collect emotional data, watch customized content, send emotional data and feedback

[2110] User:

[2111] Interact: Watch content, provide feedback

[2112] System flow and processing

[2113] The server generates new content from the materials provided by the generative AI. Creators upload video footage, audio, and text data to the server, which receives it, checks the format and quality of the materials, and performs pre-processing such as encoding and trimming. At this stage, the content provided by creators might be, for example, footage of natural scenery and music files.

[2114] Next, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice while they are watching the content. This emotional data is analyzed using facial expression recognition technology to identify various emotions such as joy, surprise, and sadness. The collected emotional data is then sent to a server.

[2115] The server analyzes this emotional data along with the viewer's behavioral history, preferences, and device information to generate a viewer profile. For example, it can identify scenes from past viewing data that evoked positive emotional responses and add them to the viewer's profile as "preferences."

[2116] Based on the viewer profile, the server issues instructions to the AI ​​to generate new customized content, taking into account emotional data. For example, the server can instruct the AI ​​to prioritize scenes that make viewers feel happy, generating new content that includes many scenes that evoke positive emotions.

[2117] The generated customized content is delivered from the server to the device, where the viewer watches it. The device collects the viewer's engagement data (viewing time, number of plays, reactions) and emotional data and sends it back to the server.

[2118] The server selects the most suitable advertisements based on viewer profiles and emotional data and inserts them into the video. It also tracks and optimizes the advertisement performance (click-through rate, viewing completion rate). For example, it can insert a relevant advertisement immediately after a scene that evokes positive emotions, increasing the effectiveness of the advertisement.

[2119] It collects feedback from viewers and updates their profiles, including their preferences and opinions, which the server then incorporates into the next content generation.

[2120] Specific examples

[2121] This is an example of a generative AI model generating new content based on a provided prompt.

[2122] Example prompt sentence:

[2123] "The input video contains multiple scenes, each with elements that elicit different emotional responses. Please generate a new video by focusing on scenes in which viewers express the 'Happy' emotion. Also, please add upbeat music and fun characters as additional elements to elicit the 'Happy' emotion."

[2124] In this way, a highly personalized viewing experience based on the viewer's emotions can be provided.

[2125] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2126] Step 1:

[2127] The server generates new content based on the materials provided by artificial intelligence. Specifically, creators upload video material, audio, and text data to the server, which then receives it and performs pre-processing (format and quality checks, encoding, trimming).

[2128] Input: Video material, audio, text data

[2129] Output: Encoded and trimmed material

[2130] Step 2:

[2131] While the viewer is watching the content, the device uses an emotion engine to collect emotional data from the viewer's facial expressions and voice. For example, the device's camera captures the viewer's face and uses facial recognition technology to analyze their emotions (happiness, surprise, sadness, etc.). The analysis results are sent to the server.

[2132] Input: Viewer's facial expressions and voice

[2133] Output: Parsed emotion data

[2134] Step 3:

[2135] The server analyzes the viewer's behavioral history, preferences, device information, and emotional data to generate a viewer profile. Based on the viewing and emotional data, scenes that evoked a positive emotional response are identified and added to the profile as the viewer's "preferences."

[2136] Input: Viewer behavior history, preferences, device information, emotional data

[2137] Output: Audience profile

[2138] Step 4:

[2139] The server issues prompts to the AI ​​based on the viewer profile to generate new customized content. It also takes into account emotional data, and instructs the AI ​​to prioritize scenes that elicit positive emotions, for example, generating content that includes many scenes that evoke positive emotions.

[2140] Input: Viewer profile, prompt

[2141] Output: New, customized content

[2142] Step 5:

[2143] The server delivers the generated customized content to the device, which then provides the content to the viewer, collecting viewer engagement data (watching time, number of plays, reactions) and emotional data and sending it back to the server.

[2144] Input: Customized Content

[2145] Output: Audience engagement data, sentiment data

[2146] Step 6:

[2147] The server selects the most suitable advertisement based on the viewer profile and emotional data, and inserts it into the middle of the video. It also tracks the performance of the advertisement (click-through rate, completion rate), and optimizes the effectiveness of the advertisement based on the results. For example, it inserts a relevant advertisement immediately after a scene that evokes positive emotions.

[2148] Input: Viewer profile, emotional data

[2149] Output: Optimized Ad

[2150] Step 7:

[2151] The server collects feedback from viewers and updates viewer profiles, including information such as viewer preferences and opinions, to reflect in the next content generation.

[2152] Input: Viewer feedback, sentiment data

[2153] Output: Updated viewer profile

[2154] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2155] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2156] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2157] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2158] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2159] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2160] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2161] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2162] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2163] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2164] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2165] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2166] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2167] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2168] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2169] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2170] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2171] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2172] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2173] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2174] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2175] The following is further disclosed regarding the above embodiment.

[2176] (Claim 1)

[2177] A means for generating new content based on the provided material by a generative artificial intelligence;

[2178] a means for creating viewer profiles based on viewer behavior history and preferences;

[2179] means for generating customized content based on an audience profile;

[2180] A means of delivering customized content to your audience

[2181] A system including:

[2182] (Claim 2)

[2183] means for inserting advertisements into the generated customized content;

[2184] A means to track advertising performance and

[2185] The system of claim 1 further comprising:

[2186] (Claim 3)

[2187] 10. The system of claim 1, further comprising means for collecting viewer feedback and updating viewer profiles.

[2188] "Example 1"

[2189] (Claim 1)

[2190] A means for generating new content based on the provided material by a generative artificial intelligence;

[2191] A means for creating a user profile based on the user's behavioral history and preferences;

[2192] means for generating customized content based on a user profile;

[2193] A means to check the format and quality of the material, and to encode and trim it.

[2194] a means for collecting user engagement data and updating user profiles;

[2195] a means for delivering customized content to users;

[2196] A system including:

[2197] (Claim 2)

[2198] means for inserting advertisements into the generated customized content;

[2199] A means to track and optimize advertising performance;

[2200] The system of claim 1 further comprising:

[2201] (Claim 3)

[2202] 10. The system of claim 1, further comprising means for collecting feedback from users and updating user profiles.

[2203] "Application Example 1"

[2204] (Claim 1)

[2205] A means for generating new content based on the provided material by a generative artificial intelligence;

[2206] a means for creating viewer profiles based on viewer behavior history and preferences;

[2207] means for generating customized content based on an audience profile;

[2208] a means for delivering customized content to viewers;

[2209] A way to collect viewer engagement data and use it for your next profile update,

[2210] A system including:

[2211] (Claim 2)

[2212] means for inserting advertisements into the generated customized content;

[2213] a means of tracking the performance of your ads;

[2214] A means to select the best ads based on viewer profile and video context;

[2215] The system of claim 1 further comprising:

[2216] (Claim 3)

[2217] a means of collecting viewer feedback and updating viewer profiles;

[2218] The collected feedback is reflected in the profile and used to generate the next video.

[2219] 10. The system of claim 1, comprising:

[2220] "Example 2: Combining Emotion Engines"

[2221] (Claim 1)

[2222] A means to pre-process materials uploaded by creators,

[2223] A means for collecting and analyzing viewer emotional data;

[2224] means for generating new content based on generative artificial intelligence;

[2225] means for creating a viewer profile based on the viewer's behavioral history and emotional data;

[2226] means for generating customized content based on an audience profile;

[2227] A means of delivering customized content to your audience

[2228] A system including:

[2229] (Claim 2)

[2230] means for inserting advertisements into the generated customized content;

[2231] a means to track and optimize advertising performance;

[2232] The system of claim 1 further comprising:

[2233] (Claim 3)

[2234] Includes a means to collect viewer engagement data and update viewer profiles

[2235] 10. The system of claim 1.

[2236] "Application example 2 when combining emotion engines"

[2237] (Claim 1)

[2238] A means for generating new content based on the provided material by a generative artificial intelligence;

[2239] a means for creating viewer profiles based on viewer behavior history and preferences;

[2240] means for generating customized content based on an audience profile;

[2241] a means for collecting viewer sentiment data using a sentiment engine;

[2242] means for generating customized content taking into account the emotion data;

[2243] a means for delivering customized content to viewers;

[2244] A system including:

[2245] (Claim 2)

[2246] means for inserting advertisements into the generated customized content;

[2247] a means of tracking the performance of your ads;

[2248] a means for optimizing advertising based on emotion data;

[2249] The system of claim 1 further comprising:

[2250] (Claim 3)

[2251] a means of collecting viewer feedback and updating viewer profiles;

[2252] 10. The system of claim 1, further comprising means for updating a viewer profile based on the emotion data. [Explanation of symbols]

[2253] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for generating new content based on the provided material by a generative artificial intelligence; a means for creating viewer profiles based on viewer behavior history and preferences; means for generating customized content based on an audience profile; A means of delivering customized content to your audience A system including:

2. means for inserting advertisements into the generated customized content; A means to track advertising performance and The system of claim 1 further comprising:

3. 10. The system of claim 1, further comprising means for collecting viewer feedback and updating viewer profiles.

Images