System

Abstract

A system is provided.SOLUTION: A system comprising: emotion analysis means; real-time video analysis means; highlight scene identification means; automatic editing means; collaborative editing means; video approval means; video sharing means; monthly subscription processing means; advertisement linking means; and revenue sharing means.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Video creators and event organizers face the problem of having to spend a lot of time and effort producing video content. For busy people in their late 20s to 40s, manually editing videos can be a significant burden, which can discourage them from engaging in creative activities. Furthermore, there are limited opportunities for viewers to share and edit videos in real time, and there is a lack of ways to encourage interactive participation. Furthermore, limited monetization options make it difficult for creators to continue producing videos sustainably. A system that solves these issues is needed. [Means for solving the problem]

[0005] The present invention is a system that includes a sentiment analysis means, a real-time video analysis means, a highlight scene identification means, an automatic editing means, a collaborative editing means, a video approval means, a video sharing means, a monthly subscription processing means, an advertising integration means, and a revenue sharing means. In this system, users upload live event or streaming video footage they film to a server, which performs real-time video analysis and sentiment analysis to automatically identify highlight scenes. The system then automatically edits the video using an AI algorithm and collects collaborative editing suggestions from viewers. Content providers review and approve these suggestions, and the video is finally shared on the streaming platform. As a revenue model, a professional plan monthly subscription, advertising integration, and revenue sharing are introduced, allowing creators to efficiently create high-quality videos and earn sustainable revenue. This system significantly improves the efficiency of video production and promotes interactive participation from viewers.

[0006] "Emotion analysis means" is a technology that analyzes the facial expressions and tone of voice of people in a video to identify their emotional state.

[0007] "Real-time video analysis means" is a technology for analyzing video data in real time and detecting specific events or scenes.

[0008] The "highlight scene identification means" is a technology for automatically selecting particularly important scenes or scenes with emotional excitement from video data.

[0009] "Automatic editing method" is a technology that uses an AI algorithm to automatically edit selected highlight scenes and optimally arrange video transitions, effects, and background music.

[0010] "Collaborative editing means" is a technology that allows viewers to participate in editing of video via a server and propose their own opinions and effects.

[0011] "Video approval means" refers to a technology that allows content providers to review and approve video editing suggestions from viewers.

[0012] "Video Sharing Means" means the technology used to share the final approved video content on a streaming platform.

[0013] "Monthly subscription processing means" means technology that allows users to subscribe to a monthly subscription plan and manage recurring payments.

[0014] "Advertising collaboration means" refers to technology that inserts advertisements into video content and enables revenue collaboration with advertising partners.

[0015] "Revenue sharing vehicle" means a technology for sharing generated revenue between users and platform operators. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming video shot by users. The specific processing content and operation of the program are described below.

[0038] 1. Users shoot and upload footage

[0039] Users capture footage of live events using their smartphones or cameras.

[0040] The device uploads the captured video to a server via an application.

[0041] The server receives the uploaded video data and stores it in cloud storage.

[0042] Examples:

[0043] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[0044] 2. Real-time video analysis

[0045] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[0046] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[0047] The server identifies important scenes based on the analysis results.

[0048] Examples:

[0049] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[0050] 3. Automatic Editing

[0051] The server performs automatic editing based on the identified highlight scenes.

[0052] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[0053] Examples:

[0054] The server edits the video based on the identified exciting scenes and applies transitions and effects to generate videos that will attract the viewer's attention.

[0055] 4. Collaborative editing

[0056] Viewers use their terminals to access the highlight footage stored on the server.

[0057] Viewers submit their opinions and suggestions for adding effects to the server.

[0058] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[0059] Examples:

[0060] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[0061] 5. Video Approval and Sharing

[0062] The content provider checks the final edited video from the server and gives their approval.

[0063] Once approved, the footage is shared to the streaming platform via a server.

[0064] The device accesses the shared video and begins viewing it.

[0065] Examples:

[0066] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[0067] 6. Monthly subscription, advertising integration, revenue sharing

[0068] The server processes the monthly subscription for the Professional plan selected by the user.

[0069] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0070] Examples:

[0071] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[0072] In this way, the system of the present invention utilizes AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers and providing a sustainable revenue model.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[0076] Step 2:

[0077] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[0078] Step 3:

[0079] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[0080] Step 4:

[0081] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[0082] Step 5:

[0083] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[0084] Step 6:

[0085] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[0086] Step 7:

[0087] The server sends the identified highlights to the auto-editing module, which uses AI algorithms to apply transitions, effects, and background music to the footage for optimal editing.

[0088] Step 8:

[0089] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[0090] Step 9:

[0091] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[0092] Step 10:

[0093] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[0094] Step 11:

[0095] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[0096] Step 12:

[0097] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[0098] Step 13:

[0099] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[0100] Step 14:

[0101] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[0102] In this way, the present invention realizes a system that achieves both efficiency and quality in video production and significantly reduces the burden on users and viewers.

[0103] Example 1

[0104] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0105] Conventional video editing systems require a lot of manual work when processing large amounts of video data, resulting in a lack of efficiency. It is also difficult to incorporate feedback from users and viewers in real time, making it difficult to quickly produce high-quality editing results. Furthermore, advertising collaboration and revenue sharing as part of the revenue model are too time-consuming. To solve these issues, a system that uses advanced AI technology to simultaneously improve the efficiency and quality of video editing is needed.

[0106] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0107] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, a unit for storing video data in cloud storage, a unit for retrieving video data from cloud storage, a unit for editing video including transitions and effects using an AI algorithm, a unit for collecting viewer suggestions, selecting and integrating optimal editing proposals, and a unit for uploading video to a streaming platform. This enables fast and efficient editing of video data and reflects feedback from viewers and users in real time. It also facilitates advertising collaboration and revenue sharing as part of a revenue model.

[0108] An "emotion analysis means" is a means for analyzing information such as facial expressions and voices of people in a video and determining their emotional state.

[0109] The "real-time video analysis means" is a means for analyzing video data in real time and recognizing target scenes and objects.

[0110] The "highlight scene identification means" is a means for extracting and identifying important scenes from video data.

[0111] The "automatic editing means" is a means for automatically editing video by applying transitions and effects based on the identified highlight scenes.

[0112] "Collaborative editing means" refers to a means for collecting suggestions from viewers and users and collaboratively editing videos based on those suggestions.

[0113] The "video approval means" is a means by which the content provider checks and approves the final edited video.

[0114] "Video sharing means" means a means for uploading and sharing approved video to a streaming platform or the like.

[0115] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[0116] An "advertising collaboration means" is a means for placing advertisements with advertising partners and generating revenue.

[0117] A "revenue share vehicle" is a vehicle for sharing revenue generated from advertising or services with users.

[0118] "Means for saving video data in cloud storage" refers to means for saving captured video data in a cloud storage service.

[0119] The "means for obtaining video data from cloud storage" refers to a means for retrieving stored video data from cloud storage.

[0120] "Means for editing video, including transitions and effects, using AI algorithms" refers to means for automatically applying transitions and effects to video and editing it using AI.

[0121] "Means for collecting viewer suggestions and selecting and integrating the most appropriate editing plan" refers to a means for receiving suggestions from viewers, selecting the most appropriate editing plan based on those suggestions, and reflecting them in the video.

[0122] "Means for uploading footage to a streaming platform" means means for uploading edited and approved footage to a streaming platform.

[0123] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming videos shot by users. This system performs data processing and calculations using the following specific hardware and software:

[0124] Users use their smartphones or cameras to capture footage of live events. The devices then input the video data into the system's application and send it to the server via the "upload" button. The server then stores the received video data in cloud storage such as AWS S3.

[0125] The server retrieves video data from cloud storage and sends it to an analytics engine such as the Google Cloud Video Intelligence API. The analytics engine performs sentiment analysis, scene recognition, and audience reaction analysis of the video. The server receives the analysis results and automatically edits the video based on the identified highlight scenes using a video editing library such as FFmpeg. Transitions, effects, and background music are applied during editing.

[0126] Viewers can access the server from their devices and watch the saved highlight footage. Viewers can also send their opinions and suggestions for adding effects to the server through the app. The server then collects the viewers' suggestions, and AI selects and integrates the optimal editing ideas.

[0127] The content provider reviews the final edited footage stored on the server and approves it through a preview screen, after which the server uploads the approved footage to a streaming platform (e.g., YouTube or Vimeo) where it is accessible to viewers.

[0128] The system also processes monthly subscription fees from users using payment gateways such as Stripe and PayPal, and generates revenue through advertising integration via advertising platforms such as Google AdSense. Based on a revenue share model, revenues from advertising and subscriptions can be shared with users.

[0129] As a concrete example, consider the process of a user filming a music concert with their smartphone and pressing the "upload" button in an application to send the video to a server. The server stores the received video data in AWS S3, retrieves it from cloud storage, and analyzes it using the Google Cloud Video Intelligence API. Based on the analysis results, FFmpeg automatically edits the video to generate a video with transitions, effects, and background music applied. Viewer suggestions are then collected, the optimal editing suggestions are integrated, and the final video is uploaded to a streaming platform.

[0130] An example of a prompt using a generative AI model is, "Detect the exciting moments in a music concert and edit them to optimally position the transitions and background music."

[0131] In this way, the system of the present invention makes full use of AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers while also providing a sustainable revenue model.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1:

[0134] Users capture footage of live events using their smartphones or cameras.

[0135] Input: Live event footage

[0136] Output: Recorded video data

[0137] Specific operation: The user takes a video using a smartphone or camera and saves the video data on the device via the application.

[0138] Step 2:

[0139] The video data captured by the device is entered into the application, and the "Upload" button is pressed to send the video to the server.

[0140] Input: Recorded video data

[0141] Output: Video data uploaded to the server

[0142] Specific operation: When the user presses the "upload" button in the application, the device sends the video data to the server, which receives the data and stores it in cloud storage.

[0143] Step 3:

[0144] The server retrieves the video data from cloud storage and sends it to the analysis engine.

[0145] Input: Video data stored in cloud storage

[0146] Output: Video data sent to the analysis engine

[0147] How it works: The server retrieves video data from cloud storage such as AWS S3 and sends it to an analytics engine such as Google Cloud Video Intelligence API for real-time video analysis.

[0148] Step 4:

[0149] The analytics engine performs video sentiment analysis, scene recognition, and audience reaction analysis.

[0150] Input: Video data sent from the server

[0151] Output: Analysis results (emotional state, scene recognition, audience reaction)

[0152] Specific operation: The analysis engine uses a deep learning model to analyze facial expressions, voices, and scene changes in the video and evaluate emotions and reactions.

[0153] Step 5:

[0154] The server receives the analysis results and automatically edits the video based on the identified highlight scenes.

[0155] Input: Analysis results (emotional state, scene recognition, audience reaction)

[0156] Output: Automatically edited video data

[0157] Specific operation: The server uses a video editing library such as FFmpeg to extract highlight scenes based on the analysis results, and automatically applies transitions, effects, and background music to edit the video.

[0158] Step 6:

[0159] Viewers access the server from their devices and view the saved highlight footage.

[0160] Input: Automatically edited video data

[0161] Output: Viewers watch highlight footage

[0162] Specific operation: Viewers access the server via the application and watch the highlight footage in streaming format.

[0163] Step 7:

[0164] Viewers send their opinions and suggestions for adding effects to the server.

[0165] Input: Viewer opinions and suggestions for adding effects

[0166] Output: Proposal data sent to the server

[0167] How it works: Viewers use the editing function within the application to input their opinions and the effects they want to add, and then send them to the server.

[0168] Step 8:

[0169] The server collects viewer suggestions, and the AI ​​selects and integrates the best editing ideas.

[0170] Input: Viewer-submitted suggestion data

[0171] Output: New footage based on optimized editing ideas

[0172] How it works: The server analyzes the suggestions from viewers, and the AI ​​selects the optimal editing plan. It then uses the video editing library again to generate new video data.

[0173] Step 9:

[0174] The content provider checks and approves the final edited video stored on the server.

[0175] Input: Final edited video data

[0176] Output: Content provider approval

[0177] Specific operation: The content provider uses the preview function in the server to check the final video and approves it by pressing the approval button.

[0178] Step 10:

[0179] The server uploads the approved footage to a streaming platform where it can be accessed by viewers.

[0180] Input: Approved video data

[0181] Output: Footage uploaded to a streaming platform

[0182] What it does: The server uploads the approved video to a streaming platform like YouTube or Vimeo and generates a viewing link.

[0183] Step 11:

[0184] The server processes the user's monthly subscription to the Professional plan.

[0185] Input: User's selected subscription plan information

[0186] Output: Processed monthly fee data

[0187] What it does: Your server automatically processes the user's subscription fee using a payment gateway such as Stripe or PayPal.

[0188] Step 12:

[0189] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0190] Input: Advertising data from advertising partners

[0191] Output: Revenue and share distribution data to users

[0192] How it works: The server works with advertising platforms such as Google AdSense to track ad impressions and clicks, calculate revenue based on that, and distribute a portion of the revenue to users.

[0193] (Application example 1)

[0194] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0195] To effectively improve customer engagement during in-store events and promotional activities, there is a need for real-time editing of filmed video and rapid sharing on social media. However, traditional manual video editing is time-consuming and often results in events losing their freshness. Furthermore, efficient video analysis and identifying highlight scenes are difficult, making it difficult to generate content that will attract viewers' attention. It is necessary to solve these issues and maximize the effectiveness of in-store promotional activities while increasing customer engagement.

[0196] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0197] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, an event video shooting unit, a cloud upload unit, and a social media posting unit. This enables real-time editing of event videos in physical stores and rapid sharing to social media. It also efficiently identifies highlight scenes that will attract viewers' attention, increasing customer engagement.

[0198] An "emotion analysis means" is a means for analyzing the facial expressions and tone of voice of people in a video to recognize their emotional state.

[0199] "Real-time video analysis means" is a means for analyzing captured video in real time and recognizing specific events or situations.

[0200] The "highlight scene identification means" is a means for identifying important scenes or exciting moments in a video.

[0201] The "automatic editing means" is a means for automatically arranging and editing transitions, effects, and background music based on the identified highlight scenes.

[0202] "Collaborative editing means" refers to a means by which viewers and other stakeholders can submit additional editing suggestions and opinions, and then integrate these to create optimal video editing.

[0203] The "video approval means" is a means by which the content provider checks and finally approves the edited video.

[0204] "Video sharing means" means a means for sharing approved videos to streaming platforms or social media.

[0205] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[0206] "Advertising collaboration means" refers to a means for displaying advertisements within videos and generating revenue through collaboration with advertising partners.

[0207] "Revenue Share Vehicle" means a vehicle for sharing generated revenue with Users.

[0208] "Event video shooting means" refers to a means for shooting video of events and promotional activities held within a physical store.

[0209] The "cloud uploading means" is a means for uploading and storing the captured video to a cloud server.

[0210] "SNS posting means" refers to a means for automatically posting edited video to SNS.

[0211] This invention is a system that combines a generative AI model and real-time video analysis technology to automatically edit videos of events and promotional activities in physical stores and share them on social media, etc. To implement this invention, the following specific hardware and software configurations are required.

[0212] First, users hold events or promotional activities in physical stores. At that time, they use devices (e.g., smartphones) to capture video of the event. This means of capturing video is important for capturing scenes that attract customers' interest.

[0213] Next, the device uploads the captured video to a cloud server. This cloud uploading means sends the video data to the server and stores it in cloud storage. In this case, a common cloud service such as Amazon S3 is used for cloud storage.

[0214] The cloud server retrieves the video data from the cloud storage and uses real-time video analysis to analyze scenes in the video and customer reactions. This analysis uses emotion analysis to recognize the emotional state of people in the video by analyzing their facial expressions and tone of voice. This provides data to identify exciting moments and important scenes.

[0215] Based on the analysis results, the server extracts important scenes using a highlight scene identification method, and edits the video by adding transitions, effects, and background music using an automatic editing method, using a generative AI model to perform optimal editing.

[0216] Once editing is complete, viewers and other interested parties can submit additional editing suggestions and opinions using the collaborative editing tool. These suggestions are collected on the server and integrated into the optimal video edit. The content provider then reviews and approves the edited video using the video approval tool.

[0217] Finally, the approved footage is automatically posted to streaming platforms and social media via a video sharing tool, which quickly shares the edited footage to pre-defined social media accounts.

[0218] Furthermore, the system processes the monthly fee for the subscription plan selected by the user through a monthly subscription processing means. The advertising collaboration means and revenue sharing means are used to share the generated revenue with the user.

[0219] As a concrete example, consider a new product launch event held at a brick-and-mortar store. Video of the event is shot with a smartphone and uploaded to a cloud server. After that, real-time video analysis is used to identify highlight scenes and automatically edit them. The generated video is then posted to social media after incorporating viewer feedback.

[0220] An example of a prompt is, "Generate real-time event highlights for new products." This prompt triggers the generative AI model to analyze and edit the video to generate the optimal highlight video.

[0221] As described above, the present invention provides a concrete means for effectively editing and sharing footage of events and promotional activities at physical stores.

[0222] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0223] Step 1:

[0224] A user uses a smartphone to capture video of an event in a physical store. The user uses the smartphone to capture a new product launch event or demonstration, launching an application to capture the video. The input is real-time video captured by the smartphone camera, and the output is video data stored in the smartphone.

[0225] Step 2:

[0226] The device uploads the captured video to the cloud server. Using the cloud upload function, the captured video data is sent to the server and saved in cloud storage. Specifically, the video file is transferred to cloud storage by pressing the "upload" button in the application. The input is the video data saved on the smartphone, and the output is the video data saved in cloud storage.

[0227] Step 3:

[0228] The server retrieves video data from cloud storage and begins real-time video analysis. The cloud server sends the video data to an analysis engine, which uses software to analyze emotions, scenes, and audience reactions. Based on the analysis results, highlight scenes are identified. The input is the video data stored in cloud storage, and the output is the analysis results and identified highlight scene data.

[0229] Step 4:

[0230] The server performs automatic editing based on the identified highlight scenes. Using the automatic editing method, the generative AI model optimally arranges transitions, effects, and background music to edit the video. The input is the identified highlight scene data, and the output is edited video data.

[0231] Step 5:

[0232] Viewers use the application to access the edited highlight footage and provide their opinions and effect suggestions. The collaborative editing method sends the viewers' suggestions to the server, which then collects these suggestions and integrates the best ones. The input is the viewers' opinions and effect suggestion data, and the output is the optimized video data.

[0233] Step 6:

[0234] The server provides the final edited video to the content provider for review and approval. The content provider reviews the video and requests corrections as necessary. Once all corrections and final confirmation are complete, the video is officially approved using the video approval process. The input is the optimized video data, and the output is the approved video data.

[0235] Step 7:

[0236] The server automatically posts the approved video to a pre-set SNS account using the SNS posting means. The SNS posting means accesses the SNS platform via API and shares the video. The input is the approved video data and SNS account information, and the output is the completion status of the SNS posting.

[0237] Step 8:

[0238] The server is a monthly subscription processing means that processes the monthly fee for the professional plan selected by the user. Payment is processed using a common online payment system. The input is the user's subscription information and payment information, and the output is the status that the payment has been completed.

[0239] Step 9:

[0240] The server uses an advertising collaboration mechanism to generate revenue with advertising partners and a revenue sharing mechanism to share the generated revenue with users. The inputs are advertising impression data and revenue information, and the output is the revenue sharing result.

[0241] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0242] The present invention is a platform that automatically edits live events and streaming videos shot by users by combining emotion analysis means, real-time video analysis means, highlight scene identification means, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly subscription processing means, advertising collaboration means, revenue sharing means, and an emotion engine that recognizes user emotions. The specific processing content and operation of the program are described below.

[0243] 1. Users shoot and upload footage

[0244] Users capture footage of live events using their smartphones or cameras.

[0245] The device uploads the captured video to a server via an application.

[0246] The server receives the uploaded video data and stores it in cloud storage.

[0247] Examples:

[0248] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[0249] 2. Real-time video analysis

[0250] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[0251] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[0252] The server identifies important scenes based on the analysis results.

[0253] Examples:

[0254] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[0255] 3. Emotion Recognition by Emotion Engine

[0256] The server uses an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice in the video to identify their emotional state.

[0257] The server collects and analyzes emotion data in real time.

[0258] Examples:

[0259] The emotion engine recognizes the user's smiling or surprised expressions in the video and sends this information to the server, which then collects the user's emotional data.

[0260] 4. Automatic Editing

[0261] The server performs automatic editing based on the identified highlight scenes and emotion data.

[0262] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[0263] Examples:

[0264] The server edits the video based on the identified exciting scenes and the user's emotional data, and applies transitions and effects to generate videos that will attract the viewer's attention.

[0265] 5. Collaborative editing

[0266] Viewers use their terminals to access the highlight footage stored on the server.

[0267] Viewers submit their opinions and suggestions for adding effects to the server.

[0268] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[0269] Examples:

[0270] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[0271] 6. Video Approval and Sharing

[0272] The content provider checks the final edited video from the server and gives their approval.

[0273] Once approved, the footage is shared to the streaming platform via a server.

[0274] The device accesses the shared video and begins viewing it.

[0275] Examples:

[0276] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[0277] 7. Monthly Subscription, Advertising Integration, Revenue Sharing

[0278] The server processes the monthly subscription for the Professional plan selected by the user.

[0279] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0280] Examples:

[0281] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[0282] In this way, the system of the present invention utilizes AI technology and an emotion engine to significantly improve the efficiency of video editing, reduce the burden on creators and viewers, and provide a sustainable revenue model.

[0283] The processing flow will be explained below.

[0284] Step 1:

[0285] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[0286] Step 2:

[0287] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[0288] Step 3:

[0289] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[0290] Step 4:

[0291] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[0292] Step 5:

[0293] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[0294] Step 6:

[0295] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[0296] Step 7:

[0297] The server runs an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. The recognized emotion data is sent to the server.

[0298] Step 8:

[0299] The server automatically edits the video based on the identified highlight scenes and recognized emotional data. The auto-editing module uses AI algorithms to optimally position transitions, effects, and background music.

[0300] Step 9:

[0301] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[0302] Step 10:

[0303] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[0304] Step 11:

[0305] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[0306] Step 12:

[0307] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[0308] Step 13:

[0309] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[0310] Step 14:

[0311] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[0312] Step 15:

[0313] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[0314] In this way, the present invention uses an emotion engine to recognize user emotions, significantly improving the efficiency of video editing, while reducing the burden on creators and viewers and providing a sustainable revenue model.

[0315] Example 2

[0316] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0317] Conventional video editing systems require a lot of manual editing work by the user, which is time-consuming and labor-intensive. Furthermore, they lack sufficient support for sentiment analysis to increase viewer engagement, automatic highlight scene identification, and monetization. This places a heavy burden on creators and viewers, making it difficult to build a sustainable revenue model.

[0318] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion analysis means, a real-time video analysis means, a highlight scene identification means, an emotion recognition means using an emotion engine, an automatic editing means, a collaborative editing means, a video approval means, a video sharing means, a monthly service processing means, an advertising partner collaboration means, and a revenue sharing means. This significantly improves the efficiency of automatic video analysis and editing, reduces the burden on creators and viewers, and enables the provision of a sustainable revenue model.

[0319] "Emotion analysis means" is a technology for analyzing the facial expressions and tone of voice of users or viewers to identify their emotional state.

[0320] "Real-time video analysis means" is a technology for analyzing live and recorded video in real time to detect important scenes and events.

[0321] The "highlight scene identification means" is a technology for automatically identifying important scenes in a video or moments that will attract the viewer's attention.

[0322] "Emotion recognition means using an emotion engine" refers to dedicated algorithms and software for recognizing the emotions of people in video.

[0323] "Automatic editing methods" are technologies that use AI algorithms to automatically edit video based on highlight scenes and emotional data.

[0324] "Collaborative editing means" is a technology that allows viewers and users to give their opinions and suggestions for adding effects to a video, and then carry out optimal editing based on those opinions.

[0325] "Video approval means" refers to a technology that allows the content provider to check and approve the final edited video.

[0326] "Video sharing tools" are technologies for sharing edited videos via streaming platforms, social media, etc.

[0327] "Monthly service processing means" refers to technology for managing and processing subscriptions to monthly services selected by users.

[0328] "Advertising partner collaboration means" refers to technology for collaborating with advertising partners to generate and manage advertising revenue.

[0329] A "revenue sharing method" is a technology that allows revenue generated from advertising, subscriptions, etc. to be distributed fairly among the parties involved.

[0330] The present invention is a system that includes emotion analysis means, real-time video analysis means, highlight scene identification means, emotion recognition means using an emotion engine, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly service processing means, advertising partner collaboration means, and revenue sharing means. This system automatically analyzes and edits live event and streaming video shot by users, and provides a platform that supports monetization.

[0331] First, a user uses a smartphone or camera to capture video of a live event. After capturing the video, the device uploads the video to a server. The server receives the uploaded video data and stores it in cloud storage, ensuring secure storage of the data needed for subsequent processing steps.

[0332] The server then retrieves the video data from the cloud storage and sends it to an analytics engine, which performs real-time sentiment analysis, scene recognition, and audience reaction analysis. Based on the results of this analysis, the server identifies important scenes within the video. For example, it can detect exciting moments by analyzing the audience's facial expressions and tone of voice.

[0333] The server also uses an emotion engine to recognize the user's emotions. It analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. Emotional data is collected in real time and automatically edited based on the analysis results. For example, the emotion engine can recognize the user's smiling or surprised expressions and edit the video based on that information to generate a video that will attract the viewer's attention.

[0334] The automated editing uses AI algorithms based on identified highlights and emotional data to optimally position transitions, effects, and background music to create highly engaging videos. Specific editing operations include adding transitions during exciting moments and applying visual effects.

[0335] With the collaborative editing feature, viewers can access highlight footage using their devices. Viewers can send their opinions and suggestions for adding effects to the server. The server collects these suggestions, and AI selects the best ones and incorporates them into the editing. Viewers can suggest improvements and effects while watching the highlight footage through the app.

[0336] After the final editing is complete, the content provider checks and approves the final edited video from the server. The approved video is then shared via the server to streaming platforms and social media. Viewers can then access the shared video from their devices and begin watching.

[0337] Users select the professional plan in the application, pay a monthly fee, and the server processes this monthly service. The server also works with advertising partners to generate advertising revenue, which is then shared with users through a fair revenue share mechanism.

[0338] This system will significantly reduce the burden on creators and viewers, enabling the creation of high-quality video content and the creation of a sustainable revenue model.

[0339] Examples of prompts include:

[0340] "Automatically detect exciting scenes and apply transitions and effects to edit them."

[0341] "Add the best effects to your footage based on your audience's feedback."

[0342] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0343] Step 1:

[0344] Input: Footage of a live event captured by a user using a smartphone or camera.

[0345] How it works: Users upload footage of live events to a server via an application on their device.

[0346] Specific operation: A user takes a photo of an event with their smartphone and presses the "upload" button within the application.

[0347] Output: Uploaded video data is sent to the server and stored in cloud storage.

[0348] Step 2:

[0349] Input: Video data stored in cloud storage.

[0350] Operation: The server retrieves video data from cloud storage and sends it to the analysis engine.

[0351] Specific operation: The server retrieves video data from storage and transfers it to the AI ​​analysis engine.

[0352] Output: The video data is passed to the analysis engine, which then performs emotion analysis, scene recognition, and audience reaction analysis to generate highlight scene information.

[0353] Step 3:

[0354] Input: Emotion data sent from the analysis engine, scene recognition information, and audience reaction analysis results.

[0355] How it works: The server uses an emotion engine to further recognize the user's emotions and analyze the user's facial expressions and tone of voice in the video.

[0356] Specific operation: The emotion engine analyzes the user's smiling or surprised expressions in the video and sends the emotion data to the server.

[0357] Output: The emotion data sent to the server is converted into information that represents the user's detailed emotional state.

[0358] Step 4:

[0359] Input: Emotion data, highlight scene information.

[0360] How it works: The server automatically edits the video based on this data, optimally positioning transitions, effects, and background music.

[0361] What it does: The AI ​​algorithm adds transitions and effects to the identified scenes and selects appropriate background music.

[0362] Output: An attractive highlight video is generated.

[0363] Step 5:

[0364] Input: Auto-edited highlight footage.

[0365] How it works: Viewers use their devices to access highlight footage stored on the server and offer their opinions and suggest additional effects.

[0366] Specific operation: Viewers watch highlight footage through the application and suggest improvements.

[0367] Output: Viewer suggestions are sent to the server, and the AI ​​selects and integrates the best suggestions.

[0368] Step 6:

[0369] Input: Final edited highlight reel.

[0370] How it works: The content provider reviews and approves the final edited footage.

[0371] Specific operation: The content provider checks the final video on the server and presses the "Approve" button.

[0372] Output: Approved footage is uploaded to a streaming platform via a server.

[0373] Step 7:

[0374] Input: Service usage data, advertising data.

[0375] What it does: The server processes the monthly service plan selected by the user and works with advertising partners to generate revenue.

[0376] What happens: A user selects the Professional plan and pays a monthly fee. The server calculates the revenue earned from ad impressions.

[0377] Output: Revenue is generated and shared with users.

[0378] (Application example 2)

[0379] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Conventional video editing systems require users to manually edit the footage they shoot, which takes time and effort. It's also difficult to incorporate viewer opinions and feedback in real time, which can lead to a decline in content quality. Furthermore, revenue is limited, making it difficult to build a sustainable model.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0382] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising linkage unit, a revenue sharing unit, a cloud storage unit, a key scene detection unit using emotion data, a unit for editing video based on viewer suggestions for improvement, and a unit for delivering the final edited video. This makes it possible to automatically analyze and edit video shot by users and reflect viewer feedback in real time. It also enables the creation of a sustainable revenue model.

[0383] An "emotion analysis means" is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[0384] The "real-time video analysis means" is a device that analyzes uploaded video in real time and detects scene features and important moments.

[0385] The "highlight scene identifying means" is a device that identifies scenes from the video that are particularly important and have high engagement.

[0386] The "automatic editing means" is an algorithm that automatically edits video based on the identified highlight scenes and emotion data.

[0387] The "collaborative editing tool" is a device that edits video based on suggestions and opinions from viewers and generates optimal content.

[0388] The "video approval means" is a device that allows the content provider to check and approve the final edited video.

[0389] "Video sharing means" refers to a device for sharing approved edited footage on streaming platforms or social media.

[0390] The "monthly subscription processing means" is a device that processes the monthly fee for the professional plan selected by the user.

[0391] The "advertising linking means" is a device that links and displays advertisements with video and generates revenue from them.

[0392] A "revenue sharing vehicle" is a device for sharing generated revenue between users and application operators.

[0393] "Cloud storage means" refers to a device for storing and managing video data on the cloud.

[0394] The "means for detecting important scenes using emotion data" is a device for detecting important scenes in a video based on the results of emotion analysis.

[0395] The "means for editing video based on viewer suggestions for improvement" is a device for collecting feedback from viewers and editing video based on that content.

[0396] "Means for distributing final edited video" refers to a device for distributing approved final edited video.

[0397] This invention relates to a system that allows users to upload footage of live events or specific moments they have filmed to cloud storage, and then automatically edits the footage using emotion analysis and real-time video analysis. The system identifies important scenes based on emotion data and edits the footage by incorporating viewer feedback. It also has the ability to distribute the edited footage after approval and share revenue.

[0398] Hardware and Software Configuration

[0399] 1. Cloud Storage Solutions

[0400] This is a device that allows users to upload and store footage they have taken on the cloud.

[0401] Specifically, this includes Amazon S3 and Google Cloud Storage.

[0402] 2. Sentiment analysis method

[0403] This is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[0404] Specifically, this includes Emotion APIs (e.g., Microsoft Azure Emotion API).

[0405] 3. Real-time video analysis methods

[0406] This device analyzes uploaded video in real time and detects scene features and important moments.

[0407] Specifically, this includes libraries such as OpenCV and TensorFlow.

[0408] 4. Automated Editing Methods

[0409] This device uses an AI algorithm to automatically edit video based on identified highlight scenes and emotional data.

[0410] Specifically, this includes FFmpeg and other video editing libraries.

[0411] 5. Collaborative editing tools

[0412] This device edits video based on suggestions and opinions from viewers and generates optimal content.

[0413] Specifically, this includes a web interface for collecting user feedback and analysis using AI models.

[0414] 6. Video Approval Method

[0415] This is a device that allows content providers to check and approve the final edited footage.

[0416] Specifically, this applies to dedicated review interfaces.

[0417] 7. Video sharing method

[0418] It is a device for sharing approved edited footage on streaming platforms and social media.

[0419] Specifically, this includes the YouTube API and Facebook API.

[0420] 8. Monthly Subscription Processing Method

[0421] This is the device that processes the monthly fee for the professional plan selected by the user.

[0422] Specifically, this includes subscription management systems such as Stripe and PayPal.

[0423] 9. Advertising Linkage Methods

[0424] This is a device that displays advertisements in conjunction with video and generates revenue from them.

[0425] Specifically, this includes Google AdSense and Facebook Audience Network.

[0426] 10. Revenue Share Instruments

[0427] It is a device for sharing generated revenue between users and application operators.

[0428] Specifically, this applies to revenue calculation algorithms and automatic transfer systems.

[0429] Specific examples

[0430] 1. Users film the concert and upload it to the app

[0431] Example: A user films a music concert with their smartphone and presses the "upload" button to send the footage to the server.

[0432] The server stores the received video data in cloud storage.

[0433] 2. Incorporating viewer feedback

[0434] Example: Viewers can watch highlights on the app and suggest effects and edits.

[0435] The server collects this information and the AI ​​performs optimal editing.

[0436] 3. Distribution and Revenue Share

[0437] For example: Once approved, the final edited footage will be shared to YouTube and Instagram, generating advertising revenue.

[0438] Revenues are shared between users and app operators.

[0439] Examples of prompt statements

[0440] "Video editing using emotion analysis in the EmotionClipper app"

[0441] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0442] Step 1:

[0443] A user films a live event and uploads the footage to a server via a smartphone app. Specifically, the user sends the video data to cloud storage using the "Upload" button in the app. The input of this step is the filmed video data, and the output is the video data saved in cloud storage.

[0444] Step 2:

[0445] The server retrieves the uploaded video data from cloud storage and sends it to the emotion analysis means. Specifically, the server sends the video data to the Emotion Engine, which then performs emotion analysis. The input of this step is the video data retrieved from cloud storage, and the output is the analyzed emotion data.

[0446] Step 3:

[0447] The server sends the emotion analysis data to the real-time video analysis means to detect important scenes in the video. Specifically, the server uses an analysis engine to recognize specific scenes and features from the video data and identify important scenes. The inputs to this step are emotion analysis data and video data, and the output is the identification of important scenes.

[0448] Step 4:

[0449] The server performs automatic editing based on the identified important scenes and emotional data. Specifically, the server uses AI algorithms to apply transitions and effects to generate optimal footage. The input for this step is the identification of important scenes and emotional data, and the output is automatically edited footage.

[0450] Step 5:

[0451] The server collects user and viewer feedback and performs additional editing using collaborative editing methods. Specifically, viewers send effects and correction suggestions through an in-app interface, and the server then optimizes the edits based on that. The inputs for this step are viewer feedback and the automatically edited video, and the output is the collaboratively edited video.

[0452] Step 6:

[0453] The content provider checks the final edited video on the server and approves it through the video approval means. Specifically, the content provider watches the final video on a dedicated interface and presses the approval button. The input of this step is the collaboratively edited video, and the output is the approved final video.

[0454] Step 7:

[0455] The server distributes the approved video to streaming platforms and social media platforms to share the video. Specifically, the server uploads the video using the YouTube API or Facebook API and makes it available to viewers. The input of this step is the final approved video, and the output is the shared video URL.

[0456] Step 8:

[0457] The server processes monthly subscriptions and advertising, and shares the generated revenue with users. Specifically, the server processes subscription fees using Stripe or PayPal APIs, calculates advertising revenue, and distributes it to users. The input for this step is subscription information and ad impression data, and the output is the revenue share result.

[0458] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0459] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0460] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0461] [Second embodiment]

[0462] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0463] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0464] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0465] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0466] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0467] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0468] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0469] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0470] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0471] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0472] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0473] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0474] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming video shot by users. The specific processing content and operation of the program are described below.

[0475] 1. Users shoot and upload footage

[0476] Users capture footage of live events using their smartphones or cameras.

[0477] The device uploads the captured video to a server via an application.

[0478] The server receives the uploaded video data and stores it in cloud storage.

[0479] Examples:

[0480] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[0481] 2. Real-time video analysis

[0482] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[0483] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[0484] The server identifies important scenes based on the analysis results.

[0485] Examples:

[0486] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[0487] 3. Automatic Editing

[0488] The server performs automatic editing based on the identified highlight scenes.

[0489] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[0490] Examples:

[0491] The server edits the video based on the identified exciting scenes and applies transitions and effects to generate videos that will attract the viewer's attention.

[0492] 4. Collaborative editing

[0493] Viewers use their terminals to access the highlight footage stored on the server.

[0494] Viewers submit their opinions and suggestions for adding effects to the server.

[0495] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[0496] Examples:

[0497] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[0498] 5. Video Approval and Sharing

[0499] The content provider checks the final edited video from the server and gives their approval.

[0500] Once approved, the footage is shared to the streaming platform via a server.

[0501] The device accesses the shared video and begins viewing it.

[0502] Examples:

[0503] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[0504] 6. Monthly subscription, advertising integration, revenue sharing

[0505] The server processes the monthly subscription for the Professional plan selected by the user.

[0506] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0507] Examples:

[0508] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[0509] In this way, the system of the present invention utilizes AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers and providing a sustainable revenue model.

[0510] The processing flow will be explained below.

[0511] Step 1:

[0512] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[0513] Step 2:

[0514] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[0515] Step 3:

[0516] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[0517] Step 4:

[0518] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[0519] Step 5:

[0520] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[0521] Step 6:

[0522] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[0523] Step 7:

[0524] The server sends the identified highlights to the auto-editing module, which uses AI algorithms to apply transitions, effects, and background music to the footage for optimal editing.

[0525] Step 8:

[0526] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[0527] Step 9:

[0528] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[0529] Step 10:

[0530] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[0531] Step 11:

[0532] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[0533] Step 12:

[0534] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[0535] Step 13:

[0536] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[0537] Step 14:

[0538] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[0539] In this way, the present invention realizes a system that achieves both efficiency and quality in video production and significantly reduces the burden on users and viewers.

[0540] Example 1

[0541] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0542] Conventional video editing systems require a lot of manual work when processing large amounts of video data, resulting in a lack of efficiency. It is also difficult to incorporate feedback from users and viewers in real time, making it difficult to quickly produce high-quality editing results. Furthermore, advertising collaboration and revenue sharing as part of the revenue model are too time-consuming. To solve these issues, a system that uses advanced AI technology to simultaneously improve the efficiency and quality of video editing is needed.

[0543] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0544] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, a unit for storing video data in cloud storage, a unit for retrieving video data from cloud storage, a unit for editing video including transitions and effects using an AI algorithm, a unit for collecting viewer suggestions, selecting and integrating optimal editing proposals, and a unit for uploading video to a streaming platform. This enables fast and efficient editing of video data and reflects feedback from viewers and users in real time. It also facilitates advertising collaboration and revenue sharing as part of a revenue model.

[0545] An "emotion analysis means" is a means for analyzing information such as facial expressions and voices of people in a video and determining their emotional state.

[0546] The "real-time video analysis means" is a means for analyzing video data in real time and recognizing target scenes and objects.

[0547] The "highlight scene identification means" is a means for extracting and identifying important scenes from video data.

[0548] The "automatic editing means" is a means for automatically editing video by applying transitions and effects based on the identified highlight scenes.

[0549] "Collaborative editing means" refers to a means for collecting suggestions from viewers and users and collaboratively editing videos based on those suggestions.

[0550] The "video approval means" is a means by which the content provider checks and approves the final edited video.

[0551] "Video sharing means" means a means for uploading and sharing approved video to a streaming platform or the like.

[0552] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[0553] An "advertising collaboration means" is a means for placing advertisements with advertising partners and generating revenue.

[0554] A "revenue share vehicle" is a vehicle for sharing revenue generated from advertising or services with users.

[0555] "Means for saving video data in cloud storage" refers to means for saving captured video data in a cloud storage service.

[0556] The "means for obtaining video data from cloud storage" refers to a means for retrieving stored video data from cloud storage.

[0557] "Means for editing video, including transitions and effects, using AI algorithms" refers to means for automatically applying transitions and effects to video and editing it using AI.

[0558] "Means for collecting viewer suggestions and selecting and integrating the most appropriate editing plan" refers to a means for receiving suggestions from viewers, selecting the most appropriate editing plan based on those suggestions, and reflecting them in the video.

[0559] "Means for uploading footage to a streaming platform" means means for uploading edited and approved footage to a streaming platform.

[0560] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming videos shot by users. This system performs data processing and calculations using the following specific hardware and software:

[0561] Users use their smartphones or cameras to capture footage of live events. The devices then input the video data into the system's application and send it to the server via the "upload" button. The server then stores the received video data in cloud storage such as AWS S3.

[0562] The server retrieves video data from cloud storage and sends it to an analytics engine such as the Google Cloud Video Intelligence API. The analytics engine performs sentiment analysis, scene recognition, and audience reaction analysis of the video. The server receives the analysis results and automatically edits the video based on the identified highlight scenes using a video editing library such as FFmpeg. Transitions, effects, and background music are applied during editing.

[0563] Viewers can access the server from their devices and watch the saved highlight footage. Viewers can also send their opinions and suggestions for adding effects to the server through the app. The server then collects the viewers' suggestions, and AI selects and integrates the optimal editing ideas.

[0564] The content provider reviews the final edited footage stored on the server and approves it through a preview screen, after which the server uploads the approved footage to a streaming platform (e.g., YouTube or Vimeo) where it is accessible to viewers.

[0565] The system also processes monthly subscription fees from users using payment gateways such as Stripe and PayPal, and generates revenue through advertising integration via advertising platforms such as Google AdSense. Based on a revenue share model, revenues from advertising and subscriptions can be shared with users.

[0566] As a concrete example, consider the process of a user filming a music concert with their smartphone and pressing the "upload" button in an application to send the video to a server. The server stores the received video data in AWS S3, retrieves it from cloud storage, and analyzes it using the Google Cloud Video Intelligence API. Based on the analysis results, FFmpeg automatically edits the video to generate a video with transitions, effects, and background music applied. Viewer suggestions are then collected, the optimal editing suggestions are integrated, and the final video is uploaded to a streaming platform.

[0567] An example of a prompt using a generative AI model is, "Detect the exciting moments in a music concert and edit them to optimally position the transitions and background music."

[0568] In this way, the system of the present invention makes full use of AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers while also providing a sustainable revenue model.

[0569] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0570] Step 1:

[0571] Users capture footage of live events using their smartphones or cameras.

[0572] Input: Live event footage

[0573] Output: Recorded video data

[0574] Specific operation: The user takes a video using a smartphone or camera and saves the video data on the device via the application.

[0575] Step 2:

[0576] The video data captured by the device is entered into the application, and the "Upload" button is pressed to send the video to the server.

[0577] Input: Recorded video data

[0578] Output: Video data uploaded to the server

[0579] Specific operation: When the user presses the "upload" button in the application, the device sends the video data to the server, which receives the data and stores it in cloud storage.

[0580] Step 3:

[0581] The server retrieves the video data from cloud storage and sends it to the analysis engine.

[0582] Input: Video data stored in cloud storage

[0583] Output: Video data sent to the analysis engine

[0584] How it works: The server retrieves video data from cloud storage such as AWS S3 and sends it to an analytics engine such as Google Cloud Video Intelligence API for real-time video analysis.

[0585] Step 4:

[0586] The analytics engine performs video sentiment analysis, scene recognition, and audience reaction analysis.

[0587] Input: Video data sent from the server

[0588] Output: Analysis results (emotional state, scene recognition, audience reaction)

[0589] Specific operation: The analysis engine uses a deep learning model to analyze facial expressions, voices, and scene changes in the video and evaluate emotions and reactions.

[0590] Step 5:

[0591] The server receives the analysis results and automatically edits the video based on the identified highlight scenes.

[0592] Input: Analysis results (emotional state, scene recognition, audience reaction)

[0593] Output: Automatically edited video data

[0594] Specific operation: The server uses a video editing library such as FFmpeg to extract highlight scenes based on the analysis results, and automatically applies transitions, effects, and background music to edit the video.

[0595] Step 6:

[0596] Viewers access the server from their devices and view the saved highlight footage.

[0597] Input: Automatically edited video data

[0598] Output: Viewers watch highlight footage

[0599] Specific operation: Viewers access the server via the application and watch the highlight footage in streaming format.

[0600] Step 7:

[0601] Viewers send their opinions and suggestions for adding effects to the server.

[0602] Input: Viewer opinions and suggestions for adding effects

[0603] Output: Proposal data sent to the server

[0604] How it works: Viewers use the editing function within the application to input their opinions and the effects they want to add, and then send them to the server.

[0605] Step 8:

[0606] The server collects viewer suggestions, and the AI ​​selects and integrates the best editing ideas.

[0607] Input: Viewer-submitted suggestion data

[0608] Output: New footage based on optimized editing ideas

[0609] How it works: The server analyzes the suggestions from viewers, and the AI ​​selects the optimal editing plan. It then uses the video editing library again to generate new video data.

[0610] Step 9:

[0611] The content provider checks and approves the final edited video stored on the server.

[0612] Input: Final edited video data

[0613] Output: Content provider approval

[0614] Specific operation: The content provider uses the preview function in the server to check the final video and approves it by pressing the approval button.

[0615] Step 10:

[0616] The server uploads the approved footage to a streaming platform where it can be accessed by viewers.

[0617] Input: Approved video data

[0618] Output: Footage uploaded to a streaming platform

[0619] What it does: The server uploads the approved video to a streaming platform like YouTube or Vimeo and generates a viewing link.

[0620] Step 11:

[0621] The server processes the user's monthly subscription to the Professional plan.

[0622] Input: User's selected subscription plan information

[0623] Output: Processed monthly fee data

[0624] What it does: Your server automatically processes the user's subscription fee using a payment gateway such as Stripe or PayPal.

[0625] Step 12:

[0626] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0627] Input: Advertising data from advertising partners

[0628] Output: Revenue and share distribution data to users

[0629] How it works: The server works with advertising platforms such as Google AdSense to track ad impressions and clicks, calculate revenue based on that, and distribute a portion of the revenue to users.

[0630] (Application example 1)

[0631] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0632] To effectively improve customer engagement during in-store events and promotional activities, there is a need for real-time editing of filmed video and rapid sharing on social media. However, traditional manual video editing is time-consuming and often results in events losing their freshness. Furthermore, efficient video analysis and identifying highlight scenes are difficult, making it difficult to generate content that will attract viewers' attention. It is necessary to solve these issues and maximize the effectiveness of in-store promotional activities while increasing customer engagement.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0634] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, an event video shooting unit, a cloud upload unit, and a social media posting unit. This enables real-time editing of event videos in physical stores and rapid sharing to social media. It also efficiently identifies highlight scenes that will attract viewers' attention, increasing customer engagement.

[0635] An "emotion analysis means" is a means for analyzing the facial expressions and tone of voice of people in a video to recognize their emotional state.

[0636] "Real-time video analysis means" is a means for analyzing captured video in real time and recognizing specific events or situations.

[0637] The "highlight scene identification means" is a means for identifying important scenes or exciting moments in a video.

[0638] The "automatic editing means" is a means for automatically arranging and editing transitions, effects, and background music based on the identified highlight scenes.

[0639] "Collaborative editing means" refers to a means by which viewers and other stakeholders can submit additional editing suggestions and opinions, and then integrate these to create optimal video editing.

[0640] The "video approval means" is a means by which the content provider checks and finally approves the edited video.

[0641] "Video sharing means" means a means for sharing approved videos to streaming platforms or social media.

[0642] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[0643] "Advertising collaboration means" refers to a means for displaying advertisements within videos and generating revenue through collaboration with advertising partners.

[0644] "Revenue Share Vehicle" means a vehicle for sharing generated revenue with Users.

[0645] "Event video shooting means" refers to a means for shooting video of events and promotional activities held within a physical store.

[0646] The "cloud uploading means" is a means for uploading and storing the captured video to a cloud server.

[0647] "SNS posting means" refers to a means for automatically posting edited video to SNS.

[0648] This invention is a system that combines a generative AI model and real-time video analysis technology to automatically edit videos of events and promotional activities in physical stores and share them on social media, etc. To implement this invention, the following specific hardware and software configurations are required.

[0649] First, users hold events or promotional activities in physical stores. At that time, they use devices (e.g., smartphones) to capture video of the event. This means of capturing video is important for capturing scenes that attract customers' interest.

[0650] Next, the device uploads the captured video to a cloud server. This cloud uploading means sends the video data to the server and stores it in cloud storage. In this case, a common cloud service such as Amazon S3 is used for cloud storage.

[0651] The cloud server retrieves the video data from the cloud storage and uses real-time video analysis to analyze scenes in the video and customer reactions. This analysis uses emotion analysis to recognize the emotional state of people in the video by analyzing their facial expressions and tone of voice. This provides data to identify exciting moments and important scenes.

[0652] Based on the analysis results, the server extracts important scenes using a highlight scene identification method, and edits the video by adding transitions, effects, and background music using an automatic editing method, using a generative AI model to perform optimal editing.

[0653] Once editing is complete, viewers and other interested parties can submit additional editing suggestions and opinions using the collaborative editing tool. These suggestions are collected on the server and integrated into the optimal video edit. The content provider then reviews and approves the edited video using the video approval tool.

[0654] Finally, the approved footage is automatically posted to streaming platforms and social media via a video sharing tool, which quickly shares the edited footage to pre-defined social media accounts.

[0655] Furthermore, the system processes the monthly fee for the subscription plan selected by the user through a monthly subscription processing means. The advertising collaboration means and revenue sharing means are used to share the generated revenue with the user.

[0656] As a concrete example, consider a new product launch event held at a brick-and-mortar store. Video of the event is shot with a smartphone and uploaded to a cloud server. After that, real-time video analysis is used to identify highlight scenes and automatically edit them. The generated video is then posted to social media after incorporating viewer feedback.

[0657] An example of a prompt is, "Generate real-time event highlights for new products." This prompt triggers the generative AI model to analyze and edit the video to generate the optimal highlight video.

[0658] As described above, the present invention provides a concrete means for effectively editing and sharing footage of events and promotional activities at physical stores.

[0659] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0660] Step 1:

[0661] A user uses a smartphone to capture video of an event in a physical store. The user uses the smartphone to capture a new product launch event or demonstration, launching an application to capture the video. The input is real-time video captured by the smartphone camera, and the output is video data stored in the smartphone.

[0662] Step 2:

[0663] The device uploads the captured video to the cloud server. Using the cloud upload function, the captured video data is sent to the server and saved in cloud storage. Specifically, the video file is transferred to cloud storage by pressing the "upload" button in the application. The input is the video data saved on the smartphone, and the output is the video data saved in cloud storage.

[0664] Step 3:

[0665] The server retrieves video data from cloud storage and begins real-time video analysis. The cloud server sends the video data to an analysis engine, which uses software to analyze emotions, scenes, and audience reactions. Based on the analysis results, highlight scenes are identified. The input is the video data stored in cloud storage, and the output is the analysis results and identified highlight scene data.

[0666] Step 4:

[0667] The server performs automatic editing based on the identified highlight scenes. Using the automatic editing method, the generative AI model optimally arranges transitions, effects, and background music to edit the video. The input is the identified highlight scene data, and the output is edited video data.

[0668] Step 5:

[0669] Viewers use the application to access the edited highlight footage and provide their opinions and effect suggestions. The collaborative editing method sends the viewers' suggestions to the server, which then collects these suggestions and integrates the best ones. The input is the viewers' opinions and effect suggestion data, and the output is the optimized video data.

[0670] Step 6:

[0671] The server provides the final edited video to the content provider for review and approval. The content provider reviews the video and requests corrections as necessary. Once all corrections and final confirmation are complete, the video is officially approved using the video approval process. The input is the optimized video data, and the output is the approved video data.

[0672] Step 7:

[0673] The server automatically posts the approved video to a pre-set SNS account using the SNS posting means. The SNS posting means accesses the SNS platform via API and shares the video. The input is the approved video data and SNS account information, and the output is the completion status of the SNS posting.

[0674] Step 8:

[0675] The server is a monthly subscription processing means that processes the monthly fee for the professional plan selected by the user. Payment is processed using a common online payment system. The input is the user's subscription information and payment information, and the output is the status that the payment has been completed.

[0676] Step 9:

[0677] The server uses an advertising collaboration mechanism to generate revenue with advertising partners and a revenue sharing mechanism to share the generated revenue with users. The inputs are advertising impression data and revenue information, and the output is the revenue sharing result.

[0678] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0679] The present invention is a platform that automatically edits live events and streaming videos shot by users by combining emotion analysis means, real-time video analysis means, highlight scene identification means, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly subscription processing means, advertising collaboration means, revenue sharing means, and an emotion engine that recognizes user emotions. The specific processing content and operation of the program are described below.

[0680] 1. Users shoot and upload footage

[0681] Users capture footage of live events using their smartphones or cameras.

[0682] The device uploads the captured video to a server via an application.

[0683] The server receives the uploaded video data and stores it in cloud storage.

[0684] Examples:

[0685] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[0686] 2. Real-time video analysis

[0687] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[0688] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[0689] The server identifies important scenes based on the analysis results.

[0690] Examples:

[0691] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[0692] 3. Emotion Recognition by Emotion Engine

[0693] The server uses an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice in the video to identify their emotional state.

[0694] The server collects and analyzes emotion data in real time.

[0695] Examples:

[0696] The emotion engine recognizes the user's smiling or surprised expressions in the video and sends this information to the server, which then collects the user's emotional data.

[0697] 4. Automatic Editing

[0698] The server performs automatic editing based on the identified highlight scenes and emotion data.

[0699] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[0700] Examples:

[0701] The server edits the video based on the identified exciting scenes and the user's emotional data, and applies transitions and effects to generate videos that will attract the viewer's attention.

[0702] 5. Collaborative editing

[0703] Viewers use their terminals to access the highlight footage stored on the server.

[0704] Viewers submit their opinions and suggestions for adding effects to the server.

[0705] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[0706] Examples:

[0707] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[0708] 6. Video Approval and Sharing

[0709] The content provider checks the final edited video from the server and gives their approval.

[0710] Once approved, the footage is shared to the streaming platform via a server.

[0711] The device accesses the shared video and begins viewing it.

[0712] Examples:

[0713] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[0714] 7. Monthly Subscription, Advertising Integration, Revenue Sharing

[0715] The server processes the monthly subscription for the Professional plan selected by the user.

[0716] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0717] Examples:

[0718] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[0719] In this way, the system of the present invention utilizes AI technology and an emotion engine to significantly improve the efficiency of video editing, reduce the burden on creators and viewers, and provide a sustainable revenue model.

[0720] The processing flow will be explained below.

[0721] Step 1:

[0722] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[0723] Step 2:

[0724] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[0725] Step 3:

[0726] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[0727] Step 4:

[0728] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[0729] Step 5:

[0730] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[0731] Step 6:

[0732] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[0733] Step 7:

[0734] The server runs an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. The recognized emotion data is sent to the server.

[0735] Step 8:

[0736] The server automatically edits the video based on the identified highlight scenes and recognized emotional data. The auto-editing module uses AI algorithms to optimally position transitions, effects, and background music.

[0737] Step 9:

[0738] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[0739] Step 10:

[0740] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[0741] Step 11:

[0742] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[0743] Step 12:

[0744] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[0745] Step 13:

[0746] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[0747] Step 14:

[0748] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[0749] Step 15:

[0750] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[0751] In this way, the present invention uses an emotion engine to recognize user emotions, significantly improving the efficiency of video editing, while reducing the burden on creators and viewers and providing a sustainable revenue model.

[0752] Example 2

[0753] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0754] Conventional video editing systems require a lot of manual editing work by the user, which is time-consuming and labor-intensive. Furthermore, they lack sufficient support for sentiment analysis to increase viewer engagement, automatic highlight scene identification, and monetization. This places a heavy burden on creators and viewers, making it difficult to build a sustainable revenue model.

[0755] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion analysis means, a real-time video analysis means, a highlight scene identification means, an emotion recognition means using an emotion engine, an automatic editing means, a collaborative editing means, a video approval means, a video sharing means, a monthly service processing means, an advertising partner collaboration means, and a revenue sharing means. This significantly improves the efficiency of automatic video analysis and editing, reduces the burden on creators and viewers, and enables the provision of a sustainable revenue model.

[0756] "Emotion analysis means" is a technology for analyzing the facial expressions and tone of voice of users or viewers to identify their emotional state.

[0757] "Real-time video analysis means" is a technology for analyzing live and recorded video in real time to detect important scenes and events.

[0758] The "highlight scene identification means" is a technology for automatically identifying important scenes in a video or moments that will attract the viewer's attention.

[0759] "Emotion recognition means using an emotion engine" refers to dedicated algorithms and software for recognizing the emotions of people in video.

[0760] "Automatic editing methods" are technologies that use AI algorithms to automatically edit video based on highlight scenes and emotional data.

[0761] "Collaborative editing means" is a technology that allows viewers and users to give their opinions and suggestions for adding effects to a video, and then carry out optimal editing based on those opinions.

[0762] "Video approval means" refers to a technology that allows the content provider to check and approve the final edited video.

[0763] "Video sharing tools" are technologies for sharing edited videos via streaming platforms, social media, etc.

[0764] "Monthly service processing means" refers to technology for managing and processing subscriptions to monthly services selected by users.

[0765] "Advertising partner collaboration means" refers to technology for collaborating with advertising partners to generate and manage advertising revenue.

[0766] A "revenue sharing method" is a technology that allows revenue generated from advertising, subscriptions, etc. to be distributed fairly among the parties involved.

[0767] The present invention is a system that includes emotion analysis means, real-time video analysis means, highlight scene identification means, emotion recognition means using an emotion engine, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly service processing means, advertising partner collaboration means, and revenue sharing means. This system automatically analyzes and edits live event and streaming video shot by users, and provides a platform that supports monetization.

[0768] First, a user uses a smartphone or camera to capture video of a live event. After capturing the video, the device uploads the video to a server. The server receives the uploaded video data and stores it in cloud storage, ensuring secure storage of the data needed for subsequent processing steps.

[0769] The server then retrieves the video data from the cloud storage and sends it to an analytics engine, which performs real-time sentiment analysis, scene recognition, and audience reaction analysis. Based on the results of this analysis, the server identifies important scenes within the video. For example, it can detect exciting moments by analyzing the audience's facial expressions and tone of voice.

[0770] The server also uses an emotion engine to recognize the user's emotions. It analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. Emotional data is collected in real time and automatically edited based on the analysis results. For example, the emotion engine can recognize the user's smiling or surprised expressions and edit the video based on that information to generate a video that will attract the viewer's attention.

[0771] The automated editing uses AI algorithms based on identified highlights and emotional data to optimally position transitions, effects, and background music to create highly engaging videos. Specific editing operations include adding transitions during exciting moments and applying visual effects.

[0772] With the collaborative editing feature, viewers can access highlight footage using their devices. Viewers can send their opinions and suggestions for adding effects to the server. The server collects these suggestions, and AI selects the best ones and incorporates them into the editing. Viewers can suggest improvements and effects while watching the highlight footage through the app.

[0773] After the final editing is complete, the content provider checks and approves the final edited video from the server. The approved video is then shared via the server to streaming platforms and social media. Viewers can then access the shared video from their devices and begin watching.

[0774] Users select the professional plan in the application, pay a monthly fee, and the server processes this monthly service. The server also works with advertising partners to generate advertising revenue, which is then shared with users through a fair revenue share mechanism.

[0775] This system will significantly reduce the burden on creators and viewers, enabling the creation of high-quality video content and the creation of a sustainable revenue model.

[0776] Examples of prompts include:

[0777] "Automatically detect exciting scenes and apply transitions and effects to edit them."

[0778] "Add the best effects to your footage based on your audience's feedback."

[0779] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0780] Step 1:

[0781] Input: Footage of a live event captured by a user using a smartphone or camera.

[0782] How it works: Users upload footage of live events to a server via an application on their device.

[0783] Specific operation: A user takes a photo of an event with their smartphone and presses the "upload" button within the application.

[0784] Output: Uploaded video data is sent to the server and stored in cloud storage.

[0785] Step 2:

[0786] Input: Video data stored in cloud storage.

[0787] Operation: The server retrieves video data from cloud storage and sends it to the analysis engine.

[0788] Specific operation: The server retrieves video data from storage and transfers it to the AI ​​analysis engine.

[0789] Output: The video data is passed to the analysis engine, which then performs emotion analysis, scene recognition, and audience reaction analysis to generate highlight scene information.

[0790] Step 3:

[0791] Input: Emotion data sent from the analysis engine, scene recognition information, and audience reaction analysis results.

[0792] How it works: The server uses an emotion engine to further recognize the user's emotions and analyze the user's facial expressions and tone of voice in the video.

[0793] Specific operation: The emotion engine analyzes the user's smiling or surprised expressions in the video and sends the emotion data to the server.

[0794] Output: The emotion data sent to the server is converted into information that represents the user's detailed emotional state.

[0795] Step 4:

[0796] Input: Emotion data, highlight scene information.

[0797] How it works: The server automatically edits the video based on this data, optimally positioning transitions, effects, and background music.

[0798] What it does: The AI ​​algorithm adds transitions and effects to the identified scenes and selects appropriate background music.

[0799] Output: An attractive highlight video is generated.

[0800] Step 5:

[0801] Input: Auto-edited highlight footage.

[0802] How it works: Viewers use their devices to access highlight footage stored on the server and offer their opinions and suggest additional effects.

[0803] Specific operation: Viewers watch highlight footage through the application and suggest improvements.

[0804] Output: Viewer suggestions are sent to the server, and the AI ​​selects and integrates the best suggestions.

[0805] Step 6:

[0806] Input: Final edited highlight reel.

[0807] How it works: The content provider reviews and approves the final edited footage.

[0808] Specific operation: The content provider checks the final video on the server and presses the "Approve" button.

[0809] Output: Approved footage is uploaded to a streaming platform via a server.

[0810] Step 7:

[0811] Input: Service usage data, advertising data.

[0812] What it does: The server processes the monthly service plan selected by the user and works with advertising partners to generate revenue.

[0813] What happens: A user selects the Professional plan and pays a monthly fee. The server calculates the revenue earned from ad impressions.

[0814] Output: Revenue is generated and shared with users.

[0815] (Application example 2)

[0816] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0817] Conventional video editing systems require users to manually edit the footage they shoot, which takes time and effort. It's also difficult to incorporate viewer opinions and feedback in real time, which can lead to a decline in content quality. Furthermore, revenue is limited, making it difficult to build a sustainable model.

[0818] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0819] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising linkage unit, a revenue sharing unit, a cloud storage unit, a key scene detection unit using emotion data, a unit for editing video based on viewer suggestions for improvement, and a unit for delivering the final edited video. This makes it possible to automatically analyze and edit video shot by users and reflect viewer feedback in real time. It also enables the creation of a sustainable revenue model.

[0820] An "emotion analysis means" is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[0821] The "real-time video analysis means" is a device that analyzes uploaded video in real time and detects scene features and important moments.

[0822] The "highlight scene identifying means" is a device that identifies scenes from the video that are particularly important and have high engagement.

[0823] The "automatic editing means" is an algorithm that automatically edits video based on the identified highlight scenes and emotion data.

[0824] The "collaborative editing tool" is a device that edits video based on suggestions and opinions from viewers and generates optimal content.

[0825] The "video approval means" is a device that allows the content provider to check and approve the final edited video.

[0826] "Video sharing means" refers to a device for sharing approved edited footage on streaming platforms or social media.

[0827] The "monthly subscription processing means" is a device that processes the monthly fee for the professional plan selected by the user.

[0828] The "advertising linking means" is a device that links and displays advertisements with video and generates revenue from them.

[0829] A "revenue sharing vehicle" is a device for sharing generated revenue between users and application operators.

[0830] "Cloud storage means" refers to a device for storing and managing video data on the cloud.

[0831] The "means for detecting important scenes using emotion data" is a device for detecting important scenes in a video based on the results of emotion analysis.

[0832] The "means for editing video based on viewer suggestions for improvement" is a device for collecting feedback from viewers and editing video based on that content.

[0833] "Means for distributing final edited video" refers to a device for distributing approved final edited video.

[0834] This invention relates to a system that allows users to upload footage of live events or specific moments they have filmed to cloud storage, and then automatically edits the footage using emotion analysis and real-time video analysis. The system identifies important scenes based on emotion data and edits the footage by incorporating viewer feedback. It also has the ability to distribute the edited footage after approval and share revenue.

[0835] Hardware and Software Configuration

[0836] 1. Cloud Storage Solutions

[0837] This is a device that allows users to upload and store footage they have taken on the cloud.

[0838] Specifically, this includes Amazon S3 and Google Cloud Storage.

[0839] 2. Sentiment analysis method

[0840] This is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[0841] Specifically, this includes Emotion APIs (e.g., Microsoft Azure Emotion API).

[0842] 3. Real-time video analysis methods

[0843] This device analyzes uploaded video in real time and detects scene features and important moments.

[0844] Specifically, this includes libraries such as OpenCV and TensorFlow.

[0845] 4. Automated Editing Methods

[0846] This device uses an AI algorithm to automatically edit video based on identified highlight scenes and emotional data.

[0847] Specifically, this includes FFmpeg and other video editing libraries.

[0848] 5. Collaborative editing tools

[0849] This device edits video based on suggestions and opinions from viewers and generates optimal content.

[0850] Specifically, this includes a web interface for collecting user feedback and analysis using AI models.

[0851] 6. Video Approval Method

[0852] This is a device that allows content providers to check and approve the final edited footage.

[0853] Specifically, this applies to dedicated review interfaces.

[0854] 7. Video sharing method

[0855] It is a device for sharing approved edited footage on streaming platforms and social media.

[0856] Specifically, this includes the YouTube API and Facebook API.

[0857] 8. Monthly Subscription Processing Method

[0858] This is the device that processes the monthly fee for the professional plan selected by the user.

[0859] Specifically, this includes subscription management systems such as Stripe and PayPal.

[0860] 9. Advertising Linkage Methods

[0861] This is a device that displays advertisements in conjunction with video and generates revenue from them.

[0862] Specifically, this includes Google AdSense and Facebook Audience Network.

[0863] 10. Revenue Share Instruments

[0864] It is a device for sharing generated revenue between users and application operators.

[0865] Specifically, this applies to revenue calculation algorithms and automatic transfer systems.

[0866] Specific examples

[0867] 1. Users film the concert and upload it to the app

[0868] Example: A user films a music concert with their smartphone and presses the "upload" button to send the footage to the server.

[0869] The server stores the received video data in cloud storage.

[0870] 2. Incorporating viewer feedback

[0871] Example: Viewers can watch highlights on the app and suggest effects and edits.

[0872] The server collects this information and the AI ​​performs optimal editing.

[0873] 3. Distribution and Revenue Share

[0874] For example: Once approved, the final edited footage will be shared to YouTube and Instagram, generating advertising revenue.

[0875] Revenues are shared between users and app operators.

[0876] Examples of prompt statements

[0877] "Video editing using emotion analysis in the EmotionClipper app"

[0878] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0879] Step 1:

[0880] A user films a live event and uploads the footage to a server via a smartphone app. Specifically, the user sends the video data to cloud storage using the "Upload" button in the app. The input of this step is the filmed video data, and the output is the video data saved in cloud storage.

[0881] Step 2:

[0882] The server retrieves the uploaded video data from cloud storage and sends it to the emotion analysis means. Specifically, the server sends the video data to the Emotion Engine, which then performs emotion analysis. The input of this step is the video data retrieved from cloud storage, and the output is the analyzed emotion data.

[0883] Step 3:

[0884] The server sends the emotion analysis data to the real-time video analysis means to detect important scenes in the video. Specifically, the server uses an analysis engine to recognize specific scenes and features from the video data and identify important scenes. The inputs to this step are emotion analysis data and video data, and the output is the identification of important scenes.

[0885] Step 4:

[0886] The server performs automatic editing based on the identified important scenes and emotional data. Specifically, the server uses AI algorithms to apply transitions and effects to generate optimal footage. The input for this step is the identification of important scenes and emotional data, and the output is automatically edited footage.

[0887] Step 5:

[0888] The server collects user and viewer feedback and performs additional editing using collaborative editing methods. Specifically, viewers submit effects and correction suggestions through an in-app interface, and the server then optimizes the edits based on those suggestions. The inputs for this step are viewer feedback and the automatically edited video, and the output is the collaboratively edited video.

[0889] Step 6:

[0890] The content provider checks the final edited video on the server and approves it through the video approval means. Specifically, the content provider watches the final video on a dedicated interface and presses the approval button. The input of this step is the collaboratively edited video, and the output is the approved final video.

[0891] Step 7:

[0892] The server distributes the approved video to streaming platforms and social media platforms to share the video. Specifically, the server uploads the video using the YouTube API or Facebook API and makes it available to viewers. The input of this step is the final approved video, and the output is the shared video URL.

[0893] Step 8:

[0894] The server processes monthly subscriptions and advertising, and shares the generated revenue with users. Specifically, the server processes subscription fees using Stripe or PayPal APIs, calculates advertising revenue, and distributes it to users. The input for this step is subscription information and ad impression data, and the output is the revenue share result.

[0895] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0896] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0897] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0898] [Third embodiment]

[0899] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0900] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0901] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0902] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0903] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0904] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0905] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0906] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0907] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0908] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0909] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0910] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0911] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming video shot by users. The specific processing content and operation of the program are described below.

[0912] 1. Users shoot and upload footage

[0913] Users capture footage of live events using their smartphones or cameras.

[0914] The device uploads the captured video to a server via an application.

[0915] The server receives the uploaded video data and stores it in cloud storage.

[0916] Examples:

[0917] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[0918] 2. Real-time video analysis

[0919] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[0920] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[0921] The server identifies important scenes based on the analysis results.

[0922] Examples:

[0923] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[0924] 3. Automatic Editing

[0925] The server performs automatic editing based on the identified highlight scenes.

[0926] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[0927] Examples:

[0928] The server edits the video based on the identified exciting scenes and applies transitions and effects to generate videos that will attract the viewer's attention.

[0929] 4. Collaborative editing

[0930] Viewers use their terminals to access the highlight footage stored on the server.

[0931] Viewers submit their opinions and suggestions for adding effects to the server.

[0932] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[0933] Examples:

[0934] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[0935] 5. Video Approval and Sharing

[0936] The content provider checks the final edited video from the server and gives their approval.

[0937] Once approved, the footage is shared to the streaming platform via a server.

[0938] The device accesses the shared video and begins viewing it.

[0939] Examples:

[0940] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[0941] 6. Monthly subscription, advertising integration, revenue sharing

[0942] The server processes the monthly subscription for the Professional plan selected by the user.

[0943] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[0944] Examples:

[0945] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[0946] In this way, the system of the present invention utilizes AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers and providing a sustainable revenue model.

[0947] The processing flow will be explained below.

[0948] Step 1:

[0949] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[0950] Step 2:

[0951] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[0952] Step 3:

[0953] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[0954] Step 4:

[0955] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[0956] Step 5:

[0957] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[0958] Step 6:

[0959] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[0960] Step 7:

[0961] The server sends the identified highlights to the auto-editing module, which uses AI algorithms to apply transitions, effects, and background music to the footage for optimal editing.

[0962] Step 8:

[0963] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[0964] Step 9:

[0965] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[0966] Step 10:

[0967] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[0968] Step 11:

[0969] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[0970] Step 12:

[0971] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[0972] Step 13:

[0973] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[0974] Step 14:

[0975] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[0976] In this way, the present invention realizes a system that achieves both efficiency and quality in video production and significantly reduces the burden on users and viewers.

[0977] Example 1

[0978] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0979] Conventional video editing systems require a lot of manual work when processing large amounts of video data, resulting in a lack of efficiency. It is also difficult to incorporate feedback from users and viewers in real time, making it difficult to quickly produce high-quality editing results. Furthermore, advertising collaboration and revenue sharing as part of the revenue model are too time-consuming. To solve these issues, a system that uses advanced AI technology to simultaneously improve the efficiency and quality of video editing is needed.

[0980] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0981] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, a unit for storing video data in cloud storage, a unit for retrieving video data from cloud storage, a unit for editing video including transitions and effects using an AI algorithm, a unit for collecting viewer suggestions, selecting and integrating optimal editing proposals, and a unit for uploading video to a streaming platform. This enables fast and efficient editing of video data and reflects feedback from viewers and users in real time. It also facilitates advertising collaboration and revenue sharing as part of a revenue model.

[0982] An "emotion analysis means" is a means for analyzing information such as facial expressions and voices of people in a video and determining their emotional state.

[0983] The "real-time video analysis means" is a means for analyzing video data in real time and recognizing target scenes and objects.

[0984] The "highlight scene identification means" is a means for extracting and identifying important scenes from video data.

[0985] The "automatic editing means" is a means for automatically editing video by applying transitions and effects based on the identified highlight scenes.

[0986] "Collaborative editing means" refers to a means for collecting suggestions from viewers and users and collaboratively editing videos based on those suggestions.

[0987] The "video approval means" is a means by which the content provider checks and approves the final edited video.

[0988] "Video sharing means" means a means for uploading and sharing approved video to a streaming platform or the like.

[0989] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[0990] An "advertising collaboration means" is a means for placing advertisements with advertising partners and generating revenue.

[0991] A "revenue share vehicle" is a vehicle for sharing revenue generated from advertising or services with users.

[0992] "Means for saving video data in cloud storage" refers to means for saving captured video data in a cloud storage service.

[0993] The "means for obtaining video data from cloud storage" refers to a means for retrieving stored video data from cloud storage.

[0994] "Means for editing video, including transitions and effects, using AI algorithms" refers to means for automatically applying transitions and effects to video and editing it using AI.

[0995] "Means for collecting viewer suggestions and selecting and integrating the most appropriate editing plan" refers to a means for receiving suggestions from viewers, selecting the most appropriate editing plan based on those suggestions, and reflecting them in the video.

[0996] "Means for uploading footage to a streaming platform" means means for uploading edited and approved footage to a streaming platform.

[0997] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming videos shot by users. This system performs data processing and calculations using the following specific hardware and software:

[0998] Users use their smartphones or cameras to capture footage of live events. The devices then input the video data into the system's application and send it to the server via the "upload" button. The server then stores the received video data in cloud storage such as AWS S3.

[0999] The server retrieves video data from cloud storage and sends it to an analytics engine such as the Google Cloud Video Intelligence API. The analytics engine performs sentiment analysis, scene recognition, and audience reaction analysis of the video. The server receives the analysis results and automatically edits the video based on the identified highlight scenes using a video editing library such as FFmpeg. Transitions, effects, and background music are applied during editing.

[1000] Viewers can access the server from their devices and watch the saved highlight footage. Viewers can also send their opinions and suggestions for adding effects to the server through the app. The server then collects the viewers' suggestions, and AI selects and integrates the optimal editing ideas.

[1001] The content provider reviews the final edited footage stored on the server and approves it through a preview screen, after which the server uploads the approved footage to a streaming platform (e.g., YouTube or Vimeo) where it is accessible to viewers.

[1002] The system also processes monthly subscription fees from users using payment gateways such as Stripe and PayPal, and generates revenue through advertising integration via advertising platforms such as Google AdSense. Based on a revenue share model, revenues from advertising and subscriptions can be shared with users.

[1003] As a concrete example, consider the process of a user filming a music concert with their smartphone and pressing the "upload" button in an application to send the video to a server. The server stores the received video data in AWS S3, retrieves it from cloud storage, and analyzes it using the Google Cloud Video Intelligence API. Based on the analysis results, FFmpeg automatically edits the video to generate a video with transitions, effects, and background music applied. Viewer suggestions are then collected, the optimal editing suggestions are integrated, and the final video is uploaded to a streaming platform.

[1004] An example of a prompt using a generative AI model is, "Detect the exciting moments in a music concert and edit them to optimally position the transitions and background music."

[1005] In this way, the system of the present invention makes full use of AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers while also providing a sustainable revenue model.

[1006] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1007] Step 1:

[1008] Users capture footage of live events using their smartphones or cameras.

[1009] Input: Live event footage

[1010] Output: Recorded video data

[1011] Specific operation: The user takes a video using a smartphone or camera and saves the video data on the device via the application.

[1012] Step 2:

[1013] The video data captured by the device is entered into the application, and the "Upload" button is pressed to send the video to the server.

[1014] Input: Recorded video data

[1015] Output: Video data uploaded to the server

[1016] Specific operation: When the user presses the "upload" button in the application, the device sends the video data to the server, which receives the data and stores it in cloud storage.

[1017] Step 3:

[1018] The server retrieves the video data from cloud storage and sends it to the analysis engine.

[1019] Input: Video data stored in cloud storage

[1020] Output: Video data sent to the analysis engine

[1021] How it works: The server retrieves video data from cloud storage such as AWS S3 and sends it to an analytics engine such as Google Cloud Video Intelligence API for real-time video analysis.

[1022] Step 4:

[1023] The analytics engine performs video sentiment analysis, scene recognition, and audience reaction analysis.

[1024] Input: Video data sent from the server

[1025] Output: Analysis results (emotional state, scene recognition, audience reaction)

[1026] Specific operation: The analysis engine uses a deep learning model to analyze facial expressions, voices, and scene changes in the video and evaluate emotions and reactions.

[1027] Step 5:

[1028] The server receives the analysis results and automatically edits the video based on the identified highlight scenes.

[1029] Input: Analysis results (emotional state, scene recognition, audience reaction)

[1030] Output: Automatically edited video data

[1031] Specific operation: The server uses a video editing library such as FFmpeg to extract highlight scenes based on the analysis results, and automatically applies transitions, effects, and background music to edit the video.

[1032] Step 6:

[1033] Viewers access the server from their devices and view the saved highlight footage.

[1034] Input: Automatically edited video data

[1035] Output: Viewers watch highlight footage

[1036] Specific operation: Viewers access the server via the application and watch the highlight footage in streaming format.

[1037] Step 7:

[1038] Viewers send their opinions and suggestions for adding effects to the server.

[1039] Input: Viewer opinions and suggestions for adding effects

[1040] Output: Proposal data sent to the server

[1041] How it works: Viewers use the editing function within the application to input their opinions and the effects they want to add, and then send them to the server.

[1042] Step 8:

[1043] The server collects viewer suggestions, and the AI ​​selects and integrates the best editing ideas.

[1044] Input: Viewer-submitted suggestion data

[1045] Output: New footage based on optimized editing ideas

[1046] How it works: The server analyzes the suggestions from viewers, and the AI ​​selects the optimal editing plan. It then uses the video editing library again to generate new video data.

[1047] Step 9:

[1048] The content provider checks and approves the final edited video stored on the server.

[1049] Input: Final edited video data

[1050] Output: Content provider approval

[1051] Specific operation: The content provider uses the preview function in the server to check the final video and approves it by pressing the approval button.

[1052] Step 10:

[1053] The server uploads the approved footage to a streaming platform where it can be accessed by viewers.

[1054] Input: Approved video data

[1055] Output: Footage uploaded to a streaming platform

[1056] What it does: The server uploads the approved video to a streaming platform like YouTube or Vimeo and generates a viewing link.

[1057] Step 11:

[1058] The server processes the user's monthly subscription to the Professional plan.

[1059] Input: User's selected subscription plan information

[1060] Output: Processed monthly fee data

[1061] What it does: Your server automatically processes the user's subscription fee using a payment gateway such as Stripe or PayPal.

[1062] Step 12:

[1063] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[1064] Input: Advertising data from advertising partners

[1065] Output: Revenue and share distribution data to users

[1066] How it works: The server works with advertising platforms such as Google AdSense to track ad impressions and clicks, calculate revenue based on that, and distribute a portion of the revenue to users.

[1067] (Application example 1)

[1068] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1069] To effectively improve customer engagement during in-store events and promotional activities, there is a need for real-time editing of filmed video and rapid sharing on social media. However, traditional manual video editing is time-consuming and often results in events losing their freshness. Furthermore, efficient video analysis and identifying highlight scenes are difficult, making it difficult to generate content that will attract viewers' attention. It is necessary to solve these issues and maximize the effectiveness of in-store promotional activities while increasing customer engagement.

[1070] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1071] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, an event video shooting unit, a cloud upload unit, and a social media posting unit. This enables real-time editing of event videos in physical stores and rapid sharing to social media. It also efficiently identifies highlight scenes that will attract viewers' attention, increasing customer engagement.

[1072] An "emotion analysis means" is a means for analyzing the facial expressions and tone of voice of people in a video to recognize their emotional state.

[1073] "Real-time video analysis means" is a means for analyzing captured video in real time and recognizing specific events or situations.

[1074] The "highlight scene identification means" is a means for identifying important scenes or exciting moments in a video.

[1075] The "automatic editing means" is a means for automatically arranging and editing transitions, effects, and background music based on the identified highlight scenes.

[1076] "Collaborative editing means" refers to a means by which viewers and other stakeholders can submit additional editing suggestions and opinions, and then integrate these to create optimal video editing.

[1077] The "video approval means" is a means by which the content provider checks and finally approves the edited video.

[1078] "Video sharing means" means a means for sharing approved videos to streaming platforms or social media.

[1079] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[1080] "Advertising collaboration means" refers to a means for displaying advertisements within videos and generating revenue through collaboration with advertising partners.

[1081] "Revenue Share Vehicle" means a vehicle for sharing generated revenue with Users.

[1082] "Event video shooting means" refers to a means for shooting video of events and promotional activities held within a physical store.

[1083] The "cloud uploading means" is a means for uploading and storing the captured video to a cloud server.

[1084] "SNS posting means" refers to a means for automatically posting edited video to SNS.

[1085] This invention is a system that combines a generative AI model and real-time video analysis technology to automatically edit videos of events and promotional activities in physical stores and share them on social media, etc. To implement this invention, the following specific hardware and software configurations are required.

[1086] First, users hold events or promotional activities in physical stores. At that time, they use devices (e.g., smartphones) to capture video of the event. This means of capturing video is important for capturing scenes that attract customers' interest.

[1087] Next, the device uploads the captured video to a cloud server. This cloud uploading means sends the video data to the server and stores it in cloud storage. In this case, a common cloud service such as Amazon S3 is used for cloud storage.

[1088] The cloud server retrieves the video data from the cloud storage and uses real-time video analysis to analyze scenes in the video and customer reactions. This analysis uses emotion analysis to recognize the emotional state of people in the video by analyzing their facial expressions and tone of voice. This provides data to identify exciting moments and important scenes.

[1089] Based on the analysis results, the server extracts important scenes using a highlight scene identification method, and edits the video by adding transitions, effects, and background music using an automatic editing method, using a generative AI model to optimize the editing.

[1090] Once editing is complete, viewers and other interested parties can submit additional editing suggestions and opinions using the collaborative editing tool. These suggestions are collected on the server and integrated into the optimal video edit. The content provider then reviews and approves the edited video using the video approval tool.

[1091] Finally, the approved footage is automatically posted to streaming platforms and social media via a video sharing tool, which quickly shares the edited footage to pre-defined social media accounts.

[1092] Furthermore, the system processes the monthly fee for the subscription plan selected by the user through a monthly subscription processing means. The advertising collaboration means and revenue sharing means are used to share the generated revenue with the user.

[1093] As a concrete example, consider a new product launch event held at a brick-and-mortar store. Video of the event is shot with a smartphone and uploaded to a cloud server. After that, real-time video analysis is used to identify highlight scenes and automatically edit them. The generated video is then posted to social media after incorporating viewer feedback.

[1094] An example of a prompt is, "Generate real-time event highlights for new products." This prompt triggers the generative AI model to analyze and edit the video to generate the optimal highlight video.

[1095] As described above, the present invention provides a concrete means for effectively editing and sharing footage of events and promotional activities at physical stores.

[1096] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1097] Step 1:

[1098] A user uses a smartphone to capture video of an event in a physical store. The user uses the smartphone to capture a new product launch event or demonstration, launching an application to capture the video. The input is real-time video captured by the smartphone camera, and the output is video data stored in the smartphone.

[1099] Step 2:

[1100] The device uploads the captured video to the cloud server. Using the cloud upload function, the captured video data is sent to the server and saved in cloud storage. Specifically, the video file is transferred to cloud storage by pressing the "upload" button in the application. The input is the video data saved on the smartphone, and the output is the video data saved in cloud storage.

[1101] Step 3:

[1102] The server retrieves video data from cloud storage and begins real-time video analysis. The cloud server sends the video data to an analysis engine, which uses software to analyze emotions, scenes, and audience reactions. Based on the analysis results, highlight scenes are identified. The input is the video data stored in cloud storage, and the output is the analysis results and identified highlight scene data.

[1103] Step 4:

[1104] The server performs automatic editing based on the identified highlight scenes. Using the automatic editing method, the generative AI model optimally arranges transitions, effects, and background music to edit the video. The input is the identified highlight scene data, and the output is edited video data.

[1105] Step 5:

[1106] Viewers use the application to access the edited highlight footage and provide their opinions and effect suggestions. The collaborative editing method sends the viewers' suggestions to the server, which then collects these suggestions and integrates the best ones. The input is the viewers' opinions and effect suggestion data, and the output is the optimized video data.

[1107] Step 6:

[1108] The server provides the final edited video to the content provider for review and approval. The content provider reviews the video and requests corrections as necessary. Once all corrections and final confirmation are complete, the video is officially approved using the video approval process. The input is the optimized video data, and the output is the approved video data.

[1109] Step 7:

[1110] The server automatically posts the approved video to a pre-set SNS account using the SNS posting means. The SNS posting means accesses the SNS platform via API and shares the video. The input is the approved video data and SNS account information, and the output is the completion status of the SNS posting.

[1111] Step 8:

[1112] The server is a monthly subscription processing means that processes the monthly fee for the professional plan selected by the user. Payment is processed using a common online payment system. The input is the user's subscription information and payment information, and the output is the status that the payment has been completed.

[1113] Step 9:

[1114] The server uses an advertising collaboration mechanism to generate revenue with advertising partners and a revenue sharing mechanism to share the generated revenue with users. The inputs are advertising impression data and revenue information, and the output is the revenue sharing results.

[1115] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1116] The present invention is a platform that automatically edits live events and streaming videos shot by users by combining emotion analysis means, real-time video analysis means, highlight scene identification means, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly subscription processing means, advertising collaboration means, revenue sharing means, and an emotion engine that recognizes user emotions. The specific processing content and operation of the program are described below.

[1117] 1. Users shoot and upload footage

[1118] Users capture footage of live events using their smartphones or cameras.

[1119] The device uploads the captured video to a server via an application.

[1120] The server receives the uploaded video data and stores it in cloud storage.

[1121] Examples:

[1122] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[1123] 2. Real-time video analysis

[1124] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[1125] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[1126] The server identifies important scenes based on the analysis results.

[1127] Examples:

[1128] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[1129] 3. Emotion Recognition by Emotion Engine

[1130] The server uses an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice in the video to identify their emotional state.

[1131] The server collects and analyzes emotion data in real time.

[1132] Examples:

[1133] The emotion engine recognizes the user's smiling or surprised expressions in the video and sends this information to the server, which then collects the user's emotional data.

[1134] 4. Automatic Editing

[1135] The server performs automatic editing based on the identified highlight scenes and emotion data.

[1136] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[1137] Examples:

[1138] The server edits the video based on the identified exciting scenes and the user's emotional data, and applies transitions and effects to generate videos that will attract the viewer's attention.

[1139] 5. Collaborative editing

[1140] Viewers use their terminals to access the highlight footage stored on the server.

[1141] Viewers submit their opinions and suggestions for adding effects to the server.

[1142] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[1143] Examples:

[1144] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[1145] 6. Video Approval and Sharing

[1146] The content provider checks the final edited video from the server and gives their approval.

[1147] Once approved, the footage is shared to the streaming platform via a server.

[1148] The device accesses the shared video and begins viewing it.

[1149] Examples:

[1150] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[1151] 7. Monthly Subscription, Advertising Integration, Revenue Sharing

[1152] The server processes the monthly subscription for the Professional plan selected by the user.

[1153] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[1154] Examples:

[1155] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[1156] In this way, the system of the present invention utilizes AI technology and an emotion engine to significantly improve the efficiency of video editing, reduce the burden on creators and viewers, and provide a sustainable revenue model.

[1157] The processing flow will be explained below.

[1158] Step 1:

[1159] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[1160] Step 2:

[1161] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[1162] Step 3:

[1163] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[1164] Step 4:

[1165] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[1166] Step 5:

[1167] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[1168] Step 6:

[1169] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[1170] Step 7:

[1171] The server runs an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. The recognized emotion data is sent to the server.

[1172] Step 8:

[1173] The server automatically edits the video based on the identified highlight scenes and recognized emotional data. The auto-editing module uses AI algorithms to optimally position transitions, effects, and background music.

[1174] Step 9:

[1175] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[1176] Step 10:

[1177] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[1178] Step 11:

[1179] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[1180] Step 12:

[1181] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[1182] Step 13:

[1183] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[1184] Step 14:

[1185] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[1186] Step 15:

[1187] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[1188] In this way, the present invention uses an emotion engine to recognize user emotions, significantly improving the efficiency of video editing, while reducing the burden on creators and viewers and providing a sustainable revenue model.

[1189] Example 2

[1190] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1191] Conventional video editing systems require a lot of manual editing work by the user, which is time-consuming and labor-intensive. Furthermore, they lack sufficient support for sentiment analysis to increase viewer engagement, automatic highlight scene identification, and monetization. This places a heavy burden on creators and viewers, making it difficult to build a sustainable revenue model.

[1192] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion analysis means, a real-time video analysis means, a highlight scene identification means, an emotion recognition means using an emotion engine, an automatic editing means, a collaborative editing means, a video approval means, a video sharing means, a monthly service processing means, an advertising partner collaboration means, and a revenue sharing means. This significantly improves the efficiency of automatic video analysis and editing, reduces the burden on creators and viewers, and enables the provision of a sustainable revenue model.

[1193] "Emotion analysis means" is a technology for analyzing the facial expressions and tone of voice of users or viewers to identify their emotional state.

[1194] "Real-time video analysis means" is a technology for analyzing live and recorded video in real time to detect important scenes and events.

[1195] The "highlight scene identification means" is a technology for automatically identifying important scenes in a video or moments that will attract the viewer's attention.

[1196] "Emotion recognition means using an emotion engine" refers to dedicated algorithms and software for recognizing the emotions of people in video.

[1197] "Automatic editing methods" are technologies that use AI algorithms to automatically edit video based on highlight scenes and emotional data.

[1198] "Collaborative editing means" is a technology that allows viewers and users to give their opinions and suggestions for adding effects to a video, and then carry out optimal editing based on those opinions.

[1199] "Video approval means" refers to a technology that allows the content provider to check and approve the final edited video.

[1200] "Video sharing tools" are technologies for sharing edited videos via streaming platforms, social media, etc.

[1201] "Monthly service processing means" refers to technology for managing and processing subscriptions to monthly services selected by users.

[1202] "Advertising partner collaboration means" refers to technology for collaborating with advertising partners to generate and manage advertising revenue.

[1203] A "revenue sharing method" is a technology that allows revenue generated from advertising, subscriptions, etc. to be distributed fairly among the parties involved.

[1204] The present invention is a system that includes emotion analysis means, real-time video analysis means, highlight scene identification means, emotion recognition means using an emotion engine, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly service processing means, advertising partner collaboration means, and revenue sharing means. This system automatically analyzes and edits live event and streaming video shot by users, and provides a platform that supports monetization.

[1205] First, a user uses a smartphone or camera to capture video of a live event. After capturing the video, the device uploads the video to a server. The server receives the uploaded video data and stores it in cloud storage, ensuring secure storage of the data needed for subsequent processing steps.

[1206] The server then retrieves the video data from the cloud storage and sends it to an analytics engine, which performs real-time sentiment analysis, scene recognition, and audience reaction analysis. Based on the results of this analysis, the server identifies important scenes within the video. For example, it can detect exciting moments by analyzing the audience's facial expressions and tone of voice.

[1207] The server also uses an emotion engine to recognize the user's emotions. It analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. Emotional data is collected in real time and automatically edited based on the analysis results. For example, the emotion engine can recognize the user's smiling or surprised expressions and edit the video based on that information to generate a video that will attract the viewer's attention.

[1208] The automated editing uses AI algorithms based on identified highlights and emotional data to optimally position transitions, effects, and background music to create highly engaging videos. Specific editing operations include adding transitions during exciting moments and applying visual effects.

[1209] With the collaborative editing feature, viewers can access highlight footage using their devices. Viewers can send their opinions and suggestions for adding effects to the server. The server collects these suggestions, and AI selects the best ones and incorporates them into the editing. Viewers can suggest improvements and effects while watching the highlight footage through the app.

[1210] After the final editing is complete, the content provider checks and approves the final edited video from the server. The approved video is then shared via the server to streaming platforms and social media. Viewers can then access the shared video from their devices and begin watching.

[1211] Users select the professional plan in the application, pay a monthly fee, and the server processes this monthly service. The server also works with advertising partners to generate advertising revenue, which is then shared with users through a fair revenue share mechanism.

[1212] This system will significantly reduce the burden on creators and viewers, enabling the creation of high-quality video content and the creation of a sustainable revenue model.

[1213] Examples of prompts include:

[1214] "Automatically detect exciting scenes and apply transitions and effects to edit them."

[1215] "Add the best effects to your footage based on your audience's feedback."

[1216] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1217] Step 1:

[1218] Input: Footage of a live event captured by a user using a smartphone or camera.

[1219] How it works: Users upload footage of live events to a server via an application on their device.

[1220] Specific operation: A user takes a photo of an event with their smartphone and presses the "upload" button within the application.

[1221] Output: Uploaded video data is sent to the server and stored in cloud storage.

[1222] Step 2:

[1223] Input: Video data stored in cloud storage.

[1224] Operation: The server retrieves video data from cloud storage and sends it to the analysis engine.

[1225] Specific operation: The server retrieves video data from storage and transfers it to the AI ​​analysis engine.

[1226] Output: The video data is passed to the analysis engine, which then performs emotion analysis, scene recognition, and audience reaction analysis to generate highlight scene information.

[1227] Step 3:

[1228] Input: Emotion data sent from the analysis engine, scene recognition information, and audience reaction analysis results.

[1229] How it works: The server uses an emotion engine to further recognize the user's emotions and analyze the user's facial expressions and tone of voice in the video.

[1230] Specific operation: The emotion engine analyzes the user's smiling or surprised expressions in the video and sends the emotion data to the server.

[1231] Output: The emotion data sent to the server is converted into information that represents the user's detailed emotional state.

[1232] Step 4:

[1233] Input: Emotion data, highlight scene information.

[1234] How it works: The server automatically edits the video based on this data, optimally positioning transitions, effects, and background music.

[1235] What it does: The AI ​​algorithm adds transitions and effects to the identified scenes and selects appropriate background music.

[1236] Output: An attractive highlight video is generated.

[1237] Step 5:

[1238] Input: Auto-edited highlight footage.

[1239] How it works: Viewers use their devices to access highlight footage stored on the server and offer their opinions and suggest additional effects.

[1240] Specific operation: Viewers watch highlight footage through the application and suggest improvements.

[1241] Output: Viewer suggestions are sent to the server, and the AI ​​selects and integrates the best suggestions.

[1242] Step 6:

[1243] Input: Final edited highlight reel.

[1244] How it works: The content provider reviews and approves the final edited footage.

[1245] Specific operation: The content provider checks the final video on the server and presses the "Approve" button.

[1246] Output: Approved footage is uploaded to a streaming platform via a server.

[1247] Step 7:

[1248] Input: Service usage data, advertising data.

[1249] What it does: The server processes the monthly service plan selected by the user and works with advertising partners to generate revenue.

[1250] What happens: A user selects the Professional plan and pays a monthly fee. The server calculates the revenue earned from ad impressions.

[1251] Output: Revenue is generated and shared with users.

[1252] (Application example 2)

[1253] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1254] Conventional video editing systems require users to manually edit the footage they shoot, which takes time and effort. It's also difficult to incorporate viewer opinions and feedback in real time, which can lead to a decline in content quality. Furthermore, revenue is limited, making it difficult to build a sustainable model.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1256] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising linkage unit, a revenue sharing unit, a cloud storage unit, a key scene detection unit using emotion data, a unit for editing video based on viewer suggestions for improvement, and a unit for delivering the final edited video. This makes it possible to automatically analyze and edit video shot by users and reflect viewer feedback in real time. It also enables the creation of a sustainable revenue model.

[1257] An "emotion analysis means" is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[1258] The "real-time video analysis means" is a device that analyzes uploaded video in real time and detects scene features and important moments.

[1259] The "highlight scene identifying means" is a device that identifies scenes from the video that are particularly important and have high engagement.

[1260] The "automatic editing means" is an algorithm that automatically edits video based on the identified highlight scenes and emotion data.

[1261] The "collaborative editing tool" is a device that edits video based on suggestions and opinions from viewers and generates optimal content.

[1262] The "video approval means" is a device that allows the content provider to check and approve the final edited video.

[1263] "Video sharing means" refers to a device for sharing approved edited footage on streaming platforms or social media.

[1264] The "monthly subscription processing means" is a device that processes the monthly fee for the professional plan selected by the user.

[1265] The "advertising linking means" is a device that links and displays advertisements with video and generates revenue from them.

[1266] A "revenue sharing vehicle" is a device for sharing generated revenue between users and application operators.

[1267] "Cloud storage means" refers to a device for storing and managing video data on the cloud.

[1268] The "means for detecting important scenes using emotion data" is a device for detecting important scenes in a video based on the results of emotion analysis.

[1269] The "means for editing video based on viewer suggestions for improvement" is a device for collecting feedback from viewers and editing video based on that content.

[1270] "Means for distributing final edited video" refers to a device for distributing approved final edited video.

[1271] This invention relates to a system that allows users to upload footage of live events or specific moments they have filmed to cloud storage, and then automatically edits the footage using emotion analysis and real-time video analysis. The system identifies important scenes based on emotion data and edits the footage by incorporating viewer feedback. It also has the ability to distribute the edited footage after approval and share revenue.

[1272] Hardware and Software Configuration

[1273] 1. Cloud Storage Solutions

[1274] This is a device that allows users to upload and store footage they have taken on the cloud.

[1275] Specifically, this includes Amazon S3 and Google Cloud Storage.

[1276] 2. Sentiment analysis method

[1277] This is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[1278] Specifically, this includes Emotion APIs (e.g., Microsoft Azure Emotion API).

[1279] 3. Real-time video analysis methods

[1280] This device analyzes uploaded video in real time and detects scene features and important moments.

[1281] Specifically, this includes libraries such as OpenCV and TensorFlow.

[1282] 4. Automated Editing Methods

[1283] This device uses an AI algorithm to automatically edit video based on identified highlight scenes and emotional data.

[1284] Specifically, this includes FFmpeg and other video editing libraries.

[1285] 5. Collaborative editing tools

[1286] This device edits video based on suggestions and opinions from viewers and generates optimal content.

[1287] Specifically, this includes a web interface for collecting user feedback and analysis using AI models.

[1288] 6. Video Approval Method

[1289] This is a device that allows content providers to check and approve the final edited footage.

[1290] Specifically, this applies to dedicated review interfaces.

[1291] 7. Video sharing method

[1292] It is a device for sharing approved edited footage on streaming platforms and social media.

[1293] Specifically, this includes the YouTube API and Facebook API.

[1294] 8. Monthly Subscription Processing Method

[1295] This is the device that processes the monthly fee for the professional plan selected by the user.

[1296] Specifically, this includes subscription management systems such as Stripe and PayPal.

[1297] 9. Advertising Linkage Methods

[1298] This is a device that displays advertisements in conjunction with video and generates revenue from them.

[1299] Specifically, this includes Google AdSense and Facebook Audience Network.

[1300] 10. Revenue Share Instruments

[1301] It is a device for sharing generated revenue between users and application operators.

[1302] Specifically, this applies to revenue calculation algorithms and automatic transfer systems.

[1303] Specific examples

[1304] 1. Users film the concert and upload it to the app

[1305] Example: A user films a music concert with their smartphone and presses the "upload" button to send the footage to the server.

[1306] The server stores the received video data in cloud storage.

[1307] 2. Incorporating viewer feedback

[1308] Example: Viewers can watch highlights on the app and suggest effects and edits.

[1309] The server collects this information and the AI ​​performs optimal editing.

[1310] 3. Distribution and Revenue Share

[1311] For example: Once approved, the final edited footage will be shared to YouTube and Instagram, generating advertising revenue.

[1312] Revenues are shared between users and app operators.

[1313] Examples of prompt statements

[1314] "Video editing using emotion analysis in the EmotionClipper app"

[1315] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1316] Step 1:

[1317] A user films a live event and uploads the footage to a server via a smartphone app. Specifically, the user sends the video data to cloud storage using the "Upload" button in the app. The input of this step is the filmed video data, and the output is the video data saved in cloud storage.

[1318] Step 2:

[1319] The server retrieves the uploaded video data from cloud storage and sends it to the emotion analysis means. Specifically, the server sends the video data to the Emotion Engine, which then performs emotion analysis. The input of this step is the video data retrieved from cloud storage, and the output is the analyzed emotion data.

[1320] Step 3:

[1321] The server sends the emotion analysis data to the real-time video analysis means to detect important scenes in the video. Specifically, the server uses an analysis engine to recognize specific scenes and features from the video data and identify important scenes. The inputs to this step are emotion analysis data and video data, and the output is the identification of important scenes.

[1322] Step 4:

[1323] The server performs automatic editing based on the identified important scenes and emotional data. Specifically, the server uses AI algorithms to apply transitions and effects to generate optimal footage. The input for this step is the identification of important scenes and emotional data, and the output is automatically edited footage.

[1324] Step 5:

[1325] The server collects user and viewer feedback and performs additional editing using collaborative editing methods. Specifically, viewers submit effects and correction suggestions through an in-app interface, and the server then optimizes the edits based on those suggestions. The inputs for this step are viewer feedback and the automatically edited video, and the output is the collaboratively edited video.

[1326] Step 6:

[1327] The content provider checks the final edited video on the server and approves it through the video approval means. Specifically, the content provider watches the final video on a dedicated interface and presses the approval button. The input of this step is the collaboratively edited video, and the output is the approved final video.

[1328] Step 7:

[1329] The server distributes the approved video to streaming platforms and social media platforms to share the video. Specifically, the server uploads the video using the YouTube API or Facebook API and makes it available to viewers. The input of this step is the final approved video, and the output is the shared video URL.

[1330] Step 8:

[1331] The server processes monthly subscriptions and advertising, and shares the generated revenue with users. Specifically, the server processes subscription fees using Stripe or PayPal APIs, calculates advertising revenue, and distributes it to users. The input for this step is subscription information and ad impression data, and the output is the revenue share result.

[1332] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1333] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1334] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1335] [Fourth embodiment]

[1336] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1337] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1338] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1339] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1340] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1341] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1342] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1343] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1344] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1347] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1348] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1349] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming video shot by users. The specific processing content and operation of the program are described below.

[1350] 1. Users shoot and upload footage

[1351] Users capture footage of live events using their smartphones or cameras.

[1352] The device uploads the captured video to a server via an application.

[1353] The server receives the uploaded video data and stores it in cloud storage.

[1354] Examples:

[1355] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[1356] 2. Real-time video analysis

[1357] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[1358] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[1359] The server identifies important scenes based on the analysis results.

[1360] Examples:

[1361] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[1362] 3. Automatic Editing

[1363] The server performs automatic editing based on the identified highlight scenes.

[1364] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[1365] Examples:

[1366] The server edits the video based on the identified exciting scenes and applies transitions and effects to generate videos that will attract the viewer's attention.

[1367] 4. Collaborative editing

[1368] Viewers use their terminals to access the highlight footage stored on the server.

[1369] Viewers submit their opinions and suggestions for adding effects to the server.

[1370] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[1371] Examples:

[1372] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[1373] 5. Video Approval and Sharing

[1374] The content provider checks the final edited video from the server and gives their approval.

[1375] Once approved, the footage is shared to the streaming platform via a server.

[1376] The device accesses the shared video and begins viewing it.

[1377] Examples:

[1378] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[1379] 6. Monthly subscription, advertising integration, revenue sharing

[1380] The server processes the monthly subscription for the Professional plan selected by the user.

[1381] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[1382] Examples:

[1383] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[1384] In this way, the system of the present invention utilizes AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers and providing a sustainable revenue model.

[1385] The processing flow will be explained below.

[1386] Step 1:

[1387] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[1388] Step 2:

[1389] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[1390] Step 3:

[1391] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[1392] Step 4:

[1393] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[1394] Step 5:

[1395] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[1396] Step 6:

[1397] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[1398] Step 7:

[1399] The server sends the identified highlights to the auto-editing module, which uses AI algorithms to apply transitions, effects, and background music to the footage for optimal editing.

[1400] Step 8:

[1401] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[1402] Step 9:

[1403] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[1404] Step 10:

[1405] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[1406] Step 11:

[1407] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[1408] Step 12:

[1409] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[1410] Step 13:

[1411] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[1412] Step 14:

[1413] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[1414] In this way, the present invention realizes a system that achieves both efficiency and quality in video production and significantly reduces the burden on users and viewers.

[1415] Example 1

[1416] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1417] Conventional video editing systems require a lot of manual work when processing large amounts of video data, resulting in a lack of efficiency. It is also difficult to incorporate feedback from users and viewers in real time, making it difficult to quickly produce high-quality editing results. Furthermore, advertising collaboration and revenue sharing as part of the revenue model are too time-consuming. To solve these issues, a system that uses advanced AI technology to simultaneously improve the efficiency and quality of video editing is needed.

[1418] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1419] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, a unit for storing video data in cloud storage, a unit for retrieving video data from cloud storage, a unit for editing video including transitions and effects using an AI algorithm, a unit for collecting viewer suggestions, selecting and integrating optimal editing proposals, and a unit for uploading video to a streaming platform. This enables fast and efficient editing of video data and reflects feedback from viewers and users in real time. It also facilitates advertising collaboration and revenue sharing as part of a revenue model.

[1420] An "emotion analysis means" is a means for analyzing information such as facial expressions and voices of people in a video and determining their emotional state.

[1421] The "real-time video analysis means" is a means for analyzing video data in real time and recognizing target scenes and objects.

[1422] The "highlight scene identification means" is a means for extracting and identifying important scenes from video data.

[1423] The "automatic editing means" is a means for automatically editing video by applying transitions and effects based on the identified highlight scenes.

[1424] "Collaborative editing means" refers to a means for collecting suggestions from viewers and users and collaboratively editing videos based on those suggestions.

[1425] The "video approval means" is a means by which the content provider checks and approves the final edited video.

[1426] "Video sharing means" means a means for uploading and sharing approved video to a streaming platform or the like.

[1427] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[1428] An "advertising collaboration means" is a means for placing advertisements with advertising partners and generating revenue.

[1429] A "revenue share vehicle" is a vehicle for sharing revenue generated from advertising or services with users.

[1430] "Means for saving video data in cloud storage" refers to means for saving captured video data in a cloud storage service.

[1431] The "means for obtaining video data from cloud storage" refers to a means for retrieving stored video data from cloud storage.

[1432] "Means for editing video, including transitions and effects, using AI algorithms" refers to means for automatically applying transitions and effects to video and editing it using AI.

[1433] "Means for collecting viewer suggestions and selecting and integrating the most appropriate editing plan" refers to a means for receiving suggestions from viewers, selecting the most appropriate editing plan based on those suggestions, and reflecting them in the video.

[1434] "Means for uploading footage to a streaming platform" means means for uploading edited and approved footage to a streaming platform.

[1435] This invention is a platform that combines generative AI and real-time video analysis technology to automatically edit live events and streaming videos shot by users. This system performs data processing and calculations using the following specific hardware and software:

[1436] Users use their smartphones or cameras to capture footage of live events. The devices then input the video data into the system's application and send it to the server via the "upload" button. The server then stores the received video data in cloud storage such as AWS S3.

[1437] The server retrieves video data from cloud storage and sends it to an analytics engine such as the Google Cloud Video Intelligence API. The analytics engine performs sentiment analysis, scene recognition, and audience reaction analysis of the video. The server receives the analysis results and automatically edits the video based on the identified highlight scenes using a video editing library such as FFmpeg. Transitions, effects, and background music are applied during editing.

[1438] Viewers can access the server from their devices and watch the saved highlight footage. Viewers can also send their opinions and suggestions for adding effects to the server through the app. The server then collects the viewers' suggestions, and AI selects and integrates the optimal editing ideas.

[1439] The content provider reviews the final edited footage stored on the server and approves it through a preview screen, after which the server uploads the approved footage to a streaming platform (e.g., YouTube or Vimeo) where it is accessible to viewers.

[1440] The system also processes monthly subscription fees from users using payment gateways such as Stripe and PayPal, and generates revenue through advertising integration via advertising platforms such as Google AdSense. Based on a revenue share model, revenues from advertising and subscriptions can be shared with users.

[1441] As a concrete example, consider the process of a user filming a music concert with their smartphone and pressing the "upload" button in an application to send the video to a server. The server stores the received video data in AWS S3, retrieves it from cloud storage, and analyzes it using the Google Cloud Video Intelligence API. Based on the analysis results, FFmpeg automatically edits the video to generate a video with transitions, effects, and background music applied. Viewer suggestions are then collected, the optimal editing suggestions are integrated, and the final video is uploaded to a streaming platform.

[1442] An example of a prompt using a generative AI model is, "Detect the exciting moments in a music concert and edit them to optimally position the transitions and background music."

[1443] In this way, the system of the present invention makes full use of AI technology to significantly improve the efficiency of video editing, reducing the burden on creators and viewers while also providing a sustainable revenue model.

[1444] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1445] Step 1:

[1446] Users capture footage of live events using their smartphones or cameras.

[1447] Input: Live event footage

[1448] Output: Recorded video data

[1449] Specific operation: The user takes a video using a smartphone or camera and saves the video data on the device via the application.

[1450] Step 2:

[1451] The video data captured by the device is entered into the application, and the "Upload" button is pressed to send the video to the server.

[1452] Input: Recorded video data

[1453] Output: Video data uploaded to the server

[1454] Specific operation: When the user presses the "upload" button in the application, the device sends the video data to the server, which receives the data and stores it in cloud storage.

[1455] Step 3:

[1456] The server retrieves the video data from cloud storage and sends it to the analysis engine.

[1457] Input: Video data stored in cloud storage

[1458] Output: Video data sent to the analysis engine

[1459] How it works: The server retrieves video data from cloud storage such as AWS S3 and sends it to an analytics engine such as Google Cloud Video Intelligence API for real-time video analysis.

[1460] Step 4:

[1461] The analytics engine performs video sentiment analysis, scene recognition, and audience reaction analysis.

[1462] Input: Video data sent from the server

[1463] Output: Analysis results (emotional state, scene recognition, audience reaction)

[1464] Specific operation: The analysis engine uses a deep learning model to analyze facial expressions, voices, and scene changes in the video and evaluate emotions and reactions.

[1465] Step 5:

[1466] The server receives the analysis results and automatically edits the video based on the identified highlight scenes.

[1467] Input: Analysis results (emotional state, scene recognition, audience reaction)

[1468] Output: Automatically edited video data

[1469] Specific operation: The server uses a video editing library such as FFmpeg to extract highlight scenes based on the analysis results, and automatically applies transitions, effects, and background music to edit the video.

[1470] Step 6:

[1471] Viewers access the server from their devices and view the saved highlight footage.

[1472] Input: Automatically edited video data

[1473] Output: Viewers watch highlight footage

[1474] Specific operation: Viewers access the server via the application and watch the highlight footage in streaming format.

[1475] Step 7:

[1476] Viewers send their opinions and suggestions for adding effects to the server.

[1477] Input: Viewer opinions and suggestions for adding effects

[1478] Output: Proposal data sent to the server

[1479] How it works: Viewers use the editing function within the application to input their opinions and the effects they want to add, and then send them to the server.

[1480] Step 8:

[1481] The server collects viewer suggestions, and the AI ​​selects and integrates the best editing ideas.

[1482] Input: Viewer-submitted suggestion data

[1483] Output: New footage based on optimized editing ideas

[1484] How it works: The server analyzes the suggestions from viewers, and the AI ​​selects the optimal editing plan. It then uses the video editing library again to generate new video data.

[1485] Step 9:

[1486] The content provider checks and approves the final edited video stored on the server.

[1487] Input: Final edited video data

[1488] Output: Content provider approval

[1489] Specific operation: The content provider uses the preview function in the server to check the final video and approves it by pressing the approval button.

[1490] Step 10:

[1491] The server uploads the approved footage to a streaming platform where it can be accessed by viewers.

[1492] Input: Approved video data

[1493] Output: Footage uploaded to a streaming platform

[1494] What it does: The server uploads the approved video to a streaming platform like YouTube or Vimeo and generates a viewing link.

[1495] Step 11:

[1496] The server processes the user's monthly subscription to the Professional plan.

[1497] Input: User's selected subscription plan information

[1498] Output: Processed monthly fee data

[1499] What it does: Your server automatically processes the user's subscription fee using a payment gateway such as Stripe or PayPal.

[1500] Step 12:

[1501] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[1502] Input: Advertising data from advertising partners

[1503] Output: Revenue and share distribution data to users

[1504] How it works: The server works with advertising platforms such as Google AdSense to track ad impressions and clicks, calculate revenue based on that, and distribute a portion of the revenue to users.

[1505] (Application example 1)

[1506] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1507] To effectively improve customer engagement during in-store events and promotional activities, there is a need for real-time editing of filmed video and rapid sharing on social media. However, traditional manual video editing is time-consuming and often results in events losing their freshness. Furthermore, efficient video analysis and identifying highlight scenes are difficult, making it difficult to generate content that will attract viewers' attention. It is necessary to solve these issues and maximize the effectiveness of in-store promotional activities while increasing customer engagement.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1509] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising collaboration unit, a revenue sharing unit, an event video shooting unit, a cloud upload unit, and a social media posting unit. This enables real-time editing of event videos in physical stores and rapid sharing to social media. It also efficiently identifies highlight scenes that will attract viewers' attention, increasing customer engagement.

[1510] An "emotion analysis means" is a means for analyzing the facial expressions and tone of voice of people in a video to recognize their emotional state.

[1511] "Real-time video analysis means" is a means for analyzing captured video in real time and recognizing specific events or situations.

[1512] The "highlight scene identification means" is a means for identifying important scenes or exciting moments in a video.

[1513] The "automatic editing means" is a means for automatically arranging and editing transitions, effects, and background music based on the identified highlight scenes.

[1514] "Collaborative editing means" refers to a means by which viewers and other stakeholders can submit additional editing suggestions and opinions, and then integrate these to create optimal video editing.

[1515] The "video approval means" is a means by which the content provider checks and finally approves the edited video.

[1516] "Video sharing means" means a means for sharing approved videos to streaming platforms or social media.

[1517] "Monthly Subscription Processing Means" means a means for processing the monthly fee for the subscription plan selected by the User.

[1518] "Advertising collaboration means" refers to a means for displaying advertisements within videos and generating revenue through collaboration with advertising partners.

[1519] "Revenue Share Vehicle" means a vehicle for sharing generated revenue with Users.

[1520] "Event video shooting means" refers to a means for shooting video of events and promotional activities held within a physical store.

[1521] The "cloud uploading means" is a means for uploading and storing the captured video to a cloud server.

[1522] "SNS posting means" refers to a means for automatically posting edited video to SNS.

[1523] This invention is a system that combines a generative AI model and real-time video analysis technology to automatically edit videos of events and promotional activities in physical stores and share them on social media, etc. To implement this invention, the following specific hardware and software configurations are required.

[1524] First, users hold events or promotional activities in physical stores. At that time, they use devices (e.g., smartphones) to capture video of the event. This means of capturing video is important for capturing scenes that attract customers' interest.

[1525] Next, the device uploads the captured video to a cloud server. This cloud uploading means sends the video data to the server and stores it in cloud storage. In this case, a common cloud service such as Amazon S3 is used for cloud storage.

[1526] The cloud server retrieves the video data from the cloud storage and uses real-time video analysis to analyze scenes in the video and customer reactions. This analysis uses emotion analysis to recognize the emotional state of people in the video by analyzing their facial expressions and tone of voice. This provides data to identify exciting moments and important scenes.

[1527] Based on the analysis results, the server extracts important scenes using a highlight scene identification method, and edits the video by adding transitions, effects, and background music using an automatic editing method, using a generative AI model to optimize the editing.

[1528] Once editing is complete, viewers and other interested parties can submit additional editing suggestions and opinions using the collaborative editing tool. These suggestions are collected on the server and integrated into the optimal video edit. The content provider then reviews and approves the edited video using the video approval tool.

[1529] Finally, the approved footage is automatically posted to streaming platforms and social media via a video sharing tool, which quickly shares the edited footage to pre-defined social media accounts.

[1530] Furthermore, the system processes the monthly fee for the subscription plan selected by the user through a monthly subscription processing means. The advertising collaboration means and revenue sharing means are used to share the generated revenue with the user.

[1531] As a concrete example, consider a new product launch event held at a brick-and-mortar store. Video of the event is shot with a smartphone and uploaded to a cloud server. After that, real-time video analysis is used to identify highlight scenes and automatically edit them. The generated video is then posted to social media after incorporating viewer feedback.

[1532] An example of a prompt is, "Generate real-time event highlights for new products." This prompt triggers the generative AI model to analyze and edit the video to generate the optimal highlight video.

[1533] As described above, the present invention provides a concrete means for effectively editing and sharing footage of events and promotional activities at physical stores.

[1534] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1535] Step 1:

[1536] A user uses a smartphone to capture video of an event in a physical store. The user uses the smartphone to capture a new product launch event or demonstration, launching an application to capture the video. The input is real-time video captured by the smartphone camera, and the output is video data stored in the smartphone.

[1537] Step 2:

[1538] The device uploads the captured video to the cloud server. Using the cloud upload function, the captured video data is sent to the server and saved in cloud storage. Specifically, the video file is transferred to cloud storage by pressing the "upload" button in the application. The input is the video data saved on the smartphone, and the output is the video data saved in cloud storage.

[1539] Step 3:

[1540] The server retrieves video data from cloud storage and begins real-time video analysis. The cloud server sends the video data to an analysis engine, which uses software to analyze emotions, scenes, and audience reactions. Based on the analysis results, highlight scenes are identified. The input is the video data stored in cloud storage, and the output is the analysis results and identified highlight scene data.

[1541] Step 4:

[1542] The server performs automatic editing based on the identified highlight scenes. Using the automatic editing method, the generative AI model optimally arranges transitions, effects, and background music to edit the video. The input is the identified highlight scene data, and the output is edited video data.

[1543] Step 5:

[1544] Viewers use the application to access the edited highlight footage and provide their opinions and effect suggestions. The collaborative editing method sends the viewers' suggestions to the server, which then collects these suggestions and integrates the best ones. The input is the viewers' opinions and effect suggestion data, and the output is the optimized video data.

[1545] Step 6:

[1546] The server provides the final edited video to the content provider for review and approval. The content provider reviews the video and requests corrections as necessary. Once all corrections and final confirmation are complete, the video is officially approved using the video approval process. The input is the optimized video data, and the output is the approved video data.

[1547] Step 7:

[1548] The server automatically posts the approved video to a pre-set SNS account using the SNS posting means. The SNS posting means accesses the SNS platform via API and shares the video. The input is the approved video data and SNS account information, and the output is the completion status of the SNS posting.

[1549] Step 8:

[1550] The server is a monthly subscription processing means that processes the monthly fee for the professional plan selected by the user. Payment is processed using a common online payment system. The input is the user's subscription information and payment information, and the output is the status that the payment has been completed.

[1551] Step 9:

[1552] The server uses an advertising collaboration mechanism to generate revenue with advertising partners and a revenue sharing mechanism to share the generated revenue with users. The inputs are advertising impression data and revenue information, and the output is the revenue sharing results.

[1553] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1554] The present invention is a platform that automatically edits live events and streaming videos shot by users by combining emotion analysis means, real-time video analysis means, highlight scene identification means, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly subscription processing means, advertising collaboration means, revenue sharing means, and an emotion engine that recognizes user emotions. The specific processing content and operation of the program are described below.

[1555] 1. Users shoot and upload footage

[1556] Users capture footage of live events using their smartphones or cameras.

[1557] The device uploads the captured video to a server via an application.

[1558] The server receives the uploaded video data and stores it in cloud storage.

[1559] Examples:

[1560] Users film a music concert with their smartphone and press the "upload" button in the app to send the footage to the server, which then stores the received video data in the cloud.

[1561] 2. Real-time video analysis

[1562] The server retrieves the video data from the cloud storage and sends it to the analysis engine.

[1563] AI performs emotion analysis within the video, scene recognition, and audience reaction analysis.

[1564] The server identifies important scenes based on the analysis results.

[1565] Examples:

[1566] The server sends the acquired video data to an analysis engine, and the AI ​​analyzes the audience's facial expressions and tone of voice to detect the moment when excitement rises.

[1567] 3. Emotion Recognition by Emotion Engine

[1568] The server uses an emotion engine to recognize the user's emotions, analyzing the user's facial expressions and tone of voice in the video to identify their emotional state.

[1569] The server collects and analyzes emotion data in real time.

[1570] Examples:

[1571] The emotion engine recognizes the user's smiling or surprised expressions in the video and sends this information to the server, which then collects the user's emotional data.

[1572] 4. Automatic Editing

[1573] The server performs automatic editing based on the identified highlight scenes and emotion data.

[1574] AI algorithms optimally place transitions, effects, and background music to edit the video to increase engagement.

[1575] Examples:

[1576] The server edits the video based on the identified exciting scenes and the user's emotional data, and applies transitions and effects to generate videos that will attract the viewer's attention.

[1577] 5. Collaborative editing

[1578] Viewers use their terminals to access the highlight footage stored on the server.

[1579] Viewers submit their opinions and suggestions for adding effects to the server.

[1580] The server collects viewer suggestions, and AI selects and integrates the best suggestions.

[1581] Examples:

[1582] Viewers can watch highlight footage through the app and suggest improvements or effects they would like to add. The server then collects these suggestions and performs optimal editing.

[1583] 6. Video Approval and Sharing

[1584] The content provider checks the final edited video from the server and gives their approval.

[1585] Once approved, the footage is shared to the streaming platform via a server.

[1586] The device accesses the shared video and begins viewing it.

[1587] Examples:

[1588] The content provider reviews the final video stored on the server and presses the approval button, which then uploads the approved video to the streaming platform, where it becomes accessible to viewers through the application.

[1589] 7. Monthly Subscription, Advertising Integration, Revenue Sharing

[1590] The server processes the monthly subscription for the Professional plan selected by the user.

[1591] The server generates revenue in collaboration with advertising partners and implements a revenue share model.

[1592] Examples:

[1593] A user selects the professional plan in the application, pays a monthly fee, and the server calculates the revenue generated from ad impressions and shares it with the user.

[1594] In this way, the system of the present invention utilizes AI technology and an emotion engine to significantly improve the efficiency of video editing, reduce the burden on creators and viewers, and provide a sustainable revenue model.

[1595] The processing flow will be explained below.

[1596] Step 1:

[1597] Users use their smartphones or cameras to capture footage of live events, and the captured video data is stored on their devices.

[1598] Step 2:

[1599] The user launches the application, selects the captured video file, and presses the "Upload" button. The device then sends the selected video data to the server via an HTTP request.

[1600] Step 3:

[1601] The server receives the uploaded video data, stores it in cloud storage, and then notifies the user that the upload was successful.

[1602] Step 4:

[1603] The server sends the video data stored in cloud storage to an analysis engine, which processes the video data frame by frame to perform emotion analysis, scene recognition, and audience reaction analysis.

[1604] Step 5:

[1605] The AI ​​analysis engine uses facial expression recognition algorithms and audio analysis to identify emotions and key scenes in the video, and the analysis results are sent to a server and stored in a database.

[1606] Step 6:

[1607] The server uses the analysis results to identify highlight scenes from the video, and a trending algorithm compares the latest trend information collected from social media and other media to prioritize the identified scenes.

[1608] Step 7:

[1609] The server runs an emotion engine to recognize the user's emotions. The emotion engine analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. The recognized emotion data is sent to the server.

[1610] Step 8:

[1611] The server automatically edits the video based on the identified highlight scenes and recognized emotional data. The auto-editing module uses AI algorithms to optimally position transitions, effects, and background music.

[1612] Step 9:

[1613] The edited footage is sent back to the server and saved in cloud storage. The user is notified when editing is complete and provided with a link to access the highlights.

[1614] Step 10:

[1615] Viewers can access the highlight footage stored on the server using their devices, and use the application's editing functions to create their own opinions and suggestions for adding effects.

[1616] Step 11:

[1617] The device sends the viewer's suggestions to the server, which then collects them and uses AI to select the best ones to create the final edited video.

[1618] Step 12:

[1619] The content provider accesses the server to review the final edited footage, and once the provider approves the footage, the server uploads the approved footage to the streaming platform.

[1620] Step 13:

[1621] The device will access the video uploaded to the streaming platform and begin watching, and the user will be notified that the video is available for viewing.

[1622] Step 14:

[1623] The server processes the monthly subscription for the Professional plan selected by the user, securely managing and periodically updating subscription payment information.

[1624] Step 15:

[1625] The server inserts advertisements provided by advertising partners into the videos and generates revenue based on ad impressions and clicks, which is shared with users via a revenue share mechanism.

[1626] In this way, the present invention uses an emotion engine to recognize user emotions, significantly improving the efficiency of video editing, while reducing the burden on creators and viewers and providing a sustainable revenue model.

[1627] Example 2

[1628] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1629] Conventional video editing systems require a lot of manual editing work by the user, which is time-consuming and labor-intensive. Furthermore, they lack sufficient support for sentiment analysis to increase viewer engagement, automatic highlight scene identification, and monetization. This places a heavy burden on creators and viewers, making it difficult to build a sustainable revenue model.

[1630] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes an emotion analysis means, a real-time video analysis means, a highlight scene identification means, an emotion recognition means using an emotion engine, an automatic editing means, a collaborative editing means, a video approval means, a video sharing means, a monthly service processing means, an advertising partner collaboration means, and a revenue sharing means. This significantly improves the efficiency of automatic video analysis and editing, reduces the burden on creators and viewers, and enables the provision of a sustainable revenue model.

[1631] "Emotion analysis means" is a technology for analyzing the facial expressions and tone of voice of users or viewers to identify their emotional state.

[1632] "Real-time video analysis means" is a technology for analyzing live and recorded video in real time to detect important scenes and events.

[1633] The "highlight scene identification means" is a technology for automatically identifying important scenes in a video or moments that will attract the viewer's attention.

[1634] "Emotion recognition means using an emotion engine" refers to dedicated algorithms and software for recognizing the emotions of people in video.

[1635] "Automatic editing methods" are technologies that use AI algorithms to automatically edit video based on highlight scenes and emotional data.

[1636] "Collaborative editing means" is a technology that allows viewers and users to give their opinions and suggestions for adding effects to a video, and then carry out optimal editing based on those opinions.

[1637] "Video approval means" refers to a technology that allows the content provider to check and approve the final edited video.

[1638] "Video sharing tools" are technologies for sharing edited videos via streaming platforms, social media, etc.

[1639] "Monthly service processing means" refers to technology for managing and processing subscriptions to monthly services selected by users.

[1640] "Advertising partner collaboration means" refers to technology for collaborating with advertising partners to generate and manage advertising revenue.

[1641] A "revenue sharing method" is a technology that allows revenue generated from advertising, subscriptions, etc. to be distributed fairly among the parties involved.

[1642] The present invention is a system that includes emotion analysis means, real-time video analysis means, highlight scene identification means, emotion recognition means using an emotion engine, automatic editing means, collaborative editing means, video approval means, video sharing means, monthly service processing means, advertising partner collaboration means, and revenue sharing means. This system automatically analyzes and edits live event and streaming video shot by users, and provides a platform that supports monetization.

[1643] First, a user uses a smartphone or camera to capture video of a live event. After capturing the video, the device uploads the video to a server. The server receives the uploaded video data and stores it in cloud storage, ensuring secure storage of the data needed for subsequent processing steps.

[1644] The server then retrieves the video data from the cloud storage and sends it to an analytics engine, which performs real-time sentiment analysis, scene recognition, and audience reaction analysis. Based on the results of this analysis, the server identifies important scenes within the video. For example, it can detect exciting moments by analyzing the audience's facial expressions and tone of voice.

[1645] The server also uses an emotion engine to recognize the user's emotions. It analyzes the user's facial expressions and tone of voice in the video to identify their emotional state. Emotional data is collected in real time and automatically edited based on the analysis results. For example, the emotion engine can recognize the user's smiling or surprised expressions and edit the video based on that information to generate a video that will attract the viewer's attention.

[1646] The automated editing uses AI algorithms based on identified highlights and emotional data to optimally position transitions, effects, and background music to create highly engaging videos. Specific editing operations include adding transitions during exciting moments and applying visual effects.

[1647] With the collaborative editing feature, viewers can access highlight footage using their devices. Viewers can send their opinions and suggestions for adding effects to the server. The server collects these suggestions, and AI selects the best ones and incorporates them into the editing. Viewers can suggest improvements and effects while watching the highlight footage through the app.

[1648] After the final editing is complete, the content provider checks and approves the final edited video from the server. The approved video is then shared via the server to streaming platforms and social media. Viewers can then access the shared video from their devices and begin watching.

[1649] Users select the professional plan in the application, pay a monthly fee, and the server processes this monthly service. The server also works with advertising partners to generate advertising revenue, which is then shared with users through a fair revenue share mechanism.

[1650] This system will significantly reduce the burden on creators and viewers, enabling the creation of high-quality video content and the creation of a sustainable revenue model.

[1651] Examples of prompts include:

[1652] "Automatically detect exciting scenes and apply transitions and effects to edit them."

[1653] "Add the best effects to your footage based on your audience's feedback."

[1654] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1655] Step 1:

[1656] Input: Footage of a live event captured by a user using a smartphone or camera.

[1657] How it works: Users upload footage of live events to a server via an application on their device.

[1658] Specific operation: A user takes a photo of an event with their smartphone and presses the "upload" button within the application.

[1659] Output: Uploaded video data is sent to the server and stored in cloud storage.

[1660] Step 2:

[1661] Input: Video data stored in cloud storage.

[1662] Operation: The server retrieves video data from cloud storage and sends it to the analysis engine.

[1663] Specific operation: The server retrieves video data from storage and transfers it to the AI ​​analysis engine.

[1664] Output: The video data is passed to the analysis engine, which then performs emotion analysis, scene recognition, and audience reaction analysis to generate highlight scene information.

[1665] Step 3:

[1666] Input: Emotion data sent from the analysis engine, scene recognition information, and audience reaction analysis results.

[1667] How it works: The server uses an emotion engine to further recognize the user's emotions and analyze the user's facial expressions and tone of voice in the video.

[1668] Specific operation: The emotion engine analyzes the user's smiling or surprised expressions in the video and sends the emotion data to the server.

[1669] Output: The emotion data sent to the server is converted into information that represents the user's detailed emotional state.

[1670] Step 4:

[1671] Input: Emotion data, highlight scene information.

[1672] How it works: The server automatically edits the video based on this data, optimally positioning transitions, effects, and background music.

[1673] What it does: The AI ​​algorithm adds transitions and effects to the identified scenes and selects appropriate background music.

[1674] Output: An attractive highlight video is generated.

[1675] Step 5:

[1676] Input: Auto-edited highlight footage.

[1677] How it works: Viewers use their devices to access highlight footage stored on the server and offer their opinions and suggest additional effects.

[1678] Specific operation: Viewers watch highlight footage through the application and suggest improvements.

[1679] Output: Viewer suggestions are sent to the server, and the AI ​​selects and integrates the best suggestions.

[1680] Step 6:

[1681] Input: Final edited highlight reel.

[1682] How it works: The content provider reviews and approves the final edited footage.

[1683] Specific operation: The content provider checks the final video on the server and presses the "Approve" button.

[1684] Output: Approved footage is uploaded to a streaming platform via a server.

[1685] Step 7:

[1686] Input: Service usage data, advertising data.

[1687] What it does: The server processes the monthly service plan selected by the user and works with advertising partners to generate revenue.

[1688] What happens: A user selects the Professional plan and pays a monthly fee. The server calculates the revenue earned from ad impressions.

[1689] Output: Revenue is generated and shared with users.

[1690] (Application example 2)

[1691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1692] Conventional video editing systems require users to manually edit the footage they shoot, which takes time and effort. It's also difficult to incorporate viewer opinions and feedback in real time, which can lead to a decline in content quality. Furthermore, revenue is limited, making it difficult to build a sustainable model.

[1693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1694] In this invention, the server includes a sentiment analysis unit, a real-time video analysis unit, a highlight scene identification unit, an automatic editing unit, a collaborative editing unit, a video approval unit, a video sharing unit, a monthly subscription processing unit, an advertising linkage unit, a revenue sharing unit, a cloud storage unit, a key scene detection unit using emotion data, a unit for editing video based on viewer suggestions for improvement, and a unit for delivering the final edited video. This makes it possible to automatically analyze and edit video shot by users and reflect viewer feedback in real time. It also enables the creation of a sustainable revenue model.

[1695] An "emotion analysis means" is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[1696] The "real-time video analysis means" is a device that analyzes uploaded video in real time and detects scene features and important moments.

[1697] The "highlight scene identifying means" is a device that identifies scenes from the video that are particularly important and have high engagement.

[1698] The "automatic editing means" is an algorithm that automatically edits video based on the identified highlight scenes and emotion data.

[1699] The "collaborative editing tool" is a device that edits video based on suggestions and opinions from viewers and generates optimal content.

[1700] The "video approval means" is a device that allows the content provider to check and approve the final edited video.

[1701] "Video sharing means" refers to a device for sharing approved edited footage on streaming platforms or social media.

[1702] The "monthly subscription processing means" is a device that processes the monthly fee for the professional plan selected by the user.

[1703] The "advertising linking means" is a device that links and displays advertisements with video and generates revenue from them.

[1704] A "revenue sharing vehicle" is a device for sharing generated revenue between users and application operators.

[1705] "Cloud storage means" refers to a device for storing and managing video data on the cloud.

[1706] The "means for detecting important scenes using emotion data" is a device for detecting important scenes in a video based on the results of emotion analysis.

[1707] The "means for editing video based on viewer suggestions for improvement" is a device for collecting feedback from viewers and editing video based on that content.

[1708] "Means for distributing final edited video" refers to a device for distributing approved final edited video.

[1709] This invention relates to a system that allows users to upload footage of live events or specific moments they have filmed to cloud storage, and then automatically edits the footage using emotion analysis and real-time video analysis. The system identifies important scenes based on emotion data and edits the footage by incorporating viewer feedback. It also has the ability to distribute the edited footage after approval and share revenue.

[1710] Hardware and Software Configuration

[1711] 1. Cloud Storage Solutions

[1712] This is a device that allows users to upload and store footage they have taken on the cloud.

[1713] Specifically, this includes Amazon S3 and Google Cloud Storage.

[1714] 2. Sentiment analysis method

[1715] This is a device that analyzes the facial expressions and tone of voice of users and audience members in a video to identify their emotional state.

[1716] Specifically, this includes Emotion APIs (e.g., Microsoft Azure Emotion API).

[1717] 3. Real-time video analysis methods

[1718] This device analyzes uploaded video in real time and detects scene features and important moments.

[1719] Specifically, this includes libraries such as OpenCV and TensorFlow.

[1720] 4. Automated Editing Methods

[1721] This device uses an AI algorithm to automatically edit video based on identified highlight scenes and emotional data.

[1722] Specifically, this includes FFmpeg and other video editing libraries.

[1723] 5. Collaborative editing tools

[1724] This device edits video based on suggestions and opinions from viewers and generates optimal content.

[1725] Specifically, this includes a web interface for collecting user feedback and analysis using AI models.

[1726] 6. Video Approval Method

[1727] This is a device that allows content providers to check and approve the final edited footage.

[1728] Specifically, this applies to dedicated review interfaces.

[1729] 7. Video sharing method

[1730] It is a device for sharing approved edited footage on streaming platforms and social media.

[1731] Specifically, this includes the YouTube API and Facebook API.

[1732] 8. Monthly Subscription Processing Method

[1733] This is the device that processes the monthly fee for the professional plan selected by the user.

[1734] Specifically, this includes subscription management systems such as Stripe and PayPal.

[1735] 9. Advertising Linkage Methods

[1736] This is a device that displays advertisements in conjunction with video and generates revenue from them.

[1737] Specifically, this includes Google AdSense and Facebook Audience Network.

[1738] 10. Revenue Share Instruments

[1739] It is a device for sharing generated revenue between users and application operators.

[1740] Specifically, this applies to revenue calculation algorithms and automatic transfer systems.

[1741] Specific examples

[1742] 1. Users film the concert and upload it to the app

[1743] Example: A user films a music concert with their smartphone and presses the "upload" button to send the footage to the server.

[1744] The server stores the received video data in cloud storage.

[1745] 2. Incorporating viewer feedback

[1746] Example: Viewers can watch highlights on the app and suggest effects and edits.

[1747] The server collects this information and the AI ​​performs optimal editing.

[1748] 3. Distribution and Revenue Share

[1749] For example: Once approved, the final edited footage will be shared to YouTube and Instagram, generating advertising revenue.

[1750] Revenues are shared between users and app operators.

[1751] Examples of prompt statements

[1752] "Video editing using emotion analysis in the EmotionClipper app"

[1753] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1754] Step 1:

[1755] A user films a live event and uploads the footage to a server via a smartphone app. Specifically, the user sends the video data to cloud storage using the "Upload" button in the app. The input of this step is the filmed video data, and the output is the video data saved in cloud storage.

[1756] Step 2:

[1757] The server retrieves the uploaded video data from cloud storage and sends it to the emotion analysis means. Specifically, the server sends the video data to the Emotion Engine, which then performs emotion analysis. The input of this step is the video data retrieved from cloud storage, and the output is the analyzed emotion data.

[1758] Step 3:

[1759] The server sends the emotion analysis data to the real-time video analysis means to detect important scenes in the video. Specifically, the server uses an analysis engine to recognize specific scenes and features from the video data and identify important scenes. The inputs to this step are emotion analysis data and video data, and the output is the identification of important scenes.

[1760] Step 4:

[1761] The server performs automatic editing based on the identified important scenes and emotional data. Specifically, the server uses AI algorithms to apply transitions and effects to generate optimal footage. The input for this step is the identification of important scenes and emotional data, and the output is automatically edited footage.

[1762] Step 5:

[1763] The server collects user and viewer feedback and performs additional editing using collaborative editing methods. Specifically, viewers send effects and correction suggestions through an in-app interface, and the server then optimizes the edits based on that. The inputs for this step are viewer feedback and the automatically edited video, and the output is the collaboratively edited video.

[1764] Step 6:

[1765] The content provider checks the final edited video on the server and approves it through the video approval means. Specifically, the content provider watches the final video on a dedicated interface and presses the approval button. The input of this step is the collaboratively edited video, and the output is the approved final video.

[1766] Step 7:

[1767] The server distributes the approved video to streaming platforms and social media platforms to share the video. Specifically, the server uploads the video using the YouTube API or Facebook API and makes it available to viewers. The input of this step is the final approved video, and the output is the shared video URL.

[1768] Step 8:

[1769] The server processes monthly subscriptions and advertising, and shares the generated revenue with users. Specifically, the server processes subscription fees using Stripe or PayPal APIs, calculates advertising revenue, and distributes it to users. The input for this step is subscription information and ad impression data, and the output is the revenue share result.

[1770] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1771] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1772] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1773] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1774] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1775] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1776] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1777] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, motorcycles, and other devices, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1778] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1779] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1780] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1781] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1782] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1783] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1784] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1785] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1786] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1787] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1788] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1789] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1790] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1791] The following is further disclosed regarding the above embodiment.

[1792] (Claim 1)

[1793] A sentiment analysis means;

[1794] Real-time video analysis means;

[1795] highlight scene identification means;

[1796] an automatic editing means;

[1797] Collaborative editing tools and

[1798] a video approval means;

[1799] Video sharing means,

[1800] a monthly subscription processing means;

[1801] Advertisement linkage means;

[1802] Revenue sharing instruments;

[1803] A system including:

[1804] (Claim 2)

[1805] 10. The system of claim 1, wherein video editing is performed based on sentiment analysis.

[1806] (Claim 3)

[1807] 10. The system of claim 1, wherein the content provider approves the highlights collaboratively edited by the viewers.

[1808] "Example 1"

[1809] (Claim 1)

[1810] A sentiment analysis means;

[1811] Real-time video analysis means;

[1812] highlight scene identification means;

[1813] an automatic editing means;

[1814] Collaborative editing tools and

[1815] a video approval means;

[1816] Video sharing means,

[1817] a monthly subscription processing means;

[1818] Advertisement linkage means;

[1819] Revenue sharing instruments;

[1820] A means for storing video data in cloud storage;

[1821] A means for acquiring video data from cloud storage;

[1822] A method to edit video using AI algorithms, including transitions and effects,

[1823] A means of collecting viewer suggestions and selecting and integrating the best editing ideas;

[1824] A way to upload footage to a streaming platform,

[1825] A system including:

[1826] (Claim 2)

[1827] 10. The system of claim 1, wherein video editing is performed based on sentiment analysis.

[1828] (Claim 3)

[1829] 10. The system of claim 1, wherein the content provider approves the highlights collaboratively edited by the viewers.

[1830] "Application Example 1"

[1831] (Claim 1)

[1832] A sentiment analysis means;

[1833] Real-time video analysis means;

[1834] highlight scene identification means;

[1835] an automatic editing means;

[1836] Collaborative editing tools and

[1837] a video approval means;

[1838] Video sharing means,

[1839] a monthly subscription processing means;

[1840] Advertisement linkage means;

[1841] Revenue sharing instruments;

[1842] Event video recording means;

[1843] Cloud upload means;

[1844] SNS posting methods and

[1845] A system including:

[1846] (Claim 2)

[1847] 10. The system of claim 1, wherein video editing is performed based on sentiment analysis.

[1848] (Claim 3)

[1849] 10. The system of claim 1, wherein the content provider approves the highlights collaboratively edited by the viewers.

[1850] "Example 2: Combining Emotion Engines"

[1851] (Claim 1)

[1852] A sentiment analysis means;

[1853] Real-time video analysis means;

[1854] highlight scene identification means;

[1855] an emotion recognition means using an emotion engine;

[1856] an automatic editing means;

[1857] Collaborative editing tools and

[1858] a video approval means;

[1859] Video sharing means,

[1860] a monthly service processing means;

[1861] Advertising partner collaboration means;

[1862] Revenue sharing instruments;

[1863] A system including:

[1864] (Claim 2)

[1865] 2. The system according to claim 1, wherein video editing is performed based on emotion analysis and emotion recognition by an emotion engine.

[1866] (Claim 3)

[1867] 10. The system of claim 1, wherein the content provider approves the highlights collaboratively edited by the viewers.

[1868] "Application example 2 when combining emotion engines"

[1869] (Claim 1)

[1870] A sentiment analysis means;

[1871] Real-time video analysis means;

[1872] highlight scene identification means;

[1873] an automatic editing means;

[1874] Collaborative editing tools and

[1875] a video approval means;

[1876] Video sharing means,

[1877] a monthly subscription processing means;

[1878] Advertisement linkage means;

[1879] Revenue sharing instruments;

[1880] Cloud storage means;

[1881] An important scene detection means using emotion data;

[1882] A means of editing the video based on viewers' suggestions for improvement,

[1883] A means of distributing the final edited footage;

[1884] A system including:

[1885] (Claim 2)

[1886] 10. The system of claim 1, wherein video editing is performed based on sentiment analysis.

[1887] (Claim 3)

[1888] 10. The system of claim 1, wherein the content provider approves the highlights collaboratively edited by the viewers. [Explanation of symbols]

[1889] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A sentiment analysis means; Real-time video analysis means; highlight scene identification means; an automatic editing means; Collaborative editing tools and a video approval means; Video sharing means, a monthly subscription processing means; Advertisement linkage means; Revenue sharing instruments; A system including:

2. The system of claim 1 , wherein video editing is performed based on emotion analysis.

3. The system of claim 1 , wherein the content provider approves the highlights collaboratively edited by the viewers.

Images