System

Abstract

An object of the system according to the embodiment is to automatically edit a live event or a streaming video and provide a highlight reflecting an emotion or a trend of a viewer.SOLUTION: A system according to an embodiment includes a generation AI, a collaborative editing function, an approval function, an emotion analyzing function, and a video trend prediction function. The generation AI automatically edits the live event or streaming video using the generation AI and real-time video analysis techniques. The collaborative editing feature allows viewers to collaboratively edit highlights of an event. In the approval function, the event organizer or the creator approves the highlight edited by the collaborative editing function. The emotion analysis function analyzes the emotion of the viewer. The video trend prediction function predicts a trend of a video.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, editing of live events and streaming video is done manually, which is inefficient and makes it difficult to edit in a way that reflects viewer emotions and trends.

[0005] The system according to the embodiment aims to automatically edit live events and streaming video to provide highlights that reflect the emotions and trends of viewers. [Means for solving the problem]

[0006] The system according to the embodiment comprises a generation AI, a collaborative editing function, an approval function, a sentiment analysis function, and a video trend prediction function. The generation AI automatically edits live events and streaming video using the generation AI and real-time video analysis technology. The collaborative editing function allows viewers to collaboratively edit event highlights. The approval function allows event organizers and creators to approve the highlights edited by the collaborative editing function. The sentiment analysis function analyzes viewer sentiment. The video trend prediction function predicts video trends. [Effects of the Invention]

[0007] The system according to the embodiment can automatically edit live events and streaming video to provide highlights that reflect the emotions and trends of viewers. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The automated editing platform of an embodiment of the present invention is a system that uses generative AI and real-time video analysis technology to automatically edit live events and streaming videos, allows viewers to collaboratively approve edited highlights, and performs sentiment analysis and video trend prediction. As a result, the automated editing platform improves the efficiency of editing live events and streaming videos, increases viewer participation, and increases revenue.

[0029] An automatic editing platform according to an embodiment includes a generation AI, real-time video analysis technology, a collaborative editing function, an approval function, a sentiment analysis function, and a video trend prediction function. The generation AI automatically edits live events and streaming videos. For example, the generation AI can analyze video of a live concert to capture and edit the most exciting moments and audience reactions. The generation AI can also analyze video of a sporting event to edit the video to highlight important play scenes. The generation AI can also analyze news video and edit important news as highlights. The real-time video analysis technology analyzes video in real time. For example, the real-time video analysis technology can detect specific objects in the video using object detection technology. The real-time video analysis technology can also recognize specific people in the video using facial recognition technology. The real-time video analysis technology can also analyze movements in the video using motion analysis technology. The collaborative editing function allows viewers to collaboratively edit event highlights. For example, viewers can select specific scenes and add comments and effects. Viewers can also use a voting function to decide which scenes are the most important. Viewers can also edit in real time and share with other viewers. The approval function allows event organizers or creators to approve highlights edited by viewers. For example, if a video edited by a viewer is approved by the organizer, the video is published as an official highlight. The approval function can also set an approval flow and approve videos based on approval criteria. The approval function can also notify users of the approval results. The sentiment analysis function analyzes viewer emotions. For example, the sentiment analysis function analyzes viewers' facial expressions using facial recognition technology. The sentiment analysis function can also analyze viewers' tone of voice using voice analysis technology. The sentiment analysis function can also analyze viewers' comments using text analysis technology. The video trend prediction function predicts video trends. For example, the video trend prediction function predicts future trends based on past data. The video trend prediction function can also predict trends by analyzing social media data.The video trend prediction function can also predict trends using machine learning models. As a result, the automatic editing platform according to the embodiment can improve the efficiency of editing live events and streaming videos, increase viewer participation, and improve revenue. For example, viewers can share highlights they have edited themselves, which can increase the appeal of the event and increase the number of viewers. Furthermore, sentiment analysis and trend prediction can provide content that viewers want.

[0030] Generative AI can automatically recognize specific objects or people in video and edit the video based on the recognition results. For example, generative AI can track specific players in video of a sporting event and edit the video to highlight their playing scenes. Generative AI can also use deep learning technology to train itself to recognize specific objects or people in video, achieving high accuracy. For example, it can use facial recognition technology to capture the facial expressions of specific people and highlight those scenes. Generative AI can also recognize specific objects or people in video and automatically change the camera angle based on the recognition results. For example, in video of a live concert, it can edit the video to focus on the singer's performance. This makes it possible to edit the video to highlight specific objects or people.

[0031] Generative AI can analyze the audio data of a video and edit the video to match the rhythm and tone of the music. For example, generative AI can analyze the audio data of a video and automatically adjust video cuts and transitions to match the rhythm and tone of the music. For example, it can switch video scenes to match the beat of the music. Generative AI can also analyze the audio data to understand the emotional tone of the music and adjust the color tone and effects of the video based on that. For example, it can apply a warm color filter to emotional scenes. Generative AI can also analyze the audio data and adjust the speed of the video to match the rhythm of the music. For example, it can speed up the playback speed of the video to match fast-paced music. This makes it possible to edit video to match the rhythm and tone of the music.

[0032] Generative AI can automatically switch between footage from different camera angles to provide the optimal perspective. For example, generative AI can analyze footage from multiple camera angles and automatically select and switch between the optimal perspectives. For example, it can display important play scenes at a sporting event from the optimal angle. Generative AI can also analyze footage from different camera angles in real time and build a system that automatically switches between angles to match the viewer's perspective. For example, if a viewer is paying attention to a particular player, it will prioritize the angle of that player. Furthermore, when switching camera angles, generative AI takes into account the viewer's emotional data and provides a perspective that is likely to resonate emotionally. For example, in a scene that excites the viewer, it will select a close-up angle. This makes it possible to automatically switch camera angles to provide the optimal perspective.

[0033] Generative AI can automatically adjust the color tone and filters of video to generate visually appealing video. For example, generative AI can automatically analyze the color tone of video and adjust it to a visually appealing color tone. For example, it can optimize the brightness and contrast of the video. Generative AI can also automatically select filters to apply to video to generate visually appealing video. For example, it can apply a vintage filter to specific scenes. When adjusting the color tone and filters of video, generative AI can also take into account the viewer's emotional data to generate video that is more likely to resonate emotionally. For example, it can apply a warm color filter to moving scenes. This makes it possible to automatically generate visually appealing video.

[0034] The collaborative editing feature allows the generative AI to automatically add effects and transitions to videos edited by viewers. For example, the collaborative editing feature will build a system in which the generative AI automatically adds effects and transitions to videos edited by viewers. For example, fade-in and fade-out effects can be applied to scenes selected by the viewer. The generative AI will also analyze the viewer's edits and automatically select the effects and transitions that are best suited to that content. For example, dynamic transitions can be applied to action scenes. The generative AI will also develop a system that adds effects and transitions in real time to videos edited by viewers. For example, effects can be automatically applied each time a viewer completes an edit. This makes it possible to automatically add effects and transitions to videos edited by viewers.

[0035] The collaborative editing function allows viewers to reflect real-time feedback from other viewers in videos they have edited. For example, the collaborative editing function will build a system that reflects real-time feedback from other viewers in videos they have edited. For example, it will add a function that allows viewers to post comments and ratings. The collaborative editing function will also develop a system that collects feedback from other viewers in real time and adjusts the editing content based on that feedback. For example, it will highlight scenes that receive a lot of positive feedback. The collaborative editing function will also provide an interface that allows viewers to reflect feedback from other viewers in videos they have edited. For example, it will adjust effects and transitions based on the feedback. This will allow viewer feedback to be reflected in real time.

[0036] The collaborative editing function will enable viewers to seamlessly share edited videos across different devices. For example, the collaborative editing function will build a system that allows viewers to seamlessly share edited videos across different devices. For example, it will allow videos edited on a smartphone to be played on a PC. The collaborative editing function will also develop a cloud-based platform for synchronizing edited content across different devices. For example, it will allow viewers to store edited videos in the cloud and access them from any device. The collaborative editing function will also add a function that allows viewers to share edited videos across different devices in real time. For example, it will allow videos edited on a smartphone to be instantly played on a TV. This will enable seamless sharing of edited videos across different devices.

[0037] The collaborative editing feature allows generative AI to automatically add subtitles and captions to videos edited by viewers. The collaborative editing feature will build a system in which generative AI automatically adds subtitles and captions to videos edited by viewers. For example, it will analyze the audio in the video and automatically generate subtitles. The generative AI will also analyze the edits made by the viewer and automatically select the subtitles and captions that best suit that content. For example, it will add highlighted subtitles to important scenes. The generative AI will also develop a system that adds subtitles and captions in real time to videos edited by viewers. For example, subtitles will be automatically applied each time a viewer completes editing. This will allow subtitles and captions to be automatically added to videos edited by viewers.

[0038] The approval function uses generative AI to check the quality of video and point out problems. For example, the approval function will build a system in which generative AI automatically analyzes video quality and points out problems with image quality and sound quality. For example, it will detect video resolution and audio noise. When checking video quality, the approval function will have the generative AI evaluate it based on specific criteria and list problems. For example, it will point out video blurring or inconsistencies in color tone. The approval function will also develop a system in which generative AI analyzes video quality in real time and immediately points out problems. For example, it will detect quality issues during live streaming and display a warning. This will allow video quality checks to be performed automatically and problems to be pointed out.

[0039] The approval function uses generative AI to check the copyright of videos and detect possible violations. For example, the approval function involves building a system in which generative AI analyzes the content of videos and automatically detects possible copyright violations. For example, it checks the copyright of music and images within videos. When conducting copyright checks on videos, the approval function also involves the generative AI comparing the results with a specific database and listing possible violations. For example, it detects parts that match existing copyrighted works. The approval function also involves developing a system in which generative AI analyzes the copyright of videos in real time and immediately points out possible violations. For example, it could detect copyright violations during live streaming and display a warning. This makes it possible to automatically check the copyright of videos and detect possible violations.

[0040] The approval function can automatically distribute approved videos to different streaming platforms. For example, the approval function builds a system that automatically distributes approved videos to different streaming platforms. For example, it can simultaneously distribute to multiple platforms such as YouTube and Twitch. When distributing to different streaming platforms, the approval function also uses generation AI to automatically convert the video to fit the format of each platform. For example, it adjusts the resolution and bitrate. The approval function also adds the ability to distribute approved videos to different streaming platforms in real time. For example, it can simultaneously distribute to multiple platforms during live streaming. This allows approved videos to be automatically distributed to different streaming platforms.

[0041] The approval function uses generative AI to generate video metadata and improve searchability. For example, the approval function builds a system in which generative AI analyzes the content of a video and automatically generates metadata. For example, it automatically extracts keywords and tags within the video. In addition, when generating video metadata, the approval function has the generative AI evaluate it based on specific criteria, improving searchability. For example, it generates metadata based on the genre or theme of the video. In addition, the approval function develops a system in which generative AI generates video metadata in real time, instantly improving searchability. For example, it generates metadata during live streaming and reflects it in search results. This generates video metadata and improves searchability.

[0042] The emotion analysis function analyzes a viewer's past viewing history and can predict individual emotional tendencies. For example, the emotion analysis function builds a system in which a generation AI analyzes a viewer's past viewing history and predicts individual emotional tendencies. For example, it analyzes tendencies based on the emotional data of videos the viewer has previously watched. In addition, when analyzing viewing history, the generation AI detects specific patterns and predicts the viewer's emotional tendencies. For example, it identifies a tendency for viewers to prefer moving scenes. In addition, the emotion analysis function develops a system in which a generation AI analyzes a viewer's past viewing history in real time and instantly predicts individual emotional tendencies. For example, the emotional tendencies are updated every time the viewer watches a new video. This makes it possible to predict emotional tendencies based on the viewer's past viewing history.

[0043] The video trend prediction function can analyze the emotional data of viewers from different regions and cultural spheres to predict global trends. For example, in this function, a generation AI collects emotional data from viewers from different regions and cultural spheres and builds a system to predict global trends based on that data. For example, it analyzes emotional trends by region. In addition, when analyzing the emotional data of viewers from different cultural spheres, the generation AI detects specific patterns and predicts global trends. For example, it identifies scenes that are popular in specific cultural spheres. In addition, in this function, a generation AI analyzes the emotional data of viewers from different regions and cultural spheres in real time to develop a system that instantly predicts global trends. For example, the trend is updated every time a viewer watches a new video. This makes it possible to analyze the emotional data of viewers from different regions and cultural spheres and predict global trends.

[0044] The video trend prediction function can make personalized video recommendations based on viewer emotional data. For example, the video trend prediction function builds a system in which a generation AI collects viewer emotional data and makes personalized video recommendations based on that data. For example, it can recommend videos that include scenes that move the viewer. Furthermore, when analyzing viewer emotional data, the generation AI detects specific patterns and makes personalized video recommendations. For example, it can recommend videos that include scenes that excite the viewer. Furthermore, the video trend prediction function develops a system in which the generation AI analyzes viewer emotional data in real time and makes personalized video recommendations instantly. For example, the recommendation content can be updated every time the viewer watches a new video. This makes it possible to make personalized video recommendations based on the viewer's emotional data.

[0045] In the Professional plan, the revenue model allows the generation AI to automatically suggest optimal ad insertion points. For example, the revenue model involves building a system in which the generation AI analyzes the content of a video and automatically suggests optimal ad insertion points. For example, it detects natural breaks in the video and scene transitions. Furthermore, when suggesting ad insertion points, the generation AI considers viewer emotional data and selects timing that is likely to resonate emotionally. For example, it inserts ads in scenes where the viewer is relaxing. Furthermore, the revenue model involves developing a system in which the generation AI analyzes the content of a video and viewer emotional data and suggests optimal ad insertion points in real time. For example, it dynamically adjusts ad insertion points during live streaming. This allows the optimal ad insertion points to be automatically suggested.

[0046] In the Professional Plan, the revenue model allows the generation AI to measure the effectiveness of advertisements in real time based on viewer emotional data. For example, the revenue model involves building a system in which the generation AI collects viewer emotional data and measures the effectiveness of advertisements in real time based on that data. For example, it analyzes changes in viewer emotion and evaluates the effectiveness of advertisements. In addition, when measuring the effectiveness of advertisements, the generation AI analyzes viewer emotional data in real time and evaluates the performance of advertisements. For example, if the viewer is excited, it determines that the advertisement is effective. In addition, the revenue model involves developing a system in which the generation AI measures the effectiveness of advertisements in real time based on viewer emotional data and provides the results to advertisers. For example, if the effectiveness of an advertisement is low, it will make suggestions to adjust the advertisement content. This makes it possible to measure the effectiveness of advertisements in real time based on viewer emotional data.

[0047] In a revenue sharing model, a generation AI can automatically calculate the optimal revenue sharing rate. For example, the revenue model will build a system in which a generation AI automatically calculates the optimal revenue sharing rate in a revenue sharing model. For example, the distribution rate will be determined based on the viewer's viewing time and engagement. In addition, when calculating the revenue sharing rate, the generation AI will take into account viewer emotional data and set a higher distribution rate for content that is likely to resonate emotionally. For example, a higher distribution rate will be set for scenes that move viewers. In addition, the revenue model will develop a system in which a generation AI calculates the optimal revenue sharing rate in a revenue sharing model in real time and provides the results to creators. For example, the revenue sharing rate will be dynamically adjusted during live streaming. This will allow the optimal revenue sharing rate to be calculated automatically.

[0048] In advertising collaboration, the revenue model allows the generative AI to optimize ad targeting based on viewer emotional data. For example, the revenue model involves building a system in which the generative AI collects viewer emotional data and optimizes ad targeting based on that data. For example, it might display ads related to scenes that move the viewer. In addition, when optimizing ad targeting, the revenue model involves the generative AI analyzing viewer emotional data in real time to improve targeting accuracy. For example, if the viewer is excited, it might display ads for action movies. In addition, the revenue model involves developing a system in which the generative AI optimizes ad targeting in real time based on viewer emotional data and provides the results to advertisers. For example, if the effectiveness of an ad is low, it might make suggestions to adjust the targeting. This makes it possible to optimize ad targeting based on viewer emotional data.

[0049] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0050] An automatic editing platform can automatically switch between footage from different camera angles to provide the optimal perspective. For example, it can display important play scenes from the best angle at a sporting event. It can also build a system that automatically switches based on the viewer's perspective. For example, if a viewer is paying attention to a particular player, it can prioritize the angle of that player. It can also take into account the viewer's emotional data when switching camera angles to provide a perspective that is likely to resonate with them emotionally. For example, in a scene that excites the viewer, it can select a close-up angle. This makes it possible to automatically switch camera angles to provide the optimal perspective.

[0051] An automated editing platform can automatically adjust the color tone and filters of video to generate visually appealing video. For example, it can optimize the brightness and contrast of video. It can also automatically select filters to apply to video to generate visually appealing video. For example, it can apply a vintage filter to specific scenes. It can also take into account the viewer's emotional data when adjusting the color tone and filters of video to generate video that is more likely to resonate emotionally. For example, it can apply a warm color filter to emotionally appealing scenes. In this way, it can automatically generate visually appealing video.

[0052] An automated editing platform would enable generative AI to automatically add effects and transitions to footage edited by viewers. For example, it could apply fade-in and fade-out effects to scenes selected by the viewer. It could also analyze the viewer's edits and automatically select the most suitable effects and transitions for that content. For example, it could apply dynamic transitions to action scenes. It would also be possible to develop a system that adds effects and transitions in real time to footage edited by viewers. For example, effects could be automatically applied each time a viewer completes editing. This would allow effects and transitions to be automatically added to footage edited by viewers.

[0053] An automated editing platform can incorporate real-time feedback from other viewers into videos edited by viewers. For example, a function could be added that allows viewers to post comments and ratings. It is also possible to develop a system that collects feedback from other viewers in real time and adjusts the editing content based on that feedback. For example, scenes that receive a lot of positive feedback could be emphasized. It is also possible to provide an interface that allows viewers to incorporate feedback from other viewers into videos they have edited. For example, effects and transitions could be adjusted based on the feedback. This allows viewer feedback to be reflected in real time.

[0054] An automated editing platform could enable viewers to seamlessly share edited footage across different devices. For example, it could allow footage edited on a smartphone to be played back on a PC. It could also develop a cloud-based platform for synchronizing edited content across different devices. For example, it could store edited footage in the cloud and make it accessible from any device. It could also add a feature that allows viewers to share edited footage in real time across different devices. For example, it could allow footage edited on a smartphone to be instantly played back on a TV. This would allow edited footage to be shared seamlessly across different devices.

[0055] An automatic editing platform can use generative AI to automatically add subtitles and captions to videos edited by viewers. For example, it can analyze the audio in the video and automatically generate subtitles. It can also analyze the viewer's edit content and automatically select the subtitles and captions that are most suitable for that content. For example, it can add highlighted subtitles to important scenes. It is also possible to develop a system that adds subtitles and captions in real time to videos edited by viewers. For example, subtitles are automatically applied each time a viewer completes editing. This makes it possible to automatically add subtitles and captions to videos edited by viewers.

[0056] The processing flow of the first embodiment will be briefly explained below.

[0057] Step 1: Generative AI automatically edits live events and streaming footage. For example, it can analyze footage of a live concert to capture and edit the most exciting moments and audience reactions. It can also analyze footage of a sporting event to edit it to emphasize important play scenes. It can also analyze news footage to edit important news as highlights. Real-time video analysis technology analyzes video in real time, using object detection technology to detect specific objects within the video, facial recognition technology to recognize specific people within the video, and motion analysis technology to analyze movements within the video. Step 2: The collaborative editing feature allows viewers to collaboratively edit the highlights of the event. For example, viewers can select specific scenes and add comments and effects. They can also use the voting function to decide which scenes are the most important. Furthermore, viewers can make edits in real time and share them with other viewers. Step 3: The approval feature allows event organizers and creators to approve the highlights edited by viewers. For example, if a video edited by a viewer is approved by the organizer, it will be published as an official highlight. You can also set up an approval flow and approve videos based on approval criteria. You can also notify the approval results. Step 4: The emotion analysis function analyzes the emotions of viewers. For example, facial expression recognition technology can be used to analyze the facial expressions of viewers. Voice analysis technology can also be used to analyze the tone of the viewers' voices. Furthermore, text analysis technology can be used to analyze the viewers' comments. Step 5: The video trend prediction function predicts video trends. For example, it predicts future trends based on past data. It can also analyze social media data and predict trends. It can also predict trends using machine learning models.

[0058] (Example 2) The automated editing platform of an embodiment of the present invention is a system that uses generative AI and real-time video analysis technology to automatically edit live events and streaming videos, allows viewers to collaboratively approve edited highlights, and performs sentiment analysis and video trend prediction. As a result, the automated editing platform improves the efficiency of editing live events and streaming videos, increases viewer participation, and increases revenue.

[0059] An automatic editing platform according to an embodiment includes a generation AI, real-time video analysis technology, a collaborative editing function, an approval function, a sentiment analysis function, and a video trend prediction function. The generation AI automatically edits live events and streaming videos. For example, the generation AI can analyze video of a live concert to capture and edit the most exciting moments and audience reactions. The generation AI can also analyze video of a sporting event to edit the video to highlight important play scenes. The generation AI can also analyze news video and edit important news as highlights. The real-time video analysis technology analyzes video in real time. For example, the real-time video analysis technology can detect specific objects in the video using object detection technology. The real-time video analysis technology can also recognize specific people in the video using facial recognition technology. The real-time video analysis technology can also analyze movements in the video using motion analysis technology. The collaborative editing function allows viewers to collaboratively edit event highlights. For example, viewers can select specific scenes and add comments and effects. Viewers can also use a voting function to decide which scenes are the most important. Viewers can also edit in real time and share with other viewers. The approval function allows event organizers or creators to approve highlights edited by viewers. For example, if a video edited by a viewer is approved by the organizer, the video is published as an official highlight. The approval function can also set an approval flow and approve videos based on approval criteria. The approval function can also notify users of the approval results. The sentiment analysis function analyzes viewer emotions. For example, the sentiment analysis function analyzes viewers' facial expressions using facial recognition technology. The sentiment analysis function can also analyze viewers' tone of voice using voice analysis technology. The sentiment analysis function can also analyze viewers' comments using text analysis technology. The video trend prediction function predicts video trends. For example, the video trend prediction function predicts future trends based on past data. The video trend prediction function can also predict trends by analyzing social media data.The video trend prediction function can also predict trends using machine learning models. As a result, the automatic editing platform according to the embodiment can improve the efficiency of editing live events and streaming videos, increase viewer participation, and improve revenue. For example, viewers can share highlights they have edited themselves, which can increase the appeal of the event and increase the number of viewers. Furthermore, sentiment analysis and trend prediction can provide content that viewers want.

[0060] Generative AI can automatically recognize specific objects or people in video and edit the video based on the recognition results. For example, generative AI can track specific players in video of a sporting event and edit the video to highlight their playing scenes. Generative AI can also use deep learning technology to train itself to recognize specific objects or people in video, achieving high accuracy. For example, it can use facial recognition technology to capture the facial expressions of specific people and highlight those scenes. Generative AI can also recognize specific objects or people in video and automatically change the camera angle based on the recognition results. For example, in video of a live concert, it can edit the video to focus on the singer's performance. This makes it possible to edit the video to highlight specific objects or people.

[0061] Generative AI can analyze the audio data of a video and edit the video to match the rhythm and tone of the music. For example, generative AI can analyze the audio data of a video and automatically adjust video cuts and transitions to match the rhythm and tone of the music. For example, it can switch video scenes to match the beat of the music. Generative AI can also analyze the audio data to understand the emotional tone of the music and adjust the color tone and effects of the video based on that. For example, it can apply a warm color filter to emotional scenes. Generative AI can also analyze the audio data and adjust the speed of the video to match the rhythm of the music. For example, it can speed up the playback speed of the video to match fast-paced music. This makes it possible to edit video to match the rhythm and tone of the music.

[0062] Generative AI can estimate the viewer's emotions in real time and edit the video accordingly. For example, if the viewer is excited, it will emphasize scenes that will uplift that emotion. Generative AI can also use its emotion estimation function to collect viewer emotional data and generate a video editing plan based on that data. For example, it can extend the scenes that move the viewer. Generative AI can also analyze the viewer's emotions and adjust the video's music and effects based on the results. For example, if the viewer is happy, it can add upbeat music and effects. This makes it possible to edit video according to the viewer's emotions.

[0063] Generative AI can automatically switch between footage from different camera angles to provide the optimal perspective. For example, generative AI can analyze footage from multiple camera angles and automatically select and switch between the optimal perspectives. For example, it can display important play scenes at a sporting event from the optimal angle. Generative AI can also analyze footage from different camera angles in real time and build a system that automatically switches between angles to match the viewer's perspective. For example, if a viewer is paying attention to a particular player, it will prioritize the angle of that player. Furthermore, when switching camera angles, generative AI takes into account the viewer's emotional data and provides a perspective that is likely to resonate emotionally. For example, in a scene that excites the viewer, it will select a close-up angle. This makes it possible to automatically switch camera angles to provide the optimal perspective.

[0064] Generative AI can automatically adjust the color tone and filters of video to generate visually appealing video. For example, generative AI can automatically analyze the color tone of video and adjust it to a visually appealing color tone. For example, it can optimize the brightness and contrast of the video. Generative AI can also automatically select filters to apply to video to generate visually appealing video. For example, it can apply a vintage filter to specific scenes. When adjusting the color tone and filters of video, generative AI can also take into account the viewer's emotional data to generate video that is more likely to resonate emotionally. For example, it can apply a warm color filter to moving scenes. This makes it possible to automatically generate visually appealing video.

[0065] Generative AI can adjust the speed and tempo of a video based on the viewer's emotions. For example, generative AI can estimate the viewer's emotions in real time and adjust the speed and tempo of a video based on those emotions. For example, if the viewer is excited, the playback speed of the video can be increased. Generative AI can also use its emotion estimation function to collect viewer emotional data and adjust the tempo of the video based on that data. For example, if the viewer is relaxed, the playback speed of the video can be decreased. Generative AI can also analyze the viewer's emotions and build a system that dynamically adjusts the speed and tempo of a video based on the results. For example, the speed of a video can be adjusted each time the viewer's emotions change. This makes it possible to adjust the speed and tempo of a video based on the viewer's emotions.

[0066] The collaborative editing feature allows the generative AI to automatically add effects and transitions to videos edited by viewers. For example, the collaborative editing feature will build a system in which the generative AI automatically adds effects and transitions to videos edited by viewers. For example, fade-in and fade-out effects can be applied to scenes selected by the viewer. The generative AI will also analyze the viewer's edits and automatically select the effects and transitions that are best suited to that content. For example, dynamic transitions can be applied to action scenes. The generative AI will also develop a system that adds effects and transitions in real time to videos edited by viewers. For example, effects can be automatically applied each time a viewer completes an edit. This makes it possible to automatically add effects and transitions to videos edited by viewers.

[0067] The collaborative editing function allows viewers to reflect real-time feedback from other viewers in videos they have edited. For example, the collaborative editing function will build a system that reflects real-time feedback from other viewers in videos they have edited. For example, it will add a function that allows viewers to post comments and ratings. The collaborative editing function will also develop a system that collects feedback from other viewers in real time and adjusts the editing content based on that feedback. For example, it will highlight scenes that receive a lot of positive feedback. The collaborative editing function will also provide an interface that allows viewers to reflect feedback from other viewers in videos they have edited. For example, it will adjust effects and transitions based on the feedback. This will allow viewer feedback to be reflected in real time.

[0068] The collaborative editing function can use the emotion estimation function to suggest editing content based on the viewer's emotions. For example, the collaborative editing function uses the emotion estimation function to collect viewer emotion data and build a system that suggests editing content based on that data. For example, it suggests editing that emphasizes scenes that move the viewer. The collaborative editing function also analyzes the viewer's emotions in real time and suggests editing content based on those emotions. For example, if the viewer is excited, it suggests editing that emphasizes action scenes. The collaborative editing function also develops a system that suggests editing content that matches the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, it suggests editing that emphasizes calm scenes. This makes it possible to suggest editing content based on the viewer's emotions.

[0069] The collaborative editing function will enable viewers to seamlessly share edited videos across different devices. For example, the collaborative editing function will build a system that allows viewers to seamlessly share edited videos across different devices. For example, it will allow videos edited on a smartphone to be played on a PC. The collaborative editing function will also develop a cloud-based platform for synchronizing edited content across different devices. For example, it will allow viewers to store edited videos in the cloud and access them from any device. The collaborative editing function will also add a function that allows viewers to share edited videos across different devices in real time. For example, it will allow videos edited on a smartphone to be instantly played on a TV. This will enable seamless sharing of edited videos across different devices.

[0070] The collaborative editing feature allows generative AI to automatically add subtitles and captions to videos edited by viewers. The collaborative editing feature will build a system in which generative AI automatically adds subtitles and captions to videos edited by viewers. For example, it will analyze the audio in the video and automatically generate subtitles. The generative AI will also analyze the edits made by the viewer and automatically select the subtitles and captions that best suit that content. For example, it will add highlighted subtitles to important scenes. The generative AI will also develop a system that adds subtitles and captions in real time to videos edited by viewers. For example, subtitles will be automatically applied each time a viewer completes editing. This will allow subtitles and captions to be automatically added to videos edited by viewers.

[0071] The collaborative editing function can use the emotion estimation function to suggest editing themes and styles based on the viewer's emotions. For example, the collaborative editing function uses the emotion estimation function to collect viewer emotion data and build a system to suggest editing themes and styles based on that data. For example, if the viewer is moved, an emotional theme is suggested. The collaborative editing function also analyzes the viewer's emotions in real time and suggests editing themes and styles based on those emotions. For example, if the viewer is excited, an action theme is suggested. The collaborative editing function also develops a system to suggest editing themes and styles that match the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, a calm style is suggested. This makes it possible to suggest editing themes and styles based on the viewer's emotions.

[0072] The approval function uses generative AI to check the quality of video and point out problems. For example, the approval function will build a system in which generative AI automatically analyzes video quality and points out problems with image quality and sound quality. For example, it will detect video resolution and audio noise. When checking video quality, the approval function will have the generative AI evaluate it based on specific criteria and list problems. For example, it will point out video blurring or inconsistencies in color tone. The approval function will also develop a system in which generative AI analyzes video quality in real time and immediately points out problems. For example, it will detect quality issues during live streaming and display a warning. This will allow video quality checks to be performed automatically and problems to be pointed out.

[0073] The approval function uses generative AI to check the copyright of videos and detect possible violations. For example, the approval function involves building a system in which generative AI analyzes the content of videos and automatically detects possible copyright violations. For example, it checks the copyright of music and images within videos. When conducting copyright checks on videos, the approval function also involves the generative AI comparing the results with a specific database and listing possible violations. For example, it detects parts that match existing copyrighted works. The approval function also involves developing a system in which generative AI analyzes the copyright of videos in real time and immediately points out possible violations. For example, it could detect copyright violations during live streaming and display a warning. This makes it possible to automatically check the copyright of videos and detect possible violations.

[0074] The approval function can use the emotion estimation function to determine approval priorities based on the viewer's emotions. For example, the approval function uses the emotion estimation function to collect viewer emotion data and build a system that determines approval priorities based on that data. For example, videos that make the viewer excited are given priority in approval. The approval function also analyzes the viewer's emotions in real time and determines approval priorities based on those emotions. For example, videos that make the viewer feel moved are given priority in approval. The approval function also develops a system that determines approval priorities based on the viewer's emotions based on the emotion estimation data. For example, videos that make the viewer feel relaxed are given priority in approval. In this way, approval priorities can be determined based on the viewer's emotions.

[0075] The approval function can automatically distribute approved videos to different streaming platforms. For example, the approval function builds a system that automatically distributes approved videos to different streaming platforms. For example, it can simultaneously distribute to multiple platforms such as YouTube and Twitch. When distributing to different streaming platforms, the approval function also uses generation AI to automatically convert the video to fit the format of each platform. For example, it adjusts the resolution and bitrate. The approval function also adds the ability to distribute approved videos to different streaming platforms in real time. For example, it can simultaneously distribute to multiple platforms during live streaming. This allows approved videos to be automatically distributed to different streaming platforms.

[0076] The approval function uses generative AI to generate video metadata and improve searchability. For example, the approval function builds a system in which generative AI analyzes the content of a video and automatically generates metadata. For example, it automatically extracts keywords and tags within the video. In addition, when generating video metadata, the approval function has the generative AI evaluate it based on specific criteria, improving searchability. For example, it generates metadata based on the genre or theme of the video. In addition, the approval function develops a system in which generative AI generates video metadata in real time, instantly improving searchability. For example, it generates metadata during live streaming and reflects it in search results. This generates video metadata and improves searchability.

[0077] The approval function can use the emotion estimation function to tag videos based on the viewer's emotions. For example, the approval function uses the emotion estimation function to collect viewer emotion data and build a system that tags videos based on that data. For example, a scene in which the viewer is moved is tagged with "emotional." The approval function also analyzes the viewer's emotions in real time and tags videos based on those emotions. For example, a scene in which the viewer is excited is tagged with "exciting." The approval function also develops a system that tags videos in accordance with the viewer's emotions based on the emotion estimation data. For example, a scene in which the viewer is relaxing is tagged with "relaxing." This makes it possible to tag videos based on the viewer's emotions.

[0078] The emotion analysis function analyzes viewers' facial expressions and voices, making it possible to track changes in emotions in real time. For example, the emotion analysis function will build a system in which generative AI analyzes viewers' facial expressions in real time and tracks changes in emotions. For example, it will detect viewers' smiling or surprised expressions. The emotion analysis function will also analyze viewers' voices and track changes in emotions in real time. For example, it will analyze the tone and pitch of the viewer's voice to infer emotions. The emotion analysis function will also develop a system in which generative AI simultaneously analyzes viewers' facial expressions and voices, and comprehensively tracks changes in emotions. For example, it will integrate data on viewers' facial expressions and voices to infer emotions. This will make it possible to track changes in viewers' emotions in real time.

[0079] The emotion analysis function analyzes a viewer's past viewing history and can predict individual emotional tendencies. For example, the emotion analysis function builds a system in which a generation AI analyzes a viewer's past viewing history and predicts individual emotional tendencies. For example, it analyzes tendencies based on the emotional data of videos the viewer has previously watched. In addition, when analyzing viewing history, the generation AI detects specific patterns and predicts the viewer's emotional tendencies. For example, it identifies a tendency for viewers to prefer moving scenes. In addition, the emotion analysis function develops a system in which a generation AI analyzes a viewer's past viewing history in real time and instantly predicts individual emotional tendencies. For example, the emotional tendencies are updated every time the viewer watches a new video. This makes it possible to predict emotional tendencies based on the viewer's past viewing history.

[0080] The emotion analysis function can use the emotion estimation function to adjust the editing content of video in real time based on the viewer's emotions. For example, the emotion analysis function uses the emotion estimation function to collect viewer emotion data in real time and build a system that adjusts the editing content of video based on that data. For example, scenes that move the viewer are displayed longer. The emotion analysis function also analyzes the viewer's emotions in real time and dynamically adjusts the editing content of video based on that emotion. For example, if the viewer is excited, action scenes are emphasized. The emotion analysis function also develops a system that adjusts the editing content of video in real time to match the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, calm scenes are emphasized. This makes it possible to adjust the editing content of video in real time based on the viewer's emotions.

[0081] The video trend prediction function can analyze the emotional data of viewers from different regions and cultural spheres to predict global trends. For example, in this function, a generation AI collects emotional data from viewers from different regions and cultural spheres and builds a system to predict global trends based on that data. For example, it analyzes emotional trends by region. In addition, when analyzing the emotional data of viewers from different cultural spheres, the generation AI detects specific patterns and predicts global trends. For example, it identifies scenes that are popular in specific cultural spheres. In addition, in this function, a generation AI analyzes the emotional data of viewers from different regions and cultural spheres in real time to develop a system that instantly predicts global trends. For example, the trend is updated every time a viewer watches a new video. This makes it possible to analyze the emotional data of viewers from different regions and cultural spheres and predict global trends.

[0082] The video trend prediction function can make personalized video recommendations based on viewer emotional data. For example, the video trend prediction function builds a system in which a generation AI collects viewer emotional data and makes personalized video recommendations based on that data. For example, it can recommend videos that include scenes that move the viewer. Furthermore, when analyzing viewer emotional data, the generation AI detects specific patterns and makes personalized video recommendations. For example, it can recommend videos that include scenes that excite the viewer. Furthermore, the video trend prediction function develops a system in which the generation AI analyzes viewer emotional data in real time and makes personalized video recommendations instantly. For example, the recommendation content can be updated every time the viewer watches a new video. This makes it possible to make personalized video recommendations based on the viewer's emotional data.

[0083] The video trend prediction function can use the emotion estimation function to optimize the advertising content of a video based on the viewer's emotions. For example, the video trend prediction function uses the emotion estimation function to collect viewer emotion data and build a system that optimizes the advertising content of a video based on that data. For example, advertisements related to scenes that move the viewer are displayed. The video trend prediction function also analyzes the viewer's emotions in real time and dynamically adjusts the advertising content based on those emotions. For example, if the viewer is excited, an advertisement for an action movie is displayed. The video trend prediction function also develops a system that optimizes the advertising content to match the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, an advertisement for a relaxation product is displayed. This makes it possible to optimize the advertising content of a video based on the viewer's emotions.

[0084] In the Professional plan, the revenue model allows the generation AI to automatically suggest optimal ad insertion points. For example, the revenue model involves building a system in which the generation AI analyzes the content of a video and automatically suggests optimal ad insertion points. For example, it detects natural breaks in the video and scene transitions. Furthermore, when suggesting ad insertion points, the generation AI considers viewer emotional data and selects timing that is likely to resonate emotionally. For example, it inserts ads in scenes where the viewer is relaxing. Furthermore, the revenue model involves developing a system in which the generation AI analyzes the content of a video and viewer emotional data and suggests optimal ad insertion points in real time. For example, it dynamically adjusts ad insertion points during live streaming. This allows the optimal ad insertion points to be automatically suggested.

[0085] In the Professional Plan, the revenue model allows the generation AI to measure the effectiveness of advertisements in real time based on viewer emotional data. For example, the revenue model involves building a system in which the generation AI collects viewer emotional data and measures the effectiveness of advertisements in real time based on that data. For example, it analyzes changes in viewer emotion and evaluates the effectiveness of advertisements. In addition, when measuring the effectiveness of advertisements, the generation AI analyzes viewer emotional data in real time and evaluates the performance of advertisements. For example, if the viewer is excited, it determines that the advertisement is effective. In addition, the revenue model involves developing a system in which the generation AI measures the effectiveness of advertisements in real time based on viewer emotional data and provides the results to advertisers. For example, if the effectiveness of an advertisement is low, it will make suggestions to adjust the advertisement content. This makes it possible to measure the effectiveness of advertisements in real time based on viewer emotional data.

[0086] The revenue model can use the emotion estimation function to customize the content of advertisements based on the viewer's emotions. For example, the revenue model uses the emotion estimation function to collect viewer emotion data and build a system that customizes the content of advertisements based on that data. For example, an emotional advertisement is displayed in a scene where the viewer is moved. The revenue model also analyzes the viewer's emotions in real time and dynamically adjusts the content of advertisements based on those emotions. For example, if the viewer is excited, an advertisement for an action movie is displayed. The revenue model also develops a system that customizes the content of advertisements to match the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, an advertisement for a relaxation product is displayed. This makes it possible to customize the content of advertisements based on the viewer's emotions.

[0087] In a revenue sharing model, a generation AI can automatically calculate the optimal revenue sharing rate. For example, the revenue model will build a system in which a generation AI automatically calculates the optimal revenue sharing rate in a revenue sharing model. For example, the distribution rate will be determined based on the viewer's viewing time and engagement. In addition, when calculating the revenue sharing rate, the generation AI will take into account viewer emotional data and set a higher distribution rate for content that is likely to resonate emotionally. For example, a higher distribution rate will be set for scenes that move viewers. In addition, the revenue model will develop a system in which a generation AI calculates the optimal revenue sharing rate in a revenue sharing model in real time and provides the results to creators. For example, the revenue sharing rate will be dynamically adjusted during live streaming. This will allow the optimal revenue sharing rate to be calculated automatically.

[0088] In advertising collaboration, the revenue model allows the generative AI to optimize ad targeting based on viewer emotional data. For example, the revenue model involves building a system in which the generative AI collects viewer emotional data and optimizes ad targeting based on that data. For example, it might display ads related to scenes that move the viewer. In addition, when optimizing ad targeting, the revenue model involves the generative AI analyzing viewer emotional data in real time to improve targeting accuracy. For example, if the viewer is excited, it might display ads for action movies. In addition, the revenue model involves developing a system in which the generative AI optimizes ad targeting in real time based on viewer emotional data and provides the results to advertisers. For example, if the effectiveness of an ad is low, it might make suggestions to adjust the targeting. This makes it possible to optimize ad targeting based on viewer emotional data.

[0089] The revenue model can use the emotion estimation function to propose new revenue models based on the viewer's emotions. For example, the revenue model uses the emotion estimation function to collect viewer emotion data and build a system that proposes new revenue models based on that data. For example, a revenue model related to scenes that move the viewer is proposed. The revenue model also analyzes the viewer's emotions in real time and proposes new revenue models based on those emotions. For example, if the viewer is excited, a revenue model for action movies is proposed. The revenue model also develops a system that proposes new revenue models that match the viewer's emotions based on the emotion estimation data. For example, if the viewer is relaxed, a revenue model for relaxation products is proposed. This makes it possible to propose new revenue models based on the viewer's emotions.

[0090] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0091] The automatic editing platform can estimate the viewer's emotions in real time and edit the video based on those emotions. For example, if the viewer is excited, scenes that heighten that emotion can be emphasized. It can also display scenes that move the viewer for longer. It can also analyze the viewer's emotions and adjust the music and effects of the video based on the results. For example, if the viewer is happy, upbeat music and effects can be added. This makes it possible to edit the video according to the viewer's emotions.

[0092] An automatic editing platform can automatically switch between footage from different camera angles to provide the optimal perspective. For example, it can display important play scenes from the best angle at a sporting event. It can also build a system that automatically switches based on the viewer's perspective. For example, if a viewer is paying attention to a particular player, it can prioritize the angle of that player. It can also take into account the viewer's emotional data when switching camera angles to provide a perspective that is likely to resonate with them emotionally. For example, in a scene that excites the viewer, it can select a close-up angle. This makes it possible to automatically switch camera angles to provide the optimal perspective.

[0093] An automated editing platform can automatically adjust the color tone and filters of video to generate visually appealing video. For example, it can optimize the brightness and contrast of video. It can also automatically select filters to apply to video to generate visually appealing video. For example, it can apply a vintage filter to specific scenes. It can also take into account the viewer's emotional data when adjusting the color tone and filters of video to generate video that is more likely to resonate emotionally. For example, it can apply a warm color filter to emotionally appealing scenes. In this way, it can automatically generate visually appealing video.

[0094] An automatic editing platform can adjust the speed and tempo of a video based on the viewer's emotions. For example, if the viewer is excited, the video playback speed can be increased. It can also collect viewer emotional data and adjust the video tempo based on that data. For example, if the viewer is relaxed, the video playback speed can be decreased. It is also possible to build a system that analyzes viewer emotions and dynamically adjusts the speed and tempo of a video based on the results. For example, the video speed can be adjusted every time the viewer's emotions change. This makes it possible to adjust the speed and tempo of a video based on the viewer's emotions.

[0095] An automated editing platform would enable generative AI to automatically add effects and transitions to footage edited by viewers. For example, it could apply fade-in and fade-out effects to scenes selected by the viewer. It could also analyze the viewer's edits and automatically select the most suitable effects and transitions for that content. For example, it could apply dynamic transitions to action scenes. It would also be possible to develop a system that adds effects and transitions in real time to footage edited by viewers. For example, effects could be automatically applied each time a viewer completes editing. This would allow effects and transitions to be automatically added to footage edited by viewers.

[0096] An automated editing platform can incorporate real-time feedback from other viewers into videos edited by viewers. For example, a function could be added that allows viewers to post comments and ratings. It is also possible to develop a system that collects feedback from other viewers in real time and adjusts the editing content based on that feedback. For example, scenes that receive a lot of positive feedback could be emphasized. It is also possible to provide an interface that allows viewers to incorporate feedback from other viewers into videos they have edited. For example, effects and transitions could be adjusted based on the feedback. This allows viewer feedback to be reflected in real time.

[0097] An automatic editing platform can use its emotion estimation function to suggest editing content based on the viewer's emotions. For example, a system can be built that collects viewer emotion data and suggests editing content based on that data. For example, it can suggest editing content that emphasizes scenes that move the viewer. It can also analyze viewer emotions in real time and suggest editing content based on those emotions. For example, if the viewer is excited, it can suggest editing content that emphasizes action scenes. It is also possible to develop a system that suggests editing content that matches the viewer's emotions based on emotion estimation data. For example, if the viewer is relaxed, it can suggest editing content that emphasizes calm scenes. This makes it possible to suggest editing content based on the viewer's emotions.

[0098] An automated editing platform could enable viewers to seamlessly share edited footage across different devices. For example, it could allow footage edited on a smartphone to be played back on a PC. It could also develop a cloud-based platform for synchronizing edited content across different devices. For example, it could store edited footage in the cloud and make it accessible from any device. It could also add a feature that allows viewers to share edited footage in real time across different devices. For example, it could allow footage edited on a smartphone to be instantly played back on a TV. This would allow edited footage to be shared seamlessly across different devices.

[0099] An automatic editing platform can use generative AI to automatically add subtitles and captions to videos edited by viewers. For example, it can analyze the audio in the video and automatically generate subtitles. It can also analyze the viewer's edit content and automatically select the subtitles and captions that are most suitable for that content. For example, it can add highlighted subtitles to important scenes. It is also possible to develop a system that adds subtitles and captions in real time to videos edited by viewers. For example, subtitles are automatically applied each time a viewer completes editing. This makes it possible to automatically add subtitles and captions to videos edited by viewers.

[0100] The automatic editing platform can use emotion estimation functions to suggest editing themes and styles based on the viewer's emotions. For example, a system can be built that collects viewer emotion data and suggests editing themes and styles based on that data. For example, if the viewer is moved, an emotional theme can be suggested. It is also possible to analyze the viewer's emotions in real time and suggest editing themes and styles based on those emotions. For example, if the viewer is excited, an action theme can be suggested. It is also possible to develop a system that suggests editing themes and styles that match the viewer's emotions based on emotion estimation data. For example, if the viewer is relaxed, a calm style can be suggested. This makes it possible to suggest editing themes and styles based on the viewer's emotions.

[0101] The processing flow of the second embodiment will be briefly explained below.

[0102] Step 1: Generative AI automatically edits live events and streaming footage. For example, it can analyze footage of a live concert to capture and edit the most exciting moments and audience reactions. It can also analyze footage of a sporting event to edit it to emphasize important play scenes. It can also analyze news footage to edit important news as highlights. Real-time video analysis technology analyzes video in real time, using object detection technology to detect specific objects within the video, facial recognition technology to recognize specific people within the video, and motion analysis technology to analyze movements within the video. Step 2: The collaborative editing feature allows viewers to collaboratively edit the highlights of the event. For example, viewers can select specific scenes and add comments and effects. They can also use the voting function to decide which scenes are the most important. Furthermore, viewers can make edits in real time and share them with other viewers. Step 3: The approval feature allows event organizers and creators to approve the highlights edited by viewers. For example, if a video edited by a viewer is approved by the organizer, it will be published as an official highlight. You can also set up an approval flow and approve videos based on approval criteria. You can also notify the approval results. Step 4: The emotion analysis function analyzes the emotions of viewers. For example, facial expression recognition technology can be used to analyze the facial expressions of viewers. Voice analysis technology can also be used to analyze the tone of the viewers' voices. Furthermore, text analysis technology can be used to analyze the viewers' comments. Step 5: The video trend prediction function predicts video trends. For example, it predicts future trends based on past data. It can also analyze social media data and predict trends. It can also predict trends using machine learning models.

[0103] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0104] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0105] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0106] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0107] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0108] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0109] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0110] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0111] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0112] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0113] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0114] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0115] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0116] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0117] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0118] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0119] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0120] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0121] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0122] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0124] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0125] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0126] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0128] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0129] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0130] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0131] In the headset type terminal 314, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0132] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0134] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0135] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0136] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0137] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0138] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0139] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0140] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0141] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0142] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0143] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0144] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0145] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0147] In the robot 414, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0148] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0149] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0150] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AI other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0151] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0152] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0153] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0154] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0155] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0156] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0157] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0158] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0159] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0160] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0161] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0162] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0163] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0164] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0165] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0166] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0167] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0168] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0169] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0170] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. Generative AI and real-time video analysis technology are used to automatically edit live events and streaming footage. A collaborative editing feature that allows viewers to collaborate on editing event highlights, An approval function that allows event organizers and creators to approve highlights edited by the collaborative editing function; Sentiment analysis function to analyze viewer emotions, A video trend prediction function that predicts video trends. A system characterized by:

2. The generated AI is Automatically recognize specific objects or people in the video and edit the video based on the recognition results.

2. The system of claim 1.

3. The generated AI is Analyze the audio data of the video and edit the video to match the rhythm and tone of the music.

2. The system of claim 1.

4. The generated AI is The emotion of the viewer is estimated in real time, and the video is edited in accordance with the emotion.

2. The system of claim 1.

5. The generated AI is Automatically switches between footage from different camera angles to provide the best perspective 2. The system of claim 1.

Images