system

Abstract

A system is provided.SOLUTION: A system comprising: means for logging into a service by inputting authentication information with a terminal available to a user; means for providing video streaming of a game; means for displaying generated real-time commentary and SNS posts superimposed on game video; and means for automatically generating and providing a highlight video based on a viewing history and preferences of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In conventional sports viewing, viewers have difficulty easily obtaining detailed information and background information about the game in real time, and there is a lack of ways to view highlights tailored to their individual needs even after the game has ended. Furthermore, there is insufficient integration with social media, resulting in a lack of a sense of unity in the viewing experience. Another issue is the lack of commentary to help users properly understand important moments in the game. The present invention aims to solve these issues and provide a new viewing experience that allows viewers to enjoy the game more deeply. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for users to log in to the service by entering authentication information on their device, a means for providing game video streaming, a means for overlaying generated real-time commentary and social media posts on the game video, and a means for automatically generating and providing highlight videos based on the user's viewing history and preferences. The system further provides a means for collecting social media posts in real time during game video streaming, analyzing the posts to select those with high relevance, and a means for analyzing game video in real time to detect important and specific events and automatically generating commentary based on the events, thereby improving the viewer experience. This allows viewers to obtain detailed game information in real time and enables a more immersive viewing experience through integration with social media. Furthermore, personalized highlights can be provided after the game to meet the needs of individual viewers.

[0006] "User" refers to an individual or group who uses the Service using a Terminal.

[0007] "Device" refers to an electronic device that can connect to the Internet, such as a smartphone, tablet, or PC.

[0008] "Authentication Information" refers to the username and password entered by a User to log in to the Service, or any other similar information.

[0009] "Service" refers to the entire system that provides a viewing experience that combines live streaming of sporting events and SNS functions according to the present invention.

[0010] "Match" means a competition in a sporting event.

[0011] "Video streaming" refers to the technology of delivering game footage in real time over the Internet.

[0012] "Real-time commentary" refers to AI-generated, moment-by-moment game situation explanations during a match.

[0013] "SNS posts" refer to the posting of content such as comments, images, and videos on social networking services.

[0014] "Overlaying on game footage" refers to overlaying other information (e.g., commentary or social media posts) on top of the video stream.

[0015] "Viewing history" refers to a record of the games and events that a user has watched.

[0016] "Preferences" refers to information such as the sports teams, players, and types of games that a user is particularly interested in.

[0017] "Highlight videos" refer to short video content that has been edited to extract important scenes from a game or parts that interest users.

[0018] "Auto-generation" refers to the process of using AI or algorithms to create videos and explanations without human intervention.

[0019] "Collecting" refers to obtaining and organizing data from the internet (e.g., social media posts).

[0020] "Analyzing" refers to analyzing collected data using AI and algorithms to find meaning and patterns.

[0021] "Highly relevant" refers to data that has been selected from the collected data based on specific criteria (e.g., sentiment score or content consistency).

[0022] An "event" refers to a significant action that occurs during a game (e.g., a run scored, a home run).

[0023] "Detect" refers to using AI or algorithms to recognize specific events in game footage. [Brief explanation of the drawings]

[0024] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0025] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0026] First, the terms used in the following description will be explained.

[0027] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0028] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0029] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0030] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0031] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0032] [First embodiment]

[0033] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0034] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0036] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0037] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0038] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0040] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0041] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0042] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0043] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0044] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0045] The present invention relates to a system for providing a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0046] Authentication and Login

[0047] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[0048] Select a match and start watching

[0049] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0050] Social Media Feed Integration

[0051] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0052] Real-time match commentary and highlight generation

[0053] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0054] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences.

[0055] For example, if a user is rooting for a particular player, a highlight video is generated that mainly shows scenes in which that player is playing an active role. The generated highlight video is transmitted from the server to the terminal and played on the terminal.

[0056] Specific examples

[0057] Specific examples of the present invention are shown below.

[0058] Authentication and Login

[0059] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0060] Select a match and start watching

[0061] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[0062] Social Media Feed Integration

[0063] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0064] Real-time match commentary and highlight generation

[0065] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, saying, "A player from Team A just hit a home run!" The user requests a highlight video after the game ends, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0066] This invention allows viewers to enjoy sporting events from a new perspective, creating an integrated viewing experience with detailed real-time information and social media posts, and allows viewers to easily watch personalized highlight videos after the game.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[0070] Step 2:

[0071] The device sends the entered authentication information to the server in JSON format.

[0072] Example of sending data: {"username": "example_user", "password": "example_password"}

[0073] Step 3:

[0074] The server checks the authentication information it receives against a database to verify its validity.

[0075] Step 4:

[0076] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[0077] Example response: {"token": "authtoken1234", "status": "success"}

[0078] Step 5:

[0079] The device saves the received token and transitions to the home screen.

[0080] Step 6:

[0081] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[0082] Step 7:

[0083] The device sends a request to the server containing the selected game ID (e.g., game1234).

[0084] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0085] Step 8:

[0086] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[0087] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[0088] Step 9:

[0089] The device will start displaying the game video using the live streaming URL.

[0090] Step 10:

[0091] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[0092] Step 11:

[0093] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[0094] Analysis example: Selecting posts with high positive sentiment scores.

[0095] Step 12:

[0096] The server streams the extracted social media posts to the device.

[0097] Step 13:

[0098] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[0099] Step 14:

[0100] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[0101] Step 15:

[0102] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[0103] Example: "Team A's player just scored a goal!"

[0104] Step 16:

[0105] The device displays real-time commentary below the game footage.

[0106] Step 17:

[0107] After the game ends, the user presses the "Highlights" button in the app.

[0108] Step 18:

[0109] The terminal sends a highlight generation request to the server.

[0110] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0111] Step 19:

[0112] Based on the server's match records, AI automatically generates highlight videos tailored to the user's preferences.

[0113] Example of generated content: Extract only scenes in which your favorite player shines.

[0114] Step 20:

[0115] The server returns the generated highlight video to the terminal.

[0116] Example response: {"highlight_url": "http: / / highlights.url"}

[0117] Step 21:

[0118] The device will play the highlight video.

[0119] These are the specific processing steps and detailed operation of the "Sports TV" system, which allows viewers to enjoy real-time game updates, a sense of unity through social media, and personalized highlight videos.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] Conventional methods for viewing sporting events have limited viewers' viewing experience by failing to provide detailed real-time information or a viewing experience that is fully integrated with social media. Furthermore, there is a lack of a way for viewers to easily view highlight videos tailored to their preferences after the game has ended. This has created a need for a system that offers a richer, more immersive viewing experience and allows viewers to enjoy personalized viewing content even after the game has ended.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes a means for logging in to the service by entering authentication information on a terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and posts from social networking services on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for collecting and selecting posts from social networking services that are highly relevant to the game. This allows users to enjoy detailed real-time information and a rich and integrated viewing experience linked to SNS, and enables them to easily watch personalized highlight videos even after the game is over.

[0125] A "terminal" is an electronic device that allows a user to access various services by operating it.

[0126] "Authentication Information" means data used to identify and secure a user.

[0127] "Service" is a general term for the functions and content provided to users.

[0128] "Streaming" is a technology for playing back data while transferring it in real time.

[0129] "Commentary" is an explanation of important events and actions that take place during the course of a match.

[0130] A "social networking service" is a platform that allows users to communicate with each other via the Internet.

[0131] A "post" is a message or content posted by a user on a social networking service.

[0132] "Viewing history" is a record of the content a user has viewed.

[0133] "Preferences" are the tendencies and choices that a user is particularly interested in.

[0134] A "highlight video" is a short video that summarizes the important moments of a game.

[0135] "Collection" is the act of gathering data or information.

[0136] "Sorting" is the act of analyzing collected data and information and selecting the most relevant information.

[0137] "Analysis" is the examination and decomposition of data or information to understand and derive meaning.

[0138] An "Event" is a significant occurrence or action that occurs during a match.

[0139] The present invention relates to a system that provides a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0140] Authentication and Login

[0141] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device then sends the entered authentication information (username and password) to the server. The server compares the received authentication information with its database, and if authentication is successful, the server generates a session token and returns it to the device. Using this token, the user can access other features within the service.

[0142] Select a match and start watching

[0143] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0144] Social Media Feed Integration

[0145] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0146] Real-time match commentary and highlight generation

[0147] The server analyzes the game footage in real time and detects important events (e.g., runs scored, home runs, etc.). Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0148] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences. For example, if a user is rooting for a particular player, a highlight video centered on that player's performance will be generated. The generated highlight video is sent from the server to the device and played on the device.

[0149] Specific examples

[0150] Authentication and Login

[0151] The user launches the "Sports Viewing App" on their device and enters the username "sample_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0152] Select a match and start watching

[0153] When a user selects the match "Team A vs Team B," the device sends a request including the match ID "game5678" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device then displays the live video of the match.

[0154] Social Media Feed Integration

[0155] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0156] Real-time match commentary and highlight generation

[0157] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, such as "A player from Team A just hit a home run!" After the game ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0158] In this way, the present invention allows users to enjoy sporting events from a new perspective, providing a unified viewing experience through real-time information provision and social media posting, and also allows users to easily watch personalized highlight videos after the game is over.

[0159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0160] Step 1:

[0161] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device sends this input information to the server. The input is the username and password, and the output is the authentication information sent to the server. Specifically, if the username is "sample_user" and the password is "password123", this information is encrypted and sent to the server.

[0162] Step 2:

[0163] The server checks the received authentication information against the database. It compares the user information in the database with the entered authentication information, and if they match, authentication is successful. If authentication is successful, the server generates a session token and returns it to the terminal. This token indicates authentication during the session and is used for subsequent requests. The input is authentication information and the output is a session token. Specifically, the server compares the password of "sample_user" with the value in the database, and if they match, it generates a session token in the format "token12345" and sends it to the terminal.

[0164] Step 3:

[0165] The user selects the game they want to watch on the home screen. The device then requests the ID of the selected game from the server. The input is the game ID, and the output is the game's live streaming URL and related social media data. Specifically, when the user selects the game "Team A vs Team B," the game ID "game5678" is sent to the server.

[0166] Step 4:

[0167] The server retrieves relevant data based on the match ID. It collects the match's live streaming URL and related social networking service (SNS) data and sends it to the device. The input is the match ID, and the output is the live streaming URL and SNS data. Specifically, the URL "http: / / livestream / game5678" corresponding to "game5678" and the SNS data are collected and sent to the device.

[0168] Step 5:

[0169] The device uses the received URL to start playing live video of the game. The input is the live streaming URL, and the output is the display of live video. Specifically, the device accesses the URL "http: / / livestream / game5678" and starts streaming.

[0170] Step 6:

[0171] The server collects social media posts using relevant hashtags and keywords during the game. The input is the relevant hashtags and keywords, and the output is the collected social media post data. Specifically, it collects posts from social media using hashtags such as "Team A" and "Team B."

[0172] Step 7:

[0173] The AI ​​(inside the server) analyzes the collected SNS posts and selects those with high relevance. The input is SNS post data, and the output is the selected SNS posts. Specifically, it uses natural language processing technology to select posts with high positive sentiment scores.

[0174] Step 8:

[0175] The server streams the selected SNS posts to the device. The input is the selected SNS posts, and the output is the display on the device. Specifically, the selected posts are sent to the device.

[0176] Step 9:

[0177] The device displays the selected SNS posts overlaid on the game video. The input is the selected SNS posts, and the output is an overlay display on the game video. Specifically, it displays real-time SNS posts on top of the game video.

[0178] Step 10:

[0179] The server analyzes the game footage as it progresses and detects important events (e.g., runs scored, home runs, etc.). The input is the game footage, and the output is the detected event data. Specifically, it uses video analysis technology to detect important moments.

[0180] Step 11:

[0181] The AI ​​(in the server) generates commentary in real time based on the detected events. The input is the event data, and the output is the generated commentary text. Specifically, it generates text such as "Team A's player just hit a home run!"

[0182] Step 12:

[0183] The terminal displays the commentary text received from the server. The input is the commentary text, and the output is the commentary displayed below the game video. Specifically, the commentary text is displayed below the video in real time.

[0184] Step 13:

[0185] After the game ends, the user presses the highlight button to request a highlight video. This causes the device to send a highlight generation request to the server. The input is the highlight generation request, and the output is the request data. Specifically, the request data is sent to the server when the user clicks.

[0186] Step 14:

[0187] The server automatically generates highlight videos based on the match records and the user's viewing history. The input is the match records and viewing history, and the output is the generated highlight videos. Specifically, if a user is rooting for a specific player, the server generates a highlight video edited to focus on scenes in which that player is active.

[0188] Step 15:

[0189] The server transmits the generated highlight video to the terminal, and the terminal plays the highlight video. The input is the generated highlight video, and the output is the played highlight video. Specifically, the highlight video is transmitted to the terminal and played.

[0190] (Application example 1)

[0191] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] Traditional sports event viewing systems are limited to streaming game footage and displaying simple social media posts, but lack the means to individually enhance the user's viewing experience. They also lack real-time commentary, highlighting key moments, and generating personalized highlight videos. This makes it difficult for viewers to enjoy detailed real-time information or a viewing experience tailored to their individual interests and preferences.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0194] In this invention, the server includes a means for users to log in to the service by entering authentication information on an available terminal, a means for providing game video streaming, a means for overlaying social media posts on game video, a means for adding inspiring commentary to key moments in real time using a generative AI model, and a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, thereby enabling users to enjoy individually customized viewing experiences in real time.

[0195] "User" means any individual or entity using the Service.

[0196] "Device" means an electronic device through which a User accesses the Service, such as a smartphone, tablet, computer, or smart TV.

[0197] "Authentication Information" refers to information that identifies an individual, such as a user name and password.

[0198] "Service" means the online platform that provides match video streaming and related functionality.

[0199] "Game video streaming" refers to the technology of providing live or recorded sports game footage over the Internet.

[0200] "SNS posts" refer to posts and comments made on social networking services.

[0201] A "generative AI model" refers to an algorithm or system that uses artificial intelligence to analyze and generate data.

[0202] "Real-time commentary" refers to commentary and commentary that is generated instantly as the match progresses.

[0203] "Viewing history" refers to a record of content that a user has viewed to date.

[0204] "Preferences" refers to a user's personal interests or preferences regarding a particular player, team, play, etc.

[0205] A "highlight video" is a short video that compiles important moments and highlights of a game.

[0206] An embodiment of the present invention is described below.

[0207] Authentication and Login

[0208] A user launches the "Sports TV" application on their device and enters their username and password. The device sends the authentication information to the server, which checks it against a predefined database. If the check is successful, the server generates a session token and returns it to the device. The user can use this token to access other features within the service.

[0209] Select a match and start watching

[0210] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server, which collects the game's live streaming URL and related social media data and sends it to the device, which then uses the URL to begin displaying the game's live stream.

[0211] Social Media Feed Integration

[0212] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by a generative AI model within the server, and the most relevant posts are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match video. This allows users to enjoy the match while watching social media reactions in real time.

[0213] Real-time match commentary

[0214] Game footage is analyzed in real time by the server to detect important events (e.g., runs scored, home runs, etc.). Based on this, a generative AI model in the server automatically generates compelling commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, informing users of key moments.

[0215] Generate personalized highlight videos

[0216] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. The generated highlight video is customized based on the user's viewing history and preferences. The generated highlight video is then sent from the server to the device, where it is played.

[0217] Specific examples

[0218] A user launches the "Sports TV" application and enters the username "example_user" and password. The device sends the authentication information to the server, which returns a session token on success.

[0219] The user selects the match "Team A vs Team B", and the device sends the match ID "game1234" to the server. The server returns the live streaming URL and related SNS data, and the device displays the live video of the match.

[0220] The server collects social media posts with the match-related hashtags "Team A" and "Team B," and the generative AI model selects posts with high positive sentiment scores. The selected posts are then overlaid on the match footage.

[0221] The server detects a home run event, and the generative AI model generates commentary in real time, such as, "Team A's player just hit a home run!" After the game ends, the user requests a highlight video, and the server generates a personalized highlight video and sends it to the device.

[0222] Prompt Sentence Examples

[0223] "We'd like your idea for an application that provides spectators with real-time coverage of exciting moments during a sports game. For example, the app would include functionality where a user logs in to the app, selects a game, displays a stream of that game and related social media feeds, and generates real-time commentary of key moments. Please explain these features with examples."

[0224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0225] Step 1:

[0226] The user launches the "Sports TV" application on their device. They enter their username and password on the login screen. The device sends this authentication information to the server. The username and password are used as input, and the authentication information is sent to the server as output. At this point, no data processing is done, only information is transferred.

[0227] Step 2:

[0228] The server checks the received authentication information against the database. It compares the user information in the database with the entered information to confirm that they match. If the check is successful, the server generates a session token and returns it to the terminal. The input here is authentication information, and the output is a session token. The generation process involves generating a session token.

[0229] Step 3:

[0230] The user selects a match they are interested in on the home screen. The device requests the ID of the selected match from the server. The input is the match ID, and the output is the request. The transfer of the match ID is required as data processing.

[0231] Step 4:

[0232] The server collects the match's live streaming URL and related social media data and sends it to the device. The input is the match ID and session token, and the output is the live streaming URL and social media data. Data processing involves obtaining the necessary information from multiple data sources and sending it as a single dataset.

[0233] Step 5:

[0234] The device uses the received URL to begin displaying live video of the match. The input is the live streaming URL, and the output is the live video. The data processing here involves requesting and displaying the video based on the received URL.

[0235] Step 6:

[0236] The server collects social media posts using relevant hashtags and keywords during the match. Data is acquired through the social media API. The input for this step is hashtags and keywords, and the output is a set of social media posts. Data processing involves collecting data through the social media API and analyzing the relevance of hashtags.

[0237] Step 7:

[0238] The collected SNS posts are analyzed by a generative AI model on the server, and those with high relevance are selected. The input is a group of SNS posts, and the output is the selected SNS posts. As a data calculation, the generative AI model determines the sentiment score and relevance, and selects appropriate posts.

[0239] Step 8:

[0240] The selected SNS posts are streamed by the server and displayed on the device overlaid on the game video. The input is the selected SNS posts, and the output is the video with the SNS posts overlaid. Data processing requires overlaying the video.

[0241] Step 9:

[0242] The server analyzes game footage in real time to detect important events (such as runs scored or home runs). The input is the game footage, and the output is the detected events. As a data calculation, a generative AI model analyzes the footage in real time and processes it to extract important events.

[0243] Step 10:

[0244] The generative AI model generates commentary in real time based on the detected event and sends it to the device. The input is the detected event and the output is real-time commentary. Data processing includes the process of the generative AI model generating text and sending it to the device.

[0245] Step 11:

[0246] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device then sends a highlight generation request to the server. The input is the user's preferences, and the output is the highlight generation request. Data processing here is the process of reflecting the user's preferences in the request.

[0247] Step 12:

[0248] The server automatically generates a highlight video based on the match record and sends it to the device. The generated highlight video is customized based on the user's viewing history and preferences. The input here is the viewing history and preferences, and the output is the highlight video. Data processing involves editing the highlight video based on the viewing history and user preferences.

[0249] Step 13:

[0250] The device plays the highlight video. The input is the URL or data of the highlight video, and the output is the playback of the highlight video. Data processing here is the process of decoding the video data and playing it back.

[0251] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0252] The present invention relates to a system that provides users with a new service that goes beyond the conventional viewing experience when watching sporting events. In particular, the system uses an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of the present invention are described below.

[0253] Authentication and Login

[0254] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[0255] Select a match and start watching

[0256] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0257] Social Media Feed Integration

[0258] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0259] Real-time match commentary and highlight generation

[0260] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0261] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This highlight video is customized based on the user's viewing history and preferences.

[0262] User Emotion Recognition

[0263] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is then sent to a server, where an emotion engine analyzes the data and recognizes the user's emotions.

[0264] For example, the emotion engine can detect whether a user is smiling, surprised, or focused, and the recognized emotion data is used to tailor the presentation of match commentary and social media posts.

[0265] For example, if a user is surprised, the emotion engine can provide a detailed explanation that matches that emotion. Also, if a user is having fun, it will prioritize displaying positive social media posts that will further enhance that emotion.

[0266] Accumulation and optimization of emotional data

[0267] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is analyzed in combination with the user's viewing history and preferences to provide a more personalized viewing experience.

[0268] For example, combining past viewing data with emotional data can identify which types of events a user responds to most strongly, allowing for tailored content and commentary to be delivered to that user when viewing future matches.

[0269] Specific examples

[0270] Specific examples of the present invention are shown below.

[0271] Authentication and Login

[0272] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0273] Select a match and start watching

[0274] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[0275] Social Media Feed Integration

[0276] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0277] Real-time match commentary and emotion recognition

[0278] The server detects the moment of a home run, and the AI ​​generates a commentary in real time, such as "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, and the emotion engine analyzes them to recognize the user's emotions. If the device captures a surprised expression, the emotion engine will provide a more detailed commentary.

[0279] Highlight Generation

[0280] After a match ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the match record and sends it to the device. The device then plays the highlight video. At this time, the video is edited based on the accumulated emotional data to include the scenes that most excite the user.

[0281] This invention allows viewers to enjoy sporting events from a new perspective, providing a holistic viewing experience with detailed real-time information and social media posts. Furthermore, an emotion engine provides an optimized viewing experience tailored to each viewer's individual emotions. Furthermore, after the game, viewers can easily watch personalized highlight videos.

[0282] The processing flow will be explained below.

[0283] Step 1:

[0284] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[0285] Step 2:

[0286] The device sends the entered authentication information to the server in JSON format.

[0287] Example of sending data: {"username": "example_user", "password": "example_password"}

[0288] Step 3:

[0289] The server checks the authentication information it receives against a database to verify its validity.

[0290] Step 4:

[0291] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[0292] Example response: {"token": "authtoken1234", "status": "success"}

[0293] Step 5:

[0294] The device saves the received token and transitions to the home screen.

[0295] Step 6:

[0296] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[0297] Step 7:

[0298] The device sends a request to the server containing the selected game ID (e.g., game1234).

[0299] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0300] Step 8:

[0301] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[0302] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[0303] Step 9:

[0304] The device will start displaying the game video using the live streaming URL.

[0305] Step 10:

[0306] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[0307] Step 11:

[0308] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[0309] Analysis example: Selecting posts with high positive sentiment scores.

[0310] Step 12:

[0311] The server streams the extracted social media posts to the device.

[0312] Step 13:

[0313] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[0314] Step 14:

[0315] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[0316] Step 15:

[0317] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[0318] Example: "Team A's player just hit a home run!"

[0319] Step 16:

[0320] The device displays real-time commentary below the game footage.

[0321] Step 17:

[0322] The device uses its built-in camera and microphone to capture the user's facial expressions and voice while watching.

[0323] Step 18:

[0324] The device sends the captured data to the server.

[0325] Step 19:

[0326] The emotion engine in the server analyzes the transmitted data and recognizes the user's emotion.

[0327] For example, detecting emotions such as smile, surprise, and concentration.

[0328] Step 20:

[0329] The server adjusts the commentary and social media posts provided based on the user's perceived emotions.

[0330] Example: If the user is surprised, provide a detailed explanation.

[0331] Step 21:

[0332] After the game ends, the user presses the "Highlights" button in the app.

[0333] Step 22:

[0334] The terminal sends a highlight generation request to the server.

[0335] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0336] Step 23:

[0337] Based on the game record, the server uses AI to automatically generate highlight videos tailored to the user's preferences and emotional data.

[0338] Example of generated content: Extract only scenes in which your favorite player shines.

[0339] Step 24:

[0340] The server returns the generated highlight video to the terminal.

[0341] Example response: {"highlight_url": "http: / / highlights.url"}

[0342] Step 25:

[0343] The device will play the highlight video.

[0344] Video example: Highlights edited to include the scenes that most excite users.

[0345] The above are the specific processing steps and detailed operation of the "Sports TV" system. By combining it with the emotion engine, viewers can grasp the game situation in real time, enjoy the sense of unity through social media, and enjoy an optimal viewing experience tailored to their individual emotions. Furthermore, after the game ends, viewers can watch personalized highlight videos.

[0346] Example 2

[0347] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0348] With conventional viewing systems for sporting events, viewers often could not simultaneously experience real-time commentary or the reactions of other viewers. It was also difficult to provide a personalized viewing experience that incorporated viewers' emotional responses. Therefore, there was a need to provide a new viewing experience that would allow viewers to feel more engaged.

[0349] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0350] In this invention, the server includes a means for logging in to the service by inputting authentication information on a user's device, a means for providing game video streaming, a means for overlaying generated real-time commentary and social networking service posts on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for recognizing the user's emotions and optimizing the viewing experience based on that information, thereby enabling viewers to enjoy not only the real-time game streaming and commentary, but also a personalized viewing experience based on the reactions of other viewers and their own emotions.

[0351] "Authentication information" refers to information such as an identifier and password required for a user to access a service.

[0352] "Service" refers to the provision of various digital content and functions that users can access and use through their devices.

[0353] "Logging in" is the process by which a user gains access to a service using authentication information.

[0354] "Game video streaming" refers to video of a sports game being provided in real time over the Internet.

[0355] "Real-time commentary" refers to sports commentary information that is generated instantly in response to events occurring during a game.

[0356] "Social networking service" refers to a platform on the Internet where users can share information and communicate.

[0357] A "post" is information such as messages, comments, photos, and videos posted by users on social networking services.

[0358] "Viewing history" is a record of content that a user has viewed in the past.

[0359] "Preferences" refers to a user's preference for a particular genre or theme as indicated by their past viewing history and behavior.

[0360] A "highlight video" is a short video clip that selects important moments from a game.

[0361] "Emotion recognition" is the process of determining a user's emotional state from data such as facial expressions and voice.

[0362] "Optimizing the viewing experience" means customizing the content and interface provided to maximize the user's viewing satisfaction.

[0363] This invention relates to a system that provides users with a new viewing experience that goes beyond the conventional viewing experience when watching sporting events. Specifically, it is characterized by using an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of this invention are described below.

[0364] Authentication and Login

[0365] The user launches the "sports viewing application" on their device and accesses the login screen. They enter their username and password and press the "Login" button. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores the session token, allowing the user to access the service. The hardware used in this process is a device such as a smartphone or tablet, and the software used is an authentication information management database.

[0366] Select a match and start watching

[0367] The user selects the game they want to watch on the home screen. The device requests the ID of the selected game from the server. The server collects the game's live streaming URL and related social networking service data and sends it to the device. The device then uses the received URL to begin displaying live footage of the game. The live streaming distribution service used in this case is YouTube Live or Twitch.

[0368] Social Media Feed Integration

[0369] The server collects posts from social networking services using relevant hashtags and keywords during the match. AI within the server analyzes the collected posts and selects the most relevant ones. The server streams the selected posts and sends them to the device. The device then overlays these posts on the match footage. The software used includes AI analysis software and social media APIs such as Twitter API and Instagram API.

[0370] Real-time match commentary and highlight generation

[0371] The server analyzes the game footage in real time and detects important events. The AI ​​in the server generates real-time commentary based on the detected events and sends it to the device. The commentary is displayed at the bottom of the game footage. After the game ends, the user can request a highlight video from their device. The server automatically generates a highlight video based on the game recording and sends it to the device. The device then plays the highlight video. This allows the user to watch the game without missing any important scenes.

[0372] User Emotion Recognition

[0373] While watching a match, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes the data to recognize the user's emotions. The recognized emotion data is used to adjust the display of match commentary and social media posts. For example, if the user is surprised, the emotion engine will provide detailed commentary tailored to that emotion. If the user is enjoying the match, the engine will display positive social media posts that further enhance that emotion.

[0374] Accumulation and optimization of emotional data

[0375] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is then combined with the user's viewing history and preferences to provide a more personalized viewing experience. For example, combining past viewing data with emotional data can identify which types of events a user responds most strongly to, allowing the server to provide that user with the most appropriate content and commentary when viewing future games.

[0376] Examples and behavior

[0377] The user logs in by entering "example_user" and password into the terminal. The terminal sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session token and sends it to the terminal.

[0378] The user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. The server returns the live streaming URL and related SNS data. The device starts displaying the live video from the received URL.

[0379] The server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays posts with high positive sentiment scores.

[0380] The server detects the moment a home run is hit from the game footage, and the AI ​​generates a real-time commentary such as, "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, which are analyzed by the server's emotion engine. If the user is surprised, the emotion engine provides a detailed commentary.

[0381] After the game ends, the user requests a highlight video, and the device sends the request to the server, which then generates a highlight video tailored to the user's preferences and plays it on the device.

[0382] This system allows users to enjoy a highly personalized sports viewing experience, detailed information provided in real time, and social media posts. In particular, the emotional engine's monitoring and feedback can increase viewer satisfaction.

[0383] Prompt Sentence Examples

[0384] "How can I provide detailed explanations when a user is surprised?"

[0385] "Please provide specific examples of how to filter and display positive social media posts."

[0386] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0387] Step 1:

[0388] The user launches the "Sports Viewing Application" on the device and accesses the login screen. The user enters their username and password and presses the "Login" button. This generates the entered authentication information. The device sends this authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores this session token, allowing the user to access the service. A specific example of how this works is when a user logs in by entering "example_user" and a password. In this case, the input is "username and password" and the output is "session token."

[0389] Step 2:

[0390] The user selects the match they want to watch on the home screen. This selection generates a match ID. The device then requests the selected match ID from the server. The server collects the match's live streaming URL and related social media data and sends them to the device. The device then uses the received URL to begin displaying live footage of the match. For example, a user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. In this case, the input is the "match ID" and the output is the "live streaming URL and related social media data."

[0391] Step 3:

[0392] The server collects social media posts using relevant hashtags and keywords during the match. This generates highly relevant social media post data. The collected posts are analyzed by AI (within the server) and selected for their relevance. The server streams the selected social media posts and sends them to the device. The device then overlays these posts on the match footage. A specific example of how this works is when the server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays the positive posts. In this case, the input is "social media post data" and the output is "selected social media posts."

[0393] Step 4:

[0394] The server analyzes game footage in real time and detects important events, which generates event data. The AI ​​generates real-time commentary based on this event data and sends it to the device. The device displays the commentary below the game footage. After the game ends, the user can request a highlight video from the device. The device sends the request to the server. The server automatically generates a highlight video based on the game record and sends it to the device. The device then plays the highlight video. A specific example of how this works is when the server detects the moment a home run is hit, and the AI ​​generates real-time commentary, saying, "A player from Team A just hit a home run!" In this case, the input is "game footage data" and the output is "commentary data."

[0395] Step 5:

[0396] While watching a game, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time, generating emotional data. This data is sent to a server, where an emotion engine within the server analyzes the data and recognizes the user's emotions. The recognized emotional data is used to adjust the display of game commentary and social media posts. A specific example of how this works is when the device captures the user's surprised expression, which the server's emotion engine analyzes and provides detailed commentary. In this case, the input is "user's facial and voice data," and the output is "recognized emotional data."

[0397] Step 6:

[0398] The server accumulates the user's emotional data and uses this data to provide a more personalized and optimized viewing experience. The accumulated data is then combined with the user's viewing history and preferences for analysis. A specific example of how this works is to combine past viewing data with emotional data to optimize future viewing experiences. In this case, the input is "emotional data and viewing history data," and the output is "optimized viewing experience data."

[0399] (Application example 2)

[0400] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0401] Conventional sports event viewing systems lack real-time optimization based on viewer emotions and preferences, resulting in viewing experiences that are not tailored to individual viewers. Furthermore, integration of social media data and real-time commentary cannot instantly provide viewers with the information they most desire at any given moment. This has led to issues such as insufficient viewer satisfaction.

[0402] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to log in to the service by entering authentication information on an available terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and SNS posts on the game video, a means for automatically generating and providing a highlight video based on the user's viewing history and preferences, a means for analyzing the user's facial expressions and voice in real time to recognize emotions, and a means for adjusting content to optimize the viewing experience based on the recognized emotions. This makes it possible to provide an optimal viewing experience that promptly responds to the emotions and preferences of each viewer.

[0403] "Authentication information" refers to personal identification information such as a username and password that a user enters when accessing a service.

[0404] "Game video streaming" is a technology that distributes video of a specific sports game in real time over the Internet.

[0405] "Real-time commentary" refers to commentary information that is generated instantly as the match progresses, explaining important events of the match to the viewer.

[0406] "SNS posts" are messages, images, videos, and other content posted by users on social networking services.

[0407] "Viewing history" is a record of content that a user has viewed in the past.

[0408] A "highlight video" is a shortened video that is edited to include only the important scenes and events of a game.

[0409] "Means of recognizing emotions by analyzing facial expressions and voice in real time" refers to technology that uses a camera and microphone to analyze the user's face and voice and grasp their emotions at that time (joy, surprise, concentration, etc.).

[0410] "Means to adjust content to optimize the viewing experience" refers to technology that changes and adjusts displayed content (comments, social media posts, etc.) in real time based on the user's emotions and preferences.

[0411] This invention relates to a sports event viewing system that optimizes the viewing experience for users. In particular, it provides a new service that goes beyond conventional viewing experiences by recognizing emotions in real time and adjusting viewing content according to those emotions.

[0412] System Overview and Configuration

[0413] 1. Authentication and Login

[0414] Users log in to the service by entering their authentication information (username and password) on their smartphone or other internet-enabled device, which is then sent to the server, which checks the authentication information against a database.

[0415] 2. Game Streaming

[0416] The server provides video streaming of the selected sports game and sends the streaming URL to the user's available device, which uses the URL to display the live video of the game.

[0417] 3. Real-time commentary and social media posts

[0418] The server collects relevant social media posts as the match progresses, and AI analyzes them to select the most relevant posts, which are then overlaid on the match footage.

[0419] Additionally, AI detects important events during the game (such as runs scored or home runs) and generates commentary in real time, which is then overlaid on the game footage.

[0420] 4. Automatic generation and provision of highlight videos

[0421] After the game ends, a highlight video is automatically generated based on the user's viewing history and preferences. The server generates this video in response to the user's request and sends it to the user's device, allowing the user to watch a personalized highlight video.

[0422] 5. Emotion Recognition and Viewing Experience Optimization

[0423] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time, and this data is sent to a server where an emotion engine analyzes it to recognize the user's emotions.

[0424] Based on the recognized emotion data, the server adjusts the game commentary and displayed social media posts, for example, providing more detailed commentary if the user is surprised, and prioritizing positive social media posts if the user is enjoying the game.

[0425] Hardware and software used

[0426] Hardware: Smartphone, Head-Mounted Display (HMD), Camera, Microphone

[0427] Software: OpenCV (video capture and display), Requests (HTTP communication)

[0428] In addition, the emotion engine, SNS API, and AI model run on a server or cloud environment.

[0429] Specific examples

[0430] The user launches the "next-generation sports viewing app" on their smartphone and enters their login information. After successful authentication by the server, the user begins watching the match footage of "Team A vs. Team B." During the match, the smartphone's camera and microphone continuously capture the user's facial expressions and voice. If the user's surprised expression is detected, the server's emotion engine analyzes this and generates and displays detailed commentary in real time. Related positive social media posts are also overlaid on the screen. After the match, the user requests a highlight video, which the server automatically generates and sends to the user's smartphone. The video is then edited to include the most exciting scenes based on past viewing and emotion data.

[0431] Prompt Sentence Examples

[0432] Implement a system that analyzes emotions from users' facial expressions and voice in real time to optimize the viewing experience of sporting events. Program a next-generation sports viewing app that displays social media posts and provides commentary on key moments of a match based on the recognized emotions. For example, provide more detailed commentary if the user is surprised.

[0433] This system allows for an individually optimized viewing experience for sporting events, providing viewers with the real-time information and commentary they most desire at any given moment.

[0434] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0435] Step 1:

[0436] The user enters authentication information (username and password) on the device to log in. The entered authentication information is sent from the device to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The user can use this token to access other functions within the service.

[0437] Step 2:

[0438] The user selects the match they want to watch. The user sends a request including the match ID from their device to the server. The server collects the live streaming URL of the match and related social media data and sends it to the device. The device starts displaying the live video of the match using the received URL.

[0439] Step 3:

[0440] While the game video is being streamed, the server collects related social media posts in real time. AI analyzes the data collected via the social media API and selects the most relevant posts. The selected social media posts are sent to the device and displayed over the game video, allowing users to view real-time social media reactions as the game progresses.

[0441] Step 4:

[0442] The server analyzes game footage in real time and detects important events (scores, home runs, etc.). Based on this event data, AI automatically generates commentary and sends it to the device in real time. The device then displays the generated commentary, allowing the user to visually recognize important moments.

[0443] Step 5:

[0444] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice. This data is sent to a server in real time. An emotion engine on the server analyzes the data and recognizes the user's emotions. For example, if the user is surprised, this information is reflected in the level of detail in the commentary and the selection of social media posts displayed.

[0445] Step 6:

[0446] After the game ends, the user requests the generation of a highlight video. The request, including the game ID, is sent from the device to the server, and the server automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and emotional data and sent to the device. The device plays the generated highlight video, allowing the user to enjoy a personally optimized summary of the game.

[0447] Step 7:

[0448] The server stores user emotional data and viewing history, which is used to optimize future viewing experiences. The data is analyzed to identify which types of events users respond most strongly to. This information is then used to provide users with the most appropriate content and commentary when viewing future matches.

[0449] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0450] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0451] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0452] [Second embodiment]

[0453] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0454] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0455] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0456] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0457] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0458] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0459] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0460] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0461] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0462] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0463] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0464] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0465] The present invention relates to a system for providing a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0466] Authentication and Login

[0467] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[0468] Select a match and start watching

[0469] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0470] Social Media Feed Integration

[0471] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0472] Real-time match commentary and highlight generation

[0473] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0474] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences.

[0475] For example, if a user is rooting for a particular player, a highlight video is generated that mainly shows scenes in which that player is playing an active role. The generated highlight video is transmitted from the server to the terminal and played on the terminal.

[0476] Specific examples

[0477] Specific examples of the present invention are shown below.

[0478] Authentication and Login

[0479] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0480] Select a match and start watching

[0481] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[0482] Social Media Feed Integration

[0483] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0484] Real-time match commentary and highlight generation

[0485] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, saying, "A player from Team A just hit a home run!" The user requests a highlight video after the game ends, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0486] This invention allows viewers to enjoy sporting events from a new perspective, creating an integrated viewing experience with detailed real-time information and social media posts, and allows viewers to easily watch personalized highlight videos after the game.

[0487] The processing flow will be explained below.

[0488] Step 1:

[0489] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[0490] Step 2:

[0491] The device sends the entered authentication information to the server in JSON format.

[0492] Example of sending data: {"username": "example_user", "password": "example_password"}

[0493] Step 3:

[0494] The server checks the authentication information it receives against a database to verify its validity.

[0495] Step 4:

[0496] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[0497] Example response: {"token": "authtoken1234", "status": "success"}

[0498] Step 5:

[0499] The device saves the received token and transitions to the home screen.

[0500] Step 6:

[0501] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[0502] Step 7:

[0503] The device sends a request to the server containing the selected game ID (e.g., game1234).

[0504] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0505] Step 8:

[0506] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[0507] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[0508] Step 9:

[0509] The device will start displaying the game video using the live streaming URL.

[0510] Step 10:

[0511] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[0512] Step 11:

[0513] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[0514] Analysis example: Selecting posts with high positive sentiment scores.

[0515] Step 12:

[0516] The server streams the extracted social media posts to the device.

[0517] Step 13:

[0518] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[0519] Step 14:

[0520] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[0521] Step 15:

[0522] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[0523] Example: "Team A's player just scored a goal!"

[0524] Step 16:

[0525] The device displays real-time commentary below the game footage.

[0526] Step 17:

[0527] After the game ends, the user presses the "Highlights" button in the app.

[0528] Step 18:

[0529] The terminal sends a highlight generation request to the server.

[0530] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0531] Step 19:

[0532] Based on the server's match records, AI automatically generates highlight videos tailored to the user's preferences.

[0533] Example of generated content: Extract only scenes in which your favorite player shines.

[0534] Step 20:

[0535] The server returns the generated highlight video to the terminal.

[0536] Example response: {"highlight_url": "http: / / highlights.url"}

[0537] Step 21:

[0538] The device will play the highlight video.

[0539] These are the specific processing steps and detailed operation of the "Sports TV" system, which allows viewers to enjoy real-time game updates, a sense of unity through social media, and personalized highlight videos.

[0540] Example 1

[0541] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0542] Conventional methods for viewing sporting events have limited viewers' viewing experience by failing to provide detailed real-time information or a viewing experience that is fully integrated with social media. Furthermore, there is a lack of a way for viewers to easily view highlight videos tailored to their preferences after the game has ended. This has created a need for a system that offers a richer, more immersive viewing experience and allows viewers to enjoy personalized viewing content even after the game has ended.

[0543] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0544] In this invention, the server includes a means for logging in to the service by entering authentication information on a terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and posts from social networking services on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for collecting and selecting posts from social networking services that are highly relevant to the game. This allows users to enjoy detailed real-time information and a rich and integrated viewing experience linked to SNS, and enables them to easily watch personalized highlight videos even after the game is over.

[0545] A "terminal" is an electronic device that allows a user to access various services by operating it.

[0546] "Authentication Information" means data used to identify and secure a user.

[0547] "Service" is a general term for the functions and content provided to users.

[0548] "Streaming" is a technology for playing back data while transferring it in real time.

[0549] "Commentary" is an explanation of important events and actions that take place during the course of a match.

[0550] A "social networking service" is a platform that allows users to communicate with each other via the Internet.

[0551] A "post" is a message or content posted by a user on a social networking service.

[0552] "Viewing history" is a record of the content a user has viewed.

[0553] "Preferences" are the tendencies and choices that a user is particularly interested in.

[0554] A "highlight video" is a short video that summarizes the important moments of a game.

[0555] "Collection" is the act of gathering data or information.

[0556] "Sorting" is the act of analyzing collected data and information and selecting the most relevant information.

[0557] "Analysis" is the examination and decomposition of data or information to understand and derive meaning.

[0558] An "Event" is a significant occurrence or action that occurs during a match.

[0559] The present invention relates to a system that provides a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0560] Authentication and Login

[0561] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device then sends the entered authentication information (username and password) to the server. The server compares the received authentication information with its database, and if authentication is successful, the server generates a session token and returns it to the device. Using this token, the user can access other features within the service.

[0562] Select a match and start watching

[0563] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0564] Social Media Feed Integration

[0565] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0566] Real-time match commentary and highlight generation

[0567] The server analyzes the game footage in real time and detects important events (e.g., runs scored, home runs, etc.). Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0568] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences. For example, if a user is rooting for a particular player, a highlight video centered on that player's performance will be generated. The generated highlight video is sent from the server to the device and played on the device.

[0569] Specific examples

[0570] Authentication and Login

[0571] The user launches the "Sports Viewing App" on their device and enters the username "sample_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0572] Select a match and start watching

[0573] When a user selects the match "Team A vs Team B," the device sends a request including the match ID "game5678" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device then displays the live video of the match.

[0574] Social Media Feed Integration

[0575] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0576] Real-time match commentary and highlight generation

[0577] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, such as "A player from Team A just hit a home run!" After the game ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0578] In this way, the present invention allows users to enjoy sporting events from a new perspective, providing a unified viewing experience through real-time information provision and social media posting, and also allows users to easily watch personalized highlight videos after the game is over.

[0579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0580] Step 1:

[0581] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device sends this input information to the server. The input is the username and password, and the output is the authentication information sent to the server. Specifically, if the username is "sample_user" and the password is "password123", this information is encrypted and sent to the server.

[0582] Step 2:

[0583] The server checks the received authentication information against the database. It compares the user information in the database with the entered authentication information, and if they match, authentication is successful. If authentication is successful, the server generates a session token and returns it to the terminal. This token indicates authentication during the session and is used for subsequent requests. The input is authentication information and the output is a session token. Specifically, the server compares the password of "sample_user" with the value in the database, and if they match, it generates a session token in the format "token12345" and sends it to the terminal.

[0584] Step 3:

[0585] The user selects the game they want to watch on the home screen. The device then requests the ID of the selected game from the server. The input is the game ID, and the output is the game's live streaming URL and related social media data. Specifically, when the user selects the game "Team A vs Team B," the game ID "game5678" is sent to the server.

[0586] Step 4:

[0587] The server retrieves relevant data based on the match ID. It collects the match's live streaming URL and related social networking service (SNS) data and sends it to the device. The input is the match ID, and the output is the live streaming URL and SNS data. Specifically, the URL "http: / / livestream / game5678" corresponding to "game5678" and the SNS data are collected and sent to the device.

[0588] Step 5:

[0589] The device uses the received URL to start playing live video of the game. The input is the live streaming URL, and the output is the display of live video. Specifically, the device accesses the URL "http: / / livestream / game5678" and starts streaming.

[0590] Step 6:

[0591] The server collects social media posts using relevant hashtags and keywords during the game. The input is the relevant hashtags and keywords, and the output is the collected social media post data. Specifically, it collects posts from social media using hashtags such as "Team A" and "Team B."

[0592] Step 7:

[0593] The AI ​​(inside the server) analyzes the collected SNS posts and selects those with high relevance. The input is SNS post data, and the output is the selected SNS posts. Specifically, it uses natural language processing technology to select posts with high positive sentiment scores.

[0594] Step 8:

[0595] The server streams the selected SNS posts to the device. The input is the selected SNS posts, and the output is the display on the device. Specifically, the selected posts are sent to the device.

[0596] Step 9:

[0597] The device displays the selected SNS posts overlaid on the game video. The input is the selected SNS posts, and the output is an overlay display on the game video. Specifically, it displays real-time SNS posts on top of the game video.

[0598] Step 10:

[0599] The server analyzes the game footage as it progresses and detects important events (e.g., runs scored, home runs, etc.). The input is the game footage, and the output is the detected event data. Specifically, it uses video analysis technology to detect important moments.

[0600] Step 11:

[0601] The AI ​​(in the server) generates commentary in real time based on the detected events. The input is the event data, and the output is the generated commentary text. Specifically, it generates text such as "Team A's player just hit a home run!"

[0602] Step 12:

[0603] The terminal displays the commentary text received from the server. The input is the commentary text, and the output is the commentary displayed below the game video. Specifically, the commentary text is displayed below the video in real time.

[0604] Step 13:

[0605] After the game ends, the user presses the highlight button to request a highlight video. This causes the device to send a highlight generation request to the server. The input is the highlight generation request, and the output is the request data. Specifically, the request data is sent to the server when the user clicks.

[0606] Step 14:

[0607] The server automatically generates highlight videos based on the match records and the user's viewing history. The input is the match records and viewing history, and the output is the generated highlight videos. Specifically, if a user is rooting for a specific player, the server generates a highlight video edited to focus on scenes in which that player is active.

[0608] Step 15:

[0609] The server transmits the generated highlight video to the terminal, and the terminal plays the highlight video. The input is the generated highlight video, and the output is the played highlight video. Specifically, the highlight video is transmitted to the terminal and played.

[0610] (Application example 1)

[0611] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] Traditional sports event viewing systems are limited to streaming game footage and displaying simple social media posts, but lack the means to individually enhance the user's viewing experience. They also lack real-time commentary, highlighting key moments, and generating personalized highlight videos. This makes it difficult for viewers to enjoy detailed real-time information or a viewing experience tailored to their individual interests and preferences.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0614] In this invention, the server includes a means for users to log in to the service by entering authentication information on an available terminal, a means for providing game video streaming, a means for overlaying social media posts on game video, a means for adding inspiring commentary to key moments in real time using a generative AI model, and a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, thereby enabling users to enjoy individually customized viewing experiences in real time.

[0615] "User" means any individual or entity using the Service.

[0616] "Device" means an electronic device through which a User accesses the Service, such as a smartphone, tablet, computer, or smart TV.

[0617] "Authentication Information" refers to information that identifies an individual, such as a user name and password.

[0618] "Service" means the online platform that provides match video streaming and related functionality.

[0619] "Game video streaming" refers to the technology of providing live or recorded sports game footage over the Internet.

[0620] "SNS posts" refer to posts and comments made on social networking services.

[0621] A "generative AI model" refers to an algorithm or system that uses artificial intelligence to analyze and generate data.

[0622] "Real-time commentary" refers to commentary and commentary that is generated instantly as the match progresses.

[0623] "Viewing history" refers to a record of content that a user has viewed to date.

[0624] "Preferences" refers to a user's personal interests or preferences regarding a particular player, team, play, etc.

[0625] A "highlight video" is a short video that compiles important moments and highlights of a game.

[0626] An embodiment of the present invention is described below.

[0627] Authentication and Login

[0628] A user launches the "Sports TV" application on their device and enters their username and password. The device sends the authentication information to the server, which checks it against a predefined database. If the check is successful, the server generates a session token and returns it to the device. The user can use this token to access other features within the service.

[0629] Select a match and start watching

[0630] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server, which collects the game's live streaming URL and related social media data and sends it to the device, which then uses the URL to begin displaying the game's live stream.

[0631] Social Media Feed Integration

[0632] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by a generative AI model within the server, and the most relevant posts are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match video. This allows users to enjoy the match while watching social media reactions in real time.

[0633] Real-time match commentary

[0634] Game footage is analyzed in real time by the server to detect important events (e.g., runs scored, home runs, etc.). Based on this, a generative AI model in the server automatically generates compelling commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, informing users of key moments.

[0635] Generate personalized highlight videos

[0636] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. The generated highlight video is customized based on the user's viewing history and preferences. The generated highlight video is then sent from the server to the device, where it is played.

[0637] Specific examples

[0638] A user launches the "Sports TV" application and enters the username "example_user" and password. The device sends the authentication information to the server, which returns a session token on success.

[0639] The user selects the match "Team A vs Team B", and the device sends the match ID "game1234" to the server. The server returns the live streaming URL and related SNS data, and the device displays the live video of the match.

[0640] The server collects social media posts with the match-related hashtags "Team A" and "Team B," and the generative AI model selects posts with high positive sentiment scores. The selected posts are then overlaid on the match footage.

[0641] The server detects a home run event, and the generative AI model generates commentary in real time, such as, "Team A's player just hit a home run!" After the game ends, the user requests a highlight video, and the server generates a personalized highlight video and sends it to the device.

[0642] Prompt Sentence Examples

[0643] "We'd like your idea for an application that provides spectators with real-time coverage of exciting moments during a sports game. For example, the app would include functionality where a user logs in to the app, selects a game, displays a stream of that game and related social media feeds, and generates real-time commentary of key moments. Please explain these features with examples."

[0644] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0645] Step 1:

[0646] The user launches the "Sports TV" application on their device. They enter their username and password on the login screen. The device sends this authentication information to the server. The username and password are used as input, and the authentication information is sent to the server as output. At this point, no data processing is done, only information is transferred.

[0647] Step 2:

[0648] The server checks the received authentication information against the database. It compares the user information in the database with the entered information to confirm that they match. If the check is successful, the server generates a session token and returns it to the terminal. The input here is authentication information, and the output is a session token. The generation process involves generating a session token.

[0649] Step 3:

[0650] The user selects a match they are interested in on the home screen. The device requests the ID of the selected match from the server. The input is the match ID, and the output is the request. The transfer of the match ID is required as data processing.

[0651] Step 4:

[0652] The server collects the match's live streaming URL and related social media data and sends it to the device. The input is the match ID and session token, and the output is the live streaming URL and social media data. Data processing involves obtaining the necessary information from multiple data sources and sending it as a single dataset.

[0653] Step 5:

[0654] The device uses the received URL to begin displaying live video of the match. The input is the live streaming URL, and the output is the live video. The data processing here involves requesting and displaying the video based on the received URL.

[0655] Step 6:

[0656] The server collects social media posts using relevant hashtags and keywords during the match. Data is acquired through the social media API. The input for this step is hashtags and keywords, and the output is a set of social media posts. Data processing involves collecting data through the social media API and analyzing the relevance of hashtags.

[0657] Step 7:

[0658] The collected SNS posts are analyzed by a generative AI model on the server, and those with high relevance are selected. The input is a group of SNS posts, and the output is the selected SNS posts. As a data calculation, the generative AI model determines the sentiment score and relevance, and selects appropriate posts.

[0659] Step 8:

[0660] The selected SNS posts are streamed by the server and displayed on the device overlaid on the game video. The input is the selected SNS posts, and the output is the video with the SNS posts overlaid. Data processing requires overlaying the video.

[0661] Step 9:

[0662] The server analyzes game footage in real time to detect important events (such as runs scored or home runs). The input is the game footage, and the output is the detected events. As a data calculation, a generative AI model analyzes the footage in real time and processes it to extract important events.

[0663] Step 10:

[0664] The generative AI model generates commentary in real time based on the detected event and sends it to the device. The input is the detected event and the output is real-time commentary. Data processing includes the process of the generative AI model generating text and sending it to the device.

[0665] Step 11:

[0666] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device then sends a highlight generation request to the server. The input is the user's preferences, and the output is the highlight generation request. Data processing here is the process of reflecting the user's preferences in the request.

[0667] Step 12:

[0668] The server automatically generates a highlight video based on the match record and sends it to the device. The generated highlight video is customized based on the user's viewing history and preferences. The input here is the viewing history and preferences, and the output is the highlight video. Data processing involves editing the highlight video based on the viewing history and user preferences.

[0669] Step 13:

[0670] The device plays the highlight video. The input is the URL or data of the highlight video, and the output is the playback of the highlight video. Data processing here is the process of decoding the video data and playing it back.

[0671] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0672] The present invention relates to a system that provides users with a new service that goes beyond the conventional viewing experience when watching sporting events. In particular, the system uses an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of the present invention are described below.

[0673] Authentication and Login

[0674] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[0675] Select a match and start watching

[0676] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0677] Social Media Feed Integration

[0678] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0679] Real-time match commentary and highlight generation

[0680] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0681] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This highlight video is customized based on the user's viewing history and preferences.

[0682] User Emotion Recognition

[0683] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is then sent to a server, where an emotion engine analyzes the data and recognizes the user's emotions.

[0684] For example, the emotion engine can detect whether a user is smiling, surprised, or focused, and the recognized emotion data is used to tailor the presentation of match commentary and social media posts.

[0685] For example, if a user is surprised, the emotion engine can provide a detailed explanation that matches that emotion. Also, if a user is having fun, it will prioritize displaying positive social media posts that will further enhance that emotion.

[0686] Accumulation and optimization of emotional data

[0687] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is analyzed in combination with the user's viewing history and preferences to provide a more personalized viewing experience.

[0688] For example, combining past viewing data with emotional data can identify which types of events a user responds to most strongly, allowing for tailored content and commentary to be delivered to that user when viewing future matches.

[0689] Specific examples

[0690] Specific examples of the present invention are shown below.

[0691] Authentication and Login

[0692] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0693] Select a match and start watching

[0694] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[0695] Social Media Feed Integration

[0696] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0697] Real-time match commentary and emotion recognition

[0698] The server detects the moment of a home run, and the AI ​​generates a commentary in real time, such as "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, and the emotion engine analyzes them to recognize the user's emotions. If the device captures a surprised expression, the emotion engine will provide a more detailed commentary.

[0699] Highlight Generation

[0700] After a match ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the match record and sends it to the device. The device then plays the highlight video. At this time, the video is edited based on the accumulated emotional data to include the scenes that most excite the user.

[0701] This invention allows viewers to enjoy sporting events from a new perspective, providing a holistic viewing experience with detailed real-time information and social media posts. Furthermore, an emotion engine provides an optimized viewing experience tailored to each viewer's individual emotions. Furthermore, after the game, viewers can easily watch personalized highlight videos.

[0702] The processing flow will be explained below.

[0703] Step 1:

[0704] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[0705] Step 2:

[0706] The device sends the entered authentication information to the server in JSON format.

[0707] Example of sending data: {"username": "example_user", "password": "example_password"}

[0708] Step 3:

[0709] The server checks the authentication information it receives against a database to verify its validity.

[0710] Step 4:

[0711] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[0712] Example response: {"token": "authtoken1234", "status": "success"}

[0713] Step 5:

[0714] The device saves the received token and transitions to the home screen.

[0715] Step 6:

[0716] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[0717] Step 7:

[0718] The device sends a request to the server containing the selected game ID (e.g., game1234).

[0719] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0720] Step 8:

[0721] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[0722] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[0723] Step 9:

[0724] The device will start displaying the game video using the live streaming URL.

[0725] Step 10:

[0726] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[0727] Step 11:

[0728] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[0729] Analysis example: Selecting posts with high positive sentiment scores.

[0730] Step 12:

[0731] The server streams the extracted social media posts to the device.

[0732] Step 13:

[0733] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[0734] Step 14:

[0735] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[0736] Step 15:

[0737] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[0738] Example: "Team A's player just hit a home run!"

[0739] Step 16:

[0740] The device displays real-time commentary below the game footage.

[0741] Step 17:

[0742] The device uses its built-in camera and microphone to capture the user's facial expressions and voice while watching.

[0743] Step 18:

[0744] The device sends the captured data to the server.

[0745] Step 19:

[0746] The emotion engine in the server analyzes the transmitted data and recognizes the user's emotion.

[0747] For example, detecting emotions such as smile, surprise, and concentration.

[0748] Step 20:

[0749] The server adjusts the commentary and social media posts provided based on the user's perceived emotions.

[0750] Example: If the user is surprised, provide a detailed explanation.

[0751] Step 21:

[0752] After the game ends, the user presses the "Highlights" button in the app.

[0753] Step 22:

[0754] The terminal sends a highlight generation request to the server.

[0755] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0756] Step 23:

[0757] Based on the game record, the server uses AI to automatically generate highlight videos tailored to the user's preferences and emotional data.

[0758] Example of generated content: Extract only scenes in which your favorite player shines.

[0759] Step 24:

[0760] The server returns the generated highlight video to the terminal.

[0761] Example response: {"highlight_url": "http: / / highlights.url"}

[0762] Step 25:

[0763] The device will play the highlight video.

[0764] Video example: Highlights edited to include the scenes that most excite users.

[0765] The above are the specific processing steps and detailed operation of the "Sports TV" system. By combining it with the emotion engine, viewers can grasp the game situation in real time, enjoy the sense of unity through social media, and enjoy an optimal viewing experience tailored to their individual emotions. Furthermore, after the game ends, viewers can watch personalized highlight videos.

[0766] Example 2

[0767] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0768] With conventional viewing systems for sporting events, viewers often could not simultaneously experience real-time commentary or the reactions of other viewers. It was also difficult to provide a personalized viewing experience that incorporated viewers' emotional responses. Therefore, there was a need to provide a new viewing experience that would allow viewers to feel more engaged.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0770] In this invention, the server includes a means for logging in to the service by inputting authentication information on a user's device, a means for providing game video streaming, a means for overlaying generated real-time commentary and social networking service posts on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for recognizing the user's emotions and optimizing the viewing experience based on that information, thereby enabling viewers to enjoy not only the real-time game streaming and commentary, but also a personalized viewing experience based on the reactions of other viewers and their own emotions.

[0771] "Authentication information" refers to information such as an identifier and password required for a user to access a service.

[0772] "Service" refers to the provision of various digital content and functions that users can access and use through their devices.

[0773] "Logging in" is the process by which a user gains access to a service using authentication information.

[0774] "Game video streaming" refers to video of a sports game being provided in real time over the Internet.

[0775] "Real-time commentary" refers to sports commentary information that is generated instantly in response to events occurring during a game.

[0776] "Social networking service" refers to a platform on the Internet where users can share information and communicate.

[0777] A "post" is information such as messages, comments, photos, and videos posted by users on social networking services.

[0778] "Viewing history" is a record of content that a user has viewed in the past.

[0779] "Preferences" refers to a user's preference for a particular genre or theme as indicated by their past viewing history and behavior.

[0780] A "highlight video" is a short video clip that selects important moments from a game.

[0781] "Emotion recognition" is the process of determining a user's emotional state from data such as facial expressions and voice.

[0782] "Optimizing the viewing experience" means customizing the content and interface provided to maximize the user's viewing satisfaction.

[0783] This invention relates to a system that provides users with a new viewing experience that goes beyond the conventional viewing experience when watching sporting events. Specifically, it is characterized by using an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of this invention are described below.

[0784] Authentication and Login

[0785] The user launches the "sports viewing application" on their device and accesses the login screen. They enter their username and password and press the "Login" button. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores the session token, allowing the user to access the service. The hardware used in this process is a device such as a smartphone or tablet, and the software used is an authentication information management database.

[0786] Select a match and start watching

[0787] The user selects the game they want to watch on the home screen. The device requests the ID of the selected game from the server. The server collects the game's live streaming URL and related social networking service data and sends it to the device. The device then uses the received URL to begin displaying live footage of the game. The live streaming distribution service used in this case is YouTube Live or Twitch.

[0788] Social Media Feed Integration

[0789] The server collects posts from social networking services using relevant hashtags and keywords during the match. AI within the server analyzes the collected posts and selects the most relevant ones. The server streams the selected posts and sends them to the device. The device then overlays these posts on the match footage. The software used includes AI analysis software and social media APIs such as Twitter API and Instagram API.

[0790] Real-time match commentary and highlight generation

[0791] The server analyzes the game footage in real time and detects important events. The AI ​​in the server generates real-time commentary based on the detected events and sends it to the device. The commentary is displayed at the bottom of the game footage. After the game ends, the user can request a highlight video from their device. The server automatically generates a highlight video based on the game recording and sends it to the device. The device then plays the highlight video. This allows the user to watch the game without missing any important scenes.

[0792] User Emotion Recognition

[0793] While watching a match, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes the data to recognize the user's emotions. The recognized emotion data is used to adjust the display of match commentary and social media posts. For example, if the user is surprised, the emotion engine will provide detailed commentary tailored to that emotion. If the user is enjoying the match, the engine will display positive social media posts that further enhance that emotion.

[0794] Accumulation and optimization of emotional data

[0795] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is then combined with the user's viewing history and preferences to provide a more personalized viewing experience. For example, combining past viewing data with emotional data can identify which types of events a user responds most strongly to, allowing the server to provide that user with the most appropriate content and commentary when viewing future games.

[0796] Examples and behavior

[0797] The user logs in by entering "example_user" and password into the terminal. The terminal sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session token and sends it to the terminal.

[0798] The user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. The server returns the live streaming URL and related SNS data. The device starts displaying the live video from the received URL.

[0799] The server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays posts with high positive sentiment scores.

[0800] The server detects the moment a home run is hit from the game footage, and the AI ​​generates a real-time commentary such as, "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, which are analyzed by the server's emotion engine. If the user is surprised, the emotion engine provides a detailed commentary.

[0801] After the game ends, the user requests a highlight video, and the device sends the request to the server, which then generates a highlight video tailored to the user's preferences and plays it on the device.

[0802] This system allows users to enjoy a highly personalized sports viewing experience, detailed information provided in real time, and social media posts. In particular, the emotional engine's monitoring and feedback can increase viewer satisfaction.

[0803] Prompt Sentence Examples

[0804] "How can I provide detailed explanations when a user is surprised?"

[0805] "Please provide specific examples of how to filter and display positive social media posts."

[0806] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0807] Step 1:

[0808] The user launches the "Sports Viewing Application" on the device and accesses the login screen. The user enters their username and password and presses the "Login" button. This generates the entered authentication information. The device sends this authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores this session token, allowing the user to access the service. A specific example of how this works is when a user logs in by entering "example_user" and a password. In this case, the input is "username and password" and the output is "session token."

[0809] Step 2:

[0810] The user selects the match they want to watch on the home screen. This selection generates a match ID. The device then requests the selected match ID from the server. The server collects the match's live streaming URL and related social media data and sends them to the device. The device then uses the received URL to begin displaying live footage of the match. For example, a user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. In this case, the input is the "match ID" and the output is the "live streaming URL and related social media data."

[0811] Step 3:

[0812] The server collects social media posts using relevant hashtags and keywords during the match. This generates highly relevant social media post data. The collected posts are analyzed by AI (within the server) and selected for their relevance. The server streams the selected social media posts and sends them to the device. The device then overlays these posts on the match footage. A specific example of how this works is when the server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays the positive posts. In this case, the input is "social media post data" and the output is "selected social media posts."

[0813] Step 4:

[0814] The server analyzes game footage in real time and detects important events, which generates event data. The AI ​​generates real-time commentary based on this event data and sends it to the device. The device displays the commentary below the game footage. After the game ends, the user can request a highlight video from the device. The device sends the request to the server. The server automatically generates a highlight video based on the game record and sends it to the device. The device then plays the highlight video. A specific example of how this works is when the server detects the moment a home run is hit, and the AI ​​generates real-time commentary, saying, "A player from Team A just hit a home run!" In this case, the input is "game footage data" and the output is "commentary data."

[0815] Step 5:

[0816] While watching a game, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time, generating emotional data. This data is sent to a server, where an emotion engine within the server analyzes the data and recognizes the user's emotions. The recognized emotional data is used to adjust the display of game commentary and social media posts. A specific example of how this works is when the device captures the user's surprised expression, which the server's emotion engine analyzes and provides detailed commentary. In this case, the input is "user's facial and voice data," and the output is "recognized emotional data."

[0817] Step 6:

[0818] The server accumulates the user's emotional data and uses this data to provide a more personalized and optimized viewing experience. The accumulated data is then combined with the user's viewing history and preferences for analysis. A specific example of how this works is to combine past viewing data with emotional data to optimize future viewing experiences. In this case, the input is "emotional data and viewing history data," and the output is "optimized viewing experience data."

[0819] (Application example 2)

[0820] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0821] Conventional sports event viewing systems lack real-time optimization based on viewer emotions and preferences, resulting in viewing experiences that are not tailored to individual viewers. Furthermore, integration of social media data and real-time commentary cannot instantly provide viewers with the information they most desire at any given moment. This has led to issues such as insufficient viewer satisfaction.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to log in to the service by entering authentication information on an available terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and SNS posts on the game video, a means for automatically generating and providing a highlight video based on the user's viewing history and preferences, a means for analyzing the user's facial expressions and voice in real time to recognize emotions, and a means for adjusting content to optimize the viewing experience based on the recognized emotions. This makes it possible to provide an optimal viewing experience that promptly responds to the emotions and preferences of each viewer.

[0823] "Authentication information" refers to personal identification information such as a username and password that a user enters when accessing a service.

[0824] "Game video streaming" is a technology that distributes video of a specific sports game in real time over the Internet.

[0825] "Real-time commentary" refers to commentary information that is generated instantly as the match progresses, explaining important events of the match to the viewer.

[0826] "SNS posts" are messages, images, videos, and other content posted by users on social networking services.

[0827] "Viewing history" is a record of content that a user has viewed in the past.

[0828] A "highlight video" is a shortened video that is edited to include only the important scenes and events of a game.

[0829] "Means of recognizing emotions by analyzing facial expressions and voice in real time" refers to technology that uses a camera and microphone to analyze the user's face and voice and grasp their emotions at that time (joy, surprise, concentration, etc.).

[0830] "Means to adjust content to optimize the viewing experience" refers to technology that changes and adjusts displayed content (comments, social media posts, etc.) in real time based on the user's emotions and preferences.

[0831] This invention relates to a sports event viewing system that optimizes the viewing experience for users. In particular, it provides a new service that goes beyond conventional viewing experiences by recognizing emotions in real time and adjusting viewing content according to those emotions.

[0832] System Overview and Configuration

[0833] 1. Authentication and Login

[0834] Users log in to the service by entering their authentication information (username and password) on their smartphone or other internet-enabled device, which is then sent to the server, which checks the authentication information against a database.

[0835] 2. Game Streaming

[0836] The server provides video streaming of the selected sports game and sends the streaming URL to the user's available device, which uses the URL to display the live video of the game.

[0837] 3. Real-time commentary and social media posts

[0838] The server collects relevant social media posts as the match progresses, and AI analyzes them to select the most relevant posts, which are then overlaid on the match footage.

[0839] Additionally, AI detects important events during the game (such as runs scored or home runs) and generates commentary in real time, which is then overlaid on the game footage.

[0840] 4. Automatic generation and provision of highlight videos

[0841] After the game ends, a highlight video is automatically generated based on the user's viewing history and preferences. The server generates this video in response to the user's request and sends it to the user's device, allowing the user to watch a personalized highlight video.

[0842] 5. Emotion Recognition and Viewing Experience Optimization

[0843] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time, and this data is sent to a server where an emotion engine analyzes it to recognize the user's emotions.

[0844] Based on the recognized emotion data, the server adjusts the game commentary and displayed social media posts, for example, providing more detailed commentary if the user is surprised, and prioritizing positive social media posts if the user is enjoying the game.

[0845] Hardware and software used

[0846] Hardware: Smartphone, Head-Mounted Display (HMD), Camera, Microphone

[0847] Software: OpenCV (video capture and display), Requests (HTTP communication)

[0848] In addition, the emotion engine, SNS API, and AI model run on a server or cloud environment.

[0849] Specific examples

[0850] The user launches the "next-generation sports viewing app" on their smartphone and enters their login information. After successful authentication by the server, the user begins watching the match footage of "Team A vs. Team B." During the match, the smartphone's camera and microphone continuously capture the user's facial expressions and voice. If the user's surprised expression is detected, the server's emotion engine analyzes this and generates and displays detailed commentary in real time. Related positive social media posts are also overlaid on the screen. After the match, the user requests a highlight video, which the server automatically generates and sends to the user's smartphone. The video is then edited to include the most exciting scenes based on past viewing and emotion data.

[0851] Prompt Sentence Examples

[0852] Implement a system that analyzes emotions from users' facial expressions and voice in real time to optimize the viewing experience of sporting events. Program a next-generation sports viewing app that displays social media posts and provides commentary on key moments of a match based on the recognized emotions. For example, provide more detailed commentary if the user is surprised.

[0853] This system allows for an individually optimized viewing experience for sporting events, providing viewers with the real-time information and commentary they most desire at any given moment.

[0854] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0855] Step 1:

[0856] The user enters authentication information (username and password) on the device to log in. The entered authentication information is sent from the device to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The user can use this token to access other functions within the service.

[0857] Step 2:

[0858] The user selects the match they want to watch. The user sends a request including the match ID from their device to the server. The server collects the live streaming URL of the match and related social media data and sends it to the device. The device starts displaying the live video of the match using the received URL.

[0859] Step 3:

[0860] While the game video is being streamed, the server collects related social media posts in real time. AI analyzes the data collected via the social media API and selects the most relevant posts. The selected social media posts are sent to the device and displayed over the game video, allowing users to view real-time social media reactions as the game progresses.

[0861] Step 4:

[0862] The server analyzes game footage in real time and detects important events (scores, home runs, etc.). Based on this event data, AI automatically generates commentary and sends it to the device in real time. The device then displays the generated commentary, allowing the user to visually recognize important moments.

[0863] Step 5:

[0864] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice. This data is sent to a server in real time. An emotion engine on the server analyzes the data and recognizes the user's emotions. For example, if the user is surprised, this information is reflected in the level of detail in the commentary and the selection of social media posts displayed.

[0865] Step 6:

[0866] After the game ends, the user requests the generation of a highlight video. The request, including the game ID, is sent from the device to the server, and the server automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and emotional data and sent to the device. The device plays the generated highlight video, allowing the user to enjoy a personally optimized summary of the game.

[0867] Step 7:

[0868] The server stores user emotional data and viewing history, which is used to optimize future viewing experiences. The data is analyzed to identify which types of events users respond most strongly to. This information is then used to provide users with the most appropriate content and commentary when viewing future matches.

[0869] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0870] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0871] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0872] [Third embodiment]

[0873] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0874] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0875] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0876] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0877] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0878] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0879] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0880] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0881] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0882] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0883] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0884] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0885] The present invention relates to a system for providing a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0886] Authentication and Login

[0887] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[0888] Select a match and start watching

[0889] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0890] Social Media Feed Integration

[0891] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0892] Real-time match commentary and highlight generation

[0893] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0894] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences.

[0895] For example, if a user is rooting for a particular player, a highlight video is generated that mainly shows scenes in which that player is playing an active role. The generated highlight video is transmitted from the server to the terminal and played on the terminal.

[0896] Specific examples

[0897] Specific examples of the present invention are shown below.

[0898] Authentication and Login

[0899] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0900] Select a match and start watching

[0901] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[0902] Social Media Feed Integration

[0903] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0904] Real-time match commentary and highlight generation

[0905] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, saying, "A player from Team A just hit a home run!" The user requests a highlight video after the game ends, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0906] This invention allows viewers to enjoy sporting events from a new perspective, creating an integrated viewing experience with detailed real-time information and social media posts, and allows viewers to easily watch personalized highlight videos after the game.

[0907] The processing flow will be explained below.

[0908] Step 1:

[0909] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[0910] Step 2:

[0911] The device sends the entered authentication information to the server in JSON format.

[0912] Example of sending data: {"username": "example_user", "password": "example_password"}

[0913] Step 3:

[0914] The server checks the authentication information it receives against a database to verify its validity.

[0915] Step 4:

[0916] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[0917] Example response: {"token": "authtoken1234", "status": "success"}

[0918] Step 5:

[0919] The device saves the received token and transitions to the home screen.

[0920] Step 6:

[0921] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[0922] Step 7:

[0923] The device sends a request to the server containing the selected game ID (e.g., game1234).

[0924] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0925] Step 8:

[0926] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[0927] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[0928] Step 9:

[0929] The device will start displaying the game video using the live streaming URL.

[0930] Step 10:

[0931] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[0932] Step 11:

[0933] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[0934] Analysis example: Selecting posts with high positive sentiment scores.

[0935] Step 12:

[0936] The server streams the extracted social media posts to the device.

[0937] Step 13:

[0938] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[0939] Step 14:

[0940] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[0941] Step 15:

[0942] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[0943] Example: "Team A's player just scored a goal!"

[0944] Step 16:

[0945] The device displays real-time commentary below the game footage.

[0946] Step 17:

[0947] After the game ends, the user presses the "Highlights" button in the app.

[0948] Step 18:

[0949] The terminal sends a highlight generation request to the server.

[0950] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[0951] Step 19:

[0952] Based on the server's match records, AI automatically generates highlight videos tailored to the user's preferences.

[0953] Example of generated content: Extract only scenes in which your favorite player shines.

[0954] Step 20:

[0955] The server returns the generated highlight video to the terminal.

[0956] Example response: {"highlight_url": "http: / / highlights.url"}

[0957] Step 21:

[0958] The device will play the highlight video.

[0959] These are the specific processing steps and detailed operation of the "Sports TV" system, which allows viewers to enjoy real-time game updates, a sense of unity through social media, and personalized highlight videos.

[0960] Example 1

[0961] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0962] Conventional methods for viewing sporting events have limited viewers' viewing experience by failing to provide detailed real-time information or a viewing experience that is fully integrated with social media. Furthermore, there is a lack of a way for viewers to easily view highlight videos tailored to their preferences after the game has ended. This has created a need for a system that offers a richer, more immersive viewing experience and allows viewers to enjoy personalized viewing content even after the game has ended.

[0963] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0964] In this invention, the server includes a means for logging in to the service by entering authentication information on a terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and posts from social networking services on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for collecting and selecting posts from social networking services that are highly relevant to the game. This allows users to enjoy detailed real-time information and a rich and integrated viewing experience linked to SNS, and enables them to easily watch personalized highlight videos even after the game is over.

[0965] A "terminal" is an electronic device that allows a user to access various services by operating it.

[0966] "Authentication Information" means data used to identify and secure a user.

[0967] "Service" is a general term for the functions and content provided to users.

[0968] "Streaming" is a technology for playing back data while transferring it in real time.

[0969] "Commentary" is an explanation of important events and actions that take place during the course of a match.

[0970] A "social networking service" is a platform that allows users to communicate with each other via the Internet.

[0971] A "post" is a message or content posted by a user on a social networking service.

[0972] "Viewing history" is a record of the content a user has viewed.

[0973] "Preferences" are the tendencies and choices that a user is particularly interested in.

[0974] A "highlight video" is a short video that summarizes the important moments of a game.

[0975] "Collection" is the act of gathering data or information.

[0976] "Sorting" is the act of analyzing collected data and information and selecting the most relevant information.

[0977] "Analysis" is the examination and decomposition of data or information to understand and derive meaning.

[0978] An "Event" is a significant occurrence or action that occurs during a match.

[0979] The present invention relates to a system that provides a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[0980] Authentication and Login

[0981] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device then sends the entered authentication information (username and password) to the server. The server compares the received authentication information with its database, and if authentication is successful, the server generates a session token and returns it to the device. Using this token, the user can access other features within the service.

[0982] Select a match and start watching

[0983] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[0984] Social Media Feed Integration

[0985] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[0986] Real-time match commentary and highlight generation

[0987] The server analyzes the game footage in real time and detects important events (e.g., runs scored, home runs, etc.). Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[0988] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences. For example, if a user is rooting for a particular player, a highlight video centered on that player's performance will be generated. The generated highlight video is sent from the server to the device and played on the device.

[0989] Specific examples

[0990] Authentication and Login

[0991] The user launches the "Sports Viewing App" on their device and enters the username "sample_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[0992] Select a match and start watching

[0993] When a user selects the match "Team A vs Team B," the device sends a request including the match ID "game5678" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device then displays the live video of the match.

[0994] Social Media Feed Integration

[0995] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[0996] Real-time match commentary and highlight generation

[0997] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, such as "A player from Team A just hit a home run!" After the game ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[0998] In this way, the present invention allows users to enjoy sporting events from a new perspective, providing a unified viewing experience through real-time information provision and social media posting, and also allows users to easily watch personalized highlight videos after the game is over.

[0999] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1000] Step 1:

[1001] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device sends this input information to the server. The input is the username and password, and the output is the authentication information sent to the server. Specifically, if the username is "sample_user" and the password is "password123", this information is encrypted and sent to the server.

[1002] Step 2:

[1003] The server checks the received authentication information against the database. It compares the user information in the database with the entered authentication information, and if they match, authentication is successful. If authentication is successful, the server generates a session token and returns it to the terminal. This token indicates authentication during the session and is used for subsequent requests. The input is authentication information and the output is a session token. Specifically, the server compares the password of "sample_user" with the value in the database, and if they match, it generates a session token in the format "token12345" and sends it to the terminal.

[1004] Step 3:

[1005] The user selects the game they want to watch on the home screen. The device then requests the ID of the selected game from the server. The input is the game ID, and the output is the game's live streaming URL and related social media data. Specifically, when the user selects the game "Team A vs Team B," the game ID "game5678" is sent to the server.

[1006] Step 4:

[1007] The server retrieves relevant data based on the match ID. It collects the match's live streaming URL and related social networking service (SNS) data and sends it to the device. The input is the match ID, and the output is the live streaming URL and SNS data. Specifically, the URL "http: / / livestream / game5678" corresponding to "game5678" and the SNS data are collected and sent to the device.

[1008] Step 5:

[1009] The device uses the received URL to start playing live video of the game. The input is the live streaming URL, and the output is the display of live video. Specifically, the device accesses the URL "http: / / livestream / game5678" and starts streaming.

[1010] Step 6:

[1011] The server collects social media posts using relevant hashtags and keywords during the game. The input is the relevant hashtags and keywords, and the output is the collected social media post data. Specifically, it collects posts from social media using hashtags such as "Team A" and "Team B."

[1012] Step 7:

[1013] The AI ​​(inside the server) analyzes the collected SNS posts and selects those with high relevance. The input is SNS post data, and the output is the selected SNS posts. Specifically, it uses natural language processing technology to select posts with high positive sentiment scores.

[1014] Step 8:

[1015] The server streams the selected SNS posts to the device. The input is the selected SNS posts, and the output is the display on the device. Specifically, the selected posts are sent to the device.

[1016] Step 9:

[1017] The device displays the selected SNS posts overlaid on the game video. The input is the selected SNS posts, and the output is an overlay display on the game video. Specifically, it displays real-time SNS posts on top of the game video.

[1018] Step 10:

[1019] The server analyzes the game footage as it progresses and detects important events (e.g., runs scored, home runs, etc.). The input is the game footage, and the output is the detected event data. Specifically, it uses video analysis technology to detect important moments.

[1020] Step 11:

[1021] The AI ​​(in the server) generates commentary in real time based on the detected events. The input is the event data, and the output is the generated commentary text. Specifically, it generates text such as "Team A's player just hit a home run!"

[1022] Step 12:

[1023] The terminal displays the commentary text received from the server. The input is the commentary text, and the output is the commentary displayed below the game video. Specifically, the commentary text is displayed below the video in real time.

[1024] Step 13:

[1025] After the game ends, the user presses the highlight button to request a highlight video. This causes the device to send a highlight generation request to the server. The input is the highlight generation request, and the output is the request data. Specifically, the request data is sent to the server when the user clicks.

[1026] Step 14:

[1027] The server automatically generates highlight videos based on the match records and the user's viewing history. The input is the match records and viewing history, and the output is the generated highlight videos. Specifically, if a user is rooting for a specific player, the server generates a highlight video edited to focus on scenes in which that player is active.

[1028] Step 15:

[1029] The server transmits the generated highlight video to the terminal, and the terminal plays the highlight video. The input is the generated highlight video, and the output is the played highlight video. Specifically, the highlight video is transmitted to the terminal and played.

[1030] (Application example 1)

[1031] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1032] Traditional sports event viewing systems are limited to streaming game footage and displaying simple social media posts, but lack the means to individually enhance the user's viewing experience. They also lack real-time commentary, highlighting key moments, and generating personalized highlight videos. This makes it difficult for viewers to enjoy detailed real-time information or a viewing experience tailored to their individual interests and preferences.

[1033] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1034] In this invention, the server includes a means for users to log in to the service by entering authentication information on an available terminal, a means for providing game video streaming, a means for overlaying social media posts on game video, a means for adding inspiring commentary to key moments in real time using a generative AI model, and a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, thereby enabling users to enjoy individually customized viewing experiences in real time.

[1035] "User" means any individual or entity using the Service.

[1036] "Device" means an electronic device through which a User accesses the Service, such as a smartphone, tablet, computer, or smart TV.

[1037] "Authentication Information" refers to information that identifies an individual, such as a user name and password.

[1038] "Service" means the online platform that provides match video streaming and related functionality.

[1039] "Game video streaming" refers to the technology of providing live or recorded sports game footage over the Internet.

[1040] "SNS posts" refer to posts and comments made on social networking services.

[1041] A "generative AI model" refers to an algorithm or system that uses artificial intelligence to analyze and generate data.

[1042] "Real-time commentary" refers to commentary and commentary that is generated instantly as the match progresses.

[1043] "Viewing history" refers to a record of content that a user has viewed to date.

[1044] "Preferences" refers to a user's personal interests or preferences regarding a particular player, team, play, etc.

[1045] A "highlight video" is a short video that compiles important moments and highlights of a game.

[1046] An embodiment of the present invention is described below.

[1047] Authentication and Login

[1048] A user launches the "Sports TV" application on their device and enters their username and password. The device sends the authentication information to the server, which checks it against a predefined database. If the check is successful, the server generates a session token and returns it to the device. The user can use this token to access other features within the service.

[1049] Select a match and start watching

[1050] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server, which collects the game's live streaming URL and related social media data and sends it to the device, which then uses the URL to begin displaying the game's live stream.

[1051] Social Media Feed Integration

[1052] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by a generative AI model within the server, and the most relevant posts are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match video. This allows users to enjoy the match while watching social media reactions in real time.

[1053] Real-time match commentary

[1054] Game footage is analyzed in real time by the server to detect important events (e.g., runs scored, home runs, etc.). Based on this, a generative AI model in the server automatically generates compelling commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, informing users of key moments.

[1055] Generate personalized highlight videos

[1056] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. The generated highlight video is customized based on the user's viewing history and preferences. The generated highlight video is then sent from the server to the device, where it is played.

[1057] Specific examples

[1058] A user launches the "Sports TV" application and enters the username "example_user" and password. The device sends the authentication information to the server, which returns a session token on success.

[1059] The user selects the match "Team A vs Team B", and the device sends the match ID "game1234" to the server. The server returns the live streaming URL and related SNS data, and the device displays the live video of the match.

[1060] The server collects social media posts with the match-related hashtags "Team A" and "Team B," and the generative AI model selects posts with high positive sentiment scores. The selected posts are then overlaid on the match footage.

[1061] The server detects a home run event, and the generative AI model generates commentary in real time, such as, "Team A's player just hit a home run!" After the game ends, the user requests a highlight video, and the server generates a personalized highlight video and sends it to the device.

[1062] Prompt Sentence Examples

[1063] "We'd like your idea for an application that provides spectators with real-time coverage of exciting moments during a sports game. For example, the app would include functionality where a user logs in to the app, selects a game, displays a stream of that game and related social media feeds, and generates real-time commentary of key moments. Please explain these features with examples."

[1064] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1065] Step 1:

[1066] The user launches the "Sports TV" application on their device. They enter their username and password on the login screen. The device sends this authentication information to the server. The username and password are used as input, and the authentication information is sent to the server as output. At this point, no data processing is done, only information is transferred.

[1067] Step 2:

[1068] The server checks the received authentication information against the database. It compares the user information in the database with the entered information to confirm that they match. If the check is successful, the server generates a session token and returns it to the terminal. The input here is authentication information, and the output is a session token. The generation process involves generating a session token.

[1069] Step 3:

[1070] The user selects a match they are interested in on the home screen. The device requests the ID of the selected match from the server. The input is the match ID, and the output is the request. The transfer of the match ID is required as data processing.

[1071] Step 4:

[1072] The server collects the match's live streaming URL and related social media data and sends it to the device. The input is the match ID and session token, and the output is the live streaming URL and social media data. Data processing involves obtaining the necessary information from multiple data sources and sending it as a single dataset.

[1073] Step 5:

[1074] The device uses the received URL to begin displaying live video of the match. The input is the live streaming URL, and the output is the live video. The data processing here involves requesting and displaying the video based on the received URL.

[1075] Step 6:

[1076] The server collects social media posts using relevant hashtags and keywords during the match. Data is acquired through the social media API. The input for this step is hashtags and keywords, and the output is a set of social media posts. Data processing involves collecting data through the social media API and analyzing the relevance of hashtags.

[1077] Step 7:

[1078] The collected SNS posts are analyzed by a generative AI model on the server, and those with high relevance are selected. The input is a group of SNS posts, and the output is the selected SNS posts. As a data calculation, the generative AI model determines the sentiment score and relevance, and selects appropriate posts.

[1079] Step 8:

[1080] The selected SNS posts are streamed by the server and displayed on the device overlaid on the game video. The input is the selected SNS posts, and the output is the video with the SNS posts overlaid. Data processing requires overlaying the video.

[1081] Step 9:

[1082] The server analyzes game footage in real time to detect important events (such as runs scored or home runs). The input is the game footage, and the output is the detected events. As a data calculation, a generative AI model analyzes the footage in real time and processes it to extract important events.

[1083] Step 10:

[1084] The generative AI model generates commentary in real time based on the detected event and sends it to the device. The input is the detected event and the output is real-time commentary. Data processing includes the process of the generative AI model generating text and sending it to the device.

[1085] Step 11:

[1086] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device then sends a highlight generation request to the server. The input is the user's preferences, and the output is the highlight generation request. Data processing here is the process of reflecting the user's preferences in the request.

[1087] Step 12:

[1088] The server automatically generates a highlight video based on the match record and sends it to the device. The generated highlight video is customized based on the user's viewing history and preferences. The input here is the viewing history and preferences, and the output is the highlight video. Data processing involves editing the highlight video based on the viewing history and user preferences.

[1089] Step 13:

[1090] The device plays the highlight video. The input is the URL or data of the highlight video, and the output is the playback of the highlight video. Data processing here is the process of decoding the video data and playing it back.

[1091] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1092] The present invention relates to a system that provides users with a new service that goes beyond the conventional viewing experience when watching sporting events. In particular, the system uses an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of the present invention are described below.

[1093] Authentication and Login

[1094] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[1095] Select a match and start watching

[1096] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[1097] Social Media Feed Integration

[1098] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[1099] Real-time match commentary and highlight generation

[1100] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[1101] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This highlight video is customized based on the user's viewing history and preferences.

[1102] User Emotion Recognition

[1103] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is then sent to a server, where an emotion engine analyzes the data and recognizes the user's emotions.

[1104] For example, the emotion engine can detect whether a user is smiling, surprised, or focused, and the recognized emotion data is used to tailor the presentation of match commentary and social media posts.

[1105] For example, if a user is surprised, the emotion engine can provide a detailed explanation that matches that emotion. Also, if a user is having fun, it will prioritize displaying positive social media posts that will further enhance that emotion.

[1106] Accumulation and optimization of emotional data

[1107] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is analyzed in combination with the user's viewing history and preferences to provide a more personalized viewing experience.

[1108] For example, combining past viewing data with emotional data can identify which types of events a user responds to most strongly, allowing for tailored content and commentary to be delivered to that user when viewing future matches.

[1109] Specific examples

[1110] Specific examples of the present invention are shown below.

[1111] Authentication and Login

[1112] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[1113] Select a match and start watching

[1114] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[1115] Social Media Feed Integration

[1116] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[1117] Real-time match commentary and emotion recognition

[1118] The server detects the moment of a home run, and the AI ​​generates a commentary in real time, such as "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, and the emotion engine analyzes them to recognize the user's emotions. If the device captures a surprised expression, the emotion engine will provide a more detailed commentary.

[1119] Highlight Generation

[1120] After a match ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the match record and sends it to the device. The device then plays the highlight video. At this time, the video is edited based on the accumulated emotional data to include the scenes that most excite the user.

[1121] This invention allows viewers to enjoy sporting events from a new perspective, providing a holistic viewing experience with detailed real-time information and social media posts. Furthermore, an emotion engine provides an optimized viewing experience tailored to each viewer's individual emotions. Furthermore, after the game, viewers can easily watch personalized highlight videos.

[1122] The processing flow will be explained below.

[1123] Step 1:

[1124] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[1125] Step 2:

[1126] The device sends the entered authentication information to the server in JSON format.

[1127] Example of sending data: {"username": "example_user", "password": "example_password"}

[1128] Step 3:

[1129] The server checks the authentication information it receives against a database to verify its validity.

[1130] Step 4:

[1131] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[1132] Example response: {"token": "authtoken1234", "status": "success"}

[1133] Step 5:

[1134] The device saves the received token and transitions to the home screen.

[1135] Step 6:

[1136] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[1137] Step 7:

[1138] The device sends a request to the server containing the selected game ID (e.g., game1234).

[1139] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1140] Step 8:

[1141] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[1142] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[1143] Step 9:

[1144] The device will start displaying the game video using the live streaming URL.

[1145] Step 10:

[1146] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[1147] Step 11:

[1148] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[1149] Analysis example: Selecting posts with high positive sentiment scores.

[1150] Step 12:

[1151] The server streams the extracted social media posts to the device.

[1152] Step 13:

[1153] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[1154] Step 14:

[1155] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[1156] Step 15:

[1157] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[1158] Example: "Team A's player just hit a home run!"

[1159] Step 16:

[1160] The device displays real-time commentary below the game footage.

[1161] Step 17:

[1162] The device uses its built-in camera and microphone to capture the user's facial expressions and voice while watching.

[1163] Step 18:

[1164] The device sends the captured data to the server.

[1165] Step 19:

[1166] The emotion engine in the server analyzes the transmitted data and recognizes the user's emotion.

[1167] For example, detecting emotions such as smile, surprise, and concentration.

[1168] Step 20:

[1169] The server adjusts the commentary and social media posts provided based on the user's perceived emotions.

[1170] Example: If the user is surprised, provide a detailed explanation.

[1171] Step 21:

[1172] After the game ends, the user presses the "Highlights" button in the app.

[1173] Step 22:

[1174] The terminal sends a highlight generation request to the server.

[1175] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1176] Step 23:

[1177] Based on the game record, the server uses AI to automatically generate highlight videos tailored to the user's preferences and emotional data.

[1178] Example of generated content: Extract only scenes in which your favorite player shines.

[1179] Step 24:

[1180] The server returns the generated highlight video to the terminal.

[1181] Example response: {"highlight_url": "http: / / highlights.url"}

[1182] Step 25:

[1183] The device will play the highlight video.

[1184] Video example: Highlights edited to include the scenes that most excite users.

[1185] The above are the specific processing steps and detailed operation of the "Sports TV" system. By combining it with the emotion engine, viewers can grasp the game situation in real time, enjoy the sense of unity through social media, and enjoy an optimal viewing experience tailored to their individual emotions. Furthermore, after the game ends, viewers can watch personalized highlight videos.

[1186] Example 2

[1187] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1188] With conventional viewing systems for sporting events, viewers often could not simultaneously experience real-time commentary or the reactions of other viewers. It was also difficult to provide a personalized viewing experience that incorporated viewers' emotional responses. Therefore, there was a need to provide a new viewing experience that would allow viewers to feel more engaged.

[1189] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1190] In this invention, the server includes a means for logging in to the service by inputting authentication information on a user's device, a means for providing game video streaming, a means for overlaying generated real-time commentary and social networking service posts on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for recognizing the user's emotions and optimizing the viewing experience based on that information, thereby enabling viewers to enjoy not only the real-time game streaming and commentary, but also a personalized viewing experience based on the reactions of other viewers and their own emotions.

[1191] "Authentication information" refers to information such as an identifier and password required for a user to access a service.

[1192] "Service" refers to the provision of various digital content and functions that users can access and use through their devices.

[1193] "Logging in" is the process by which a user gains access to a service using authentication information.

[1194] "Game video streaming" refers to video of a sports game being provided in real time over the Internet.

[1195] "Real-time commentary" refers to sports commentary information that is generated instantly in response to events occurring during a game.

[1196] "Social networking service" refers to a platform on the Internet where users can share information and communicate.

[1197] A "post" is information such as messages, comments, photos, and videos posted by users on social networking services.

[1198] "Viewing history" is a record of content that a user has viewed in the past.

[1199] "Preferences" refers to a user's preference for a particular genre or theme as indicated by their past viewing history and behavior.

[1200] A "highlight video" is a short video clip that selects important moments from a game.

[1201] "Emotion recognition" is the process of determining a user's emotional state from data such as facial expressions and voice.

[1202] "Optimizing the viewing experience" means customizing the content and interface provided to maximize the user's viewing satisfaction.

[1203] This invention relates to a system that provides users with a new viewing experience that goes beyond the conventional viewing experience when watching sporting events. Specifically, it is characterized by using an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of this invention are described below.

[1204] Authentication and Login

[1205] The user launches the "sports viewing application" on their device and accesses the login screen. They enter their username and password and press the "Login" button. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores the session token, allowing the user to access the service. The hardware used in this process is a device such as a smartphone or tablet, and the software used is an authentication information management database.

[1206] Select a match and start watching

[1207] The user selects the game they want to watch on the home screen. The device requests the ID of the selected game from the server. The server collects the game's live streaming URL and related social networking service data and sends it to the device. The device then uses the received URL to begin displaying live footage of the game. The live streaming distribution service used in this case is YouTube Live or Twitch.

[1208] Social Media Feed Integration

[1209] The server collects posts from social networking services using relevant hashtags and keywords during the match. AI within the server analyzes the collected posts and selects the most relevant ones. The server streams the selected posts and sends them to the device. The device then overlays these posts on the match footage. The software used includes AI analysis software and social media APIs such as Twitter API and Instagram API.

[1210] Real-time match commentary and highlight generation

[1211] The server analyzes the game footage in real time and detects important events. The AI ​​in the server generates real-time commentary based on the detected events and sends it to the device. The commentary is displayed at the bottom of the game footage. After the game ends, the user can request a highlight video from their device. The server automatically generates a highlight video based on the game recording and sends it to the device. The device then plays the highlight video. This allows the user to watch the game without missing any important scenes.

[1212] User Emotion Recognition

[1213] While watching a match, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes the data to recognize the user's emotions. The recognized emotion data is used to adjust the display of match commentary and social media posts. For example, if the user is surprised, the emotion engine will provide detailed commentary tailored to that emotion. If the user is enjoying the match, the engine will display positive social media posts that further enhance that emotion.

[1214] Accumulation and optimization of emotional data

[1215] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is then combined with the user's viewing history and preferences to provide a more personalized viewing experience. For example, combining past viewing data with emotional data can identify which types of events a user responds most strongly to, allowing the server to provide that user with the most appropriate content and commentary when viewing future games.

[1216] Examples and behavior

[1217] The user logs in by entering "example_user" and password into the terminal. The terminal sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session token and sends it to the terminal.

[1218] The user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. The server returns the live streaming URL and related SNS data. The device starts displaying the live video from the received URL.

[1219] The server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays posts with high positive sentiment scores.

[1220] The server detects the moment a home run is hit from the game footage, and the AI ​​generates a real-time commentary such as, "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, which are analyzed by the server's emotion engine. If the user is surprised, the emotion engine provides a detailed commentary.

[1221] After the game ends, the user requests a highlight video, and the device sends the request to the server, which then generates a highlight video tailored to the user's preferences and plays it on the device.

[1222] This system allows users to enjoy a highly personalized sports viewing experience, detailed information provided in real time, and social media posts. In particular, the emotional engine's monitoring and feedback can increase viewer satisfaction.

[1223] Prompt Sentence Examples

[1224] "How can I provide detailed explanations when a user is surprised?"

[1225] "Please provide specific examples of how to filter and display positive social media posts."

[1226] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1227] Step 1:

[1228] The user launches the "Sports Viewing Application" on the device and accesses the login screen. The user enters their username and password and presses the "Login" button. This generates the entered authentication information. The device sends this authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores this session token, allowing the user to access the service. A specific example of how this works is when a user logs in by entering "example_user" and a password. In this case, the input is "username and password" and the output is "session token."

[1229] Step 2:

[1230] The user selects the match they want to watch on the home screen. This selection generates a match ID. The device then requests the selected match ID from the server. The server collects the match's live streaming URL and related social media data and sends them to the device. The device then uses the received URL to begin displaying live footage of the match. For example, a user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. In this case, the input is the "match ID" and the output is the "live streaming URL and related social media data."

[1231] Step 3:

[1232] The server collects social media posts using relevant hashtags and keywords during the match. This generates highly relevant social media post data. The collected posts are analyzed by AI (within the server) and selected for their relevance. The server streams the selected social media posts and sends them to the device. The device then overlays these posts on the match footage. A specific example of how this works is when the server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays the positive posts. In this case, the input is "social media post data" and the output is "selected social media posts."

[1233] Step 4:

[1234] The server analyzes game footage in real time and detects important events, which generates event data. The AI ​​generates real-time commentary based on this event data and sends it to the device. The device displays the commentary below the game footage. After the game ends, the user can request a highlight video from the device. The device sends the request to the server. The server automatically generates a highlight video based on the game record and sends it to the device. The device then plays the highlight video. A specific example of how this works is when the server detects the moment a home run is hit, and the AI ​​generates real-time commentary, saying, "A player from Team A just hit a home run!" In this case, the input is "game footage data" and the output is "commentary data."

[1235] Step 5:

[1236] While watching a game, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time, generating emotional data. This data is sent to a server, where an emotion engine within the server analyzes the data and recognizes the user's emotions. The recognized emotional data is used to adjust the display of game commentary and social media posts. A specific example of how this works is when the device captures the user's surprised expression, which the server's emotion engine analyzes and provides detailed commentary. In this case, the input is "user's facial and voice data," and the output is "recognized emotional data."

[1237] Step 6:

[1238] The server accumulates the user's emotional data and uses this data to provide a more personalized and optimized viewing experience. The accumulated data is then combined with the user's viewing history and preferences for analysis. A specific example of how this works is to combine past viewing data with emotional data to optimize future viewing experiences. In this case, the input is "emotional data and viewing history data," and the output is "optimized viewing experience data."

[1239] (Application example 2)

[1240] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1241] Conventional sports event viewing systems lack real-time optimization based on viewer emotions and preferences, resulting in viewing experiences that are not tailored to individual viewers. Furthermore, integration of social media data and real-time commentary cannot instantly provide viewers with the information they most desire at any given moment. This has led to issues such as insufficient viewer satisfaction.

[1242] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to log in to the service by entering authentication information on an available terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and SNS posts on the game video, a means for automatically generating and providing a highlight video based on the user's viewing history and preferences, a means for analyzing the user's facial expressions and voice in real time to recognize emotions, and a means for adjusting content to optimize the viewing experience based on the recognized emotions. This makes it possible to provide an optimal viewing experience that promptly responds to the emotions and preferences of each viewer.

[1243] "Authentication information" refers to personal identification information such as a username and password that a user enters when accessing a service.

[1244] "Game video streaming" is a technology that distributes video of a specific sports game in real time over the Internet.

[1245] "Real-time commentary" refers to commentary information that is generated instantly as the match progresses, explaining important events of the match to the viewer.

[1246] "SNS posts" are messages, images, videos, and other content posted by users on social networking services.

[1247] "Viewing history" is a record of content that a user has viewed in the past.

[1248] A "highlight video" is a shortened video that is edited to include only the important scenes and events of a game.

[1249] "Means of recognizing emotions by analyzing facial expressions and voice in real time" refers to technology that uses a camera and microphone to analyze the user's face and voice and grasp their emotions at that time (joy, surprise, concentration, etc.).

[1250] "Means to adjust content to optimize the viewing experience" refers to technology that changes and adjusts displayed content (comments, social media posts, etc.) in real time based on the user's emotions and preferences.

[1251] This invention relates to a sports event viewing system that optimizes the viewing experience for users. In particular, it provides a new service that goes beyond conventional viewing experiences by recognizing emotions in real time and adjusting viewing content according to those emotions.

[1252] System Overview and Configuration

[1253] 1. Authentication and Login

[1254] Users log in to the service by entering their authentication information (username and password) on their smartphone or other internet-enabled device, which is then sent to the server, which checks the authentication information against a database.

[1255] 2. Game Streaming

[1256] The server provides video streaming of the selected sports game and sends the streaming URL to the user's available device, which uses the URL to display the live video of the game.

[1257] 3. Real-time commentary and social media posts

[1258] The server collects relevant social media posts as the match progresses, and AI analyzes them to select the most relevant posts, which are then overlaid on the match footage.

[1259] Additionally, AI detects important events during the game (such as runs scored or home runs) and generates commentary in real time, which is then overlaid on the game footage.

[1260] 4. Automatic generation and provision of highlight videos

[1261] After the game ends, a highlight video is automatically generated based on the user's viewing history and preferences. The server generates this video in response to the user's request and sends it to the user's device, allowing the user to watch a personalized highlight video.

[1262] 5. Emotion Recognition and Viewing Experience Optimization

[1263] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time, and this data is sent to a server where an emotion engine analyzes it to recognize the user's emotions.

[1264] Based on the recognized emotion data, the server adjusts the game commentary and displayed social media posts, for example, providing more detailed commentary if the user is surprised, and prioritizing positive social media posts if the user is enjoying the game.

[1265] Hardware and software used

[1266] Hardware: Smartphone, Head-Mounted Display (HMD), Camera, Microphone

[1267] Software: OpenCV (video capture and display), Requests (HTTP communication)

[1268] In addition, the emotion engine, SNS API, and AI model run on a server or cloud environment.

[1269] Specific examples

[1270] The user launches the "next-generation sports viewing app" on their smartphone and enters their login information. After successful authentication by the server, the user begins watching the match footage of "Team A vs. Team B." During the match, the smartphone's camera and microphone continuously capture the user's facial expressions and voice. If the user's surprised expression is detected, the server's emotion engine analyzes this and generates and displays detailed commentary in real time. Related positive social media posts are also overlaid on the screen. After the match, the user requests a highlight video, which the server automatically generates and sends to the user's smartphone. The video is then edited to include the most exciting scenes based on past viewing and emotion data.

[1271] Prompt Sentence Examples

[1272] Implement a system that analyzes emotions from users' facial expressions and voice in real time to optimize the viewing experience of sporting events. Program a next-generation sports viewing app that displays social media posts and provides commentary on key moments of a match based on the recognized emotions. For example, provide more detailed commentary if the user is surprised.

[1273] This system allows for an individually optimized viewing experience for sporting events, providing viewers with the real-time information and commentary they most desire at any given moment.

[1274] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1275] Step 1:

[1276] The user enters authentication information (username and password) on the device to log in. The entered authentication information is sent from the device to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The user can use this token to access other functions within the service.

[1277] Step 2:

[1278] The user selects the match they want to watch. The user sends a request including the match ID from their device to the server. The server collects the live streaming URL of the match and related social media data and sends it to the device. The device starts displaying the live video of the match using the received URL.

[1279] Step 3:

[1280] While the game video is being streamed, the server collects related social media posts in real time. AI analyzes the data collected via the social media API and selects the most relevant posts. The selected social media posts are sent to the device and displayed over the game video, allowing users to view real-time social media reactions as the game progresses.

[1281] Step 4:

[1282] The server analyzes game footage in real time and detects important events (scores, home runs, etc.). Based on this event data, AI automatically generates commentary and sends it to the device in real time. The device then displays the generated commentary, allowing the user to visually recognize important moments.

[1283] Step 5:

[1284] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice. This data is sent to a server in real time. An emotion engine on the server analyzes the data and recognizes the user's emotions. For example, if the user is surprised, this information is reflected in the level of detail in the commentary and the selection of social media posts displayed.

[1285] Step 6:

[1286] After the game ends, the user requests the generation of a highlight video. The request, including the game ID, is sent from the device to the server, and the server automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and emotional data and sent to the device. The device plays the generated highlight video, allowing the user to enjoy a personally optimized summary of the game.

[1287] Step 7:

[1288] The server stores user emotional data and viewing history, which is used to optimize future viewing experiences. The data is analyzed to identify which types of events users respond most strongly to. This information is then used to provide users with the most appropriate content and commentary when viewing future matches.

[1289] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1290] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1291] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1292] [Fourth embodiment]

[1293] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1294] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1295] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1296] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1297] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1298] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1299] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1300] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1301] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1302] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1303] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1304] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1305] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1306] The present invention relates to a system for providing a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[1307] Authentication and Login

[1308] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[1309] Select a match and start watching

[1310] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[1311] Social Media Feed Integration

[1312] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[1313] Real-time match commentary and highlight generation

[1314] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[1315] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences.

[1316] For example, if a user is rooting for a particular player, a highlight video is generated that mainly shows scenes in which that player is playing an active role. The generated highlight video is transmitted from the server to the terminal and played on the terminal.

[1317] Specific examples

[1318] Specific examples of the present invention are shown below.

[1319] Authentication and Login

[1320] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[1321] Select a match and start watching

[1322] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[1323] Social Media Feed Integration

[1324] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[1325] Real-time match commentary and highlight generation

[1326] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, saying, "A player from Team A just hit a home run!" The user requests a highlight video after the game ends, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[1327] This invention allows viewers to enjoy sporting events from a new perspective, creating an integrated viewing experience with detailed real-time information and social media posts, and allows viewers to easily watch personalized highlight videos after the game.

[1328] The processing flow will be explained below.

[1329] Step 1:

[1330] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[1331] Step 2:

[1332] The device sends the entered authentication information to the server in JSON format.

[1333] Example of sending data: {"username": "example_user", "password": "example_password"}

[1334] Step 3:

[1335] The server checks the authentication information it receives against a database to verify its validity.

[1336] Step 4:

[1337] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[1338] Example response: {"token": "authtoken1234", "status": "success"}

[1339] Step 5:

[1340] The device saves the received token and transitions to the home screen.

[1341] Step 6:

[1342] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[1343] Step 7:

[1344] The device sends a request to the server containing the selected game ID (e.g., game1234).

[1345] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1346] Step 8:

[1347] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[1348] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[1349] Step 9:

[1350] The device will start displaying the game video using the live streaming URL.

[1351] Step 10:

[1352] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[1353] Step 11:

[1354] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[1355] Analysis example: Selecting posts with high positive sentiment scores.

[1356] Step 12:

[1357] The server streams the extracted social media posts to the device.

[1358] Step 13:

[1359] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[1360] Step 14:

[1361] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[1362] Step 15:

[1363] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[1364] Example: "Team A's player just scored a goal!"

[1365] Step 16:

[1366] The device displays real-time commentary below the game footage.

[1367] Step 17:

[1368] After the game ends, the user presses the "Highlights" button in the app.

[1369] Step 18:

[1370] The terminal sends a highlight generation request to the server.

[1371] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1372] Step 19:

[1373] Based on the server's match records, AI automatically generates highlight videos tailored to the user's preferences.

[1374] Example of generated content: Extract only scenes in which your favorite player shines.

[1375] Step 20:

[1376] The server returns the generated highlight video to the terminal.

[1377] Example response: {"highlight_url": "http: / / highlights.url"}

[1378] Step 21:

[1379] The device will play the highlight video.

[1380] These are the specific processing steps and detailed operation of the "Sports TV" system, which allows viewers to enjoy real-time game updates, a sense of unity through social media, and personalized highlight videos.

[1381] Example 1

[1382] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1383] Conventional methods for viewing sporting events have limited viewers' viewing experience by failing to provide detailed real-time information or a viewing experience that is fully integrated with social media. Furthermore, there is a lack of a way for viewers to easily view highlight videos tailored to their preferences after the game has ended. This has created a need for a system that offers a richer, more immersive viewing experience and allows viewers to enjoy personalized viewing content even after the game has ended.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1385] In this invention, the server includes a means for logging in to the service by entering authentication information on a terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and posts from social networking services on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for collecting and selecting posts from social networking services that are highly relevant to the game. This allows users to enjoy detailed real-time information and a rich and integrated viewing experience linked to SNS, and enables them to easily watch personalized highlight videos even after the game is over.

[1386] A "terminal" is an electronic device that allows a user to access various services by operating it.

[1387] "Authentication Information" means data used to identify and secure a user.

[1388] "Service" is a general term for the functions and content provided to users.

[1389] "Streaming" is a technology for playing back data while transferring it in real time.

[1390] "Commentary" is an explanation of important events and actions that take place during the course of a match.

[1391] A "social networking service" is a platform that allows users to communicate with each other via the Internet.

[1392] A "post" is a message or content posted by a user on a social networking service.

[1393] "Viewing history" is a record of the content a user has viewed.

[1394] "Preferences" are the tendencies and choices that a user is particularly interested in.

[1395] A "highlight video" is a short video that summarizes the important moments of a game.

[1396] "Collection" is the act of gathering data or information.

[1397] "Sorting" is the act of analyzing collected data and information and selecting the most relevant information.

[1398] "Analysis" is the examination and decomposition of data or information to understand and derive meaning.

[1399] An "Event" is a significant occurrence or action that occurs during a match.

[1400] The present invention relates to a system that provides a user with a new service that goes beyond the conventional viewing experience when viewing a sporting event. Specific embodiments of the system of the present invention will be described below.

[1401] Authentication and Login

[1402] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device then sends the entered authentication information (username and password) to the server. The server compares the received authentication information with its database, and if authentication is successful, the server generates a session token and returns it to the device. Using this token, the user can access other features within the service.

[1403] Select a match and start watching

[1404] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[1405] Social Media Feed Integration

[1406] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[1407] Real-time match commentary and highlight generation

[1408] The server analyzes the game footage in real time and detects important events (e.g., runs scored, home runs, etc.). Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[1409] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and preferences. For example, if a user is rooting for a particular player, a highlight video centered on that player's performance will be generated. The generated highlight video is sent from the server to the device and played on the device.

[1410] Specific examples

[1411] Authentication and Login

[1412] The user launches the "Sports Viewing App" on their device and enters the username "sample_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[1413] Select a match and start watching

[1414] When a user selects the match "Team A vs Team B," the device sends a request including the match ID "game5678" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device then displays the live video of the match.

[1415] Social Media Feed Integration

[1416] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[1417] Real-time match commentary and highlight generation

[1418] The server detects the moment a home run is hit, and the AI ​​generates a commentary in real time, such as "A player from Team A just hit a home run!" After the game ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the game record and sends it to the device. The device then plays the highlight video.

[1419] In this way, the present invention allows users to enjoy sporting events from a new perspective, providing a unified viewing experience through real-time information provision and social media posting, and also allows users to easily watch personalized highlight videos after the game is over.

[1420] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1421] Step 1:

[1422] The user launches the "Sports Viewing App" on their device and enters their username and password on the login screen. The device sends this input information to the server. The input is the username and password, and the output is the authentication information sent to the server. Specifically, if the username is "sample_user" and the password is "password123", this information is encrypted and sent to the server.

[1423] Step 2:

[1424] The server checks the received authentication information against the database. It compares the user information in the database with the entered authentication information, and if they match, authentication is successful. If authentication is successful, the server generates a session token and returns it to the terminal. This token indicates authentication during the session and is used for subsequent requests. The input is authentication information and the output is a session token. Specifically, the server compares the password of "sample_user" with the value in the database, and if they match, it generates a session token in the format "token12345" and sends it to the terminal.

[1425] Step 3:

[1426] The user selects the game they want to watch on the home screen. The device then requests the ID of the selected game from the server. The input is the game ID, and the output is the game's live streaming URL and related social media data. Specifically, when the user selects the game "Team A vs Team B," the game ID "game5678" is sent to the server.

[1427] Step 4:

[1428] The server retrieves relevant data based on the match ID. It collects the match's live streaming URL and related social networking service (SNS) data and sends it to the device. The input is the match ID, and the output is the live streaming URL and SNS data. Specifically, the URL "http: / / livestream / game5678" corresponding to "game5678" and the SNS data are collected and sent to the device.

[1429] Step 5:

[1430] The device uses the received URL to start playing live video of the game. The input is the live streaming URL, and the output is the display of live video. Specifically, the device accesses the URL "http: / / livestream / game5678" and starts streaming.

[1431] Step 6:

[1432] The server collects social media posts using relevant hashtags and keywords during the game. The input is the relevant hashtags and keywords, and the output is the collected social media post data. Specifically, it collects posts from social media using hashtags such as "Team A" and "Team B."

[1433] Step 7:

[1434] The AI ​​(inside the server) analyzes the collected SNS posts and selects those with high relevance. The input is SNS post data, and the output is the selected SNS posts. Specifically, it uses natural language processing technology to select posts with high positive sentiment scores.

[1435] Step 8:

[1436] The server streams the selected SNS posts to the device. The input is the selected SNS posts, and the output is the display on the device. Specifically, the selected posts are sent to the device.

[1437] Step 9:

[1438] The device displays the selected SNS posts overlaid on the game video. The input is the selected SNS posts, and the output is an overlay display on the game video. Specifically, it displays real-time SNS posts on top of the game video.

[1439] Step 10:

[1440] The server analyzes the game footage as it progresses and detects important events (e.g., runs scored, home runs, etc.). The input is the game footage, and the output is the detected event data. Specifically, it uses video analysis technology to detect important moments.

[1441] Step 11:

[1442] The AI ​​(in the server) generates commentary in real time based on the detected events. The input is the event data, and the output is the generated commentary text. Specifically, it generates text such as "Team A's player just hit a home run!"

[1443] Step 12:

[1444] The terminal displays the commentary text received from the server. The input is the commentary text, and the output is the commentary displayed below the game video. Specifically, the commentary text is displayed below the video in real time.

[1445] Step 13:

[1446] After the game ends, the user presses the highlight button to request a highlight video. This causes the device to send a highlight generation request to the server. The input is the highlight generation request, and the output is the request data. Specifically, the request data is sent to the server when the user clicks.

[1447] Step 14:

[1448] The server automatically generates highlight videos based on the match records and the user's viewing history. The input is the match records and viewing history, and the output is the generated highlight videos. Specifically, if a user is rooting for a specific player, the server generates a highlight video edited to focus on scenes in which that player is active.

[1449] Step 15:

[1450] The server transmits the generated highlight video to the terminal, and the terminal plays the highlight video. The input is the generated highlight video, and the output is the played highlight video. Specifically, the highlight video is transmitted to the terminal and played.

[1451] (Application example 1)

[1452] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1453] Traditional sports event viewing systems are limited to streaming game footage and displaying simple social media posts, but lack the means to individually enhance the user's viewing experience. They also lack real-time commentary, highlighting key moments, and generating personalized highlight videos. This makes it difficult for viewers to enjoy detailed real-time information or a viewing experience tailored to their individual interests and preferences.

[1454] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1455] In this invention, the server includes a means for users to log in to the service by entering authentication information on an available terminal, a means for providing game video streaming, a means for overlaying social media posts on game video, a means for adding inspiring commentary to key moments in real time using a generative AI model, and a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, thereby enabling users to enjoy individually customized viewing experiences in real time.

[1456] "User" means any individual or entity using the Service.

[1457] "Device" means an electronic device through which a User accesses the Service, such as a smartphone, tablet, computer, or smart TV.

[1458] "Authentication Information" refers to information that identifies an individual, such as a user name and password.

[1459] "Service" means the online platform that provides match video streaming and related functionality.

[1460] "Game video streaming" refers to the technology of providing live or recorded sports game footage over the Internet.

[1461] "SNS posts" refer to posts and comments made on social networking services.

[1462] A "generative AI model" refers to an algorithm or system that uses artificial intelligence to analyze and generate data.

[1463] "Real-time commentary" refers to commentary and commentary that is generated instantly as the match progresses.

[1464] "Viewing history" refers to a record of content that a user has viewed to date.

[1465] "Preferences" refers to a user's personal interests or preferences regarding a particular player, team, play, etc.

[1466] A "highlight video" is a short video that compiles important moments and highlights of a game.

[1467] An embodiment of the present invention is described below.

[1468] Authentication and Login

[1469] A user launches the "Sports TV" application on their device and enters their username and password. The device sends the authentication information to the server, which checks it against a predefined database. If the check is successful, the server generates a session token and returns it to the device. The user can use this token to access other features within the service.

[1470] Select a match and start watching

[1471] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server, which collects the game's live streaming URL and related social media data and sends it to the device, which then uses the URL to begin displaying the game's live stream.

[1472] Social Media Feed Integration

[1473] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by a generative AI model within the server, and the most relevant posts are selected. The selected social media posts are streamed by the server and displayed on the device overlaid on the match video. This allows users to enjoy the match while watching social media reactions in real time.

[1474] Real-time match commentary

[1475] Game footage is analyzed in real time by the server to detect important events (e.g., runs scored, home runs, etc.). Based on this, a generative AI model in the server automatically generates compelling commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, informing users of key moments.

[1476] Generate personalized highlight videos

[1477] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. The generated highlight video is customized based on the user's viewing history and preferences. The generated highlight video is then sent from the server to the device, where it is played.

[1478] Specific examples

[1479] A user launches the "Sports TV" application and enters the username "example_user" and password. The device sends the authentication information to the server, which returns a session token on success.

[1480] The user selects the match "Team A vs Team B", and the device sends the match ID "game1234" to the server. The server returns the live streaming URL and related SNS data, and the device displays the live video of the match.

[1481] The server collects social media posts with the match-related hashtags "Team A" and "Team B," and the generative AI model selects posts with high positive sentiment scores. The selected posts are then overlaid on the match footage.

[1482] The server detects a home run event, and the generative AI model generates commentary in real time, such as, "Team A's player just hit a home run!" After the game ends, the user requests a highlight video, and the server generates a personalized highlight video and sends it to the device.

[1483] Prompt Sentence Examples

[1484] "We'd like your idea for an application that provides spectators with real-time coverage of exciting moments during a sports game. For example, the app would include functionality where a user logs in to the app, selects a game, displays a stream of that game and related social media feeds, and generates real-time commentary of key moments. Please explain these features with examples."

[1485] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1486] Step 1:

[1487] The user launches the "Sports TV" application on their device. They enter their username and password on the login screen. The device sends this authentication information to the server. The username and password are used as input, and the authentication information is sent to the server as output. At this point, no data processing is done, only information is transferred.

[1488] Step 2:

[1489] The server checks the received authentication information against the database. It compares the user information in the database with the entered information to confirm that they match. If the check is successful, the server generates a session token and returns it to the terminal. The input here is authentication information, and the output is a session token. The generation process involves generating a session token.

[1490] Step 3:

[1491] The user selects a match they are interested in on the home screen. The device requests the ID of the selected match from the server. The input is the match ID, and the output is the request. The transfer of the match ID is required as data processing.

[1492] Step 4:

[1493] The server collects the match's live streaming URL and related social media data and sends it to the device. The input is the match ID and session token, and the output is the live streaming URL and social media data. Data processing involves obtaining the necessary information from multiple data sources and sending it as a single dataset.

[1494] Step 5:

[1495] The device uses the received URL to begin displaying live video of the match. The input is the live streaming URL, and the output is the live video. The data processing here involves requesting and displaying the video based on the received URL.

[1496] Step 6:

[1497] The server collects social media posts using relevant hashtags and keywords during the match. Data is acquired through the social media API. The input for this step is hashtags and keywords, and the output is a set of social media posts. Data processing involves collecting data through the social media API and analyzing the relevance of hashtags.

[1498] Step 7:

[1499] The collected SNS posts are analyzed by a generative AI model on the server, and those with high relevance are selected. The input is a group of SNS posts, and the output is the selected SNS posts. As a data calculation, the generative AI model determines the sentiment score and relevance, and selects appropriate posts.

[1500] Step 8:

[1501] The selected SNS posts are streamed by the server and displayed on the device overlaid on the game video. The input is the selected SNS posts, and the output is the video with the SNS posts overlaid. Data processing requires overlaying the video.

[1502] Step 9:

[1503] The server analyzes game footage in real time to detect important events (such as runs scored or home runs). The input is the game footage, and the output is the detected events. As a data calculation, a generative AI model analyzes the footage in real time and processes it to extract important events.

[1504] Step 10:

[1505] The generative AI model generates commentary in real time based on the detected event and sends it to the device. The input is the detected event and the output is real-time commentary. Data processing includes the process of the generative AI model generating text and sending it to the device.

[1506] Step 11:

[1507] After the game ends, the user presses the highlight button in the app to request a highlight video tailored to their individual preferences. The device then sends a highlight generation request to the server. The input is the user's preferences, and the output is the highlight generation request. Data processing here is the process of reflecting the user's preferences in the request.

[1508] Step 12:

[1509] The server automatically generates a highlight video based on the match record and sends it to the device. The generated highlight video is customized based on the user's viewing history and preferences. The input here is the viewing history and preferences, and the output is the highlight video. Data processing involves editing the highlight video based on the viewing history and user preferences.

[1510] Step 13:

[1511] The device plays the highlight video. The input is the URL or data of the highlight video, and the output is the playback of the highlight video. Data processing here is the process of decoding the video data and playing it back.

[1512] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1513] The present invention relates to a system that provides users with a new service that goes beyond the conventional viewing experience when watching sporting events. In particular, the system uses an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of the present invention are described below.

[1514] Authentication and Login

[1515] A user launches the Sports TV app on their device and enters their username and password on the login screen. The device sends the authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. This token allows the user to access other features within the service.

[1516] Select a match and start watching

[1517] When a user selects a game of interest on the home screen, the device requests the selected game's ID from the server. The server collects the game's live streaming URL and related social media data and sends it to the device. The device then uses the received URL to begin displaying the game's live stream.

[1518] Social Media Feed Integration

[1519] The server collects social media posts using relevant hashtags and keywords during the match. The collected posts are analyzed by AI (within the server) and the most relevant ones are selected. These selected social media posts are streamed by the server and displayed on the device overlaid on the match footage. This allows users to enjoy the match while watching social media reactions in real time.

[1520] Real-time match commentary and highlight generation

[1521] The game footage is analyzed in real time by the server, and important events (e.g., runs scored, home runs, etc.) are detected. Based on this, the AI ​​(in the server) generates commentary in real time and sends it to the device. The commentary is displayed at the bottom of the game footage, helping users understand important moments.

[1522] After the game, users can press the highlight button in the app to request a highlight video tailored to their individual preferences. The device sends a highlight generation request to the server, which then automatically generates a highlight video based on the game record. This highlight video is customized based on the user's viewing history and preferences.

[1523] User Emotion Recognition

[1524] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is then sent to a server, where an emotion engine analyzes the data and recognizes the user's emotions.

[1525] For example, the emotion engine can detect whether a user is smiling, surprised, or focused, and the recognized emotion data is used to tailor the presentation of match commentary and social media posts.

[1526] For example, if a user is surprised, the emotion engine can provide a detailed explanation that matches that emotion. Also, if a user is having fun, it will prioritize displaying positive social media posts that will further enhance that emotion.

[1527] Accumulation and optimization of emotional data

[1528] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is analyzed in combination with the user's viewing history and preferences to provide a more personalized viewing experience.

[1529] For example, combining past viewing data with emotional data can identify which types of events a user responds to most strongly, allowing for tailored content and commentary to be delivered to that user when viewing future matches.

[1530] Specific examples

[1531] Specific examples of the present invention are shown below.

[1532] Authentication and Login

[1533] The user launches the "Sports TV" app on their device and enters the username "example_user" and password. The device sends the authentication information to the server, and if authentication is successful, the server generates a session token and returns it to the device.

[1534] Select a match and start watching

[1535] The user selects the match "Team A vs Team B." The device sends a request including the match ID "game1234" to the server, and the server returns the live streaming URL and match-related SNS data to the device. The device displays the live video of the match.

[1536] Social Media Feed Integration

[1537] The server collects social media posts using the match-related hashtags "Team A" and "Team B," and the AI ​​selects posts with a high positive emotion score. The selected social media posts are then overlaid on the game footage by the device.

[1538] Real-time match commentary and emotion recognition

[1539] The server detects the moment of a home run, and the AI ​​generates a commentary in real time, such as "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, and the emotion engine analyzes them to recognize the user's emotions. If the device captures a surprised expression, the emotion engine will provide a more detailed commentary.

[1540] Highlight Generation

[1541] After a match ends, the user requests a highlight video, and the device sends the request to the server. The server automatically generates a highlight video tailored to the user's preferences based on the match record and sends it to the device. The device then plays the highlight video. At this time, the video is edited based on the accumulated emotional data to include the scenes that most excite the user.

[1542] This invention allows viewers to enjoy sporting events from a new perspective, providing a holistic viewing experience with detailed real-time information and social media posts. Furthermore, an emotion engine provides an optimized viewing experience tailored to each viewer's individual emotions. Furthermore, after the game, viewers can easily watch personalized highlight videos.

[1543] The processing flow will be explained below.

[1544] Step 1:

[1545] The user launches the "Sports TV" app on their device and enters their username and password on the login screen.

[1546] Step 2:

[1547] The device sends the entered authentication information to the server in JSON format.

[1548] Example of sending data: {"username": "example_user", "password": "example_password"}

[1549] Step 3:

[1550] The server checks the authentication information it receives against a database to verify its validity.

[1551] Step 4:

[1552] If the server is successful in authentication, it generates a session token (e.g., authtoken1234) and returns it to the terminal.

[1553] Example response: {"token": "authtoken1234", "status": "success"}

[1554] Step 5:

[1555] The device saves the received token and transitions to the home screen.

[1556] Step 6:

[1557] The user selects the match they want to watch, "Team A vs Team B," on the home screen.

[1558] Step 7:

[1559] The device sends a request to the server containing the selected game ID (e.g., game1234).

[1560] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1561] Step 8:

[1562] The server collects the live streaming URL and related social media data based on the match ID and returns it to the device.

[1563] Example response: {"stream_url": "http: / / streaming.url", "hashtags": ["TeamA", "TeamB"]}

[1564] Step 9:

[1565] The device will start displaying the game video using the live streaming URL.

[1566] Step 10:

[1567] The server collects posts in real time from the social media API using match-related hashtags (e.g., TeamA, TeamB).

[1568] Step 11:

[1569] AI (within the server) analyzes the collected social media posts using natural language processing (NLP) and extracts posts with high positive sentiment scores.

[1570] Analysis example: Selecting posts with high positive sentiment scores.

[1571] Step 12:

[1572] The server streams the extracted social media posts to the device.

[1573] Step 13:

[1574] The device will overlay social media posts (e.g., "Go Team A!") along with game footage.

[1575] Step 14:

[1576] The server analyzes the game video data in real time and detects important events (e.g., runs scored, home runs).

[1577] Step 15:

[1578] Based on the key moments detected by the AI ​​(in the server), commentary is generated in real time and provided to the device.

[1579] Example: "Team A's player just hit a home run!"

[1580] Step 16:

[1581] The device displays real-time commentary below the game footage.

[1582] Step 17:

[1583] The device uses its built-in camera and microphone to capture the user's facial expressions and voice while watching.

[1584] Step 18:

[1585] The device sends the captured data to the server.

[1586] Step 19:

[1587] The emotion engine in the server analyzes the transmitted data and recognizes the user's emotion.

[1588] For example, detecting emotions such as smile, surprise, and concentration.

[1589] Step 20:

[1590] The server adjusts the commentary and social media posts provided based on the user's perceived emotions.

[1591] Example: If the user is surprised, provide a detailed explanation.

[1592] Step 21:

[1593] After the game ends, the user presses the "Highlights" button in the app.

[1594] Step 22:

[1595] The terminal sends a highlight generation request to the server.

[1596] Example of sending data: {"token": "authtoken1234", "game_id": "game1234"}

[1597] Step 23:

[1598] Based on the game record, the server uses AI to automatically generate highlight videos tailored to the user's preferences and emotional data.

[1599] Example of generated content: Extract only scenes in which your favorite player shines.

[1600] Step 24:

[1601] The server returns the generated highlight video to the terminal.

[1602] Example response: {"highlight_url": "http: / / highlights.url"}

[1603] Step 25:

[1604] The device will play the highlight video.

[1605] Video example: Highlights edited to include the scenes that most excite users.

[1606] The above are the specific processing steps and detailed operation of the "Sports TV" system. By combining it with the emotion engine, viewers can grasp the game situation in real time, enjoy the sense of unity through social media, and enjoy an optimal viewing experience tailored to their individual emotions. Furthermore, after the game ends, viewers can watch personalized highlight videos.

[1607] Example 2

[1608] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1609] With conventional viewing systems for sporting events, viewers often could not simultaneously experience real-time commentary or the reactions of other viewers. It was also difficult to provide a personalized viewing experience that incorporated viewers' emotional responses. Therefore, there was a need to provide a new viewing experience that would allow viewers to feel more engaged.

[1610] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1611] In this invention, the server includes a means for logging in to the service by inputting authentication information on a user's device, a means for providing game video streaming, a means for overlaying generated real-time commentary and social networking service posts on the game video, a means for automatically generating and providing highlight videos based on the user's viewing history and preferences, and a means for recognizing the user's emotions and optimizing the viewing experience based on that information, thereby enabling viewers to enjoy not only the real-time game streaming and commentary, but also a personalized viewing experience based on the reactions of other viewers and their own emotions.

[1612] "Authentication information" refers to information such as an identifier and password required for a user to access a service.

[1613] "Service" refers to the provision of various digital content and functions that users can access and use through their devices.

[1614] "Logging in" is the process by which a user gains access to a service using authentication information.

[1615] "Game video streaming" refers to video of a sports game being provided in real time over the Internet.

[1616] "Real-time commentary" refers to sports commentary information that is generated instantly in response to events occurring during a game.

[1617] "Social networking service" refers to a platform on the Internet where users can share information and communicate.

[1618] A "post" is information such as messages, comments, photos, and videos posted by users on social networking services.

[1619] "Viewing history" is a record of content that a user has viewed in the past.

[1620] "Preferences" refers to a user's preference for a particular genre or theme as indicated by their past viewing history and behavior.

[1621] A "highlight video" is a short video clip that selects important moments from a game.

[1622] "Emotion recognition" is the process of determining a user's emotional state from data such as facial expressions and voice.

[1623] "Optimizing the viewing experience" means customizing the content and interface provided to maximize the user's viewing satisfaction.

[1624] This invention relates to a system that provides users with a new viewing experience that goes beyond the conventional viewing experience when watching sporting events. Specifically, it is characterized by using an emotion engine to recognize the user's emotions and optimize the viewing experience. Specific embodiments of the system of this invention are described below.

[1625] Authentication and Login

[1626] The user launches the "sports viewing application" on their device and accesses the login screen. They enter their username and password and press the "Login" button. The device sends the entered authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores the session token, allowing the user to access the service. The hardware used in this process is a device such as a smartphone or tablet, and the software used is an authentication information management database.

[1627] Select a match and start watching

[1628] The user selects the game they want to watch on the home screen. The device requests the ID of the selected game from the server. The server collects the game's live streaming URL and related social networking service data and sends it to the device. The device then uses the received URL to begin displaying live footage of the game. The live streaming distribution service used in this case is YouTube Live or Twitch.

[1629] Social Media Feed Integration

[1630] The server collects posts from social networking services using relevant hashtags and keywords during the match. AI within the server analyzes the collected posts and selects the most relevant ones. The server streams the selected posts and sends them to the device. The device then overlays these posts on the match footage. The software used includes AI analysis software and social media APIs such as Twitter API and Instagram API.

[1631] Real-time match commentary and highlight generation

[1632] The server analyzes the game footage in real time and detects important events. The AI ​​in the server generates real-time commentary based on the detected events and sends it to the device. The commentary is displayed at the bottom of the game footage. After the game ends, the user can request a highlight video from their device. The server automatically generates a highlight video based on the game recording and sends it to the device. The device then plays the highlight video. This allows the user to watch the game without missing any important scenes.

[1633] User Emotion Recognition

[1634] While watching a match, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time. This data is sent to a server, where an emotion engine analyzes the data to recognize the user's emotions. The recognized emotion data is used to adjust the display of match commentary and social media posts. For example, if the user is surprised, the emotion engine will provide detailed commentary tailored to that emotion. If the user is enjoying the match, the engine will display positive social media posts that further enhance that emotion.

[1635] Accumulation and optimization of emotional data

[1636] The server accumulates user emotional data and uses it to optimize future viewing experiences. The accumulated data is then combined with the user's viewing history and preferences to provide a more personalized viewing experience. For example, combining past viewing data with emotional data can identify which types of events a user responds most strongly to, allowing the server to provide that user with the most appropriate content and commentary when viewing future games.

[1637] Examples and behavior

[1638] The user logs in by entering "example_user" and password into the terminal. The terminal sends the authentication information to the server, which checks it against the database. If authentication is successful, the server generates a session token and sends it to the terminal.

[1639] The user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. The server returns the live streaming URL and related SNS data. The device starts displaying the live video from the received URL.

[1640] The server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays posts with high positive sentiment scores.

[1641] The server detects the moment a home run is hit from the game footage, and the AI ​​generates a real-time commentary such as, "Team A's player just hit a home run!" The device captures the user's facial expressions and voice, which are analyzed by the server's emotion engine. If the user is surprised, the emotion engine provides a detailed commentary.

[1642] After the game ends, the user requests a highlight video, and the device sends the request to the server, which then generates a highlight video tailored to the user's preferences and plays it on the device.

[1643] This system allows users to enjoy a highly personalized sports viewing experience, detailed information provided in real time, and social media posts. In particular, the emotional engine's monitoring and feedback can increase viewer satisfaction.

[1644] Prompt Sentence Examples

[1645] "How can I provide detailed explanations when a user is surprised?"

[1646] "Please provide specific examples of how to filter and display positive social media posts."

[1647] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1648] Step 1:

[1649] The user launches the "Sports Viewing Application" on the device and accesses the login screen. The user enters their username and password and presses the "Login" button. This generates the entered authentication information. The device sends this authentication information to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The device stores this session token, allowing the user to access the service. A specific example of how this works is when a user logs in by entering "example_user" and a password. In this case, the input is "username and password" and the output is "session token."

[1650] Step 2:

[1651] The user selects the match they want to watch on the home screen. This selection generates a match ID. The device then requests the selected match ID from the server. The server collects the match's live streaming URL and related social media data and sends them to the device. The device then uses the received URL to begin displaying live footage of the match. For example, a user selects the match "Team A vs Team B" and the device requests the match ID "game1234" from the server. In this case, the input is the "match ID" and the output is the "live streaming URL and related social media data."

[1652] Step 3:

[1653] The server collects social media posts using relevant hashtags and keywords during the match. This generates highly relevant social media post data. The collected posts are analyzed by AI (within the server) and selected for their relevance. The server streams the selected social media posts and sends them to the device. The device then overlays these posts on the match footage. A specific example of how this works is when the server collects social media posts using the hashtags "TeamA" and "TeamB" and selects and displays the positive posts. In this case, the input is "social media post data" and the output is "selected social media posts."

[1654] Step 4:

[1655] The server analyzes game footage in real time and detects important events, which generates event data. The AI ​​generates real-time commentary based on this event data and sends it to the device. The device displays the commentary below the game footage. After the game ends, the user can request a highlight video from the device. The device sends the request to the server. The server automatically generates a highlight video based on the game record and sends it to the device. The device then plays the highlight video. A specific example of how this works is when the server detects the moment a home run is hit, and the AI ​​generates real-time commentary, saying, "A player from Team A just hit a home run!" In this case, the input is "game footage data" and the output is "commentary data."

[1656] Step 5:

[1657] While watching a game, the device uses its built-in camera and microphone to capture the user's facial expressions and voice in real time, generating emotional data. This data is sent to a server, where an emotion engine within the server analyzes the data and recognizes the user's emotions. The recognized emotional data is used to adjust the display of game commentary and social media posts. A specific example of how this works is when the device captures the user's surprised expression, which the server's emotion engine analyzes and provides detailed commentary. In this case, the input is "user's facial and voice data," and the output is "recognized emotional data."

[1658] Step 6:

[1659] The server accumulates the user's emotional data and uses this data to provide a more personalized and optimized viewing experience. The accumulated data is then combined with the user's viewing history and preferences for analysis. A specific example of how this works is to combine past viewing data with emotional data to optimize future viewing experiences. In this case, the input is "emotional data and viewing history data," and the output is "optimized viewing experience data."

[1660] (Application example 2)

[1661] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1662] Conventional sports event viewing systems lack real-time optimization based on viewer emotions and preferences, resulting in viewing experiences that are not tailored to individual viewers. Furthermore, integration of social media data and real-time commentary cannot instantly provide viewers with the information they most desire at any given moment. This has led to issues such as insufficient viewer satisfaction.

[1663] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for a user to log in to the service by entering authentication information on an available terminal, a means for providing video streaming of the game, a means for overlaying generated real-time commentary and SNS posts on the game video, a means for automatically generating and providing a highlight video based on the user's viewing history and preferences, a means for analyzing the user's facial expressions and voice in real time to recognize emotions, and a means for adjusting content to optimize the viewing experience based on the recognized emotions. This makes it possible to provide an optimal viewing experience that promptly responds to the emotions and preferences of each viewer.

[1664] "Authentication information" refers to personal identification information such as a username and password that a user enters when accessing a service.

[1665] "Game video streaming" is a technology that distributes video of a specific sports game in real time over the Internet.

[1666] "Real-time commentary" refers to commentary information that is generated instantly as the match progresses, explaining important events of the match to the viewer.

[1667] "SNS posts" are messages, images, videos, and other content posted by users on social networking services.

[1668] "Viewing history" is a record of content that a user has viewed in the past.

[1669] A "highlight video" is a shortened video that is edited to include only the important scenes and events of a game.

[1670] "Means of recognizing emotions by analyzing facial expressions and voice in real time" refers to technology that uses a camera and microphone to analyze the user's face and voice and grasp their emotions at that time (joy, surprise, concentration, etc.).

[1671] "Means to adjust content to optimize the viewing experience" refers to technology that changes and adjusts displayed content (comments, social media posts, etc.) in real time based on the user's emotions and preferences.

[1672] This invention relates to a sports event viewing system that optimizes the viewing experience for users. In particular, it provides a new service that goes beyond conventional viewing experiences by recognizing emotions in real time and adjusting viewing content according to those emotions.

[1673] System Overview and Configuration

[1674] 1. Authentication and Login

[1675] Users log in to the service by entering their authentication information (username and password) on their smartphone or other internet-enabled device, which is then sent to the server, which checks the authentication information against a database.

[1676] 2. Game Streaming

[1677] The server provides video streaming of the selected sports game and sends the streaming URL to the user's available device, which uses the URL to display the live video of the game.

[1678] 3. Real-time commentary and social media posts

[1679] The server collects relevant social media posts as the match progresses, and AI analyzes them to select the most relevant posts, which are then overlaid on the match footage.

[1680] Additionally, AI detects important events during the game (such as runs scored or home runs) and generates commentary in real time, which is then overlaid on the game footage.

[1681] 4. Automatic generation and provision of highlight videos

[1682] After the game ends, a highlight video is automatically generated based on the user's viewing history and preferences. The server generates this video in response to the user's request and sends it to the user's device, allowing the user to watch a personalized highlight video.

[1683] 5. Emotion Recognition and Viewing Experience Optimization

[1684] The user's device uses a built-in camera and microphone to capture the user's facial expressions and voice in real time, and this data is sent to a server where an emotion engine analyzes it to recognize the user's emotions.

[1685] Based on the recognized emotion data, the server adjusts the game commentary and displayed social media posts, for example, providing more detailed commentary if the user is surprised, and prioritizing positive social media posts if the user is enjoying the game.

[1686] Hardware and software used

[1687] Hardware: Smartphone, Head-Mounted Display (HMD), Camera, Microphone

[1688] Software: OpenCV (video capture and display), Requests (HTTP communication)

[1689] In addition, the emotion engine, SNS API, and AI model run on a server or cloud environment.

[1690] Specific examples

[1691] The user launches the "next-generation sports viewing app" on their smartphone and enters their login information. After successful authentication by the server, the user begins watching the match footage of "Team A vs. Team B." During the match, the smartphone's camera and microphone continuously capture the user's facial expressions and voice. If the user's surprised expression is detected, the server's emotion engine analyzes this and generates and displays detailed commentary in real time. Related positive social media posts are also overlaid on the screen. After the match, the user requests a highlight video, which the server automatically generates and sends to the user's smartphone. The video is then edited to include the most exciting scenes based on past viewing and emotion data.

[1692] Prompt Sentence Examples

[1693] Implement a system that analyzes emotions from users' facial expressions and voice in real time to optimize the viewing experience of sporting events. Program a next-generation sports viewing app that displays social media posts and provides commentary on key moments of a match based on the recognized emotions. For example, provide more detailed commentary if the user is surprised.

[1694] This system allows for an individually optimized viewing experience for sporting events, providing viewers with the real-time information and commentary they most desire at any given moment.

[1695] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1696] Step 1:

[1697] The user enters authentication information (username and password) on the device to log in. The entered authentication information is sent from the device to the server, which checks it against a database. If authentication is successful, the server generates a session token and returns it to the device. The user can use this token to access other functions within the service.

[1698] Step 2:

[1699] The user selects the match they want to watch. The user sends a request including the match ID from their device to the server. The server collects the live streaming URL of the match and related social media data and sends it to the device. The device starts displaying the live video of the match using the received URL.

[1700] Step 3:

[1701] While the game video is being streamed, the server collects related social media posts in real time. AI analyzes the data collected via the social media API and selects the most relevant posts. The selected social media posts are sent to the device and displayed over the game video, allowing users to view real-time social media reactions as the game progresses.

[1702] Step 4:

[1703] The server analyzes game footage in real time and detects important events (scores, home runs, etc.). Based on this event data, AI automatically generates commentary and sends it to the device in real time. The device then displays the generated commentary, allowing the user to visually recognize important moments.

[1704] Step 5:

[1705] While watching a game, the device uses a built-in camera and microphone to capture the user's facial expressions and voice. This data is sent to a server in real time. An emotion engine on the server analyzes the data and recognizes the user's emotions. For example, if the user is surprised, this information is reflected in the level of detail in the commentary and the selection of social media posts displayed.

[1706] Step 6:

[1707] After the game ends, the user requests the generation of a highlight video. The request, including the game ID, is sent from the device to the server, and the server automatically generates a highlight video based on the game record. This video is customized based on the user's viewing history and emotional data and sent to the device. The device plays the generated highlight video, allowing the user to enjoy a personally optimized summary of the game.

[1708] Step 7:

[1709] The server stores user emotional data and v...

Claims

1. A means for a user to log in to the service by entering authentication information on an available device; a means for providing video streaming of the match; A means to overlay generated real-time commentary and social media posts onto game footage; A means for automatically generating and providing a highlight video based on a user's viewing history and preferences; A system including:

2. The system of claim 1 , further comprising means for collecting social media posts in real time during video streaming of the game, analyzing the posts, and selecting those with high relevance.

3. 10. The system of claim 1, further comprising means for analyzing game footage in real time, detecting important and specific events, and automatically generating commentary based on the events.

Images