System

Abstract

A system is provided.SOLUTION: Means for selecting a sports event for the user to spectate, means for inputting spectating settings of the user, means for sending the inputted settings information to the server, means for the server to generate a user-specific spectating profile based on the received settings information, means for the server to collect relevant data and generate a customized commentary scenario, and the terminal displaying the live streaming of the match; A system, comprising: means for providing a customized commentary in real time; means for a user to input a question during spectating; means for a server to parse the received question and generate and send the commentary to a terminal in real time; and means for the terminal to display the commentary.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional sports viewing has faced the challenge of providing a personalized experience that matches the spectator's level of knowledge and interest. Spectators who lack knowledge of technical terms and the background of the game often find it difficult to fully enjoy the game. Another issue is the lack of technology to provide appropriate commentary in response to spectator questions in real time. By solving these challenges, we aim to provide a viewing environment in which more spectators can deeply understand and enjoy the appeal of sports. [Means for solving the problem]

[0005] The present invention provides a means for a user to select a sporting event to watch and input viewing settings. The input setting information is sent to a server, which then generates a user-specific viewing profile based on the received information. The server collects relevant data based on the viewing profile and generates a customized commentary scenario. The terminal also displays live streaming of the game and provides customized commentary in real time. The present invention also provides a means for the user to input questions while watching the game. The server analyzes the questions, generates real-time commentary, and sends it to the terminal. The terminal then displays this commentary, providing the user with a personalized and interactive viewing experience. This provides a system that allows anyone, regardless of their level of knowledge or interest, to deeply enjoy watching sports.

[0006] A "user" refers to a person who watches a sporting event and uses the system of the present invention to set up viewing, watch live streaming footage, and ask questions in real time.

[0007] "Watching settings" are setting items that allow the user to specify their preferred watching style, focus on a particular player, the level and content of commentary, and the like.

[0008] The "server" is a computer network system that generates a viewing profile based on viewing setting information received from a user, collects related data, and generates a customized commentary scenario.

[0009] A "watching profile" is a data structure that includes user-specific watching setting information that is generated based on the user's watching history, preferred commentary style, feedback data, and the like.

[0010] A "customized commentary scenario" is a user-specific commentary content generated by the server, which includes commentary and explanations tailored to a specific match, player, or situation.

[0011] A "terminal" is a user-operated device, such as a smartphone, tablet, or computer, that displays live streaming footage of the game and provides customized commentary.

[0012] "Live streaming" refers to the video and audio of a game being distributed in real time over a network, and is used by users to watch the game.

[0013] "Real-time commentary" refers to commentary content that is generated in response to user questions immediately, and is information that is provided as the match progresses.

[0014] "Natural language processing" refers to algorithms and techniques used to analyze user questions and understand their intent and interests.

[0015] "Related data" refers to data searched by the server, and includes past match data, player data, explanations of technical terms, and the like. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system of the present invention is designed to allow users watching sporting events to have a viewing experience that is tailored to their preferences. The system operates by allowing users to select a sporting event to watch, input viewing preferences, and transmit the input preferences to a server.

[0038] Overall system configuration

[0039] The system of the present invention consists of the following main components:

[0040] 1. User Device

[0041] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[0042] Enter your viewing preferences, watch live streams, and enter questions.

[0043] 2. Server

[0044] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0045] 3. Communication Network

[0046] Infrastructure for realizing data communication between user terminals and servers.

[0047] Program processing flow

[0048] The user selects the event to watch and enters the settings.

[0049] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[0050] The server processes the user's settings and generates a viewing profile.

[0051] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0052] The server collects relevant data and generates a customized commentary scenario.

[0053] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[0054] The device displays live streaming of the match and provides customized commentary in real time.

[0055] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0056] Specific examples

[0057] 1. The user sets up the viewing settings

[0058] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[0059] 2. The server processes the configuration information

[0060] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[0061] 3. The device provides live streaming

[0062] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[0063] 4. Users ask questions in real time

[0064] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[0065] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[0066] As a result, the system of the present invention can provide a personalized and interactive sports viewing experience for users.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[0070] Step 2:

[0071] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[0072] Step 3:

[0073] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[0074] Step 4:

[0075] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[0076] Step 5:

[0077] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or adding more explanations of the basic rules. Once the input is complete, the device sends this setting information to the server.

[0078] Step 6:

[0079] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[0080] Step 7:

[0081] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[0082] Step 8:

[0083] The server generates a customized commentary scenario based on the collected data and according to the user's settings.

[0084] Step 9:

[0085] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[0086] Step 10:

[0087] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[0088] Step 11:

[0089] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[0090] Step 12:

[0091] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[0092] Step 13:

[0093] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[0094] Step 14:

[0095] The server generates an appropriate explanation and sends it to the device. Based on the search results, explanatory text, video, and images are generated and sent to the device.

[0096] Step 15:

[0097] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[0098] This allows users to enjoy a real-time, personalized sports viewing experience.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] Conventional sports viewing systems have struggled to provide personalized commentary and information tailored to individual users' preferences and interests. Furthermore, they were unable to quickly provide real-time questions and answers, resulting in limited information available to users and reduced satisfaction with the viewing experience. There is a need to resolve these issues and provide users with a more interactive and personalized viewing experience.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the information processing device includes means for receiving and analyzing spectator setting information, means for generating a spectator profile, means for collecting related information and generating a customized commentary scenario, and means for analyzing a user's questions and generating commentary in real time. This allows a user to receive personalized commentary in real time according to their preferences, and can obtain quick and accurate commentary when they ask a question while watching a game.

[0104] "User" refers to an individual who uses the system to watch a competitive event.

[0105] "Competitive events" refer to various sporting events and matches that users can watch.

[0106] "Sports viewing preferences" are the viewing preferences and wishes that a user inputs into the system.

[0107] An "information processing device" is a device that receives and analyzes user setting information and generates the necessary profiles and data, and refers to a computer system such as a server.

[0108] A "watching profile" is a customized data set generated based on a user's unique viewing history, preferences, feedback data, etc.

[0109] A "display device" is a device used to display video streaming of competitions and commentary information to users, and includes smartphones, tablets, computers, etc.

[0110] "Video streaming" refers to a service that provides users with real-time video footage of competitive events.

[0111] "Customized commentary" is individual commentary information generated based on a user's unique viewing profile.

[0112] A "question" refers to a knowledge-based question or request for additional information that a user enters while watching a game.

[0113] "Natural language processing" refers to a technology that enables an information processing device to analyze questions received from users in human language and generate appropriate explanations.

[0114] "Related information" refers to any data collected based on users' questions and viewing profiles, such as player information, past match data, and explanations of technical terms.

[0115] The present invention is a system that provides commentary tailored to individual preferences when a user watches a sports event, enriching the viewing experience. This system functions by allowing the user to select the sports event to watch, input viewing settings, and transmit these to an information processing device (server). Specific hardware and software layouts, as well as data processing and data calculation methods, are described below.

[0116] System configuration

[0117] Hardware

[0118] 1. User Device

[0119] The device the user operates (smartphone, tablet, computer).

[0120] The device is used to watch live streaming, enter viewing settings, and enter questions.

[0121] 2. Server (information processing device)

[0122] High-performance computer cloud systems (e.g., AWS, Google Cloud Platform, Microsoft Azure).

[0123] The server receives and analyzes the viewing setting information, generates a viewing profile, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0124] 3. Communication Network

[0125] Infrastructure (Wi-Fi, 4G / 5G) for enabling data communication between user devices and servers.

[0126] software

[0127] 1. Live Streaming Applications

[0128] Video streaming services (e.g. YouTube Live, Twitch).

[0129] 2. Generative AI Models

[0130] Generative AI using natural language processing techniques (e.g., OpenAI GPT-3, GPT-4).

[0131] Specific operation of the system

[0132] 1. Enter your spectator settings

[0133] Users log in to the application from their own device and select the sporting event they want to watch. They then enter their viewing preferences, such as "focus on a specific player," "add basic rule explanations," and "display player interviews." This preference information is sent from the device to the server.

[0134] 2. Analyzing configuration information and generating a profile

[0135] The server receives and analyzes the viewing settings information sent from the device. Based on the analyzed information, a viewing profile unique to the user is generated. This profile includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[0136] 3. Generate customized explanations

[0137] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, terminology explanations, etc.) and then uses a generative AI model (such as GPT-3 or GPT-4) to generate a customized commentary scenario specific to the user.

[0138] 4. Live streaming and commentary

[0139] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0140] 5. Responding to user inquiries

[0141] While watching a game, users can input questions in real time through the application. For example, if a question is input such as "I want to know the background of Player X's scoring ability," the device will send this to the server. The server will analyze the question and generate an explanation for the question using a generative AI model. The explanation will be sent to the device, where the user can view it in real time.

[0142] Prompt Sentence Examples

[0143] "Generate a commentary scenario for player X in the next match, taking into account the user's viewing history and preferred commentary style."

[0144] "Please provide detailed explanations behind Player X's scoring ability in response to user questions."

[0145] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[0146] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0147] Step 1:

[0148] The user selects the athletic event they want to watch and enters their viewing preferences.

[0149] Input: A user logs in to the application, selects the athletic event to watch, and enters viewing preferences such as "focus on a specific player," "add basic rules explanation," and "show player interviews."

[0150] Data processing: The device formats the entered viewing settings and converts them into data for transmission.

[0151] Output: Spectator settings are formatted and sent to the server.

[0152] Specific operations: The user selects the desired competition event from the application's event selection screen, enters the viewing settings on the settings screen, and presses the send button.

[0153] Step 2:

[0154] The server receives and analyzes the spectator setting information.

[0155] Input: Spectator settings information sent from the device

[0156] Data processing: The server analyzes the viewing setting information using an analysis engine to extract the user's wishes and preferences.

[0157] Output: Information converted into a data structure for production based on user preferences.

[0158] Specific operation: The server analyzes the received data, extracts information such as the selected player and commentary type, and records it in an internal database.

[0159] Step 3:

[0160] The server generates a spectator profile

[0161] Input: Analyzed viewing preferences, user's past viewing history, preferred commentary style, and feedback data

[0162] Data processing: The server runs an algorithm based on these inputs to generate a unique viewing profile for the user.

[0163] Output: User-specific viewing profile

[0164] How it works: The server compares the viewing settings with past viewing history and generates a unique profile for the user, which reflects the user's preferences and tendencies.

[0165] Step 4:

[0166] The server collects relevant data and generates a customized commentary scenario.

[0167] Input: User-specific viewing profile, player information, past performance data, interview information

[0168] Data processing: The server collects relevant data based on the profile and generates explanatory scenarios using generative AI models (e.g., GPT-3 or GPT-4).

[0169] Output: A customized expository scenario

[0170] Specific operation: The server collects player information and past match data, and inputs this along with the player's profile into the AI, which then generates commentary based on this information.

[0171] Step 5:

[0172] The device displays live streaming and provides customized commentary in real time.

[0173] Input: Live streaming URL and commentary scenario sent from the server

[0174] Data processing: The device uses the received streaming URL to play the live video and simultaneously display the commentary scenario.

[0175] Output: A screen displaying live streaming footage and customizable commentary

[0176] How it works: When a user starts live streaming in the application, real-time commentary along a timeline is displayed along with the video.

[0177] Step 6:

[0178] Users ask questions in real time

[0179] Input: The user enters text into the application's question input field and presses the submit button.

[0180] Data processing: The terminal formats the entered question and converts it into data to be sent to the server.

[0181] Output: The user's question is formatted and sent to the server.

[0182] Specific operation: The user enters a question and presses the send button to send the question to the server.

[0183] Step 7:

[0184] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[0185] Input: User-submitted question

[0186] Data processing: The server analyzes the question using natural language processing technology and generates an appropriate explanation using a generative AI model.

[0187] Output: Generated commentary

[0188] Specific operation: The server processes the question content using an analysis engine, generates an explanation using an AI model, and sends it to the device.

[0189] Step 8:

[0190] The device displays the explanation.

[0191] Input: Description sent from the server

[0192] Data processing: The device converts the received commentary into a format that can be displayed on the screen.

[0193] Output: Explanation displayed on the terminal screen

[0194] Specific behavior: The explanation is displayed properly in the user's field of view, and the user can get the answer in real time.

[0195] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[0196] (Application example 1)

[0197] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0198] In modern sports viewing, it is difficult for users to get a personalized experience that meets their preferences. In particular, there is a lack of real-time customized commentary and immediate responses to user questions. This makes it difficult for users to enjoy a satisfying viewing experience.

[0199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0200] In this invention, the server includes a means for generating a user-specific viewing profile, a means for collecting related data and generating a customized commentary scenario, and a means for automatically generating commentary using a generative AI model and providing appropriate commentary based on the related data, thereby enabling users to receive personalized commentary in real time and instantly respond to questions while watching sports.

[0201] A "user-specific viewing profile" is a personalized profile generated based on the viewing settings, past viewing history, preferred commentary style, and feedback data set by the user.

[0202] A "customized commentary scenario" is a scenario that includes commentary and additional information focused on specific players or plays, generated based on a user's unique viewing profile.

[0203] A "generative AI model" is a mathematical model that uses artificial intelligence technology to automatically analyze data and generate personalized explanations and answers.

[0204] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to accurately analyze user questions and provide appropriate explanations.

[0205] "Real-time commentary" refers to commentary information that is generated and provided instantly as the game progresses, making the user's viewing experience more interesting.

[0206] The present invention relates to a system that enables users to watch sporting events in a personalized manner. The specific configuration and operation of this system will be described below.

[0207] System configuration

[0208] The system consists of the following main components:

[0209] 1. User Device:

[0210] This refers to devices such as smartphones and head-mounted displays operated by users.

[0211] Enter your viewing preferences, watch live streams, and enter questions.

[0212] 2. Server:

[0213] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0214] 3. Communication Network:

[0215] Refers to the network infrastructure that enables data communication between user terminals and servers.

[0216] Data processing and calculation

[0217] The server processes the data in the following steps:

[0218] 1. Receiving and analyzing spectator settings information:

[0219] The system receives and analyzes the viewing settings information sent from the user's device, taking into account the user's preferences, past viewing history, and feedback data.

[0220] 2. Create a viewing profile:

[0221] Based on the received information, a unique viewing profile for the user is generated, which includes the user's settings and past viewing data.

[0222] 3. Collection of relevant data:

[0223] Relevant data such as player information, past performance data, and explanations of technical terms are collected from the database. This collection is done using a database management system (RDBMS).

[0224] 4. Generate customized commentary scenarios:

[0225] Based on the collected data, a commentary scenario specific to the user is generated, using natural language processing (NLP) technology and generative AI models.

[0226] 5. Providing live streaming and commentary:

[0227] Once the match begins, the user's device displays live streaming video and displays customized commentary received from the server in real time.

[0228] 6. User Questions:

[0229] When a user types a question while watching a game, the question is sent to the server, which analyzes the question using natural language processing technology and uses a generative AI model to search for relevant data and generate an appropriate commentary. The commentary is then displayed on the user's device.

[0230] Specific examples

[0231] During a game, a user inputs a question such as, "I want to know the background of Player X's scoring ability." This question is analyzed by the server, and an explanation based on Player X's scoring performance data and past interview information is generated and provided to the user in real time.

[0232] Example prompt sentence:

[0233] Question: I want to know the story behind Player X's scoring ability.

[0234] Relevant information: Player X's past scoring data, snippets of interviews.

[0235] Explanation: Player X has averaged 20 points per game over the past three seasons, thanks to changes in his training methods and close collaboration with a mental health coach.

[0236] As a result, the present invention can provide users with an interactive and personalized sports viewing experience.

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1:

[0239] The user selects the sporting event they want to watch and enters their viewing settings. A list of sporting events to watch is displayed on the user's device. The user selects an event from the list and then enters viewing settings (e.g., to focus on a specific player, or to add an explanation of the basic rules). This input data is packaged in JSON format and sent to the server. The input is the user's selection and settings information, and the output is the packaged settings information.

[0240] Step 2:

[0241] The server analyzes the received viewing setting information and generates a viewing profile specific to the user. The server analyzes the received JSON data and combines the user's past viewing history, preferred commentary style, and feedback data to generate a viewing profile. This profile is used for all subsequent customization. The input is user setting information and the output is the viewing profile.

[0242] Step 3:

[0243] The server collects relevant data and generates a customized commentary scenario. Using generative AI models and natural language processing (NLP) technology, it collects player information, past performance data, explanations of technical terms, and other information to generate a commentary scenario specific to the user. The collected data is combined with the viewing profile to create a customized commentary. The input is the viewing profile and related data, and the output is a customized commentary scenario.

[0244] Step 4:

[0245] When the match begins, the device displays live streaming video and provides customized commentary in real time. The user device displays a commentary scenario retrieved from the server while watching. The commentary is dynamically updated as the match progresses, so the device periodically retrieves new commentary data from the server. The input is the live streaming and commentary scenario, and the output is the video and commentary displayed to the user.

[0246] Step 5:

[0247] The user inputs a question while watching the game. The question is sent from the user's terminal to the server. As the game progresses, the user inputs questions about their doubts or requests for additional explanation. These questions are transferred to the server in text format. The input is the user's question, and the output is the transmission of the question data to the server.

[0248] Step 6:

[0249] The server analyzes the received question, generates an explanation in real time, and sends it to the device. The server uses natural language processing technology to analyze the received question and input a prompt sentence into the generative AI model. An answer is generated based on related data and sent in text format to the user's device. The input is the user question and the prompt sentence from the generative AI model, and the output is an explanatory text.

[0250] Step 7:

[0251] The terminal displays the explanation. The explanatory text received from the server is visually displayed on the user terminal. This allows the user to obtain a customized answer in real time. The input is the explanatory text, and the output is the explanation displayed to the user.

[0252] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0253] The system of the present invention is designed to allow users to enjoy a personalized sporting event. It features an emotion engine that recognizes a user's emotional state and provides adaptive commentary in real time. The system works by allowing a user to select a sporting event to watch, input viewing preferences, and transmit them to a server. The emotion engine then analyzes the user's emotional state and dynamically adjusts the commentary based on that data.

[0254] Overall system configuration

[0255] The system of the present invention consists of the following main components:

[0256] 1. User Device

[0257] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[0258] Enter viewing preferences, watch live streams, enter questions, and collect sentiment data.

[0259] It is equipped with an emotion engine that recognizes the user's emotional state in real time.

[0260] 2. Server

[0261] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0262] Dynamically adjust commentary content based on data from the emotion engine.

[0263] 3. Communication Network

[0264] Infrastructure for realizing data communication between user terminals and servers.

[0265] Program processing flow

[0266] The user selects the event to watch and enters the settings.

[0267] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[0268] The server processes the user's settings and generates a viewing profile.

[0269] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0270] The server collects relevant data and generates a customized commentary scenario.

[0271] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[0272] The device displays live streaming of the match and provides customized commentary in real time.

[0273] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0274] Recognizing the user's emotional state

[0275] The device is equipped with an emotion engine that analyzes the user's facial expressions, voice, and operation patterns in real time. Once the user's emotional state is recognized, the data is sent to a server.

[0276] The server adjusts the commentary based on the emotional data.

[0277] The server dynamically adjusts the commentary scenario based on the data received from the emotion engine according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical explanations.

[0278] Specific examples

[0279] 1. The user sets up the viewing settings

[0280] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[0281] 2. The server processes the configuration information

[0282] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[0283] 3. The device provides live streaming

[0284] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[0285] 4. Users ask questions in real time

[0286] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[0287] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[0288] 5. Emotion engine recognizes the user's emotional state

[0289] The emotion engine analyzes the user's facial expression data, voice data, and operation patterns to recognize the user's emotional state, such as surprise, joy, anxiety, etc. The recognition results are sent to the server.

[0290] 6. Server dynamically adjusts description

[0291] The server then analyzes the user's emotional state based on the received emotion data and dynamically adjusts the commentary accordingly. For example, if the user shows a surprised expression, the server can add a detailed tactical analysis to the commentary.

[0292] As a result, the system of the present invention can provide a personalized, interactive, and emotionally relevant sports viewing experience for the user.

[0293] The processing flow will be explained below.

[0294] Step 1:

[0295] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[0296] Step 2:

[0297] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[0298] Step 3:

[0299] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[0300] Step 4:

[0301] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[0302] Step 5:

[0303] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or increasing explanations of basic rules. Once these settings are complete, the device sends the setting information to the server.

[0304] Step 6:

[0305] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[0306] Step 7:

[0307] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[0308] Step 8:

[0309] The server generates a customized commentary scenario. The server creates a commentary scenario based on the collected data and in accordance with the user's settings.

[0310] Step 9:

[0311] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[0312] Step 10:

[0313] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[0314] Step 11:

[0315] The emotion engine recognizes the user's emotional state. The emotion engine installed in the device analyzes the user's facial expressions, voice, and operation patterns to recognize the user's emotional state in real time. The recognition results are sent to the server.

[0316] Step 12:

[0317] The server adjusts the commentary based on the emotion data. Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical commentary.

[0318] Step 13:

[0319] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[0320] Step 14:

[0321] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[0322] Step 15:

[0323] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[0324] Step 16:

[0325] The server generates an appropriate explanation and sends it to the device.The server generates explanatory text, videos, and images based on the search results and sends them to the device.

[0326] Step 17:

[0327] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[0328] This allows users to enjoy a personalized sports viewing experience in real time, and by incorporating emotion recognition functionality using an emotion engine, it is possible to provide even more interactive commentary that responds to the user's emotions.

[0329] Example 2

[0330] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0331] Existing sports viewing systems have difficulty providing real-time commentary that reflects individual user preferences and emotional states. Furthermore, few systems can respond quickly and appropriately to user questions. Therefore, providing users with an interactive and personalized viewing experience is a challenge.

[0332] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0333] In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's viewing preferences; means for transmitting the input preference information to the server; means for the server to generate a user-specific viewing profile based on the received preference information; means for the server to collect relevant data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state; means for dynamically adjusting the commentary scenario based on the emotion data; and means for the terminal to display the commentary, thereby enabling an interactive, emotionally-sensitive, and personalized sports viewing experience for the user.

[0334] "User" refers to an individual who uses the system to watch a sporting event.

[0335] "Spectator Settings" refers to information that allows a user to input a particular commentary style or focus point desired when watching a sporting event.

[0336] "Terminal" refers to a device operated by a user (e.g., a smartphone, tablet, computer, etc.).

[0337] "Server" refers to a central processing unit that receives viewing setting information and emotional data from users and generates viewing profiles and commentary scenarios based on that information.

[0338] A "watching profile" refers to user-specific data including the user's past viewing history, preferred commentary style, feedback, etc.

[0339] "Explanatory scenario" refers to a set of customized expository content generated by the server.

[0340] "Live streaming" refers to game footage that is broadcast in real time over the Internet.

[0341] "Emotion engine" refers to software or hardware that has the functionality to recognize the emotional state of a user.

[0342] "Emotion data" refers to data that represents the user's emotional state (for example, surprise, joy, anxiety, etc.).

[0343] "Question" refers to the question or information that a user wants to know that they input into the system while watching a game.

[0344] MODE FOR CARRYING OUT THE INVENTION

[0345] The system of the present invention is designed to personalize the viewing of sporting events and provide users with a richer viewing experience in real time. The main components of the system are a user terminal, a server, and a communication network.

[0346] The user terminals include devices such as smartphones, tablets, and computers, and are used to input viewing settings, watch live streaming, input questions, and collect emotional data. The terminals are also equipped with an emotion engine that recognizes emotions in real time through analysis of the user's facial expressions and voice. This emotional data is sent to a server. Specifically, facial expression recognition software and a voice analysis engine may be used.

[0347] The server receives the viewing setting information sent by the user and generates a viewing profile specific to the user based on that information. The profile includes the user's past viewing history, preferred commentary style, feedback, etc. The server also collects related data from the Internet (player information, past performance data, interview information, etc.) and generates a customized commentary scenario based on that data. When the server receives a question from the user, it analyzes the question using a natural language processing engine, generates appropriate commentary, and sends it to the user's device. Specifically, a natural language processing library and database are used.

[0348] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time. This commentary is dynamically updated as the match progresses and adjusts based on the user's excitement level and other emotional states. For example, if the user expresses surprise, the server can add more detailed tactical analysis to the commentary. The device can use existing live streaming services such as YouTube and Dailymotion.

[0349] Specific examples

[0350] 1. The user sets up the viewing settings

[0351] A user logs in to the system on their smartphone and selects a soccer match. The user enters settings such as "Focus on Player X," "More explanations of the basic rules," and "Show player interviews," and sends the settings information to the server.

[0352] 2. The server generates a profile

[0353] The server analyzes the received viewing setting information and generates a viewing profile based on the user's past viewing history and preferred commentary style. For example, the server collects information about player X and prepares a commentary scenario to be displayed during the match.

[0354] 3. The device displays the live stream.

[0355] The device uses services such as YouTube to display live streaming of the game, and simultaneously displays commentary received from the server. When Player X has the ball, information such as "Player X's special technique" is added to the commentary in real time.

[0356] 4. The user enters a question

[0357] During a game, a user types a question into their device, such as "I want to know about player X's scoring ability." The server analyzes the question using natural language processing, searches for relevant data, and generates an explanation. The explanation is then sent to the device and displayed to the user.

[0358] 5. Emotion engine recognizes the user's emotional state

[0359] The emotion engine analyzes the user's facial expressions and voice to recognize, for example, whether the user is happy, and sends that data to the server.

[0360] 6. The server adjusts the commentary based on the emotion data.

[0361] The server analyzes the emotional data and, for example, if the user is excited, adds details to the commentary such as "These are the tactics behind this play."

[0362] Prompt Sentence Examples

[0363] "Please explain in detail the processing steps of this sports viewing system program. Please break down the processing steps into smaller steps and describe each step, including the specific actions that are taken."

[0364] As a result, this system can provide users with an interactive, emotionally-driven, and personalized sports viewing experience.

[0365] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0366] Step 1:

[0367] The user selects the event to watch and enters the settings.

[0368] Input: A user operates a terminal to log in to the system, selects a sporting event to watch, and then inputs viewing settings (e.g., "Focus on player X," "More explanations of the basic rules," "Show player interviews").

[0369] Data processing: The device compiles the entered viewing settings information.

[0370] Output: Spectator settings information is sent from the device to the server.

[0371] Specific actions: The user uses their smartphone to enter viewing information into the designated fields in the app and presses the "Settings complete" button.

[0372] Step 2:

[0373] The server processes the user's settings and generates a viewing profile.

[0374] Input: Spectator settings information sent from the device.

[0375] Data processing: The server analyzes the received configuration information and updates the user's unique viewing profile, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0376] Output: The updated spectator profile.

[0377] Specific operation: The server generates a new viewing profile based on the user's viewing history and feedback information stored in the database.

[0378] Step 3:

[0379] The server collects relevant data and generates a customized commentary scenario.

[0380] Input: The user's watching profile.

[0381] Data processing: The server collects relevant data from the Internet (player information, past performance data, interview information, etc.) and generates customized commentary scenarios based on the viewing profile.

[0382] Output: A customized commentary scenario.

[0383] Specific operation: The server uses a Web API to obtain the latest player information and past match data, and inputs this information into the commentary scenario generation algorithm to generate a scenario.

[0384] Step 4:

[0385] The device will display live streaming of the match and provide customized commentary

[0386] Input: A customized commentary script received from the server, and live streaming footage.

[0387] Data processing: The device displays live streaming video while overlaying explanatory information sent from the server at any time.

[0388] Output: Live streaming video and real-time commentary.

[0389] Specific operation: The device uses the live streaming APIs of YouTube and Dailymotion to play game footage while displaying commentary text on the screen in real time.

[0390] Step 5:

[0391] Recognizing the user's emotional state

[0392] Input: User's facial expressions, voice, and operation patterns.

[0393] Data processing: The emotion engine installed in the device analyzes this data and recognizes the user's emotional state.

[0394] Output: Emotion data (e.g. surprise, joy, anxiety, etc.).

[0395] Specific operation: The user's facial expressions and voice are captured in real time through the device's camera and microphone, and emotions are analyzed using facial recognition software and a voice analysis engine.

[0396] Step 6:

[0397] The server adjusts the commentary based on the emotional data.

[0398] Input: User emotional state data.

[0399] Data processing: The server dynamically adjusts the commentary scenario based on the received emotional data.

[0400] Output: A tailored expository scenario.

[0401] Specific actions: For example, if a user is surprised, the server will add a detailed analysis to the commentary, such as "These are the tactics behind this play."

[0402] Step 7:

[0403] Users can enter questions while watching

[0404] Input: User question (e.g., "What is Player X's scoring stats?").

[0405] Data processing: The device sends the query to the server.

[0406] Output: The question sent to the server.

[0407] Specific actions: The user enters a question into the input field on the terminal and presses the send button.

[0408] Step 8:

[0409] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[0410] Input: The user's question.

[0411] Data processing: The server uses a natural language processing engine to analyze the question, search for relevant data, and generate an appropriate explanation.

[0412] Output: The generated commentary.

[0413] Specific operation: The server analyzes the user's question, extracts relevant information from the database, generates an explanation, and sends it to the terminal.

[0414] (Application example 2)

[0415] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] Current sports viewing applications lack the ability to personalize commentary based on users' emotions and preferences, resulting in a uniform viewing experience. Furthermore, there is a lack of technology that can recognize users' emotional state in real time and dynamically adjust commentary accordingly. Therefore, there is a need for a more interactive experience.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's watching settings; means for transmitting the input setting information to the server; means for the server to generate a user-specific watching profile based on the setting information received; means for the server to collect related data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the user's emotional state; means for the terminal to collect user emotion data using an emotion engine; and means for the terminal to display commentary. This enables a personalized, interactive, and emotion-responsive sports watching experience for the user.

[0418] Key Word Definitions

[0419] The "means for the user to select a sporting event to watch" refers to a device or software that provides an interface for the user to select any sporting event from among a plurality of sporting events.

[0420] The "means for inputting user viewing settings" refers to a device or software that provides an interface for the user to input viewing settings such as a particular commentary style or focus points.

[0421] The "means for transmitting input setting information to a server" refers to a device or software that transmits the watching setting information input by the user to a server via a communication network such as the Internet.

[0422] "Means for generating a user-specific viewing profile based on the setting information received by the server" refers to a device or software that analyzes the viewing setting information received from the user by the server and generates a viewing profile based on the user's interests and preferences.

[0423] "Means for the server to collect relevant data and generate customized commentary scenarios" refers to devices or software that allow the server to collect relevant data such as player information and past performance data based on a user's unique viewing profile and use that data to generate customized commentary scenarios.

[0424] "Means for a terminal to display live streaming of a match and provide customized commentary in real time" refers to a device or software that allows a terminal to display live video of a match received via the Internet and also displays customized commentary received from a server in real time.

[0425] The "means for users to input questions while watching" refers to a device or software that provides an interface that allows users watching the game to input questions they have in real time.

[0426] "Means for analyzing questions received by the server, generating explanations in real time, and transmitting them to the terminal" refers to a device or software that uses natural language processing technology to analyze questions received from users by the server, generates appropriate explanations based on the analysis, and transmits them to the terminal in real time.

[0427] "Means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the emotional state" refers to a device or software in which the terminal analyzes the user's facial expressions, voice, etc., recognizes the user's emotional state, and then dynamically adjusts the content of the commentary according to that emotional state.

[0428] "Means for a terminal to collect user emotional data using an emotion engine" refers to devices or software that allow devices such as smartphones and tablets to collect user emotional data using algorithms and sensors called emotion engines.

[0429] The "means for the terminal to display the commentary" refers to a device or software that allows the terminal to visually or audibly display the commentary generated in real time to the user.

[0430] "Form for carrying out the invention" of the specification

[0431] The system of the present invention allows users to enjoy a personalized sporting event. The system features an emotion engine that recognizes the user's emotional state and provides adaptive commentary in real time. The specific configuration for realizing this system is shown below.

[0432] Hardware and software used

[0433] This system consists of the following hardware and software:

[0434] Hardware: User devices (smartphones, tablets, computers, head-mounted displays (HMDs), etc.)

[0435] Software: Flask (Python framework), OpenCV (image processing library), Dlib (face detection and landmark prediction library), natural language processing algorithm, emotion engine

[0436] Data processing and calculation

[0437] 1. Select the event you want to watch and enter your settings:

[0438] Users log in to the system using their terminals, select the sporting event they want to watch, and enter viewing preferences such as focus points and commentary style, which are then sent from the terminal to the server.

[0439] 2. The server processes the user's preferences and generates a viewing profile:

[0440] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[0441] 3. The server collects the relevant data and generates a customized commentary scenario:

[0442] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, explanations of technical terms, etc.) and generates a customized commentary scenario specific to the user.

[0443] 4. The device displays live streaming of the match and provides customized commentary in real time:

[0444] Once the match begins, the device displays live streaming footage and simultaneously displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0445] 5. Recognize the user's emotional state:

[0446] The emotion engine analyzes the user's facial expressions, voice, and operation patterns in real time. For example, if the user's emotional state, such as excitement or surprise, is detected, the data is sent to the server and used to adjust the commentary content.

[0447] 6. The server adjusts the commentary based on the emotion data:

[0448] Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is surprised, it can add detailed tactical analysis to the commentary.

[0449] Specific examples

[0450] Front-end: The user selects a basketball game and sets preferences such as "focus on player X," "show more explanation of the basic rules," and "display player interviews."

[0451] Backend: The server receives this configuration information and generates a customized commentary scenario based on Player X's profile data and related data.

[0452] Live Operation: During the match, the device provides live video of the match in real time and displays specialized commentary received from the server. The user's emotional state is analyzed and the commentary content is dynamically adjusted as needed.

[0453] Prompt Sentence Examples

[0454] "Tell me the story behind player X's scoring ability in a basketball game. The user is currently in an emotional state of surprise. Please provide commentary that is appropriate to this emotion."

[0455] This allows for personalized commentary based on the user's emotions, resulting in a more fulfilling viewing experience.

[0456] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0457] Specific processing flow of the program

[0458] Step 1:

[0459] A user selects a sporting event to watch and enters viewing preferences.

[0460] input:

[0461] An interface for users to select sporting events

[0462] Spectator settings (e.g., focus on specific players, commentary style, interview data to display)

[0463] output:

[0464] Entered sporting event information and viewing preferences

[0465] Specific operation:

[0466] Users log into the application on their smartphone or tablet and select the sporting event they want to watch.

[0467] Operates a form where users enter viewing preferences (player focus, commentary style, interviews, etc.).

[0468] This information is compiled into a single JSON object on the terminal side.

[0469] Step 2:

[0470] The entered setting information is sent to the server.

[0471] input:

[0472] Spectator settings information from the user (JSON object)

[0473] output:

[0474] Spectator settings sent to the server

[0475] Specific operation:

[0476] The device sends the user's viewing settings information to the server via the Internet using HTTPS as the communication protocol.

[0477] Step 3:

[0478] The server generates a user-specific game watching profile based on the received setting information.

[0479] input:

[0480] Spectator settings information

[0481] User's past viewing history, preferred commentary style, and feedback data

[0482] output:

[0483] User-specific viewing profiles

[0484] Specific operation:

[0485] The server analyzes the viewing setting information and updates the user's unique profile using a viewing profile generation algorithm.

[0486] A viewing profile includes a user's past viewing history, preferred commentary style, and feedback data.

[0487] Step 4:

[0488] The server collects the relevant data and generates a customized expository scenario.

[0489] input:

[0490] Spectator Profile

[0491] output:

[0492] Customized commentary scenarios

[0493] Specific operation:

[0494] The server collects relevant data such as player information, past performance data, interview information, and explanations of technical terms based on the user's viewing profile.

[0495] Based on the collected data, a customized commentary scenario specific to the user is generated.

[0496] Step 5:

[0497] The device displays live streaming of the match and provides customized commentary in real time.

[0498] input:

[0499] Live streaming video

[0500] Customized commentary scenarios

[0501] output:

[0502] Live streaming video and commentary scenario displayed to users

[0503] Specific operation:

[0504] Once the match begins, the device will display live streaming footage.

[0505] Customized commentary received from the server is displayed in real time as an overlay on the live video.

[0506] Step 6:

[0507] Recognize the user's emotional state.

[0508] input:

[0509] User facial expression data, voice data, operation patterns

[0510] output:

[0511] User emotional state data

[0512] Specific operation:

[0513] The device's built-in camera and microphone collect the user's facial expression and voice data.

[0514] The emotion engine analyzes this data in real time to recognize the user's emotional state.

[0515] Step 7:

[0516] The server adjusts the commentary based on the emotional data.

[0517] input:

[0518] User emotional state data

[0519] Customized commentary scenarios

[0520] output:

[0521] Updated commentary scenarios according to user emotions

[0522] Specific operation:

[0523] The server analyzes the emotional state data received from the emotion engine and dynamically adjusts the commentary scenario.

[0524] For example, if the user is excited, add tactical explanations, or if the user is surprised, provide a detailed explanation of external factors.

[0525] Step 8:

[0526] The device displays an explanation.

[0527] input:

[0528] Updated commentary scenario

[0529] output:

[0530] Updated explanatory scenarios displayed to users

[0531] Specific operation:

[0532] The terminal receives the updated commentary scenario sent from the server and displays it as an overlay on the live video.

[0533] This provides a personalized sports viewing experience that is tailored to the user's emotional state.

[0534] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0535] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0536] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0537] [Second embodiment]

[0538] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0539] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0540] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0541] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0542] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0543] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0544] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0545] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0546] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0547] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0548] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0549] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0550] The system of the present invention is designed to allow users watching sporting events to have a viewing experience that is tailored to their preferences. The system operates by allowing users to select a sporting event to watch, input viewing preferences, and transmit the input preferences to a server.

[0551] Overall system configuration

[0552] The system of the present invention consists of the following main components:

[0553] 1. User Device

[0554] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[0555] Enter your viewing preferences, watch live streams, and enter questions.

[0556] 2. Server

[0557] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0558] 3. Communication Network

[0559] Infrastructure for realizing data communication between user terminals and servers.

[0560] Program processing flow

[0561] The user selects the event to watch and enters the settings.

[0562] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[0563] The server processes the user's settings and generates a viewing profile.

[0564] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0565] The server collects relevant data and generates a customized commentary scenario.

[0566] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[0567] The device displays live streaming of the match and provides customized commentary in real time.

[0568] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0569] Specific examples

[0570] 1. The user sets up the viewing settings

[0571] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[0572] 2. The server processes the configuration information

[0573] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[0574] 3. The device provides live streaming

[0575] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[0576] 4. Users ask questions in real time

[0577] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[0578] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[0579] As a result, the system of the present invention can provide a personalized and interactive sports viewing experience for users.

[0580] The processing flow will be explained below.

[0581] Step 1:

[0582] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[0583] Step 2:

[0584] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[0585] Step 3:

[0586] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[0587] Step 4:

[0588] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[0589] Step 5:

[0590] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or adding more explanations of the basic rules. Once the input is complete, the device sends this setting information to the server.

[0591] Step 6:

[0592] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[0593] Step 7:

[0594] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[0595] Step 8:

[0596] The server generates a customized commentary scenario based on the collected data and according to the user's settings.

[0597] Step 9:

[0598] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[0599] Step 10:

[0600] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[0601] Step 11:

[0602] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[0603] Step 12:

[0604] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[0605] Step 13:

[0606] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[0607] Step 14:

[0608] The server generates an appropriate explanation and sends it to the device. Based on the search results, explanatory text, video, and images are generated and sent to the device.

[0609] Step 15:

[0610] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[0611] This allows users to enjoy a real-time, personalized sports viewing experience.

[0612] Example 1

[0613] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0614] Conventional sports viewing systems have struggled to provide personalized commentary and information tailored to individual users' preferences and interests. Furthermore, they were unable to quickly provide real-time questions and answers, resulting in limited information available to users and reduced satisfaction with the viewing experience. There is a need to resolve these issues and provide users with a more interactive and personalized viewing experience.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0616] In this invention, the information processing device includes means for receiving and analyzing spectator setting information, means for generating a spectator profile, means for collecting related information and generating a customized commentary scenario, and means for analyzing a user's questions and generating commentary in real time. This allows a user to receive personalized commentary in real time according to their preferences, and can obtain quick and accurate commentary when they ask a question while watching a game.

[0617] "User" refers to an individual who uses the system to watch a competitive event.

[0618] "Competitive events" refer to various sporting events and matches that users can watch.

[0619] "Sports viewing preferences" are the viewing preferences and wishes that a user inputs into the system.

[0620] An "information processing device" is a device that receives and analyzes user setting information and generates the necessary profiles and data, and refers to a computer system such as a server.

[0621] A "watching profile" is a customized data set generated based on a user's unique viewing history, preferences, feedback data, etc.

[0622] A "display device" is a device used to display video streaming of competitions and commentary information to users, and includes smartphones, tablets, computers, etc.

[0623] "Video streaming" refers to a service that provides users with real-time video footage of competitive events.

[0624] "Customized commentary" is individual commentary information generated based on a user's unique viewing profile.

[0625] A "question" refers to a knowledge-based question or request for additional information that a user enters while watching a game.

[0626] "Natural language processing" refers to a technology that enables an information processing device to analyze questions received from users in human language and generate appropriate explanations.

[0627] "Related information" refers to any data collected based on users' questions and viewing profiles, such as player information, past match data, and explanations of technical terms.

[0628] The present invention is a system that provides commentary tailored to individual preferences when a user watches a sports event, enriching the viewing experience. This system functions by allowing the user to select the sports event to watch, input viewing settings, and transmit these to an information processing device (server). Specific hardware and software layouts, as well as data processing and data calculation methods, are described below.

[0629] System configuration

[0630] Hardware

[0631] 1. User Device

[0632] The device the user operates (smartphone, tablet, computer).

[0633] The device is used to watch live streaming, enter viewing settings, and enter questions.

[0634] 2. Server (information processing device)

[0635] High-performance computer cloud systems (e.g., AWS, Google Cloud Platform, Microsoft Azure).

[0636] The server receives and analyzes the viewing setting information, generates a viewing profile, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0637] 3. Communication Network

[0638] Infrastructure (Wi-Fi, 4G / 5G) for enabling data communication between user devices and servers.

[0639] software

[0640] 1. Live Streaming Applications

[0641] Video streaming services (e.g. YouTube Live, Twitch).

[0642] 2. Generative AI Models

[0643] Generative AI using natural language processing techniques (e.g., OpenAI GPT-3, GPT-4).

[0644] Specific operation of the system

[0645] 1. Enter your spectator settings

[0646] Users log in to the application from their own device and select the sporting event they want to watch. They then enter their viewing preferences, such as "focus on a specific player," "add basic rule explanations," and "display player interviews." This preference information is sent from the device to the server.

[0647] 2. Analyzing configuration information and generating a profile

[0648] The server receives and analyzes the viewing settings information sent from the device. Based on the analyzed information, a viewing profile unique to the user is generated. This profile includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[0649] 3. Generate customized explanations

[0650] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, terminology explanations, etc.) and then uses a generative AI model (such as GPT-3 or GPT-4) to generate a customized commentary scenario specific to the user.

[0651] 4. Live streaming and commentary

[0652] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0653] 5. Responding to user inquiries

[0654] While watching a game, users can input questions in real time through the application. For example, if a question is input such as "I want to know the background of Player X's scoring ability," the device will send this to the server. The server will analyze the question and generate an explanation for the question using a generative AI model. The explanation will be sent to the device, where the user can view it in real time.

[0655] Prompt Sentence Examples

[0656] "Generate a commentary scenario for player X in the next match, taking into account the user's viewing history and preferred commentary style."

[0657] "Please provide detailed explanations behind Player X's scoring ability in response to user questions."

[0658] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[0659] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0660] Step 1:

[0661] The user selects the athletic event they want to watch and enters their viewing preferences.

[0662] Input: A user logs in to the application, selects the athletic event to watch, and enters viewing preferences such as "focus on a specific player," "add basic rules explanation," and "show player interviews."

[0663] Data processing: The device formats the entered viewing settings and converts them into data for transmission.

[0664] Output: Spectator settings are formatted and sent to the server.

[0665] Specific operations: The user selects the desired competition event from the application's event selection screen, enters the viewing settings on the settings screen, and presses the send button.

[0666] Step 2:

[0667] The server receives and analyzes the spectator setting information.

[0668] Input: Spectator settings information sent from the device

[0669] Data processing: The server analyzes the viewing setting information using an analysis engine to extract the user's wishes and preferences.

[0670] Output: Information converted into a data structure for production based on user preferences.

[0671] Specific operation: The server analyzes the received data, extracts information such as the selected player and commentary type, and records it in an internal database.

[0672] Step 3:

[0673] The server generates a spectator profile

[0674] Input: Analyzed viewing preferences, user's past viewing history, preferred commentary style, and feedback data

[0675] Data processing: The server runs an algorithm based on these inputs to generate a unique viewing profile for the user.

[0676] Output: User-specific viewing profile

[0677] How it works: The server compares the viewing settings with past viewing history and generates a unique profile for the user, which reflects the user's preferences and tendencies.

[0678] Step 4:

[0679] The server collects relevant data and generates a customized commentary scenario.

[0680] Input: User-specific viewing profile, player information, past performance data, interview information

[0681] Data processing: The server collects relevant data based on the profile and generates explanatory scenarios using generative AI models (e.g., GPT-3 or GPT-4).

[0682] Output: A customized expository scenario

[0683] Specific operation: The server collects player information and past match data, and inputs this along with the player's profile into the AI, which then generates commentary based on this information.

[0684] Step 5:

[0685] The device displays live streaming and provides customized commentary in real time.

[0686] Input: Live streaming URL and commentary scenario sent from the server

[0687] Data processing: The device uses the received streaming URL to play the live video and simultaneously display the commentary scenario.

[0688] Output: A screen displaying live streaming footage and customizable commentary

[0689] How it works: When a user starts live streaming in the application, real-time commentary along a timeline is displayed along with the video.

[0690] Step 6:

[0691] Users ask questions in real time

[0692] Input: The user enters text into the application's question input field and presses the submit button.

[0693] Data processing: The terminal formats the entered question and converts it into data to be sent to the server.

[0694] Output: The user's question is formatted and sent to the server.

[0695] Specific operation: The user enters a question and presses the send button to send the question to the server.

[0696] Step 7:

[0697] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[0698] Input: User-submitted question

[0699] Data processing: The server analyzes the question using natural language processing technology and generates an appropriate explanation using a generative AI model.

[0700] Output: Generated commentary

[0701] Specific operation: The server processes the question content using an analysis engine, generates an explanation using an AI model, and sends it to the device.

[0702] Step 8:

[0703] The device displays the explanation.

[0704] Input: Description sent from the server

[0705] Data processing: The device converts the received commentary into a format that can be displayed on the screen.

[0706] Output: Explanation displayed on the terminal screen

[0707] Specific behavior: The explanation is displayed properly in the user's field of view, and the user can get the answer in real time.

[0708] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[0709] (Application example 1)

[0710] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0711] In modern sports viewing, it is difficult for users to get a personalized experience that meets their preferences. In particular, there is a lack of real-time customized commentary and immediate responses to user questions. This makes it difficult for users to enjoy a satisfying viewing experience.

[0712] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0713] In this invention, the server includes a means for generating a user-specific viewing profile, a means for collecting related data and generating a customized commentary scenario, and a means for automatically generating commentary using a generative AI model and providing appropriate commentary based on the related data, thereby enabling users to receive personalized commentary in real time and instantly respond to questions while watching sports.

[0714] A "user-specific viewing profile" is a personalized profile generated based on the viewing settings, past viewing history, preferred commentary style, and feedback data set by the user.

[0715] A "customized commentary scenario" is a scenario that includes commentary and additional information focused on specific players or plays, generated based on a user's unique viewing profile.

[0716] A "generative AI model" is a mathematical model that uses artificial intelligence technology to automatically analyze data and generate personalized explanations and answers.

[0717] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to accurately analyze user questions and provide appropriate explanations.

[0718] "Real-time commentary" refers to commentary information that is generated and provided instantly as the game progresses, making the user's viewing experience more interesting.

[0719] The present invention relates to a system that enables users to watch sporting events in a personalized manner. The specific configuration and operation of this system will be described below.

[0720] System configuration

[0721] The system consists of the following main components:

[0722] 1. User Device:

[0723] This refers to devices such as smartphones and head-mounted displays operated by users.

[0724] Enter your viewing preferences, watch live streams, and enter questions.

[0725] 2. Server:

[0726] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0727] 3. Communication Network:

[0728] Refers to the network infrastructure that enables data communication between user terminals and servers.

[0729] Data processing and calculation

[0730] The server processes the data in the following steps:

[0731] 1. Receiving and analyzing spectator settings information:

[0732] The system receives and analyzes the viewing settings information sent from the user's device, taking into account the user's preferences, past viewing history, and feedback data.

[0733] 2. Create a viewing profile:

[0734] Based on the received information, a unique viewing profile for the user is generated, which includes the user's settings and past viewing data.

[0735] 3. Collection of relevant data:

[0736] Relevant data such as player information, past performance data, and explanations of technical terms are collected from the database. This collection is done using a database management system (RDBMS).

[0737] 4. Generate customized commentary scenarios:

[0738] Based on the collected data, a commentary scenario specific to the user is generated, using natural language processing (NLP) technology and generative AI models.

[0739] 5. Providing live streaming and commentary:

[0740] Once the match begins, the user's device displays live streaming video and displays customized commentary received from the server in real time.

[0741] 6. User Questions:

[0742] When a user types a question while watching a game, the question is sent to the server, which analyzes the question using natural language processing technology and uses a generative AI model to search for relevant data and generate an appropriate commentary. The commentary is then displayed on the user's device.

[0743] Specific examples

[0744] During a game, a user inputs a question such as, "I want to know the background of Player X's scoring ability." This question is analyzed by the server, and an explanation based on Player X's scoring performance data and past interview information is generated and provided to the user in real time.

[0745] Example prompt sentence:

[0746] Question: I want to know the story behind Player X's scoring ability.

[0747] Relevant information: Player X's past scoring data, snippets of interviews.

[0748] Explanation: Player X has averaged 20 points per game over the past three seasons, thanks to changes in his training methods and close collaboration with a mental health coach.

[0749] As a result, the present invention can provide users with an interactive and personalized sports viewing experience.

[0750] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0751] Step 1:

[0752] The user selects the sporting event they want to watch and enters their viewing settings. A list of sporting events to watch is displayed on the user's device. The user selects an event from the list and then enters viewing settings (e.g., to focus on a specific player, or to add an explanation of the basic rules). This input data is packaged in JSON format and sent to the server. The input is the user's selection and settings information, and the output is the packaged settings information.

[0753] Step 2:

[0754] The server analyzes the received viewing setting information and generates a viewing profile specific to the user. The server analyzes the received JSON data and combines the user's past viewing history, preferred commentary style, and feedback data to generate a viewing profile. This profile is used for all subsequent customization. The input is user setting information and the output is the viewing profile.

[0755] Step 3:

[0756] The server collects relevant data and generates a customized commentary scenario. Using generative AI models and natural language processing (NLP) technology, it collects player information, past performance data, explanations of technical terms, and other information to generate a commentary scenario specific to the user. The collected data is combined with the viewing profile to create a customized commentary. The input is the viewing profile and related data, and the output is a customized commentary scenario.

[0757] Step 4:

[0758] When the match begins, the device displays live streaming video and provides customized commentary in real time. The user device displays a commentary scenario retrieved from the server while watching. The commentary is dynamically updated as the match progresses, so the device periodically retrieves new commentary data from the server. The input is the live streaming and commentary scenario, and the output is the video and commentary displayed to the user.

[0759] Step 5:

[0760] The user inputs a question while watching the game. The question is sent from the user's terminal to the server. As the game progresses, the user inputs questions about their doubts or requests for additional explanation. These questions are transferred to the server in text format. The input is the user's question, and the output is the transmission of the question data to the server.

[0761] Step 6:

[0762] The server analyzes the received question, generates an explanation in real time, and sends it to the device. The server uses natural language processing technology to analyze the received question and input a prompt sentence into the generative AI model. An answer is generated based on related data and sent in text format to the user's device. The input is the user question and the prompt sentence from the generative AI model, and the output is an explanatory text.

[0763] Step 7:

[0764] The terminal displays the explanation. The explanatory text received from the server is visually displayed on the user terminal. This allows the user to obtain a customized answer in real time. The input is the explanatory text, and the output is the explanation displayed to the user.

[0765] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0766] The system of the present invention is designed to allow users to enjoy a personalized sporting event. It features an emotion engine that recognizes a user's emotional state and provides adaptive commentary in real time. The system works by allowing a user to select a sporting event to watch, input viewing preferences, and transmit them to a server. The emotion engine then analyzes the user's emotional state and dynamically adjusts the commentary based on that data.

[0767] Overall system configuration

[0768] The system of the present invention consists of the following main components:

[0769] 1. User Device

[0770] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[0771] Enter viewing preferences, watch live streams, enter questions, and collect sentiment data.

[0772] It is equipped with an emotion engine that recognizes the user's emotional state in real time.

[0773] 2. Server

[0774] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[0775] Dynamically adjust commentary content based on data from the emotion engine.

[0776] 3. Communication Network

[0777] Infrastructure for realizing data communication between user terminals and servers.

[0778] Program processing flow

[0779] The user selects the event to watch and enters the settings.

[0780] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[0781] The server processes the user's settings and generates a viewing profile.

[0782] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0783] The server collects relevant data and generates a customized commentary scenario.

[0784] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[0785] The device displays live streaming of the match and provides customized commentary in real time.

[0786] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0787] Recognizing the user's emotional state

[0788] The device is equipped with an emotion engine that analyzes the user's facial expressions, voice, and operation patterns in real time. Once the user's emotional state is recognized, the data is sent to a server.

[0789] The server adjusts the commentary based on the emotional data.

[0790] The server dynamically adjusts the commentary scenario based on the data received from the emotion engine according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical explanations.

[0791] Specific examples

[0792] 1. The user sets up the viewing settings

[0793] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[0794] 2. The server processes the configuration information

[0795] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[0796] 3. The device provides live streaming

[0797] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[0798] 4. Users ask questions in real time

[0799] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[0800] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[0801] 5. Emotion engine recognizes the user's emotional state

[0802] The emotion engine analyzes the user's facial expression data, voice data, and operation patterns to recognize the user's emotional state, such as surprise, joy, anxiety, etc. The recognition results are sent to the server.

[0803] 6. Server dynamically adjusts description

[0804] The server then analyzes the user's emotional state based on the received emotion data and dynamically adjusts the commentary accordingly. For example, if the user shows a surprised expression, the server can add a detailed tactical analysis to the commentary.

[0805] As a result, the system of the present invention can provide a personalized, interactive, and emotionally relevant sports viewing experience for the user.

[0806] The processing flow will be explained below.

[0807] Step 1:

[0808] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[0809] Step 2:

[0810] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[0811] Step 3:

[0812] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[0813] Step 4:

[0814] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[0815] Step 5:

[0816] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or increasing explanations of basic rules. Once these settings are complete, the device sends the setting information to the server.

[0817] Step 6:

[0818] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[0819] Step 7:

[0820] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[0821] Step 8:

[0822] The server generates a customized commentary scenario. The server creates a commentary scenario based on the collected data and in accordance with the user's settings.

[0823] Step 9:

[0824] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[0825] Step 10:

[0826] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[0827] Step 11:

[0828] The emotion engine recognizes the user's emotional state. The emotion engine installed in the device analyzes the user's facial expressions, voice, and operation patterns to recognize the user's emotional state in real time. The recognition results are sent to the server.

[0829] Step 12:

[0830] The server adjusts the commentary based on the emotion data. Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical commentary.

[0831] Step 13:

[0832] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[0833] Step 14:

[0834] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[0835] Step 15:

[0836] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[0837] Step 16:

[0838] The server generates an appropriate explanation and sends it to the device.The server generates explanatory text, videos, and images based on the search results and sends them to the device.

[0839] Step 17:

[0840] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[0841] This allows users to enjoy a personalized sports viewing experience in real time, and by incorporating emotion recognition functionality using an emotion engine, it is possible to provide even more interactive commentary that responds to the user's emotions.

[0842] Example 2

[0843] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0844] Existing sports viewing systems have difficulty providing real-time commentary that reflects individual user preferences and emotional states. Furthermore, few systems can respond quickly and appropriately to user questions. Therefore, providing users with an interactive and personalized viewing experience is a challenge.

[0845] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0846] In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's viewing preferences; means for transmitting the input preference information to the server; means for the server to generate a user-specific viewing profile based on the received preference information; means for the server to collect relevant data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state; means for dynamically adjusting the commentary scenario based on the emotion data; and means for the terminal to display the commentary, thereby enabling an interactive, emotionally-sensitive, and personalized sports viewing experience for the user.

[0847] "User" refers to an individual who uses the system to watch a sporting event.

[0848] "Spectator Settings" refers to information that allows a user to input a particular commentary style or focus point desired when watching a sporting event.

[0849] "Terminal" refers to a device operated by a user (e.g., a smartphone, tablet, computer, etc.).

[0850] "Server" refers to a central processing unit that receives viewing setting information and emotional data from users and generates viewing profiles and commentary scenarios based on that information.

[0851] A "watching profile" refers to user-specific data including the user's past viewing history, preferred commentary style, feedback, etc.

[0852] "Explanatory scenario" refers to a set of customized expository content generated by the server.

[0853] "Live streaming" refers to game footage that is broadcast in real time over the Internet.

[0854] "Emotion engine" refers to software or hardware that has the functionality to recognize the emotional state of a user.

[0855] "Emotion data" refers to data that represents the user's emotional state (for example, surprise, joy, anxiety, etc.).

[0856] "Question" refers to the question or information that a user wants to know that they input into the system while watching a game.

[0857] MODE FOR CARRYING OUT THE INVENTION

[0858] The system of the present invention is designed to personalize the viewing of sporting events and provide users with a richer viewing experience in real time. The main components of the system are a user terminal, a server, and a communication network.

[0859] The user terminals include devices such as smartphones, tablets, and computers, and are used to input viewing settings, watch live streaming, input questions, and collect emotional data. The terminals are also equipped with an emotion engine that recognizes emotions in real time through analysis of the user's facial expressions and voice. This emotional data is sent to a server. Specifically, facial expression recognition software and a voice analysis engine may be used.

[0860] The server receives the viewing setting information sent by the user and generates a viewing profile specific to the user based on that information. The profile includes the user's past viewing history, preferred commentary style, feedback, etc. The server also collects related data from the Internet (player information, past performance data, interview information, etc.) and generates a customized commentary scenario based on that data. When the server receives a question from the user, it analyzes the question using a natural language processing engine, generates appropriate commentary, and sends it to the user's device. Specifically, a natural language processing library and database are used.

[0861] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time. This commentary is dynamically updated as the match progresses and adjusts based on the user's excitement level and other emotional states. For example, if the user expresses surprise, the server can add more detailed tactical analysis to the commentary. The device can use existing live streaming services such as YouTube and Dailymotion.

[0862] Specific examples

[0863] 1. The user sets up the viewing settings

[0864] A user logs in to the system on their smartphone and selects a soccer match. The user enters settings such as "Focus on Player X," "More explanations of the basic rules," and "Show player interviews," and sends the settings information to the server.

[0865] 2. The server generates a profile

[0866] The server analyzes the received viewing setting information and generates a viewing profile based on the user's past viewing history and preferred commentary style. For example, the server collects information about player X and prepares a commentary scenario to be displayed during the match.

[0867] 3. The device displays the live stream.

[0868] The device uses services such as YouTube to display live streaming of the game, and simultaneously displays commentary received from the server. When Player X has the ball, information such as "Player X's special technique" is added to the commentary in real time.

[0869] 4. The user enters a question

[0870] During a game, a user types a question into their device, such as "I want to know about player X's scoring ability." The server analyzes the question using natural language processing, searches for relevant data, and generates an explanation. The explanation is then sent to the device and displayed to the user.

[0871] 5. Emotion engine recognizes the user's emotional state

[0872] The emotion engine analyzes the user's facial expressions and voice to recognize, for example, whether the user is happy, and sends that data to the server.

[0873] 6. The server adjusts the commentary based on the emotion data.

[0874] The server analyzes the emotional data and, for example, if the user is excited, adds details to the commentary such as "These are the tactics behind this play."

[0875] Prompt Sentence Examples

[0876] "Please explain in detail the processing steps of this sports viewing system program. Please break down the processing steps into smaller steps and describe each step, including the specific actions that are taken."

[0877] As a result, this system can provide users with an interactive, emotionally-driven, and personalized sports viewing experience.

[0878] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0879] Step 1:

[0880] The user selects the event to watch and enters the settings.

[0881] Input: A user operates a terminal to log in to the system, selects a sporting event to watch, and then inputs viewing settings (e.g., "Focus on player X," "More explanations of the basic rules," "Show player interviews").

[0882] Data processing: The device compiles the entered viewing settings information.

[0883] Output: Spectator settings information is sent from the device to the server.

[0884] Specific actions: The user uses their smartphone to enter viewing information into the designated fields in the app and presses the "Settings complete" button.

[0885] Step 2:

[0886] The server processes the user's settings and generates a viewing profile.

[0887] Input: Spectator settings information sent from the device.

[0888] Data processing: The server analyzes the received configuration information and updates the user's unique viewing profile, which includes the user's past viewing history, preferred commentary style, and feedback data.

[0889] Output: The updated spectator profile.

[0890] Specific operation: The server generates a new viewing profile based on the user's viewing history and feedback information stored in the database.

[0891] Step 3:

[0892] The server collects relevant data and generates a customized commentary scenario.

[0893] Input: The user's watching profile.

[0894] Data processing: The server collects relevant data from the Internet (player information, past performance data, interview information, etc.) and generates customized commentary scenarios based on the viewing profile.

[0895] Output: A customized commentary scenario.

[0896] Specific operation: The server uses a Web API to obtain the latest player information and past match data, and inputs this information into the commentary scenario generation algorithm to generate a scenario.

[0897] Step 4:

[0898] The device will display live streaming of the match and provide customized commentary

[0899] Input: A customized commentary script received from the server, and live streaming footage.

[0900] Data processing: The device displays live streaming video while overlaying explanatory information sent from the server at any time.

[0901] Output: Live streaming video and real-time commentary.

[0902] Specific operation: The device uses the live streaming APIs of YouTube and Dailymotion to play game footage while displaying commentary text on the screen in real time.

[0903] Step 5:

[0904] Recognizing the user's emotional state

[0905] Input: User's facial expressions, voice, and operation patterns.

[0906] Data processing: The emotion engine installed in the device analyzes this data and recognizes the user's emotional state.

[0907] Output: Emotion data (e.g. surprise, joy, anxiety, etc.).

[0908] Specific operation: The user's facial expressions and voice are captured in real time through the device's camera and microphone, and emotions are analyzed using facial recognition software and a voice analysis engine.

[0909] Step 6:

[0910] The server adjusts the commentary based on the emotional data.

[0911] Input: User emotional state data.

[0912] Data processing: The server dynamically adjusts the commentary scenario based on the received emotional data.

[0913] Output: A tailored expository scenario.

[0914] Specific actions: For example, if a user is surprised, the server will add a detailed analysis to the commentary, such as "These are the tactics behind this play."

[0915] Step 7:

[0916] Users can enter questions while watching

[0917] Input: User question (e.g., "What is Player X's scoring stats?").

[0918] Data processing: The device sends the query to the server.

[0919] Output: The question sent to the server.

[0920] Specific actions: The user enters a question into the input field on the terminal and presses the send button.

[0921] Step 8:

[0922] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[0923] Input: The user's question.

[0924] Data processing: The server uses a natural language processing engine to analyze the question, search for relevant data, and generate an appropriate explanation.

[0925] Output: The generated commentary.

[0926] Specific operation: The server analyzes the user's question, extracts relevant information from the database, generates an explanation, and sends it to the terminal.

[0927] (Application example 2)

[0928] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0929] Current sports viewing applications lack the ability to personalize commentary based on users' emotions and preferences, resulting in a uniform viewing experience. Furthermore, there is a lack of technology that can recognize users' emotional state in real time and dynamically adjust commentary accordingly. Therefore, there is a need for a more interactive experience.

[0930] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's watching settings; means for transmitting the input setting information to the server; means for the server to generate a user-specific watching profile based on the setting information received; means for the server to collect related data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the user's emotional state; means for the terminal to collect user emotion data using an emotion engine; and means for the terminal to display commentary. This enables a personalized, interactive, and emotion-responsive sports watching experience for the user.

[0931] Key Word Definitions

[0932] The "means for the user to select a sporting event to watch" refers to a device or software that provides an interface for the user to select any sporting event from among a plurality of sporting events.

[0933] The "means for inputting user viewing settings" refers to a device or software that provides an interface for the user to input viewing settings such as a particular commentary style or focus points.

[0934] The "means for transmitting input setting information to a server" refers to a device or software that transmits the watching setting information input by the user to a server via a communication network such as the Internet.

[0935] "Means for generating a user-specific viewing profile based on the setting information received by the server" refers to a device or software that analyzes the viewing setting information received from the user by the server and generates a viewing profile based on the user's interests and preferences.

[0936] "Means for the server to collect relevant data and generate customized commentary scenarios" refers to devices or software that allow the server to collect relevant data such as player information and past performance data based on a user's unique viewing profile and use that data to generate customized commentary scenarios.

[0937] "Means for a terminal to display live streaming of a match and provide customized commentary in real time" refers to a device or software that allows a terminal to display live video of a match received via the Internet and also displays customized commentary received from a server in real time.

[0938] The "means for users to input questions while watching" refers to a device or software that provides an interface that allows users watching the game to input questions they have in real time.

[0939] "Means for analyzing questions received by the server, generating explanations in real time, and transmitting them to the terminal" refers to a device or software that uses natural language processing technology to analyze questions received from users by the server, generates appropriate explanations based on the analysis, and transmits them to the terminal in real time.

[0940] "Means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the emotional state" refers to a device or software in which the terminal analyzes the user's facial expressions, voice, etc., recognizes the user's emotional state, and then dynamically adjusts the content of the commentary according to that emotional state.

[0941] "Means for a terminal to collect user emotional data using an emotion engine" refers to devices or software that allow devices such as smartphones and tablets to collect user emotional data using algorithms and sensors called emotion engines.

[0942] The "means for the terminal to display the commentary" refers to a device or software that allows the terminal to visually or audibly display the commentary generated in real time to the user.

[0943] "Form for carrying out the invention" of the specification

[0944] The system of the present invention allows users to enjoy a personalized sporting event. The system features an emotion engine that recognizes the user's emotional state and provides adaptive commentary in real time. The specific configuration for realizing this system is shown below.

[0945] Hardware and software used

[0946] This system consists of the following hardware and software:

[0947] Hardware: User devices (smartphones, tablets, computers, head-mounted displays (HMDs), etc.)

[0948] Software: Flask (Python framework), OpenCV (image processing library), Dlib (face detection and landmark prediction library), natural language processing algorithm, emotion engine

[0949] Data processing and calculation

[0950] 1. Select the event you want to watch and enter your settings:

[0951] Users log in to the system using their terminals, select the sporting event they want to watch, and enter viewing preferences such as focus points and commentary style, which are then sent from the terminal to the server.

[0952] 2. The server processes the user's preferences and generates a viewing profile:

[0953] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[0954] 3. The server collects the relevant data and generates a customized commentary scenario:

[0955] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, explanations of technical terms, etc.) and generates a customized commentary scenario specific to the user.

[0956] 4. The device displays live streaming of the match and provides customized commentary in real time:

[0957] Once the match begins, the device displays live streaming footage and simultaneously displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[0958] 5. Recognize the user's emotional state:

[0959] The emotion engine analyzes the user's facial expressions, voice, and operation patterns in real time. For example, if the user's emotional state, such as excitement or surprise, is detected, the data is sent to the server and used to adjust the commentary content.

[0960] 6. The server adjusts the commentary based on the emotion data:

[0961] Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is surprised, it can add detailed tactical analysis to the commentary.

[0962] Specific examples

[0963] Front-end: The user selects a basketball game and sets preferences such as "focus on player X," "show more explanation of the basic rules," and "display player interviews."

[0964] Backend: The server receives this configuration information and generates a customized commentary scenario based on Player X's profile data and related data.

[0965] Live Operation: During the match, the device provides live video of the match in real time and displays specialized commentary received from the server. The user's emotional state is analyzed and the commentary content is dynamically adjusted as needed.

[0966] Prompt Sentence Examples

[0967] "Tell me the story behind player X's scoring ability in a basketball game. The user is currently in an emotional state of surprise. Please provide commentary that is appropriate to this emotion."

[0968] This allows for personalized commentary based on the user's emotions, resulting in a more fulfilling viewing experience.

[0969] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0970] Specific processing flow of the program

[0971] Step 1:

[0972] A user selects a sporting event to watch and enters viewing preferences.

[0973] input:

[0974] An interface for users to select sporting events

[0975] Spectator settings (e.g., focus on specific players, commentary style, interview data to display)

[0976] output:

[0977] Entered sporting event information and viewing preferences

[0978] Specific operation:

[0979] Users log into the application on their smartphone or tablet and select the sporting event they want to watch.

[0980] Operates a form where users enter viewing preferences (player focus, commentary style, interviews, etc.).

[0981] This information is compiled into a single JSON object on the terminal side.

[0982] Step 2:

[0983] The entered setting information is sent to the server.

[0984] input:

[0985] Spectator settings information from the user (JSON object)

[0986] output:

[0987] Spectator settings sent to the server

[0988] Specific operation:

[0989] The device sends the user's viewing settings information to the server via the Internet using HTTPS as the communication protocol.

[0990] Step 3:

[0991] The server generates a user-specific game watching profile based on the received setting information.

[0992] input:

[0993] Spectator settings information

[0994] User's past viewing history, preferred commentary style, and feedback data

[0995] output:

[0996] User-specific viewing profiles

[0997] Specific operation:

[0998] The server analyzes the viewing setting information and updates the user's unique profile using a viewing profile generation algorithm.

[0999] A viewing profile includes a user's past viewing history, preferred commentary style, and feedback data.

[1000] Step 4:

[1001] The server collects the relevant data and generates a customized expository scenario.

[1002] input:

[1003] Spectator Profile

[1004] output:

[1005] Customized commentary scenarios

[1006] Specific operation:

[1007] The server collects relevant data such as player information, past performance data, interview information, and explanations of technical terms based on the user's viewing profile.

[1008] Based on the collected data, a customized commentary scenario specific to the user is generated.

[1009] Step 5:

[1010] The device displays live streaming of the match and provides customized commentary in real time.

[1011] input:

[1012] Live streaming video

[1013] Customized commentary scenarios

[1014] output:

[1015] Live streaming video and commentary scenario displayed to users

[1016] Specific operation:

[1017] Once the match begins, the device will display live streaming footage.

[1018] Customized commentary received from the server is displayed in real time as an overlay on the live video.

[1019] Step 6:

[1020] Recognize the user's emotional state.

[1021] input:

[1022] User facial expression data, voice data, operation patterns

[1023] output:

[1024] User emotional state data

[1025] Specific operation:

[1026] The device's built-in camera and microphone collect the user's facial expression and voice data.

[1027] The emotion engine analyzes this data in real time to recognize the user's emotional state.

[1028] Step 7:

[1029] The server adjusts the commentary based on the emotional data.

[1030] input:

[1031] User emotional state data

[1032] Customized commentary scenarios

[1033] output:

[1034] Updated commentary scenarios according to user emotions

[1035] Specific operation:

[1036] The server analyzes the emotional state data received from the emotion engine and dynamically adjusts the commentary scenario.

[1037] For example, if the user is excited, add tactical explanations, or if the user is surprised, provide a detailed explanation of external factors.

[1038] Step 8:

[1039] The device displays an explanation.

[1040] input:

[1041] Updated commentary scenario

[1042] output:

[1043] Updated explanatory scenarios displayed to users

[1044] Specific operation:

[1045] The terminal receives the updated commentary scenario sent from the server and displays it as an overlay on the live video.

[1046] This provides a personalized sports viewing experience that is tailored to the user's emotional state.

[1047] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1048] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1049] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1050] [Third embodiment]

[1051] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1052] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1053] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1054] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1055] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1056] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1057] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1058] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1059] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1060] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1061] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1062] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1063] The system of the present invention is designed to allow users watching sporting events to have a viewing experience that is tailored to their preferences. The system operates by allowing users to select a sporting event to watch, input viewing preferences, and transmit the input preferences to a server.

[1064] Overall system configuration

[1065] The system of the present invention consists of the following main components:

[1066] 1. User Device

[1067] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[1068] Enter your viewing preferences, watch live streams, and enter questions.

[1069] 2. Server

[1070] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1071] 3. Communication Network

[1072] Infrastructure for realizing data communication between user terminals and servers.

[1073] Program processing flow

[1074] The user selects the event to watch and enters the settings.

[1075] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[1076] The server processes the user's settings and generates a viewing profile.

[1077] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1078] The server collects relevant data and generates a customized commentary scenario.

[1079] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[1080] The device displays live streaming of the match and provides customized commentary in real time.

[1081] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1082] Specific examples

[1083] 1. The user sets up the viewing settings

[1084] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[1085] 2. The server processes the configuration information

[1086] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[1087] 3. The device provides live streaming

[1088] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[1089] 4. Users ask questions in real time

[1090] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[1091] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[1092] As a result, the system of the present invention can provide a personalized and interactive sports viewing experience for users.

[1093] The processing flow will be explained below.

[1094] Step 1:

[1095] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[1096] Step 2:

[1097] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[1098] Step 3:

[1099] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[1100] Step 4:

[1101] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[1102] Step 5:

[1103] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or adding more explanations of the basic rules. Once the input is complete, the device sends this setting information to the server.

[1104] Step 6:

[1105] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[1106] Step 7:

[1107] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[1108] Step 8:

[1109] The server generates a customized commentary scenario based on the collected data and according to the user's settings.

[1110] Step 9:

[1111] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[1112] Step 10:

[1113] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[1114] Step 11:

[1115] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[1116] Step 12:

[1117] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[1118] Step 13:

[1119] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[1120] Step 14:

[1121] The server generates an appropriate explanation and sends it to the device. Based on the search results, explanatory text, video, and images are generated and sent to the device.

[1122] Step 15:

[1123] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[1124] This allows users to enjoy a real-time, personalized sports viewing experience.

[1125] Example 1

[1126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1127] Conventional sports viewing systems have struggled to provide personalized commentary and information tailored to individual users' preferences and interests. Furthermore, they were unable to quickly provide real-time questions and answers, resulting in limited information available to users and reduced satisfaction with the viewing experience. There is a need to resolve these issues and provide users with a more interactive and personalized viewing experience.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1129] In this invention, the information processing device includes means for receiving and analyzing spectator setting information, means for generating a spectator profile, means for collecting related information and generating a customized commentary scenario, and means for analyzing a user's questions and generating commentary in real time. This allows a user to receive personalized commentary in real time according to their preferences, and can obtain quick and accurate commentary when they ask a question while watching a game.

[1130] "User" refers to an individual who uses the system to watch a competitive event.

[1131] "Competitive events" refer to various sporting events and matches that users can watch.

[1132] "Sports viewing preferences" are the viewing preferences and wishes that a user inputs into the system.

[1133] An "information processing device" is a device that receives and analyzes user setting information and generates the necessary profiles and data, and refers to a computer system such as a server.

[1134] A "watching profile" is a customized data set generated based on a user's unique viewing history, preferences, feedback data, etc.

[1135] A "display device" is a device used to display video streaming of competitions and commentary information to users, and includes smartphones, tablets, computers, etc.

[1136] "Video streaming" refers to a service that provides users with real-time video footage of competitive events.

[1137] "Customized commentary" is individual commentary information generated based on a user's unique viewing profile.

[1138] A "question" refers to a knowledge-based question or request for additional information that a user enters while watching a game.

[1139] "Natural language processing" refers to a technology that enables an information processing device to analyze questions received from users in human language and generate appropriate explanations.

[1140] "Related information" refers to any data collected based on users' questions and viewing profiles, such as player information, past match data, and explanations of technical terms.

[1141] The present invention is a system that provides commentary tailored to individual preferences when a user watches a sports event, enriching the viewing experience. This system functions by allowing the user to select the sports event to watch, input viewing settings, and transmit these to an information processing device (server). Specific hardware and software layouts, as well as data processing and data calculation methods, are described below.

[1142] System configuration

[1143] Hardware

[1144] 1. User Device

[1145] The device the user operates (smartphone, tablet, computer).

[1146] The device is used to watch live streaming, enter viewing settings, and enter questions.

[1147] 2. Server (information processing device)

[1148] High-performance computer cloud systems (e.g., AWS, Google Cloud Platform, Microsoft Azure).

[1149] The server receives and analyzes the viewing setting information, generates a viewing profile, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1150] 3. Communication Network

[1151] Infrastructure (Wi-Fi, 4G / 5G) for enabling data communication between user devices and servers.

[1152] software

[1153] 1. Live Streaming Applications

[1154] Video streaming services (e.g. YouTube Live, Twitch).

[1155] 2. Generative AI Models

[1156] Generative AI using natural language processing techniques (e.g., OpenAI GPT-3, GPT-4).

[1157] Specific operation of the system

[1158] 1. Enter your spectator settings

[1159] Users log in to the application from their own device and select the sporting event they want to watch. They then enter their viewing preferences, such as "focus on a specific player," "add basic rule explanations," and "display player interviews." This preference information is sent from the device to the server.

[1160] 2. Analyzing configuration information and generating a profile

[1161] The server receives and analyzes the viewing settings information sent from the device. Based on the analyzed information, a viewing profile unique to the user is generated. This profile includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[1162] 3. Generate customized explanations

[1163] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, terminology explanations, etc.) and then uses a generative AI model (such as GPT-3 or GPT-4) to generate a customized commentary scenario specific to the user.

[1164] 4. Live streaming and commentary

[1165] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1166] 5. Responding to user inquiries

[1167] While watching a game, users can input questions in real time through the application. For example, if a question is input such as "I want to know the background of Player X's scoring ability," the device will send this to the server. The server will analyze the question and generate an explanation for the question using a generative AI model. The explanation will be sent to the device, where the user can view it in real time.

[1168] Prompt Sentence Examples

[1169] "Generate a commentary scenario for player X in the next match, taking into account the user's viewing history and preferred commentary style."

[1170] "Please provide detailed explanations behind Player X's scoring ability in response to user questions."

[1171] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[1172] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1173] Step 1:

[1174] The user selects the athletic event they want to watch and enters their viewing preferences.

[1175] Input: A user logs in to the application, selects the athletic event to watch, and enters viewing preferences such as "focus on a specific player," "add basic rules explanation," and "show player interviews."

[1176] Data processing: The device formats the entered viewing settings and converts them into data for transmission.

[1177] Output: Spectator settings are formatted and sent to the server.

[1178] Specific operations: The user selects the desired competition event from the application's event selection screen, enters the viewing settings on the settings screen, and presses the send button.

[1179] Step 2:

[1180] The server receives and analyzes the spectator setting information.

[1181] Input: Spectator settings information sent from the device

[1182] Data processing: The server analyzes the viewing setting information using an analysis engine to extract the user's wishes and preferences.

[1183] Output: Information converted into a data structure for production based on user preferences.

[1184] Specific operation: The server analyzes the received data, extracts information such as the selected player and commentary type, and records it in an internal database.

[1185] Step 3:

[1186] The server generates a spectator profile

[1187] Input: Analyzed viewing preferences, user's past viewing history, preferred commentary style, and feedback data

[1188] Data processing: The server runs an algorithm based on these inputs to generate a unique viewing profile for the user.

[1189] Output: User-specific viewing profile

[1190] How it works: The server compares the viewing settings with past viewing history and generates a unique profile for the user, which reflects the user's preferences and tendencies.

[1191] Step 4:

[1192] The server collects relevant data and generates a customized commentary scenario.

[1193] Input: User-specific viewing profile, player information, past performance data, interview information

[1194] Data processing: The server collects relevant data based on the profile and generates explanatory scenarios using generative AI models (e.g., GPT-3 or GPT-4).

[1195] Output: A customized expository scenario

[1196] Specific operation: The server collects player information and past match data, and inputs this along with the player's profile into the AI, which then generates commentary based on this information.

[1197] Step 5:

[1198] The device displays live streaming and provides customized commentary in real time.

[1199] Input: Live streaming URL and commentary scenario sent from the server

[1200] Data processing: The device uses the received streaming URL to play the live video and simultaneously display the commentary scenario.

[1201] Output: A screen displaying live streaming footage and customizable commentary

[1202] How it works: When a user starts live streaming in the application, real-time commentary along a timeline is displayed along with the video.

[1203] Step 6:

[1204] Users ask questions in real time

[1205] Input: The user enters text into the application's question input field and presses the submit button.

[1206] Data processing: The terminal formats the entered question and converts it into data to be sent to the server.

[1207] Output: The user's question is formatted and sent to the server.

[1208] Specific operation: The user enters a question and presses the send button to send the question to the server.

[1209] Step 7:

[1210] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[1211] Input: User-submitted question

[1212] Data processing: The server analyzes the question using natural language processing technology and generates an appropriate explanation using a generative AI model.

[1213] Output: Generated commentary

[1214] Specific operation: The server processes the question content using an analysis engine, generates an explanation using an AI model, and sends it to the device.

[1215] Step 8:

[1216] The device displays the explanation.

[1217] Input: Description sent from the server

[1218] Data processing: The device converts the received commentary into a format that can be displayed on the screen.

[1219] Output: Explanation displayed on the terminal screen

[1220] Specific behavior: The explanation is displayed properly in the user's field of view, and the user can get the answer in real time.

[1221] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[1222] (Application example 1)

[1223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1224] In modern sports viewing, it is difficult for users to get a personalized experience that meets their preferences. In particular, there is a lack of real-time customized commentary and immediate responses to user questions. This makes it difficult for users to enjoy a satisfying viewing experience.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1226] In this invention, the server includes a means for generating a user-specific viewing profile, a means for collecting related data and generating a customized commentary scenario, and a means for automatically generating commentary using a generative AI model and providing appropriate commentary based on the related data, thereby enabling users to receive personalized commentary in real time and instantly respond to questions while watching sports.

[1227] A "user-specific viewing profile" is a personalized profile generated based on the viewing settings, past viewing history, preferred commentary style, and feedback data set by the user.

[1228] A "customized commentary scenario" is a scenario that includes commentary and additional information focused on specific players or plays, generated based on a user's unique viewing profile.

[1229] A "generative AI model" is a mathematical model that uses artificial intelligence technology to automatically analyze data and generate personalized explanations and answers.

[1230] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to accurately analyze user questions and provide appropriate explanations.

[1231] "Real-time commentary" refers to commentary information that is generated and provided instantly as the game progresses, making the user's viewing experience more interesting.

[1232] The present invention relates to a system that enables users to watch sporting events in a personalized manner. The specific configuration and operation of this system will be described below.

[1233] System configuration

[1234] The system consists of the following main components:

[1235] 1. User Device:

[1236] This refers to devices such as smartphones and head-mounted displays operated by users.

[1237] Enter your viewing preferences, watch live streams, and enter questions.

[1238] 2. Server:

[1239] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1240] 3. Communication Network:

[1241] Refers to the network infrastructure that enables data communication between user terminals and servers.

[1242] Data processing and calculation

[1243] The server processes the data in the following steps:

[1244] 1. Receiving and analyzing spectator settings information:

[1245] The system receives and analyzes the viewing settings information sent from the user's device, taking into account the user's preferences, past viewing history, and feedback data.

[1246] 2. Create a viewing profile:

[1247] Based on the received information, a unique viewing profile for the user is generated, which includes the user's settings and past viewing data.

[1248] 3. Collection of relevant data:

[1249] Relevant data such as player information, past performance data, and explanations of technical terms are collected from the database. This collection is done using a database management system (RDBMS).

[1250] 4. Generate customized commentary scenarios:

[1251] Based on the collected data, a commentary scenario specific to the user is generated, using natural language processing (NLP) technology and generative AI models.

[1252] 5. Providing live streaming and commentary:

[1253] Once the match begins, the user's device displays live streaming video and displays customized commentary received from the server in real time.

[1254] 6. User Questions:

[1255] When a user types a question while watching a game, the question is sent to the server, which analyzes the question using natural language processing technology and uses a generative AI model to search for relevant data and generate an appropriate commentary. The commentary is then displayed on the user's device.

[1256] Specific examples

[1257] During a game, a user inputs a question such as, "I want to know the background of Player X's scoring ability." This question is analyzed by the server, and an explanation based on Player X's scoring performance data and past interview information is generated and provided to the user in real time.

[1258] Example prompt sentence:

[1259] Question: I want to know the story behind Player X's scoring ability.

[1260] Relevant information: Player X's past scoring data, snippets of interviews.

[1261] Explanation: Player X has averaged 20 points per game over the past three seasons, thanks to changes in his training methods and close collaboration with a mental health coach.

[1262] As a result, the present invention can provide users with an interactive and personalized sports viewing experience.

[1263] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1264] Step 1:

[1265] The user selects the sporting event they want to watch and enters their viewing settings. A list of sporting events to watch is displayed on the user's device. The user selects an event from the list and then enters viewing settings (e.g., to focus on a specific player, or to add an explanation of the basic rules). This input data is packaged in JSON format and sent to the server. The input is the user's selection and settings information, and the output is the packaged settings information.

[1266] Step 2:

[1267] The server analyzes the received viewing setting information and generates a viewing profile specific to the user. The server analyzes the received JSON data and combines the user's past viewing history, preferred commentary style, and feedback data to generate a viewing profile. This profile is used for all subsequent customization. The input is user setting information and the output is the viewing profile.

[1268] Step 3:

[1269] The server collects relevant data and generates a customized commentary scenario. Using generative AI models and natural language processing (NLP) technology, it collects player information, past performance data, explanations of technical terms, and other information to generate a commentary scenario specific to the user. The collected data is combined with the viewing profile to create a customized commentary. The input is the viewing profile and related data, and the output is a customized commentary scenario.

[1270] Step 4:

[1271] When the match begins, the device displays live streaming video and provides customized commentary in real time. The user device displays a commentary scenario retrieved from the server while watching. The commentary is dynamically updated as the match progresses, so the device periodically retrieves new commentary data from the server. The input is the live streaming and commentary scenario, and the output is the video and commentary displayed to the user.

[1272] Step 5:

[1273] The user inputs a question while watching the game. The question is sent from the user's terminal to the server. As the game progresses, the user inputs questions about their doubts or requests for additional explanation. These questions are transferred to the server in text format. The input is the user's question, and the output is the transmission of the question data to the server.

[1274] Step 6:

[1275] The server analyzes the received question, generates an explanation in real time, and sends it to the device. The server uses natural language processing technology to analyze the received question and input a prompt sentence into the generative AI model. An answer is generated based on related data and sent in text format to the user's device. The input is the user question and the prompt sentence from the generative AI model, and the output is an explanatory text.

[1276] Step 7:

[1277] The terminal displays the explanation. The explanatory text received from the server is visually displayed on the user terminal. This allows the user to obtain a customized answer in real time. The input is the explanatory text, and the output is the explanation displayed to the user.

[1278] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1279] The system of the present invention is designed to allow users to enjoy a personalized sporting event. It features an emotion engine that recognizes a user's emotional state and provides adaptive commentary in real time. The system works by allowing a user to select a sporting event to watch, input viewing preferences, and transmit them to a server. The emotion engine then analyzes the user's emotional state and dynamically adjusts the commentary based on that data.

[1280] Overall system configuration

[1281] The system of the present invention consists of the following main components:

[1282] 1. User Device

[1283] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[1284] Enter viewing preferences, watch live streams, enter questions, and collect sentiment data.

[1285] It is equipped with an emotion engine that recognizes the user's emotional state in real time.

[1286] 2. Server

[1287] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1288] Dynamically adjust commentary content based on data from the emotion engine.

[1289] 3. Communication Network

[1290] Infrastructure for realizing data communication between user terminals and servers.

[1291] Program processing flow

[1292] The user selects the event to watch and enters the settings.

[1293] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[1294] The server processes the user's settings and generates a viewing profile.

[1295] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1296] The server collects relevant data and generates a customized commentary scenario.

[1297] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[1298] The device displays live streaming of the match and provides customized commentary in real time.

[1299] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1300] Recognizing the user's emotional state

[1301] The device is equipped with an emotion engine that analyzes the user's facial expressions, voice, and operation patterns in real time. Once the user's emotional state is recognized, the data is sent to a server.

[1302] The server adjusts the commentary based on the emotional data.

[1303] The server dynamically adjusts the commentary scenario based on the data received from the emotion engine according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical explanations.

[1304] Specific examples

[1305] 1. The user sets up the viewing settings

[1306] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[1307] 2. The server processes the configuration information

[1308] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[1309] 3. The device provides live streaming

[1310] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[1311] 4. Users ask questions in real time

[1312] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[1313] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[1314] 5. Emotion engine recognizes the user's emotional state

[1315] The emotion engine analyzes the user's facial expression data, voice data, and operation patterns to recognize the user's emotional state, such as surprise, joy, anxiety, etc. The recognition results are sent to the server.

[1316] 6. Server dynamically adjusts description

[1317] The server then analyzes the user's emotional state based on the received emotion data and dynamically adjusts the commentary accordingly. For example, if the user shows a surprised expression, the server can add a detailed tactical analysis to the commentary.

[1318] As a result, the system of the present invention can provide a personalized, interactive, and emotionally relevant sports viewing experience for the user.

[1319] The processing flow will be explained below.

[1320] Step 1:

[1321] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[1322] Step 2:

[1323] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[1324] Step 3:

[1325] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[1326] Step 4:

[1327] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[1328] Step 5:

[1329] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or increasing explanations of basic rules. Once these settings are complete, the device sends the setting information to the server.

[1330] Step 6:

[1331] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[1332] Step 7:

[1333] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[1334] Step 8:

[1335] The server generates a customized commentary scenario. The server creates a commentary scenario based on the collected data and in accordance with the user's settings.

[1336] Step 9:

[1337] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[1338] Step 10:

[1339] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[1340] Step 11:

[1341] The emotion engine recognizes the user's emotional state. The emotion engine installed in the device analyzes the user's facial expressions, voice, and operation patterns to recognize the user's emotional state in real time. The recognition results are sent to the server.

[1342] Step 12:

[1343] The server adjusts the commentary based on the emotion data. Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical commentary.

[1344] Step 13:

[1345] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[1346] Step 14:

[1347] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[1348] Step 15:

[1349] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[1350] Step 16:

[1351] The server generates an appropriate explanation and sends it to the device.The server generates explanatory text, videos, and images based on the search results and sends them to the device.

[1352] Step 17:

[1353] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[1354] This allows users to enjoy a personalized sports viewing experience in real time, and by incorporating emotion recognition functionality using an emotion engine, it is possible to provide even more interactive commentary that responds to the user's emotions.

[1355] Example 2

[1356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1357] Existing sports viewing systems have difficulty providing real-time commentary that reflects individual user preferences and emotional states. Furthermore, few systems can respond quickly and appropriately to user questions. Therefore, providing users with an interactive and personalized viewing experience is a challenge.

[1358] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1359] In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's viewing preferences; means for transmitting the input preference information to the server; means for the server to generate a user-specific viewing profile based on the received preference information; means for the server to collect relevant data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state; means for dynamically adjusting the commentary scenario based on the emotion data; and means for the terminal to display the commentary, thereby enabling an interactive, emotionally-sensitive, and personalized sports viewing experience for the user.

[1360] "User" refers to an individual who uses the system to watch a sporting event.

[1361] "Spectator Settings" refers to information that allows a user to input a particular commentary style or focus point desired when watching a sporting event.

[1362] "Terminal" refers to a device operated by a user (e.g., a smartphone, tablet, computer, etc.).

[1363] "Server" refers to a central processing unit that receives viewing setting information and emotional data from users and generates viewing profiles and commentary scenarios based on that information.

[1364] A "watching profile" refers to user-specific data including the user's past viewing history, preferred commentary style, feedback, etc.

[1365] "Explanatory scenario" refers to a set of customized expository content generated by the server.

[1366] "Live streaming" refers to game footage that is broadcast in real time over the Internet.

[1367] "Emotion engine" refers to software or hardware that has the functionality to recognize the emotional state of a user.

[1368] "Emotion data" refers to data that represents the user's emotional state (for example, surprise, joy, anxiety, etc.).

[1369] "Question" refers to the question or information that a user wants to know that they input into the system while watching a game.

[1370] MODE FOR CARRYING OUT THE INVENTION

[1371] The system of the present invention is designed to personalize the viewing of sporting events and provide users with a richer viewing experience in real time. The main components of the system are a user terminal, a server, and a communication network.

[1372] The user terminals include devices such as smartphones, tablets, and computers, and are used to input viewing settings, watch live streaming, input questions, and collect emotional data. The terminals are also equipped with an emotion engine that recognizes emotions in real time through analysis of the user's facial expressions and voice. This emotional data is sent to a server. Specifically, facial expression recognition software and a voice analysis engine may be used.

[1373] The server receives the viewing setting information sent by the user and generates a viewing profile specific to the user based on that information. The profile includes the user's past viewing history, preferred commentary style, feedback, etc. The server also collects related data from the Internet (player information, past performance data, interview information, etc.) and generates a customized commentary scenario based on that data. When the server receives a question from the user, it analyzes the question using a natural language processing engine, generates appropriate commentary, and sends it to the user's device. Specifically, a natural language processing library and database are used.

[1374] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time. This commentary is dynamically updated as the match progresses and adjusts based on the user's excitement level and other emotional states. For example, if the user expresses surprise, the server can add more detailed tactical analysis to the commentary. The device can use existing live streaming services such as YouTube and Dailymotion.

[1375] Specific examples

[1376] 1. The user sets up the viewing settings

[1377] A user logs in to the system on their smartphone and selects a soccer match. The user enters settings such as "Focus on Player X," "More explanations of the basic rules," and "Show player interviews," and sends the settings information to the server.

[1378] 2. The server generates a profile

[1379] The server analyzes the received viewing setting information and generates a viewing profile based on the user's past viewing history and preferred commentary style. For example, the server collects information about player X and prepares a commentary scenario to be displayed during the match.

[1380] 3. The device displays the live stream.

[1381] The device uses services such as YouTube to display live streaming of the game, and simultaneously displays commentary received from the server. When Player X has the ball, information such as "Player X's special technique" is added to the commentary in real time.

[1382] 4. The user enters a question

[1383] During a game, a user types a question into their device, such as "I want to know about player X's scoring ability." The server analyzes the question using natural language processing, searches for relevant data, and generates an explanation. The explanation is then sent to the device and displayed to the user.

[1384] 5. Emotion engine recognizes the user's emotional state

[1385] The emotion engine analyzes the user's facial expressions and voice to recognize, for example, whether the user is happy, and sends that data to the server.

[1386] 6. The server adjusts the commentary based on the emotion data.

[1387] The server analyzes the emotional data and, for example, if the user is excited, adds details to the commentary such as "These are the tactics behind this play."

[1388] Prompt Sentence Examples

[1389] "Please explain in detail the processing steps of this sports viewing system program. Please break down the processing steps into smaller steps and describe each step, including the specific actions that are taken."

[1390] As a result, this system can provide users with an interactive, emotionally-driven, and personalized sports viewing experience.

[1391] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1392] Step 1:

[1393] The user selects the event to watch and enters the settings.

[1394] Input: A user operates a terminal to log in to the system, selects a sporting event to watch, and then inputs viewing settings (e.g., "Focus on player X," "More explanations of the basic rules," "Show player interviews").

[1395] Data processing: The device compiles the entered viewing settings information.

[1396] Output: Spectator settings information is sent from the device to the server.

[1397] Specific actions: The user uses their smartphone to enter viewing information into the designated fields in the app and presses the "Settings complete" button.

[1398] Step 2:

[1399] The server processes the user's settings and generates a viewing profile.

[1400] Input: Spectator settings information sent from the device.

[1401] Data processing: The server analyzes the received configuration information and updates the user's unique viewing profile, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1402] Output: The updated spectator profile.

[1403] Specific operation: The server generates a new viewing profile based on the user's viewing history and feedback information stored in the database.

[1404] Step 3:

[1405] The server collects relevant data and generates a customized commentary scenario.

[1406] Input: The user's watching profile.

[1407] Data processing: The server collects relevant data from the Internet (player information, past performance data, interview information, etc.) and generates customized commentary scenarios based on the viewing profile.

[1408] Output: A customized commentary scenario.

[1409] Specific operation: The server uses a Web API to obtain the latest player information and past match data, and inputs this information into the commentary scenario generation algorithm to generate a scenario.

[1410] Step 4:

[1411] The device will display live streaming of the match and provide customized commentary

[1412] Input: A customized commentary script received from the server, and live streaming footage.

[1413] Data processing: The device displays live streaming video while overlaying explanatory information sent from the server at any time.

[1414] Output: Live streaming video and real-time commentary.

[1415] Specific operation: The device uses the live streaming APIs of YouTube and Dailymotion to play game footage while displaying commentary text on the screen in real time.

[1416] Step 5:

[1417] Recognizing the user's emotional state

[1418] Input: User's facial expressions, voice, and operation patterns.

[1419] Data processing: The emotion engine installed in the device analyzes this data and recognizes the user's emotional state.

[1420] Output: Emotion data (e.g. surprise, joy, anxiety, etc.).

[1421] Specific operation: The user's facial expressions and voice are captured in real time through the device's camera and microphone, and emotions are analyzed using facial recognition software and a voice analysis engine.

[1422] Step 6:

[1423] The server adjusts the commentary based on the emotional data.

[1424] Input: User emotional state data.

[1425] Data processing: The server dynamically adjusts the commentary scenario based on the received emotional data.

[1426] Output: A tailored expository scenario.

[1427] Specific actions: For example, if a user is surprised, the server will add a detailed analysis to the commentary, such as "These are the tactics behind this play."

[1428] Step 7:

[1429] Users can enter questions while watching

[1430] Input: User question (e.g., "What is Player X's scoring stats?").

[1431] Data processing: The device sends the query to the server.

[1432] Output: The question sent to the server.

[1433] Specific actions: The user enters a question into the input field on the terminal and presses the send button.

[1434] Step 8:

[1435] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[1436] Input: The user's question.

[1437] Data processing: The server uses a natural language processing engine to analyze the question, search for relevant data, and generate an appropriate explanation.

[1438] Output: The generated commentary.

[1439] Specific operation: The server analyzes the user's question, extracts relevant information from the database, generates an explanation, and sends it to the terminal.

[1440] (Application example 2)

[1441] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1442] Current sports viewing applications lack the ability to personalize commentary based on users' emotions and preferences, resulting in a uniform viewing experience. Furthermore, there is a lack of technology that can recognize users' emotional state in real time and dynamically adjust commentary accordingly. Therefore, there is a need for a more interactive experience.

[1443] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's watching settings; means for transmitting the input setting information to the server; means for the server to generate a user-specific watching profile based on the setting information received; means for the server to collect related data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the user's emotional state; means for the terminal to collect user emotion data using an emotion engine; and means for the terminal to display commentary. This enables a personalized, interactive, and emotion-responsive sports watching experience for the user.

[1444] Key Word Definitions

[1445] The "means for the user to select a sporting event to watch" refers to a device or software that provides an interface for the user to select any sporting event from among a plurality of sporting events.

[1446] The "means for inputting user viewing settings" refers to a device or software that provides an interface for the user to input viewing settings such as a particular commentary style or focus points.

[1447] The "means for transmitting input setting information to a server" refers to a device or software that transmits the watching setting information input by the user to a server via a communication network such as the Internet.

[1448] "Means for generating a user-specific viewing profile based on the setting information received by the server" refers to a device or software that analyzes the viewing setting information received from the user by the server and generates a viewing profile based on the user's interests and preferences.

[1449] "Means for the server to collect relevant data and generate customized commentary scenarios" refers to devices or software that allow the server to collect relevant data such as player information and past performance data based on a user's unique viewing profile and use that data to generate customized commentary scenarios.

[1450] "Means for a terminal to display live streaming of a match and provide customized commentary in real time" refers to a device or software that allows a terminal to display live video of a match received via the Internet and also displays customized commentary received from a server in real time.

[1451] The "means for users to input questions while watching" refers to a device or software that provides an interface that allows users watching the game to input questions they have in real time.

[1452] "Means for analyzing questions received by the server, generating explanations in real time, and transmitting them to the terminal" refers to a device or software that uses natural language processing technology to analyze questions received from users by the server, generates appropriate explanations based on the analysis, and transmits them to the terminal in real time.

[1453] "Means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the emotional state" refers to a device or software in which the terminal analyzes the user's facial expressions, voice, etc., recognizes the user's emotional state, and then dynamically adjusts the content of the commentary according to that emotional state.

[1454] "Means for a device to collect user emotional data using an emotion engine" refers to devices or software that allow devices such as smartphones and tablets to collect user emotional data using algorithms and sensors called emotion engines.

[1455] The "means for the terminal to display the commentary" refers to a device or software that allows the terminal to visually or audibly display the commentary generated in real time to the user.

[1456] "Form for carrying out the invention" of the specification

[1457] The system of the present invention allows users to enjoy a personalized sporting event. The system features an emotion engine that recognizes the user's emotional state and provides adaptive commentary in real time. The specific configuration for realizing this system is shown below.

[1458] Hardware and software used

[1459] This system consists of the following hardware and software:

[1460] Hardware: User devices (smartphones, tablets, computers, head-mounted displays (HMDs), etc.)

[1461] Software: Flask (Python framework), OpenCV (image processing library), Dlib (face detection and landmark prediction library), natural language processing algorithm, emotion engine

[1462] Data processing and calculation

[1463] 1. Select the event you want to watch and enter your settings:

[1464] Users log in to the system using their terminals, select the sporting event they want to watch, and enter viewing preferences such as focus points and commentary style, which are then sent from the terminal to the server.

[1465] 2. The server processes the user's preferences and generates a viewing profile:

[1466] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[1467] 3. The server collects the relevant data and generates a customized commentary scenario:

[1468] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, explanations of technical terms, etc.) and generates a customized commentary scenario specific to the user.

[1469] 4. The device displays live streaming of the match and provides customized commentary in real time:

[1470] Once the match begins, the device displays live streaming footage and simultaneously displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1471] 5. Recognize the user's emotional state:

[1472] The emotion engine analyzes the user's facial expressions, voice, and operation patterns in real time. For example, if the user's emotional state, such as excitement or surprise, is detected, the data is sent to the server and used to adjust the commentary content.

[1473] 6. The server adjusts the commentary based on the emotion data:

[1474] Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is surprised, it can add detailed tactical analysis to the commentary.

[1475] Specific examples

[1476] Front-end: The user selects a basketball game and sets preferences such as "focus on player X," "show more explanation of the basic rules," and "display player interviews."

[1477] Backend: The server receives this configuration information and generates a customized commentary scenario based on Player X's profile data and related data.

[1478] Live Operation: During the match, the device provides live video of the match in real time and displays specialized commentary received from the server. The user's emotional state is analyzed and the commentary content is dynamically adjusted as needed.

[1479] Prompt Sentence Examples

[1480] "Tell me the story behind player X's scoring ability in a basketball game. The user is currently in an emotional state of surprise. Please provide commentary that is appropriate to this emotion."

[1481] This allows for personalized commentary based on the user's emotions, resulting in a more fulfilling viewing experience.

[1482] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1483] Specific processing flow of the program

[1484] Step 1:

[1485] A user selects a sporting event to watch and enters viewing preferences.

[1486] input:

[1487] An interface for users to select sporting events

[1488] Spectator settings (e.g., focus on specific players, commentary style, interview data to display)

[1489] output:

[1490] Entered sporting event information and viewing preferences

[1491] Specific operation:

[1492] Users log into the application on their smartphone or tablet and select the sporting event they want to watch.

[1493] Operates a form where users enter viewing preferences (player focus, commentary style, interviews, etc.).

[1494] This information is compiled into a single JSON object on the terminal side.

[1495] Step 2:

[1496] The entered setting information is sent to the server.

[1497] input:

[1498] Spectator settings information from the user (JSON object)

[1499] output:

[1500] Spectator settings sent to the server

[1501] Specific operation:

[1502] The device sends the user's viewing settings information to the server via the Internet using HTTPS as the communication protocol.

[1503] Step 3:

[1504] The server generates a user-specific game watching profile based on the received setting information.

[1505] input:

[1506] Spectator settings information

[1507] User's past viewing history, preferred commentary style, and feedback data

[1508] output:

[1509] User-specific viewing profiles

[1510] Specific operation:

[1511] The server analyzes the viewing setting information and updates the user's unique profile using a viewing profile generation algorithm.

[1512] A viewing profile includes a user's past viewing history, preferred commentary style, and feedback data.

[1513] Step 4:

[1514] The server collects the relevant data and generates a customized expository scenario.

[1515] input:

[1516] Spectator Profile

[1517] output:

[1518] Customized commentary scenarios

[1519] Specific operation:

[1520] The server collects relevant data such as player information, past performance data, interview information, and explanations of technical terms based on the user's viewing profile.

[1521] Based on the collected data, a customized commentary scenario specific to the user is generated.

[1522] Step 5:

[1523] The device displays live streaming of the match and provides customized commentary in real time.

[1524] input:

[1525] Live streaming video

[1526] Customized commentary scenarios

[1527] output:

[1528] Live streaming video and commentary scenario displayed to users

[1529] Specific operation:

[1530] Once the match begins, the device will display live streaming footage.

[1531] Customized commentary received from the server is displayed in real time as an overlay on the live video.

[1532] Step 6:

[1533] Recognize the user's emotional state.

[1534] input:

[1535] User facial expression data, voice data, operation patterns

[1536] output:

[1537] User emotional state data

[1538] Specific operation:

[1539] The device's built-in camera and microphone collect the user's facial expression and voice data.

[1540] The emotion engine analyzes this data in real time to recognize the user's emotional state.

[1541] Step 7:

[1542] The server adjusts the commentary based on the emotional data.

[1543] input:

[1544] User emotional state data

[1545] Customized commentary scenarios

[1546] output:

[1547] Updated commentary scenarios according to user emotions

[1548] Specific operation:

[1549] The server analyzes the emotional state data received from the emotion engine and dynamically adjusts the commentary scenario.

[1550] For example, if the user is excited, add tactical explanations, or if the user is surprised, provide a detailed explanation of external factors.

[1551] Step 8:

[1552] The device displays an explanation.

[1553] input:

[1554] Updated commentary scenario

[1555] output:

[1556] Updated explanatory scenarios displayed to users

[1557] Specific operation:

[1558] The terminal receives the updated commentary scenario sent from the server and displays it as an overlay on the live video.

[1559] This provides a personalized sports viewing experience that is tailored to the user's emotional state.

[1560] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1561] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1562] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1563] [Fourth embodiment]

[1564] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1565] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1566] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1567] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1568] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1569] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1570] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1571] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1572] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1573] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1574] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1575] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1576] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1577] The system of the present invention is designed to allow users watching sporting events to have a viewing experience that is tailored to their preferences. The system operates by allowing users to select a sporting event to watch, input viewing preferences, and transmit the input preferences to a server.

[1578] Overall system configuration

[1579] The system of the present invention consists of the following main components:

[1580] 1. User Device

[1581] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[1582] Enter your viewing preferences, watch live streams, and enter questions.

[1583] 2. Server

[1584] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1585] 3. Communication Network

[1586] Infrastructure for realizing data communication between user terminals and servers.

[1587] Program processing flow

[1588] The user selects the event to watch and enters the settings.

[1589] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[1590] The server processes the user's settings and generates a viewing profile.

[1591] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1592] The server collects relevant data and generates a customized commentary scenario.

[1593] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[1594] The device displays live streaming of the match and provides customized commentary in real time.

[1595] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1596] Specific examples

[1597] 1. The user sets up the viewing settings

[1598] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[1599] 2. The server processes the configuration information

[1600] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[1601] 3. The device provides live streaming

[1602] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[1603] 4. Users ask questions in real time

[1604] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[1605] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[1606] As a result, the system of the present invention can provide a personalized and interactive sports viewing experience for users.

[1607] The processing flow will be explained below.

[1608] Step 1:

[1609] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[1610] Step 2:

[1611] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[1612] Step 3:

[1613] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[1614] Step 4:

[1615] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[1616] Step 5:

[1617] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or adding more explanations of the basic rules. Once the input is complete, the device sends this setting information to the server.

[1618] Step 6:

[1619] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[1620] Step 7:

[1621] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[1622] Step 8:

[1623] The server generates a customized commentary scenario based on the collected data and according to the user's settings.

[1624] Step 9:

[1625] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[1626] Step 10:

[1627] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[1628] Step 11:

[1629] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[1630] Step 12:

[1631] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[1632] Step 13:

[1633] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[1634] Step 14:

[1635] The server generates an appropriate explanation and sends it to the device. Based on the search results, explanatory text, video, and images are generated and sent to the device.

[1636] Step 15:

[1637] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[1638] This allows users to enjoy a real-time, personalized sports viewing experience.

[1639] Example 1

[1640] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1641] Conventional sports viewing systems have struggled to provide personalized commentary and information tailored to individual users' preferences and interests. Furthermore, they were unable to quickly provide real-time questions and answers, resulting in limited information available to users and reduced satisfaction with the viewing experience. There is a need to resolve these issues and provide users with a more interactive and personalized viewing experience.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1643] In this invention, the information processing device includes means for receiving and analyzing spectator setting information, means for generating a spectator profile, means for collecting related information and generating a customized commentary scenario, and means for analyzing a user's questions and generating commentary in real time. This allows a user to receive personalized commentary in real time according to their preferences, and can obtain quick and accurate commentary when they ask a question while watching a game.

[1644] "User" refers to an individual who uses the system to watch a competitive event.

[1645] "Competitive events" refer to various sporting events and matches that users can watch.

[1646] "Sports viewing preferences" are the viewing preferences and wishes that a user inputs into the system.

[1647] An "information processing device" is a device that receives and analyzes user setting information and generates the necessary profiles and data, and refers to a computer system such as a server.

[1648] A "watching profile" is a customized data set generated based on a user's unique viewing history, preferences, feedback data, etc.

[1649] A "display device" is a device used to display video streaming of competitions and commentary information to users, and includes smartphones, tablets, computers, etc.

[1650] "Video streaming" refers to a service that provides users with real-time video footage of competitive events.

[1651] "Customized commentary" is individual commentary information generated based on a user's unique viewing profile.

[1652] A "question" refers to a knowledge-based question or request for additional information that a user enters while watching a game.

[1653] "Natural language processing" refers to a technology that enables an information processing device to analyze questions received from users in human language and generate appropriate explanations.

[1654] "Related information" refers to any data collected based on users' questions and viewing profiles, such as player information, past match data, and explanations of technical terms.

[1655] The present invention is a system that provides commentary tailored to individual preferences when a user watches a sports event, enriching the viewing experience. This system functions by allowing the user to select the sports event to watch, input viewing settings, and transmit these to an information processing device (server). Specific hardware and software layouts, as well as data processing and data calculation methods, are described below.

[1656] System configuration

[1657] Hardware

[1658] 1. User Device

[1659] The device the user operates (smartphone, tablet, computer).

[1660] The device is used to watch live streaming, enter viewing settings, and enter questions.

[1661] 2. Server (information processing device)

[1662] High-performance computer cloud systems (e.g., AWS, Google Cloud Platform, Microsoft Azure).

[1663] The server receives and analyzes the viewing setting information, generates a viewing profile, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1664] 3. Communication Network

[1665] Infrastructure (Wi-Fi, 4G / 5G) for enabling data communication between user devices and servers.

[1666] software

[1667] 1. Live Streaming Applications

[1668] Video streaming services (e.g. YouTube Live, Twitch).

[1669] 2. Generative AI Models

[1670] Generative AI using natural language processing techniques (e.g., OpenAI GPT-3, GPT-4).

[1671] Specific operation of the system

[1672] 1. Enter your spectator settings

[1673] Users log in to the application from their own device and select the sporting event they want to watch. They then enter their viewing preferences, such as "focus on a specific player," "add basic rule explanations," and "display player interviews." This preference information is sent from the device to the server.

[1674] 2. Analyzing configuration information and generating a profile

[1675] The server receives and analyzes the viewing settings information sent from the device. Based on the analyzed information, a viewing profile unique to the user is generated. This profile includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[1676] 3. Generate customized explanations

[1677] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, terminology explanations, etc.) and then uses a generative AI model (such as GPT-3 or GPT-4) to generate a customized commentary scenario specific to the user.

[1678] 4. Live streaming and commentary

[1679] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1680] 5. Responding to user inquiries

[1681] While watching a game, users can input questions in real time through the application. For example, if a question is input such as "I want to know the background of Player X's scoring ability," the device will send this to the server. The server will analyze the question and generate an explanation for the question using a generative AI model. The explanation will be sent to the device, where the user can view it in real time.

[1682] Prompt Sentence Examples

[1683] "Generate a commentary scenario for player X in the next match, taking into account the user's viewing history and preferred commentary style."

[1684] "Please provide detailed explanations behind Player X's scoring ability in response to user questions."

[1685] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[1686] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1687] Step 1:

[1688] The user selects the athletic event they want to watch and enters their viewing preferences.

[1689] Input: A user logs in to the application, selects the athletic event to watch, and enters viewing preferences such as "focus on a specific player," "add basic rules explanation," and "show player interviews."

[1690] Data processing: The device formats the entered viewing settings and converts them into data for transmission.

[1691] Output: Spectator settings are formatted and sent to the server.

[1692] Specific operations: The user selects the desired competition event from the application's event selection screen, enters the viewing settings on the settings screen, and presses the send button.

[1693] Step 2:

[1694] The server receives and analyzes the spectator setting information.

[1695] Input: Spectator settings information sent from the device

[1696] Data processing: The server analyzes the viewing setting information using an analysis engine to extract the user's wishes and preferences.

[1697] Output: Information converted into a data structure for production based on user preferences.

[1698] Specific operation: The server analyzes the received data, extracts information such as the selected player and commentary type, and records it in an internal database.

[1699] Step 3:

[1700] The server generates a spectator profile

[1701] Input: Analyzed viewing preferences, user's past viewing history, preferred commentary style, and feedback data

[1702] Data processing: The server runs an algorithm based on these inputs to generate a unique viewing profile for the user.

[1703] Output: User-specific viewing profile

[1704] How it works: The server compares the viewing settings with past viewing history and generates a unique profile for the user, which reflects the user's preferences and tendencies.

[1705] Step 4:

[1706] The server collects relevant data and generates a customized commentary scenario.

[1707] Input: User-specific viewing profile, player information, past performance data, interview information

[1708] Data processing: The server collects relevant data based on the profile and generates explanatory scenarios using generative AI models (e.g., GPT-3 or GPT-4).

[1709] Output: A customized expository scenario

[1710] Specific operation: The server collects player information and past match data, and inputs this along with the player's profile into the AI, which then generates commentary based on this information.

[1711] Step 5:

[1712] The device displays live streaming and provides customized commentary in real time.

[1713] Input: Live streaming URL and commentary scenario sent from the server

[1714] Data processing: The device uses the received streaming URL to play the live video and simultaneously display the commentary scenario.

[1715] Output: A screen displaying live streaming footage and customizable commentary

[1716] How it works: When a user starts live streaming in the application, real-time commentary along a timeline is displayed along with the video.

[1717] Step 6:

[1718] Users ask questions in real time

[1719] Input: The user enters text into the application's question input field and presses the submit button.

[1720] Data processing: The terminal formats the entered question and converts it into data to be sent to the server.

[1721] Output: The user's question is formatted and sent to the server.

[1722] Specific operation: The user enters a question and presses the send button to send the question to the server.

[1723] Step 7:

[1724] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[1725] Input: User-submitted question

[1726] Data processing: The server analyzes the question using natural language processing technology and generates an appropriate explanation using a generative AI model.

[1727] Output: Generated commentary

[1728] Specific operation: The server processes the question content using an analysis engine, generates an explanation using an AI model, and sends it to the device.

[1729] Step 8:

[1730] The device displays the explanation.

[1731] Input: Description sent from the server

[1732] Data processing: The device converts the received commentary into a format that can be displayed on the screen.

[1733] Output: Explanation displayed on the terminal screen

[1734] Specific behavior: The explanation is displayed properly in the user's field of view, and the user can get the answer in real time.

[1735] As a result, the system of the present invention can provide users with a personalized and highly interactive sports viewing experience.

[1736] (Application example 1)

[1737] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1738] In modern sports viewing, it is difficult for users to get a personalized experience that meets their preferences. In particular, there is a lack of real-time customized commentary and immediate responses to user questions. This makes it difficult for users to enjoy a satisfying viewing experience.

[1739] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1740] In this invention, the server includes a means for generating a user-specific viewing profile, a means for collecting related data and generating a customized commentary scenario, and a means for automatically generating commentary using a generative AI model and providing appropriate commentary based on the related data, thereby enabling users to receive personalized commentary in real time and instantly respond to questions while watching sports.

[1741] A "user-specific viewing profile" is a personalized profile generated based on the viewing settings, past viewing history, preferred commentary style, and feedback data set by the user.

[1742] A "customized commentary scenario" is a scenario that includes commentary and additional information focused on specific players or plays, generated based on a user's unique viewing profile.

[1743] A "generative AI model" is a mathematical model that uses artificial intelligence technology to automatically analyze data and generate personalized explanations and answers.

[1744] "Natural language processing" is a technology that enables computers to understand, analyze, and generate human language, and is used to accurately analyze user questions and provide appropriate explanations.

[1745] "Real-time commentary" refers to commentary information that is generated and provided instantly as the game progresses, making the user's viewing experience more interesting.

[1746] The present invention relates to a system that enables users to watch sporting events in a personalized manner. The specific configuration and operation of this system will be described below.

[1747] System configuration

[1748] The system consists of the following main components:

[1749] 1. User Device:

[1750] This refers to devices such as smartphones and head-mounted displays operated by users.

[1751] Enter your viewing preferences, watch live streams, and enter questions.

[1752] 2. Server:

[1753] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1754] 3. Communication Network:

[1755] Refers to the network infrastructure that enables data communication between user terminals and servers.

[1756] Data processing and calculation

[1757] The server processes the data in the following steps:

[1758] 1. Receiving and analyzing spectator settings information:

[1759] The system receives and analyzes the viewing settings information sent from the user's device, taking into account the user's preferences, past viewing history, and feedback data.

[1760] 2. Create a viewing profile:

[1761] Based on the received information, a unique viewing profile for the user is generated, which includes the user's settings and past viewing data.

[1762] 3. Collection of relevant data:

[1763] Relevant data such as player information, past performance data, and explanations of technical terms are collected from the database. This collection is done using a database management system (RDBMS).

[1764] 4. Generate customized commentary scenarios:

[1765] Based on the collected data, a commentary scenario specific to the user is generated, using natural language processing (NLP) technology and generative AI models.

[1766] 5. Providing live streaming and commentary:

[1767] Once the match begins, the user's device displays live streaming video and displays customized commentary received from the server in real time.

[1768] 6. User Questions:

[1769] When a user types a question while watching a game, the question is sent to the server, which analyzes the question using natural language processing technology and uses a generative AI model to search for relevant data and generate an appropriate commentary. The commentary is then displayed on the user's device.

[1770] Specific examples

[1771] During a game, a user inputs a question such as, "I want to know the background of Player X's scoring ability." This question is analyzed by the server, and an explanation based on Player X's scoring performance data and past interview information is generated and provided to the user in real time.

[1772] Example prompt sentence:

[1773] Question: I want to know the story behind Player X's scoring ability.

[1774] Relevant information: Player X's past scoring data, snippets of interviews.

[1775] Explanation: Player X has averaged 20 points per game over the past three seasons, thanks to changes in his training methods and close collaboration with a mental health coach.

[1776] As a result, the present invention can provide users with an interactive and personalized sports viewing experience.

[1777] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1778] Step 1:

[1779] The user selects the sporting event they want to watch and enters their viewing settings. A list of sporting events to watch is displayed on the user's device. The user selects an event from the list and then enters viewing settings (e.g., to focus on a specific player, or to add an explanation of the basic rules). This input data is packaged in JSON format and sent to the server. The input is the user's selection and settings information, and the output is the packaged settings information.

[1780] Step 2:

[1781] The server analyzes the received viewing setting information and generates a viewing profile specific to the user. The server analyzes the received JSON data and combines the user's past viewing history, preferred commentary style, and feedback data to generate a viewing profile. This profile is used for all subsequent customization. The input is user setting information and the output is the viewing profile.

[1782] Step 3:

[1783] The server collects relevant data and generates a customized commentary scenario. Using generative AI models and natural language processing (NLP) technology, it collects player information, past performance data, explanations of technical terms, and other information to generate a commentary scenario specific to the user. The collected data is combined with the viewing profile to create a customized commentary. The input is the viewing profile and related data, and the output is a customized commentary scenario.

[1784] Step 4:

[1785] When the match begins, the device displays live streaming video and provides customized commentary in real time. The user device displays a commentary scenario retrieved from the server while watching. The commentary is dynamically updated as the match progresses, so the device periodically retrieves new commentary data from the server. The input is the live streaming and commentary scenario, and the output is the video and commentary displayed to the user.

[1786] Step 5:

[1787] The user inputs a question while watching the game. The question is sent from the user's terminal to the server. As the game progresses, the user inputs questions about their doubts or requests for additional explanation. These questions are transferred to the server in text format. The input is the user's question, and the output is the transmission of the question data to the server.

[1788] Step 6:

[1789] The server analyzes the received question, generates an explanation in real time, and sends it to the device. The server uses natural language processing technology to analyze the received question and input a prompt sentence into the generative AI model. An answer is generated based on related data and sent in text format to the user's device. The input is the user question and the prompt sentence from the generative AI model, and the output is an explanatory text.

[1790] Step 7:

[1791] The terminal displays the explanation. The explanatory text received from the server is visually displayed on the user terminal. This allows the user to obtain a customized answer in real time. The input is the explanatory text, and the output is the explanation displayed to the user.

[1792] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1793] The system of the present invention is designed to allow users to enjoy a personalized sporting event. It features an emotion engine that recognizes a user's emotional state and provides adaptive commentary in real time. The system works by allowing a user to select a sporting event to watch, input viewing preferences, and transmit them to a server. The emotion engine then analyzes the user's emotional state and dynamically adjusts the commentary based on that data.

[1794] Overall system configuration

[1795] The system of the present invention consists of the following main components:

[1796] 1. User Device

[1797] It consists of devices such as smartphones, tablets, and computers that are operated by users.

[1798] Enter viewing preferences, watch live streams, enter questions, and collect sentiment data.

[1799] It is equipped with an emotion engine that recognizes the user's emotional state in real time.

[1800] 2. Server

[1801] It receives and analyzes viewing preference information, generates viewing profiles, collects relevant data, generates customized commentary scenarios, and generates real-time commentary based on user questions.

[1802] Dynamically adjust commentary content based on data from the emotion engine.

[1803] 3. Communication Network

[1804] Infrastructure for realizing data communication between user terminals and servers.

[1805] Program processing flow

[1806] The user selects the event to watch and enters the settings.

[1807] Users log in to the system using their own devices, select the sporting event they want to watch, and then enter their viewing preferences (e.g., focus on a specific player, increase explanations of basic rules, etc.). This information is sent from the device to the server.

[1808] The server processes the user's settings and generates a viewing profile.

[1809] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1810] The server collects relevant data and generates a customized commentary scenario.

[1811] Based on the user's viewing profile, the server collects relevant data (such as player information, past performance data, interview information, terminology explanations, etc.) and then uses this data to generate a customized commentary scenario specific to the user.

[1812] The device displays live streaming of the match and provides customized commentary in real time.

[1813] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1814] Recognizing the user's emotional state

[1815] The device is equipped with an emotion engine that analyzes the user's facial expressions, voice, and operation patterns in real time. Once the user's emotional state is recognized, the data is sent to a server.

[1816] The server adjusts the commentary based on the emotional data.

[1817] The server dynamically adjusts the commentary scenario based on the data received from the emotion engine according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical explanations.

[1818] Specific examples

[1819] 1. The user sets up the viewing settings

[1820] After logging in, the user selects a basketball game and sets the following: "Focus on Player X," "Expand explanation of basic rules," and "Show player interviews."

[1821] 2. The server processes the configuration information

[1822] The server analyzes the received setting information and updates the user's unique viewing profile. Based on the profile data of player X, it generates a commentary scenario of player X's movements and important plays during the match.

[1823] 3. The device provides live streaming

[1824] During the game, the device provides live streaming video to the user and displays customized commentary in real time based on the user's settings. The video and commentary are synchronized so that the user can focus on the movements of Player X.

[1825] 4. Users ask questions in real time

[1826] While watching a game, a user can input a question such as, "I want to know the background of player X's scoring ability." The device sends this question to the server, which then generates commentary based on the relevant data.

[1827] The commentary generated by the server is sent to the terminal, and the user can check the commentary in real time.

[1828] 5. Emotion engine recognizes the user's emotional state

[1829] The emotion engine analyzes the user's facial expression data, voice data, and operation patterns to recognize the user's emotional state, such as surprise, joy, anxiety, etc. The recognition results are sent to the server.

[1830] 6. Server dynamically adjusts description

[1831] The server then analyzes the user's emotional state based on the received emotion data and dynamically adjusts the commentary accordingly. For example, if the user shows a surprised expression, the server can add a detailed tactical analysis to the commentary.

[1832] As a result, the system of the present invention can provide a personalized, interactive, and emotionally relevant sports viewing experience for the user.

[1833] The processing flow will be explained below.

[1834] Step 1:

[1835] A user logs in to the system from a terminal. The user enters a username and password on the login screen and presses the "Login" button. The terminal encrypts this information and sends it to the server.

[1836] Step 2:

[1837] The server receives the login information. The server searches the database for the corresponding user information and compares it with the received information. If authentication is successful, the server creates a user session and returns a session ID to the terminal.

[1838] Step 3:

[1839] The device saves the session ID and notifies the user that they have successfully logged in. The device then displays the home screen, allowing the user to select their next action.

[1840] Step 4:

[1841] The user selects a sporting event to watch. The device displays a list of sporting events available for viewing to the user, and the user selects the event of their choice.

[1842] Step 5:

[1843] The user sets up the viewing settings. The user inputs viewing settings such as focusing on a specific player or increasing explanations of basic rules. Once these settings are complete, the device sends the setting information to the server.

[1844] Step 6:

[1845] The server receives and analyzes the viewing preferences, generating a unique viewing profile for each user based on their past viewing history, preferred commentary style, and feedback data.

[1846] Step 7:

[1847] The server collects relevant data, such as player information, past performance data, interview information, and terminology explanations.

[1848] Step 8:

[1849] The server generates a customized commentary scenario. The server creates a commentary scenario based on the collected data and in accordance with the user's settings.

[1850] Step 9:

[1851] The terminal displays the live streaming video of the game. As the game starts, the terminal receives the live streaming video and displays it to the user.

[1852] Step 10:

[1853] The device provides customized commentary in real time. The customized commentary data received from the server is synchronized with the game footage and displayed to the user in real time.

[1854] Step 11:

[1855] The emotion engine recognizes the user's emotional state. The emotion engine installed in the device analyzes the user's facial expressions, voice, and operation patterns to recognize the user's emotional state in real time. The recognition results are sent to the server.

[1856] Step 12:

[1857] The server adjusts the commentary based on the emotion data. Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is excited, the server may add more detailed tactical commentary.

[1858] Step 13:

[1859] The user inputs a question while watching the game. The terminal provides the user with an interface for inputting a question, and the user inputs any points of interest. The question is then sent from the terminal to the server.

[1860] Step 14:

[1861] The server receives and analyzes the user's question, using natural language processing to identify intent and interests.

[1862] Step 15:

[1863] The server searches for relevant data, such as past match data, related player data, and explanations of technical terms, based on the analysis results.

[1864] Step 16:

[1865] The server generates an appropriate explanation and sends it to the device.The server generates explanatory text, videos, and images based on the search results and sends them to the device.

[1866] Step 17:

[1867] The device receives and displays the commentary. The commentary received from the server is displayed on the screen in real time, allowing the user to continue watching the game.

[1868] This allows users to enjoy a personalized sports viewing experience in real time, and by incorporating emotion recognition functionality using an emotion engine, it is possible to provide even more interactive commentary that responds to the user's emotions.

[1869] Example 2

[1870] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1871] Existing sports viewing systems have difficulty providing real-time commentary that reflects individual user preferences and emotional states. Furthermore, few systems can respond quickly and appropriately to user questions. Therefore, providing users with an interactive and personalized viewing experience is a challenge.

[1872] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1873] In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's viewing preferences; means for transmitting the input preference information to the server; means for the server to generate a user-specific viewing profile based on the received preference information; means for the server to collect relevant data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state; means for dynamically adjusting the commentary scenario based on the emotion data; and means for the terminal to display the commentary, thereby enabling an interactive, emotionally-sensitive, and personalized sports viewing experience for the user.

[1874] "User" refers to an individual who uses the system to watch a sporting event.

[1875] "Spectator Settings" refers to information that allows a user to input a particular commentary style or focus point desired when watching a sporting event.

[1876] "Terminal" refers to a device operated by a user (e.g., a smartphone, tablet, computer, etc.).

[1877] "Server" refers to a central processing unit that receives viewing setting information and emotional data from users and generates viewing profiles and commentary scenarios based on that information.

[1878] A "watching profile" refers to user-specific data including the user's past viewing history, preferred commentary style, feedback, etc.

[1879] "Explanatory scenario" refers to a set of customized expository content generated by the server.

[1880] "Live streaming" refers to game footage that is broadcast in real time over the Internet.

[1881] "Emotion engine" refers to software or hardware that has the functionality to recognize the emotional state of a user.

[1882] "Emotion data" refers to data that represents the user's emotional state (for example, surprise, joy, anxiety, etc.).

[1883] "Question" refers to the question or information that a user wants to know that they input into the system while watching a game.

[1884] MODE FOR CARRYING OUT THE INVENTION

[1885] The system of the present invention is designed to personalize the viewing of sporting events and provide users with a richer viewing experience in real time. The main components of the system are a user terminal, a server, and a communication network.

[1886] The user terminals include devices such as smartphones, tablets, and computers, and are used to input viewing settings, watch live streaming, input questions, and collect emotional data. The terminals are also equipped with an emotion engine that recognizes emotions in real time through analysis of the user's facial expressions and voice. This emotional data is sent to a server. Specifically, facial expression recognition software and a voice analysis engine may be used.

[1887] The server receives the viewing setting information sent by the user and generates a viewing profile specific to the user based on that information. The profile includes the user's past viewing history, preferred commentary style, feedback, etc. The server also collects related data from the Internet (player information, past performance data, interview information, etc.) and generates a customized commentary scenario based on that data. When the server receives a question from the user, it analyzes the question using a natural language processing engine, generates appropriate commentary, and sends it to the user's device. Specifically, a natural language processing library and database are used.

[1888] Once the match begins, the device displays live streaming footage and displays customized commentary received from the server in real time. This commentary is dynamically updated as the match progresses and adjusts based on the user's excitement level and other emotional states. For example, if the user expresses surprise, the server can add more detailed tactical analysis to the commentary. The device can use existing live streaming services such as YouTube and Dailymotion.

[1889] Specific examples

[1890] 1. The user sets up the viewing settings

[1891] A user logs in to the system on their smartphone and selects a soccer match. The user enters settings such as "Focus on Player X," "More explanations of the basic rules," and "Show player interviews," and sends the settings information to the server.

[1892] 2. The server generates a profile

[1893] The server analyzes the received viewing setting information and generates a viewing profile based on the user's past viewing history and preferred commentary style. For example, the server collects information about player X and prepares a commentary scenario to be displayed during the match.

[1894] 3. The device displays the live stream.

[1895] The device uses services such as YouTube to display live streaming of the game, and simultaneously displays commentary received from the server. When Player X has the ball, information such as "Player X's special technique" is added to the commentary in real time.

[1896] 4. The user enters a question

[1897] During a game, a user types a question into their device, such as "I want to know about player X's scoring ability." The server analyzes the question using natural language processing, searches for relevant data, and generates an explanation. The explanation is then sent to the device and displayed to the user.

[1898] 5. Emotion engine recognizes the user's emotional state

[1899] The emotion engine analyzes the user's facial expressions and voice to recognize, for example, whether the user is happy, and sends that data to the server.

[1900] 6. The server adjusts the commentary based on the emotion data.

[1901] The server analyzes the emotional data and, for example, if the user is excited, adds details to the commentary such as "These are the tactics behind this play."

[1902] Prompt Sentence Examples

[1903] "Please explain in detail the processing steps of this sports viewing system program. Please break down the processing steps into smaller steps and describe each step, including the specific actions that are taken."

[1904] As a result, this system can provide users with an interactive, emotionally-driven, and personalized sports viewing experience.

[1905] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1906] Step 1:

[1907] The user selects the event to watch and enters the settings.

[1908] Input: A user operates a terminal to log in to the system, selects a sporting event to watch, and then inputs viewing settings (e.g., "Focus on player X," "More explanations of the basic rules," "Show player interviews").

[1909] Data processing: The device compiles the entered viewing settings information.

[1910] Output: Spectator settings information is sent from the device to the server.

[1911] Specific actions: The user uses their smartphone to enter viewing information into the designated fields in the app and presses the "Settings complete" button.

[1912] Step 2:

[1913] The server processes the user's settings and generates a viewing profile.

[1914] Input: Spectator settings information sent from the device.

[1915] Data processing: The server analyzes the received configuration information and updates the user's unique viewing profile, which includes the user's past viewing history, preferred commentary style, and feedback data.

[1916] Output: The updated spectator profile.

[1917] Specific operation: The server generates a new viewing profile based on the user's viewing history and feedback information stored in the database.

[1918] Step 3:

[1919] The server collects relevant data and generates a customized commentary scenario.

[1920] Input: The user's watching profile.

[1921] Data processing: The server collects relevant data from the Internet (player information, past performance data, interview information, etc.) and generates customized commentary scenarios based on the viewing profile.

[1922] Output: A customized commentary scenario.

[1923] Specific operation: The server uses a Web API to obtain the latest player information and past match data, and inputs this information into the commentary scenario generation algorithm to generate a scenario.

[1924] Step 4:

[1925] The device will display live streaming of the match and provide customized commentary

[1926] Input: A customized commentary script received from the server, and live streaming footage.

[1927] Data processing: The device displays live streaming video while overlaying explanatory information sent from the server at any time.

[1928] Output: Live streaming video and real-time commentary.

[1929] Specific operation: The device uses the live streaming APIs of YouTube and Dailymotion to play game footage while displaying commentary text on the screen in real time.

[1930] Step 5:

[1931] Recognizing the user's emotional state

[1932] Input: User's facial expressions, voice, and operation patterns.

[1933] Data processing: The emotion engine installed in the device analyzes this data and recognizes the user's emotional state.

[1934] Output: Emotion data (e.g. surprise, joy, anxiety, etc.).

[1935] Specific operation: The user's facial expressions and voice are captured in real time through the device's camera and microphone, and emotions are analyzed using facial recognition software and a voice analysis engine.

[1936] Step 6:

[1937] The server adjusts the commentary based on the emotional data.

[1938] Input: User emotional state data.

[1939] Data processing: The server dynamically adjusts the commentary scenario based on the received emotional data.

[1940] Output: A tailored expository scenario.

[1941] Specific actions: For example, if a user is surprised, the server will add a detailed analysis to the commentary, such as "These are the tactics behind this play."

[1942] Step 7:

[1943] Users can enter questions while watching

[1944] Input: User question (e.g., "What is Player X's scoring stats?").

[1945] Data processing: The device sends the query to the server.

[1946] Output: The question sent to the server.

[1947] Specific actions: The user enters a question into the input field on the terminal and presses the send button.

[1948] Step 8:

[1949] The server analyzes the received questions, generates explanations in real time, and sends them to the device.

[1950] Input: The user's question.

[1951] Data processing: The server uses a natural language processing engine to analyze the question, search for relevant data, and generate an appropriate explanation.

[1952] Output: The generated commentary.

[1953] Specific operation: The server analyzes the user's question, extracts relevant information from the database, generates an explanation, and sends it to the terminal.

[1954] (Application example 2)

[1955] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1956] Current sports viewing applications lack the ability to personalize commentary based on users' emotions and preferences, resulting in a uniform viewing experience. Furthermore, there is a lack of technology that can recognize users' emotional state in real time and dynamically adjust commentary accordingly. Therefore, there is a need for a more interactive experience.

[1957] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for a user to select a sport event to watch; means for inputting the user's watching settings; means for transmitting the input setting information to the server; means for the server to generate a user-specific watching profile based on the setting information received; means for the server to collect related data and generate a customized commentary scenario; means for the terminal to display live streaming of the game and provide customized commentary in real time; means for the user to input a question while watching the game; means for the server to analyze the received question, generate commentary in real time, and transmit it to the terminal; means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the user's emotional state; means for the terminal to collect user emotion data using an emotion engine; and means for the terminal to display commentary. This enables a personalized, interactive, and emotion-responsive sports watching experience for the user.

[1958] Key Word Definitions

[1959] The "means for the user to select a sporting event to watch" refers to a device or software that provides an interface for the user to select any sporting event from among a plurality of sporting events.

[1960] The "means for inputting user viewing settings" refers to a device or software that provides an interface for the user to input viewing settings such as a particular commentary style or focus points.

[1961] The "means for transmitting input setting information to a server" refers to a device or software that transmits the watching setting information input by the user to a server via a communication network such as the Internet.

[1962] "Means for generating a user-specific viewing profile based on the setting information received by the server" refers to a device or software that analyzes the viewing setting information received from the user by the server and generates a viewing profile based on the user's interests and preferences.

[1963] "Means for the server to collect relevant data and generate customized commentary scenarios" refers to devices or software that allow the server to collect relevant data such as player information and past performance data based on a user's unique viewing profile and use that data to generate customized commentary scenarios.

[1964] "Means for a terminal to display live streaming of a match and provide customized commentary in real time" refers to a device or software that allows a terminal to display live video of a match received via the Internet and also displays customized commentary received from a server in real time.

[1965] The "means for users to input questions while watching" refers to a device or software that provides an interface that allows users watching the game to input questions they have in real time.

[1966] "Means for analyzing questions received by the server, generating explanations in real time, and transmitting them to the terminal" refers to a device or software that uses natural language processing technology to analyze questions received from users by the server, generates appropriate explanations based on the analysis, and transmits them to the terminal in real time.

[1967] "Means for recognizing the user's emotional state and dynamically adjusting the commentary scenario based on the emotional state" refers to a device or software in which the terminal analyzes the user's facial expressions, voice, etc., recognizes the user's emotional state, and then dynamically adjusts the content of the commentary according to that emotional state.

[1968] "Means for a terminal to collect user emotional data using an emotion engine" refers to devices or software that allow devices such as smartphones and tablets to collect user emotional data using algorithms and sensors called emotion engines.

[1969] The "means for the terminal to display the commentary" refers to a device or software that allows the terminal to visually or audibly display the commentary generated in real time to the user.

[1970] "Form for carrying out the invention" of the specification

[1971] The system of the present invention allows users to enjoy a personalized sporting event. The system features an emotion engine that recognizes the user's emotional state and provides adaptive commentary in real time. The specific configuration for realizing this system is shown below.

[1972] Hardware and software used

[1973] This system consists of the following hardware and software:

[1974] Hardware: User devices (smartphones, tablets, computers, head-mounted displays (HMDs), etc.)

[1975] Software: Flask (Python framework), OpenCV (image processing library), Dlib (face detection and landmark prediction library), natural language processing algorithm, emotion engine

[1976] Data processing and calculation

[1977] 1. Select the event you want to watch and enter your settings:

[1978] Users log in to the system using their terminals, select the sporting event they want to watch, and enter viewing preferences such as focus points and commentary style, which are then sent from the terminal to the server.

[1979] 2. The server processes the user's preferences and generates a viewing profile:

[1980] The server generates a user-specific viewing profile based on the settings received from the user, which includes the user's past viewing history, preferred commentary style, feedback data, and so on.

[1981] 3. The server collects the relevant data and generates a customized commentary scenario:

[1982] Based on the viewing profile, the server collects relevant data (player information, past performance data, interview information, explanations of technical terms, etc.) and generates a customized commentary scenario specific to the user.

[1983] 4. The device displays live streaming of the match and provides customized commentary in real time:

[1984] Once the match begins, the device displays live streaming footage and simultaneously displays customized commentary received from the server in real time, dynamically updating based on player movements and key plays.

[1985] 5. Recognize the user's emotional state:

[1986] The emotion engine analyzes the user's facial expressions, voice, and operation patterns in real time. For example, if the user's emotional state, such as excitement or surprise, is detected, the data is sent to the server and used to adjust the commentary content.

[1987] 6. The server adjusts the commentary based on the emotion data:

[1988] Based on the data received from the emotion engine, the server dynamically adjusts the commentary scenario according to the user's emotional state. For example, if the user is surprised, it can add detailed tactical analysis to the commentary.

[1989] Specific examples

[1990] Front-end: The user selects a basketball game and sets preferences such as "focus on player X," "show more explanation of the basic rules," and "display player interviews."

[1991] Backend: The server receives this configuration information and generates a customized commentary scenario based on Player X's profile data and related data.

[1992] Live Operation: During the match, the device provides live video of the match in real time and displays specialized commentary received from the server. The user's emotional state is analyzed and the commentary content is dynamically adjusted as needed.

[1993] Prompt Sentence Examples

[1994] "Tell me the story behind player X's scoring ability in a basketball game. The user is currently in an emotional state of surprise. Please provide commentary that is appropriate to this emotion."

[1995] This allows for personalized commentary based on the user's emotions, resulting in a more fulfilling viewing experience.

[1996] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1997] Specific processing flow of the program

[1998] Step 1:

[1999] A user selects a sporting event to watch and enters viewing preferences.

[2000] input:

[2001] An interface for users to select sporting events

[2002] Spectator settings (e.g., focus on specific players, commentary style, interview data to display)

[2003] output:

[2004] Entered sporting event information and viewing preferences

[2005] Specific operation:

[2006] Users log into the application on their smartphone or tablet and select the sporting event they want to watch.

[2007] Operates a form where users enter viewing preferences (player focus, commentary style, interviews, etc.).

[2008] This information is compiled into a single JSON object on the terminal side.

[2009] Step 2:

[2010] The entered setting information is sent to the server.

[2011] input:

[2012] Spectator settings information from the user (JSON object)

[2013] output:

[2014] Spectator settings sent to the server

[2015] Specific operation:

[2016] The device sends the user's viewing settings information to the server via the Internet using HTTPS as the communication protocol.

[2017] Step 3:

[2018] The server generates a user-specific game watching profile based on the received setting information.

[2019] input:

[2020] Spectator settings information

[2021] User's past viewing history, preferred commentary style, and feedback data

[2022] output:

[2023] User-specific viewing profiles

[2024] Specific operation:

[2025] The server analyzes the viewing setting information and updates the user's unique profile using a viewing profile generation algorithm.

[2026] A viewing profile includes a user's past viewing history, preferred commentary style, and feedback data.

[2027] Step 4:

[2028] The server collects the relevant data and generates a customized expository scenario.

[2029] input:

[2030] Spectator Profile

[2031] output:

[2032] Customized commentary scenarios

[2033] Specific operation:

[2034] The server collects relevant data such as player information, past performance data, interview information, and explanations of technical terms based on the user's viewing profile.

[2035] Based on the collected data, a customized commentary scenario specific to the user is generated.

[2036] Step 5:

[2037] The device displays live streaming of the match and provides customized commentary in real time.

[2038] input:

[2039] Live streaming video

[2040] Customized commentary scenarios

[2041] output:

[2042] Live streaming video and commentary scenario displayed to users

[2043] Specific operation:

[2044] Once the match begins, the device will display live streaming footage.

[2045] Customized commentary received from the server is displayed in real time as an overlay on the live video.

[2046] Step 6:

[2047] Recognize the user's emotional state.

[2048] input:

[2049] User facial expression data, voice data, operation patterns

[2050] output:

[2051] User emotional state data

[2052] Specific operation:

[2053] The device's built-in camera and microphone collect the user's facial expression and voice data.

[2054] The emotion engine analyzes this data in real time to recognize the user's emotional state.

[2055] Step 7:

[2056] The server adjusts the commentary based on the emotional data.

[2057] input:

[2058] User emotional state data

[2059] Customized commentary scenarios

[2060] output:

[2061] Updated commentary scenarios according to user emotions

[2062] Specific operation:

[2063] The server analyzes the emotional state data received from the emotion engine and dynamically adjusts the commentary scenario.

[2064] For example, if the user is excited, add tactical explanations, or if the user is surprised, provide a detailed explanation of external factors.

[2065] Step 8:

[2066] The device displays an explanation.

[2067] input:

[2068] Updated commentary scenario

[2069] output:

[2070] Updated explanatory scenarios displayed to users

[2071] Specific operation:

[2072] The terminal receives the updated commentary scenario sent from the server and displays it as an overlay on the live video.

[2073] This provides a personalized sports viewing experience that is tailored to the user's emotional state.

[2074] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2075] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2076] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2077] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2078] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2079] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2080] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2081] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2082] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2083] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2084] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2085] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2086] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2087] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2088] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2089] The hardware resource for executing a speci...

Claims

1. means for a user to select a sporting event to watch; means for inputting a user's viewing preferences; means for transmitting the input setting information to a server; means for generating a user-specific viewing profile based on the setting information received by the server; a means for the server to collect relevant data and generate a customized commentary scenario; A means for the device to display live streaming of the match and provide customized commentary in real time; a means for a user to input a question while watching; A means for the server to analyze the received question, generate an explanation in real time, and transmit the explanation to the terminal; a means for the device to display a description; A system including:

2. 2. The system of claim 1, wherein the server includes means for analyzing a user's viewing history, preferred commentary style, and feedback data and updating the viewing profile.

3. 2. The system according to claim 1, wherein the server includes means for analyzing the user's question using natural language processing, searching related data, and generating an appropriate explanation.

Images