System

Abstract

To provide a system.SOLUTION: A system includes means for receiving video data and audio data of a match, means for analyzing the received video data; means for analyzing the received audio data, means for extracting scoring situations and foul situations from results of analysis of the video data and the audio data, means for generating a running score based on the extracted scoring situations and foul situations, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing a user's emotions and adjusting a display of the running score based on the user's emotions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When live commentary for basketball games is done by humans, it is difficult to cover youth and mini-basketball games across the country. It is also difficult to automatically generate running scores, which are created manually by officials and for each team. Furthermore, it is difficult to visualize Japan's registered basketball players by creating a database of teams and players across the country and use it for training. [Means for solving the problem]

[0005] We provide a means to use AI to provide commentary for live broadcasts of basketball games. We provide a means to broadcast commentary for youth and mini-basketball games across the country with AI commentary. We provide a means to automatically generate running scores created by humans for each official and team. We provide a means to create a database of teams and players across the country, and a means to use the information in the database to visualize and develop registered basketball players in Japan. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] A first embodiment of the present invention is a system in which AI provides commentary for live coverage of a basketball game. In this system, AI analyzes video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, it automatically generates commentary for the game and distributes it along with the live coverage.

[0029] "Example 2"

[0030] A second embodiment of the present invention is a system that distributes commentary on youth and mini-basketball games from around the country, accompanied by live coverage by AI. This system collects video and audio data from youth and mini-basketball games held across the country, and, as in the first embodiment, AI analyzes and generates commentary. Then, by distributing this commentary together with the game video, games from around the country can be provided to a wide audience.

[0031] "Example 3"

[0032] A third embodiment of the present invention is a system that uses AI to automatically generate running scores that are currently created manually by officials or teams. In this system, AI analyzes the score and foul situation from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

[0033] "Example 4"

[0034] A fourth embodiment of the present invention is a system that creates a database of teams and players nationwide and utilizes the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats of teams and players nationwide and creates a database. The database is then used to analyze player performance and provide feedback for development.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: AI collects video and audio data from basketball games.

[0038] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[0039] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[0040] Step 4: The generated commentary is distributed along with the live broadcast.

[0041] "Example 2"

[0042] Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

[0043] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[0044] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[0045] Step 4: The generated commentary is distributed along with the game footage.

[0046] "Example 3"

[0047] Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

[0048] Step 2: Based on the analysis results, the AI ​​automatically generates a running score.

[0049] "Example 4"

[0050] Step 1: Collect information such as game results and stats for teams and players across the country.

[0051] Step 2: Create a database of the collected information.

[0052] Step 3: Use the databased information to analyze player performance and provide feedback for development.

[0053] Example 1

[0054] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0055] Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. Furthermore, providing commentary for youth and mini-basketball games across the country requires a large number of people, which is costly and labor-intensive. Furthermore, running scores created by officials and by each team are done manually, which is time-consuming and can lack accuracy. To solve these issues, an automated system utilizing AI is needed.

[0056] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0057] In this invention, the server includes a means for collecting video and audio data of the game in real time, a means for analyzing the collected video and audio data, a means for generating commentary on the game based on the analysis results, and a means for distributing the generated commentary along with a live broadcast. This enables accurate real-time reporting of detailed game situations and player movements. Furthermore, commentary on youth and mini-basketball games across the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores, which are currently created manually by officials and for each team, using AI can reduce time and improve accuracy.

[0058] "Game video data" refers to digital data that visually records the progress of a basketball game.

[0059] "Audio data" refers to digitally recorded data of sounds occurring during a basketball game.

[0060] "Means of collecting in real time" refers to technology and devices for acquiring video and audio data as the match progresses.

[0061] "Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

[0062] "Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of the players based on the analysis results.

[0063] "Means for distribution along with live coverage" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

[0064] "Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

[0065] "Running score" is data that records the scores and player performances that are updated in real time during the game.

[0066] "AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

[0067] "Means of database creation" refers to the technology and software used to organize collected information and store it in a format that is easy to search and use.

[0068] "Visualization" refers to techniques and methods for visually displaying data to make it easier to understand.

[0069] "Means for use in development" refers to technologies and methods that use data to support the training and development of athletes.

[0070] This invention is a system that uses AI to automatically generate commentary for live broadcasts of basketball games and delivers it in real time. Specific embodiments of this system are described below.

[0071] Server Operation

[0072] 1. Data Collection

[0073] The server collects video and audio data of the match in real time using a camera and microphone to capture the video and audio of the match, with the video data collected at a resolution of 1080p and the audio data collected at a sampling rate of 44.1kHz.

[0074] 2. Data Analysis

[0075] The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g., OpenCV) and voice recognition software (e.g., Google® Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the commentary audio at that moment into text.

[0076] 3. Explanation Generation

[0077] The server generates commentary on the game based on the analysis results. A generative AI model (e.g., GPT-4 (registered trademark)) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, if the information "Player A made a successful three-point shot" is input, the server generates commentary such as "Player A made a brilliant three-point shot."

[0078] 4. Distribution

[0079] The server distributes the generated commentary along with the live broadcast. A streaming server (e.g., Wowza Streaming Engine) is used for distribution, allowing users to watch the game footage and commentary in real time via the Internet.

[0080] Device behavior

[0081] 1. Data Reception

[0082] The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g., VLC Media Player).

[0083] 2.Display

[0084] The device displays the received video and commentary to the user. This is done using hardware such as a display and speakers. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played from the built-in speaker.

[0085] User Actions

[0086] 1. Viewing

[0087] Users can watch game footage and commentary through their devices. Users can receive detailed commentary on the progress of the game and the movements of the players in real time. For example, if a user is watching on a television in their living room, the game footage will be displayed on the TV screen and commentary will be played from the TV speakers.

[0088] Specific examples

[0089] Example 1

[0090] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[0091] Player Action: Home team player A makes a three-point shot.

[0092] Example prompts for generative AI models

[0093] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[0094] Player Action: Home team player A makes a three-point shot.

[0095] Example of generated commentary

[0096] "With five minutes left in the third quarter, the home team is tied at 75 and the away team is tied at 70. Player A on the home team makes a brilliant three-point shot, widening the gap."

[0097] In this way, users can view detailed commentary on the progress of the match and the movements of the players in real time.

[0098] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0099] Step 1:

[0100] The server collects video and audio data of the match in real time.

[0101] Specifically, it uses cameras and microphones to capture video and audio of the game.

[0102] The input is video and audio from a camera and microphone, and the output is video and audio data in digital format.

[0103] Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

[0104] Step 2:

[0105] The server analyzes the collected video and audio data.

[0106] Specifically, image recognition software (e.g., OpenCV) is used to detect player movements from video data, and voice recognition software (e.g., Google Speech-to-Text API) is used to convert the audio data into text to provide commentary.

[0107] The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

[0108] For example, the system detects when player A takes a shot and converts the live audio at that moment into text.

[0109] Step 3:

[0110] The server generates commentary on the match based on the analysis results.

[0111] Specifically, a prompt sentence is input into a generative AI model (e.g., GPT-4) to generate an explanatory sentence.

[0112] The input is the player movement information obtained in step 2 and the commentary converted into text, and the output is the generated commentary.

[0113] For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

[0114] Step 4:

[0115] The server distributes the generated commentary along with the live broadcast.

[0116] Specifically, a streaming server (e.g., Wowza Streaming Engine) is used to stream video and commentary audio in real time.

[0117] The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and audio commentary delivered in real time.

[0118] Step 5:

[0119] The device receives real-time game footage and commentary streamed from the server.

[0120] Specifically, it requires an internet connection and streaming software (e.g., VLC media player).

[0121] The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played back.

[0122] Step 6:

[0123] The terminal displays the received video and commentary to the user.

[0124] Specifically, the system displays video on a screen and plays explanatory audio from a speaker.

[0125] The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user visually and audibly.

[0126] Step 7:

[0127] Users can watch the game footage and commentary through their devices.

[0128] Specifically, the user watches the game footage on the device's display while listening to commentary coming from the speaker.

[0129] The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

[0130] For example, when a user is watching on a television in their living room, the video of the game is displayed on the television screen and commentary audio is played from the television speakers.

[0131] (Application example 1)

[0132] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0133] Conventional basketball game commentary systems lack real-time commentary, post-game highlight generation, and detailed analysis of player performance. Furthermore, they face challenges in providing accurate commentary immediately when users ask questions about specific players or plays.

[0134] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0135] In this invention, the server includes means for AI to provide commentary for live broadcasts of basketball games, means for distributing commentary for youth and mini-basketball games across the country with AI commentary, means for AI to automatically generate running scores that are created manually for each official and team, means for analyzing game video data and generating commentary in real time, means for automatically extracting important scenes after the game ends and generating a highlight video, means for analyzing player movements and performance and providing detailed statistical information, and means for AI to provide commentary when a user asks a question about a specific player or play. This enables real-time commentary, post-game highlight generation, detailed analysis of player performance, and immediate commentary in response to user questions.

[0136] "A means for AI to provide commentary for live broadcasts of basketball games" refers to a technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

[0137] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

[0138] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

[0139] "Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

[0140] "Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after it has ended, automatically extracts important scenes, and generates a highlight video.

[0141] "Means for analyzing player movements and performance and providing detailed statistical information" refers to technology for analyzing player movements and performance and providing detailed statistical information based on the results of the analysis.

[0142] "A means for AI to provide commentary when a user asks a question about a specific player or play" is a technology in which AI provides appropriate commentary in response to a question when a user asks a question about a specific player or play.

[0143] To implement the present invention, the following system configuration and processing procedures are required.

[0144] System Configuration

[0145] Hardware

[0146] Server: A high-performance computer for running AI models and analyzing data.

[0147] Smartphone: The device through which users view and interact with the match commentary.

[0148] Camera: A device used to capture footage of the match.

[0149] software

[0150] OpenCV: A library used to read and analyze video data.

[0151] TENSORFLOW(R): A framework used to run AI models.

[0152] Transformers: A library that generates explanations using the GPT-2 model.

[0153] Processing flow

[0154] 1. Loading video data

[0155] The server uses OpenCV to read the game video data obtained from the camera frame by frame.

[0156] 2. Frame Analysis

[0157] The server analyzes the loaded frames to detect player movements and important actions, using a deep learning model powered by TensorFlow.

[0158] 3. Generating Explanations

[0159] The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

[0160] 4. Streaming commentary

[0161] The generated commentary is delivered to smartphones in real time, allowing users to view and listen to the commentary on the match.

[0162] 5. Highlight video generation

[0163] After the match ends, the server automatically extracts important scenes and generates a highlight video, which is also delivered to smartphones.

[0164] 6. Player performance analysis

[0165] The server analyzes the players' movements and performance in detail and generates statistical information that users can view on their smartphones.

[0166] 7. User Interaction

[0167] When a user asks a question about a specific player or play, the server generates an appropriate commentary for the question and delivers it to the smartphone.

[0168] Specific examples

[0169] Scoring scene in the game: Detects the moment when Player A makes a three-point shot and generates a commentary such as "Player A scores a three-pointer."

[0170] Defensive highlights: Detects when Player B successfully blocks a shot and generates commentary such as "Player B makes a great block."

[0171] Prompt Sentence Examples

[0172] "Player A scores a three-pointer. Generate a commentary for this action."

[0173] "Player B makes a great block. Generate a commentary for this action."

[0174] In this way, the server can analyze the match video data in real time and automatically provide commentary to users, enabling real-time commentary, post-match highlights, detailed analysis of players' performance, and instant commentary in response to user questions.

[0175] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0176] Step 1:

[0177] The server uses OpenCV to read the video data of the game captured by the camera, frame by frame. The input is the video data from the camera, and the output is individual frame images. Specifically, the server divides the video data into a series of still images and stores each frame in memory.

[0178] Step 2:

[0179] The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow. The input is the frame image obtained in step 1, and the output is information about player movements and actions. Specifically, the server detects the position and movement of players within the frame and identifies important actions such as points and blocks.

[0180] Step 3:

[0181] The server generates commentary based on the analysis results. Specifically, it runs a GPT-2 model using the Transformers library and generates commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain commentary.

[0182] Step 4:

[0183] The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server sends the commentary to the smartphone via the network so that the user can view it.

[0184] Step 5:

[0185] After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

[0186] Step 6:

[0187] The server analyzes the players' movements and performance in detail and generates statistical information. The input is frame images from the game and information about the players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it into statistical information.

[0188] Step 7:

[0189] When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt, and the output is the generated commentary. Specifically, the server analyzes the user's question, generates a corresponding prompt, inputs it into the GPT-2 model, obtains the commentary, and sends it to the smartphone.

[0190] Example 2

[0191] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] Traditional basketball game commentary requires specialized commentators, and securing commentators for youth and mini-basketball games across the country has been a challenge. It's also difficult to communicate game progress and player movements to viewers in real time, limiting the viewing experience. Furthermore, game data collection and analysis is done manually, resulting in inefficiencies and a lack of data consistency and accuracy.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0194] In this invention, the server includes means for collecting video and audio data of games held nationwide, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model to generate game commentary, means for synchronizing the generated commentary with the game video, means for streaming the game video with commentary, means for providing an interface for users to view, means for receiving the game video with commentary streamed from the server, and means for displaying the received video to users. This enables live broadcasts with high-quality commentary by AI, even for youth and mini-basketball games held nationwide, improving the viewing experience. Furthermore, the automation of game data collection and analysis enables efficient and accurate data management.

[0195] "Means for collecting video and audio data from matches held across the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

[0196] "Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

[0197] "Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompts to a generative AI model to analyze the data, and generates commentary on a match.

[0198] "Means for synchronizing generated commentary with game footage" refers to a device or system that inserts generated commentary text at specific timestamps in game footage, allowing the footage and commentary to be played in a synchronized state.

[0199] A "means for streaming commentary-enhanced game footage" is a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

[0200] "Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through devices such as smartphones, tablets, and PCs.

[0201] The "means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

[0202] The "means for displaying the received video to the user" refers to a device or system for decoding the received video data and displaying it on the screen of the user's device.

[0203] This invention is a system that broadcasts youth and mini-basketball matches held across the country with AI commentary. A specific embodiment of this system is described below.

[0204] Server Roles

[0205] The server collects video and audio data from matches held across the country. Cameras and microphones installed at match venues capture video and audio of matches in real time and send this data to the server. The collected data is then stored in cloud storage such as Amazon S3 and Google Cloud Storage.

[0206] Next, the server sends the saved data to a generative AI model. Examples of generative AI models used include OpenAI's (registered trademark) GPT-4 and Google's BERT. The server retrieves the data from cloud storage and sends prompts to the generative AI model. The prompts include instructions for analyzing the progress of the game, player movements, score status, and so on.

[0207] The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, ensuring that the footage and commentary are synchronized when played.

[0208] Finally, the server streams the match footage with commentary to a streaming platform such as YouTube® Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

[0209] Device Role

[0210] The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

[0211] The device receives game video with commentary streamed from the server. This requires an internet connection. The device receives video data from the streaming platform in real time, buffers it, decodes the received video data, and displays it on the user's device screen.

[0212] User Roles

[0213] Users access the system using a terminal. To access the system, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

[0214] Users select the match they want to watch. They are provided with a list of matches and a search function. Users can scroll through the list of matches on the interface and click on the match they want to watch. They can also use the search function to find a specific match.

[0215] The user watches the commentary video of the selected match. While watching, the user can perform operations such as play, pause, and rewind. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

[0216] Specific examples

[0217] For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they might take the following steps:

[0218] 1. The user launches the dedicated app on their smartphone.

[0219] 2. Use the in-app search function and type in "National Youth Basketball Tournament."

[0220] 3. Select the match you want to watch from the search results.

[0221] 4. The app streams game footage with commentary from the server.

[0222] 5. Users can watch the game on their smartphone screen and enjoy the commentary.

[0223] An example of a prompt to be input to the generative AI model is, "Analyze the game video and audio data from the national youth basketball tournament and generate commentary that includes the progress of the game, player movements, score status, etc." This prompt is used to have the AI ​​analysis system generate commentary for the game.

[0224] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0225] Step 1:

[0226] The server collects video and audio data from games held across the country. As input, it receives real-time data from cameras and microphones installed at game venues around the country. Specifically, it sends the video and audio captured by these devices to the server via the Internet. As output, it obtains the collected video and audio data.

[0227] Step 2:

[0228] The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

[0229] Step 3:

[0230] The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends prompts to the generative AI model to analyze the data. The prompts include instructions for analyzing the progress of the match, player movements, and the score. The output is commentary text returned by the generative AI model.

[0231] Step 4:

[0232] The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

[0233] Step 5:

[0234] The server streams the game video with commentary. As input, it receives the game video synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and sends it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to viewers.

[0235] Step 6:

[0236] The terminal provides an interface for users to watch games. It receives user operations as input. Specific operations include displaying the viewing interface through a dedicated application or web browser. The interface includes a list of games and a search function. The output provides information that allows users to select the game they want to watch.

[0237] Step 7:

[0238] The device receives game video with commentary streamed from the server. The input is the streaming data from the server. Specifically, the device receives the video data in real time via an Internet connection and buffers it. The output is the received video data.

[0239] Step 8:

[0240] The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

[0241] Step 9:

[0242] Users access the system using a terminal. As input, they use a dedicated application or a web browser. Specifically, they launch a dedicated application on a smartphone or PC, or access the system's URL in a web browser. As output, they gain access to the system.

[0243] Step 10:

[0244] The user selects the match they want to watch. The input is from the list of matches or the search function on the viewing interface. The specific action is to scroll through the list of matches and click on the match they want to watch. They can also use the search function to search for a specific match. The output is the selected match.

[0245] Step 11:

[0246] The user watches the commentary video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

[0247] (Application example 2)

[0248] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Conventional basketball game commentary systems rely on manual broadcasting and commentary, making it difficult to provide a wide audience with information on youth and mini-basketball games held across the country. Furthermore, they lacked game highlights and replay features, as well as post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, they lacked the ability for users to follow their favorite teams and players, making it difficult to meet individual needs.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for AI to provide commentary on live broadcasts of basketball games, means for distributing commentary on youth and mini-basketball games across the country with AI commentary, means for AI to automatically generate running scores that are created manually by officials and for each team, means for collecting video and audio data of the games and analyzing it with AI to generate commentary, means for distributing games in real time and providing commentary by AI through an application installed on a smartphone, means for providing game highlights and replay functions, means for enabling users to follow their favorite teams and players, and means for providing post-game statistical data and analysis reports. This makes it possible to widely provide youth and mini-basketball games held across the country to viewers and increase viewer satisfaction.

[0251] "A means for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

[0252] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that broadcasts youth and mini-basketball games held all over the country in real time with commentary by artificial intelligence.

[0253] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[0254] "Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to a technology that collects video and audio data from the match and analyzes it with artificial intelligence to generate commentary.

[0255] "Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

[0256] "Means for providing match highlight and replay functions" refers to technology that provides a function for compiling important scenes and plays during a match as highlights or playing them as replays.

[0257] "Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

[0258] "Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match has ended and provides them to users.

[0259] The following system configuration will be described as an embodiment of the present invention.

[0260] System Configuration

[0261] The system includes a server, a smartphone, and an AI model. The server collects video and audio data from matches, analyzes it using the AI ​​model, and generates commentary. The smartphone provides an application that allows users to watch matches in real time and receive commentary from the AI.

[0262] Hardware and software used

[0263] Server: A server with high-performance data processing capabilities (e.g., AWS (registered trademark) EC2 instance)

[0264] Smartphone: iOS or ANDROID (registered trademark) device

[0265] AI model: OpenAI's GPT-4

[0266] Data processing libraries: OpenCV (for image processing), Python (programming language)

[0267] Data processing and calculation

[0268] 1. Collection of match video and audio data:

[0269] The server collects video and audio data from youth and mini-basketball matches held across the country in real time, using input devices such as cameras and microphones.

[0270] 2. AI-based analysis and commentary generation:

[0271] The server analyzes the collected video and audio data using OpenCV to extract important scenes and plays from the match. Based on the extracted information, it generates commentary using OpenAI's GPT-4. Specifically, it uses the following prompt:

[0272] Example prompt sentence:

[0273] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[0274] 3. Real-time streaming and commentary:

[0275] The server delivers the game footage along with the generated commentary in real time to smartphones, where an application allows users to receive AI-generated commentary while watching the game.

[0276] 4. Providing highlight and replay features:

[0277] The server compiles key scenes from the match as highlights and provides a replay function, allowing users to rewatch key moments of the match.

[0278] 5. User following feature:

[0279] The smartphone application allows users to follow their favorite teams and players, which allows them to receive information about those teams and players on a priority basis.

[0280] 6. Providing post-match statistics and analysis reports:

[0281] After the match, the server generates statistical data and analysis reports and provides them to users through a smartphone application, allowing them to obtain detailed analysis information about the match.

[0282] Specific examples

[0283] For example, if a user is watching a youth basketball game through a smartphone application, the server will deliver the game footage and AI commentary in real time. The moment Player A scores a goal during the game, the server will analyze the scene and generate the following prompt:

[0284] Example prompt sentence:

[0285] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[0286] Based on this prompt, the AI ​​model generates commentary and provides it to users. Users can enjoy the AI ​​commentary in real time while watching the game. After the game, they can also get detailed information about the game through statistical data and analysis reports.

[0287] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0288] Step 1:

[0289] The server collects video and audio data from youth and mini-basketball matches held across the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. Specifically, the server receives video and audio data sent from each match venue, converts it into the appropriate format, and stores it.

[0290] Step 2:

[0291] The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input and extracts information about important scenes and plays from the game as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify player movements and the position of the ball. It also extracts events such as goals and fouls from the audio data.

[0292] Step 3:

[0293] The server generates a prompt sentence based on the analysis results. It uses information about important scenes and plays in the game extracted in step 2 as input, and generates a prompt sentence to be passed to the AI ​​model as output. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the one below.

[0294] Example prompt sentence:

[0295] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[0296] Step 4:

[0297] The server sends the generated prompt sentence to the AI ​​model to generate an explanation. It uses the prompt sentence generated in step 3 as input and receives the explanation text generated from the AI ​​model as output. Specifically, the server calls OpenAI's GPT-4 API, sends the prompt sentence, and generates an explanation.

[0298] Step 5:

[0299] The server delivers the generated commentary and match video to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. Specifically, the server overlays the commentary text on the video data and sends it to the smartphone in streaming format.

[0300] Step 6:

[0301] Users watch the game through a smartphone application and receive AI commentary. The input is the video and commentary data delivered from the server, and the output is the display of the game video and commentary to the user. Specifically, the smartphone application decodes the received data and displays it on the screen.

[0302] Step 7:

[0303] The server compiles important scenes from the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates a highlight video as output. Specifically, the server edits the important scenes and saves them in a format that users can replay.

[0304] Step 8:

[0305] Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input and prioritizes displaying information about the followed teams and players as output. Specifically, the application records the user's selection and prioritizes retrieving and displaying relevant information.

[0306] Step 9:

[0307] The server generates statistical data and analysis reports after the match ends and provides them to users. It uses data collected during the match as input and generates statistical data and analysis reports as output. Specifically, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to users through a smartphone application.

[0308] Example 3

[0309] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] In traditional basketball games, keeping track of the score and foul situation in real time required the creation of a running score manually, which was time-consuming and labor-intensive. Additionally, there was a lack of a system to effectively collect and analyze data on teams and players nationwide and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0312] In this invention, the server includes means for receiving video data and audio data of a game, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score and foul status from the analysis results, means for generating a running score based on the extracted information, means for updating the generated running score in real time and transmitting it to the terminal, means for displaying the running score transmitted to the terminal, means for collecting information such as game results and stats of teams and players nationwide, means for compiling the collected information into a database, means for analyzing the databased information and evaluating player performance, means for generating feedback for player development based on the analysis results, means for transmitting the generated feedback to the terminal, and means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the game in real time, and to effectively visualize player performance and provide feedback for development.

[0313] "Game video data" refers to a video file of a basketball game captured on camera.

[0314] "Audio data" refers to audio files recorded by a microphone during a basketball game.

[0315] "Means for receiving" is a function that allows the server to obtain data from the outside.

[0316] The "analyzing means" is a function for processing received data and extracting necessary information.

[0317] "Score status" is information indicating how many points each team has scored during the game.

[0318] "Foul situation" is information that indicates which player committed what kind of foul during the game.

[0319] A "running score" is a record of points and fouls that is updated in real time as the game progresses.

[0320] "Means of updating in real time" refers to a function that instantly reflects data as the game progresses.

[0321] A "terminal" is a device that a user uses to view information.

[0322] "Display means" refers to a function for visually presenting information on a terminal.

[0323] "Match result" is information indicating the final score and outcome after the match has finished.

[0324] "Stats" are statistical data that quantify the performance of a player or team.

[0325] "Means of collection" refers to the function for collecting the necessary data.

[0326] "Means for creating a database" is a function for organizing and storing collected data.

[0327] "Means of analysis" refers to the function of processing databased information and deriving meaningful results.

[0328] "Feedback" refers to advice or evaluation provided based on the analysis results.

[0329] "Visualization" refers to displaying data visually to make it easier to understand.

[0330] "Development" refers to training and instruction to improve a player's abilities.

[0331] This invention is a system for evaluating and developing player performance by grasping the score and foul situation in a basketball game in real time. A specific embodiment of this system will be described below.

[0332] System configuration

[0333] Hardware

[0334] Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

[0335] Terminal: A device on which a user checks information, including a smartphone, tablet, or PC.

[0336] Camera: A device used to capture video data of a match.

[0337] Microphone: A device used to capture audio data from a match.

[0338] software

[0339] Video analysis software: Uses libraries such as OpenCV to analyze video data.

[0340] Audio analysis software: Use a library such as LibROSA to analyze audio data.

[0341] Database management system: Collected data is stored in a database using MySQL (registered trademark) or similar.

[0342] Data analysis tools: Analyze data using tools such as Pandas and NumPy.

[0343] Generative AI models, such as GPT-4, are used to generate running scores and feedback.

[0344] System Operation

[0345] Data reception and analysis

[0346] The server receives video and audio data from the camera and microphone in real time. The video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and audience reactions.

[0347] Information extraction and running score generation

[0348] The server extracts the score and foul situations from the analysis results. Based on the extracted information, it uses a generative AI model to generate a running score. The generated running score is updated in real time and sent to the device.

[0349] Data Display

[0350] The terminal displays the received running score, allowing the user to grasp the progress of the match in real time.

[0351] Data collection and database creation

[0352] The server collects information such as match results and stats for teams and players from around the country, and the collected information is stored in a database using a database management system.

[0353] Data analysis and feedback generation

[0354] The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for player development. The generated feedback is sent to the device, which displays it. This allows the user to check the analysis results of the player's performance and receive advice for development.

[0355] Specific examples

[0356] When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

[0357] The server generates information such as "Team A has scored" or "Team B has committed a foul" in real time and sends it to the terminal.

[0358] The device displays this information, allowing users to check the progress of the match in real time.

[0359] Prompt Sentence Examples

[0360] "Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

[0361] "Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of the players, and generate feedback for training." The flow of the specific process in the third embodiment will be described with reference to FIG.

[0362] Step 1:

[0363] The server receives the video and audio data of the match.

[0364] Input: Real-time data from camera and microphone

[0365] Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

[0366] Output: Received video and audio data

[0367] Step 2:

[0368] The server analyzes the received video data.

[0369] Input: Received video data

[0370] Specific operation: Using video analysis software (e.g., OpenCV), score and foul scenes are detected from video data.

[0371] Output: Time-stamped data of goals and fouls

[0372] Step 3:

[0373] The server analyzes the received audio data.

[0374] Input: Received audio data

[0375] What it does: Use audio analysis software (e.g., LibROSA) to analyze commentary and audience reactions from audio data.

[0376] Output: Timestamped data of commentary and audience reactions

[0377] Step 4:

[0378] The server extracts the score and foul situations from the analysis results.

[0379] Input: Timestamps of goals and fouls, as well as timestamped commentary and crowd reactions

[0380] Specific operations: Integrate the results of video analysis and audio analysis to extract the occurrence of goals and fouls.

[0381] Output: Scoring and foul data

[0382] Step 5:

[0383] The server generates a running score based on the extracted information.

[0384] Input: Scoring and foul data

[0385] What it does: Uses a generative AI model (e.g., GPT-4) to generate score and foul information in text format.

[0386] Output: Text data of running score

[0387] Step 6:

[0388] The server updates the generated running score in real time and transmits it to the terminal.

[0389] Input: Text data of running score

[0390] Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

[0391] Output: Running score sent to the device

[0392] Step 7:

[0393] The terminal displays the received running score.

[0394] Input: Running score sent to the device

[0395] Specific operation: Visually display the running score on the device display.

[0396] Output: A running score that can be visually checked by the user

[0397] Step 8:

[0398] The server collects information such as match results and stats for teams and players across the country.

[0399] Input: Data from each team's official website and sports data provider services

[0400] What it does: Collects match results and stats using web scraping and APIs.

[0401] Output: Collected match results and stats data

[0402] Step 9:

[0403] The server stores the collected information in a database.

[0404] Input: Collected match results and stats data

[0405] Specific behavior: Use a database management system (e.g. MySQL) to store data in tables.

[0406] Output: Match results and stats stored in a database

[0407] Step 10:

[0408] The server analyzes the databased information and evaluates the players' performance.

[0409] Input: Match results and stats data stored in a database

[0410] Specific operations: Use data analysis tools (e.g., Pandas, NumPy) to calculate players' shooting percentages, defensive strength, etc.

[0411] Output: Player performance evaluation data

[0412] Step 11:

[0413] The server generates feedback for player development based on the analysis results.

[0414] Input: Player performance evaluation data

[0415] What it does: Uses a generative AI model (e.g., GPT-4) to generate textual feedback for player development.

[0416] Output: Text data of feedback for training

[0417] Step 12:

[0418] The server transmits the generated feedback to the terminal.

[0419] Input: Text data of feedback for training

[0420] Specific behavior: Sends feedback to the device using WebSocket or HTTP protocols.

[0421] Output: Feedback sent to the terminal

[0422] Step 13:

[0423] The device displays the feedback it receives.

[0424] Input: Feedback sent to the device

[0425] Specific behavior: Provide visual feedback on the device display.

[0426] Output: Visual feedback for the user

[0427] (Application example 3)

[0428] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0429] Traditional live broadcasts and commentary of basketball games rely on human commentators, making it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human intervention, making it difficult to provide information efficiently. Furthermore, detecting anomalies and improving production efficiency are key issues when monitoring production lines in factories. To solve these issues, an automated system utilizing AI is needed.

[0430] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0431] In this invention, the server includes a means for AI to provide commentary on live broadcasts of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI play-by-play, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormalities in real time and provide scores, and a means for storing past production data in a database and providing feedback for improving efficiency. This makes it possible to accurately grasp the progress of games and the status of production lines in real time, enabling efficient information provision and improved production efficiency.

[0432] "AI-based commentary for live broadcasts of basketball games" refers to a technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and the plays in real time.

[0433] "A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

[0434] "A means of automatically generating running scores using AI, which are currently created manually by officials and for each team," is a technology in which artificial intelligence automatically analyzes scores such as points scored and fouls while the game is in progress, and generates scores in real time.

[0435] "A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" is a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within a factory, detects production line status and abnormalities in real time, and displays the results as a score.

[0436] "Means of creating a database of past production data and providing feedback to improve efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes that data, and provides feedback to improve production efficiency.

[0437] The following system configuration will be described as an embodiment of the present invention.

[0438] System Configuration

[0439] Hardware

[0440] Camera: Cameras installed inside the factory capture video data.

[0441] Microphone: A microphone installed in the factory captures voice data.

[0442] Server: A server for analyzing and storing data.

[0443] Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

[0444] software

[0445] OpenCV: A library for acquiring and preprocessing camera footage.

[0446] TensorFlow / Keras: Libraries for loading AI models and making predictions.

[0447] SQLite: A database management system for storing production data.

[0448] Data processing and calculation

[0449] The server captures camera images and audio data from inside the factory in real time and preprocesses them using OpenCV. The preprocessed data is input into an AI model using TensorFlow / Keras to detect production status and abnormalities. The detection results are displayed as scores and stored in an SQLite database.

[0450] Specific examples

[0451] For example, if an abnormality occurs in a factory, it can detect it from camera footage and notify managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

[0452] Prompt Sentence Examples

[0453] An example of a prompt to input to a generative AI model is as follows:

[0454] "Please create an AI model that analyzes camera footage within the factory and detects abnormalities on the production line in real time. Please also add a function that notifies the administrator when an abnormality is detected. Also, please build a system that stores past production data in a database and provides feedback to improve efficiency."

[0455] In this way, the server can monitor the status of the production line in real time, detect abnormalities, and provide feedback for efficiency improvements based on past data.

[0456] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0457] Step 1:

[0458] The server acquires video and audio data in real time from cameras and microphones installed in the factory. The input is raw data from the cameras and microphones, and the output is the acquired video and audio data. Specifically, the server connects to the cameras and microphones and starts the data stream.

[0459] Step 2:

[0460] The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

[0461] Step 3:

[0462] The server inputs the preprocessed video data into the AI ​​model using TensorFlow / Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. Specifically, the server loads the AI ​​model, inputs the preprocessed data into the model, and makes predictions.

[0463] Step 4:

[0464] The server displays the prediction result as a score and saves it in an SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. Specifically, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

[0465] Step 5:

[0466] The terminal receives the score and anomaly detection notification sent from the server and displays them to the administrator. The input is the score and notification sent from the server, and the output is the display information that the administrator can check. Specifically, the terminal receives data from the server and displays it on the user interface.

[0467] Step 6:

[0468] The server analyzes past production data and generates feedback for efficiency improvement. The input is past production data stored in a database, and the output is feedback information for efficiency improvement. Specifically, the server retrieves past data from the database and generates feedback by applying an analysis algorithm.

[0469] Step 7:

[0470] The terminal receives the feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives the feedback information from the server and displays it on the user interface.

[0471] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0472] "Example 1"

[0473] In one embodiment of the present invention, an AI system for commentating on live broadcasts of basketball games is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (joy, sadness, surprise, etc.) expressed by the user while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user is happy with the progress of the game, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the game.

[0474] "Example 2"

[0475] The emotion engine is also incorporated into a system that broadcasts live commentary of youth and mini-basketball games from around the country, complete with AI commentary. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary accordingly. For example, if a user is moved by their child's performance, the commentary will also share that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

[0476] "Example 3"

[0477] Furthermore, emotion engines are also being incorporated into systems that automatically generate running scores created by AI for each official or team. Specifically, the emotion engine recognizes the user's emotions regarding the score of the game and adjusts the score display method according to those emotions. For example, if the user expresses joy at a score, the score display will also emphasize that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the game.

[0478] The processing flow of each embodiment will be described below.

[0479] "Example 1"

[0480] Step 1: The AI ​​begins commentating on a live basketball game.

[0481] Step 2: The emotion engine recognizes the user's emotions in real time.

[0482] Step 3: The AI ​​adjusts the content and tone of the commentary based on the emotions recognized by the emotion engine.

[0483] "Example 2"

[0484] Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

[0485] Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

[0486] Step 3: The AI ​​adjusts the content and tone of the commentary based on the emotions recognized by the emotion engine.

[0487] "Example 3"

[0488] Step 1: AI automatically generates a running score based on the score of the match.

[0489] Step 2: The emotion engine recognizes the user's emotions regarding the score of the game in real time.

[0490] Step 3: The AI ​​adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

[0491] Example 1

[0492] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0493] Conventional basketball game commentary systems have struggled to analyze the game situation and player movements in real time and provide commentary that responds to the user's emotions. Furthermore, there are lack of efficient ways to deliver commentary for youth and mini-basketball games across the country, and a lack of ways to automatically generate running scores that are currently created manually by officials and teams. Furthermore, there is insufficient utilization of emotion recognition technology to improve the user viewing experience.

[0494] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0495] In this invention, the server includes means for providing commentary for live coverage of basketball games using artificial intelligence, means for distributing commentary for youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores created by humans for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotion, and means for distributing the adjusted commentary along with the live coverage. This makes it possible to analyze the game situation and player movements in real time and provide commentary according to the user's emotion.

[0496] "Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to a technology that analyzes video and audio data of basketball games, and automatically generates commentary using artificial intelligence based on the results of that analysis, which is then distributed along with the live broadcast.

[0497] "A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" is a technology that uses artificial intelligence to automatically generate commentary for youth and mini-basketball games nationwide and broadcasts that commentary along with live coverage.

[0498] "A means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[0499] "Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

[0500] "Means for analyzing audio data of a match" refers to a technology that analyzes audio data of a match and identifies the content of the commentary and the reactions of the audience.

[0501] The "means for receiving and analyzing user emotion data" refers to a technology that captures the user's facial expressions and voice, analyzes the data, and identifies the user's emotion.

[0502] "Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

[0503] The "means for delivering adjusted commentary together with live broadcast" is a technology for delivering adjusted commentary to a user's terminal in real time together with live broadcast.

[0504] MODE FOR CARRYING OUT THE INVENTION

[0505] This invention is a system that uses artificial intelligence to provide commentary for live coverage of basketball games, analyzing video and audio data of the game and providing commentary that responds to the user's emotions. A specific embodiment of this system is described below.

[0506] Hardware and software used

[0507] Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

[0508] Software: Video analysis software (e.g., OpenCV), audio analysis software (e.g., Google Cloud Speech-to-Text API), emotion engine (e.g., Microsoft® Azure® Emotion API), generative AI model (e.g., GPT-4)

[0509] System configuration

[0510] 1. The server receives the video and audio data of the match in real time. It uses a high-speed network connection to receive the streaming data sent from the match venue.

[0511] 2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, it uses computer vision techniques, such as OpenCV, to identify the positions and movements of players and the position of the ball.

[0512] 3. The server uses voice analysis software to analyze the received audio data. Specifically, it uses voice recognition technology, such as the Google Cloud Speech-to-Text API, to identify the content of the commentary and the audience's reactions.

[0513] 4. The server automatically generates commentary on the match using a generative AI model based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

[0514] 5. The device captures the user's facial expressions and voice in real time and sends them to the server as emotional data, specifically using a camera and microphone.

[0515] 6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology, such as the Microsoft Azure Emotion API, to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

[0516] 7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary will be brighter and more positive expressions will be used.

[0517] 8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers it in real time using streaming technology, such as WebRTC.

[0518] Specific examples

[0519] For example, if a user expresses joy while watching a game, the emotion engine recognizes that joy. Based on that information, the server adjusts the tone of the commentary generated by the generative AI model to be more cheerful and share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

[0520] Prompt Sentence Examples

[0521] "Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and player movements, and incorporate an emotion engine to adjust the content and tone of the commentary based on the user's emotions."

[0522] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0523] Step 1:

[0524] The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in the appropriate format.

[0525] Step 2:

[0526] The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the position of players, their movements, and the position of the ball. Specifically, it uses OpenCV to analyze each frame and identify the positions of players and the ball.

[0527] Step 3:

[0528] The server uses speech analysis software to analyze the received audio data. The input is the stored audio data, and the output is the analysis results, which are a text version of the commentary and audience reactions. Specifically, the server converts the audio data into text using the Google Cloud Speech-to-Text API, and extracts key information.

[0529] Step 4:

[0530] The server automatically generates commentary on the match using a generative AI model based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate text that explains the situation of the match and the movements of the players.

[0531] Step 5:

[0532] The device captures the user's facial expressions and voice in real time and sends them to the server as emotional data. The input is the user's facial expressions and voice, and the output is the emotional data sent to the server. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice and sends them to the server in real time.

[0533] Step 6:

[0534] The server uses an emotion engine to analyze the received emotion data. The input is the transmitted emotion data, and the output is the analysis result that identifies the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

[0535] Step 7:

[0536] The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user expresses joy, the tone of the commentary becomes brighter and more positive expressions are included.

[0537] Step 8:

[0538] The server delivers the adjusted commentary along with the live broadcast to the user's device. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. Specifically, WebRTC is used to deliver the adjusted commentary in real time.

[0539] (Application example 1)

[0540] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0541] Conventional basketball game commentary systems have the problem that commentary is uniform and cannot flexibly respond to viewer emotions. Furthermore, improving the quality of live game coverage and commentary requires a large number of people, which increases costs and effort. Furthermore, there is a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

[0542] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live broadcasts of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes user emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting prompt sentences. This enables flexible commentary that responds to viewer emotions, improving the quality of live broadcasts and commentary. Furthermore, commentary on youth and mini-basketball games nationwide can be efficiently provided.

[0543] "A method for AI to provide commentary for live broadcasts of basketball games" refers to a method that uses artificial intelligence technology to analyze video and audio data from basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

[0544] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" means a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

[0545] "A means for AI to automatically generate running scores that are created manually by officials and teams" refers to a means for AI to automatically generate scores that are manually created by officials and teams during a match.

[0546] "Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (joy, sadness, surprise, etc.) and adjust the content and tone of commentary according to those emotions.

[0547] "Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (e.g., GPT-3 (registered trademark)).

[0548] "Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

[0549] A system for implementing this invention includes a means for AI to provide commentary for live broadcasts of basketball games, a means for providing live commentary of youth and mini-basketball games across the country with AI, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt sentence.

[0550] Hardware and software used

[0551] Hardware: Smartphone camera and microphone

[0552] Software: OpenCV, Keras, Transformers

[0553] Data processing and calculation

[0554] server

[0555] The server receives and analyzes video and audio data from basketball games in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as joy, sadness, and surprise.

[0556] Terminal

[0557] The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

[0558] User

[0559] Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if a user expresses joy at a highlight scene, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if a user expresses sadness at a comeback, the commentary might say, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

[0560] This allows for flexible commentary that responds to the emotions of viewers, improving the quality of live broadcasts and commentary. It also allows for efficient commentary of youth and mini-basketball games across the country.

[0561] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0562] Step 1:

[0563] The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is a state where this data is ready for analysis.

[0564] Step 2:

[0565] The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

[0566] Step 3:

[0567] The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

[0568] Step 4:

[0569] The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, the server selects a template sentence according to the emotion and generates the prompt sentence.

[0570] Step 5:

[0571] The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt generated in step 4, and the output is the generated explanation. Specifically, the prompt is input into the generative AI model, and an explanation is generated.

[0572] Step 6:

[0573] The server delivers the generated commentary to the user in real time. The input is the commentary generated in step 5, and the output is the commentary audio delivered to the user's smartphone. Specifically, the commentary is input into a speech synthesis engine, and audio data is generated and delivered.

[0574] Step 7:

[0575] Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is to improve the user's viewing experience. Specifically, the commentary audio is played from the smartphone's speaker.

[0576] Example 2

[0577] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0578] Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the viewer's emotions. Furthermore, there was a lack of systems to provide nationwide youth and mini-basketball games to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for a game was time-consuming and labor-intensive, requiring efficient management.

[0579] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0580] In this invention, the server includes means for providing commentary on live broadcasts of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live broadcasts by artificial intelligence, means for automatically generating running scores created by humans for each official and team using artificial intelligence, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing viewer emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve the satisfaction of watching the game.

[0581] "Means for AI to provide commentary for live broadcasts of basketball games" refers to a technology in which AI analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

[0582] "A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" is a technology that uses artificial intelligence to analyze youth and mini-basketball games held across the country, generate commentary, and broadcast it in real time.

[0583] "A means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[0584] "Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

[0585] The "means of analyzing collected data and generating commentary" refers to a technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

[0586] "Means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice and adjusts the content and tone of the commentary according to those emotions.

[0587] "Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

[0588] This invention is a system that uses artificial intelligence (AI) to broadcast live basketball games and also provides commentary on youth and mini-basketball games from around the country. Furthermore, it includes a function that automatically generates running scores, which are currently created by humans for each official and team.

[0589] The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue, specifically using hardware such as Sony 4K cameras and Shure high-sensitivity microphones, and stores the collected data in a database on the server.

[0590] The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI ​​model analyzes the input data and generates commentary based on the progress of the game and the performance of the players. For example, it analyzes the moment Player A makes a shot and generates commentary such as, "Player A made a great shot!"

[0591] Furthermore, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. This is done using Apple's Face ID technology or Amazon's Alexa, for example. The device will then send the captured data to a server.

[0592] The server receives the user's emotion data sent from the device and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson (registered trademark) emotion analysis APIs. If the user is emotional, the server adjusts the tone of the commentary to be more emotional, generating a commentary such as, "Player A's shot is truly amazing! The crowd is really excited!"

[0593] Finally, the server distributes the generated commentary along with the game footage via streaming platforms such as YouTube and Twitch, allowing users to watch the game and enjoy the emotional commentary.

[0594] As a concrete example, consider a youth basketball game. The server collects video and audio data in real time from a Sony 4K camera and Shure high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4 to analyze the moment Player A makes a shot. The AI ​​model generates a commentary such as, "Player A made a great shot!"

[0595] Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognize that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot is truly amazing! The crowd is really excited!" and distributes it via YouTube.

[0596] An example of a prompt might be:

[0597] "Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also, recognize viewer emotions and adjust the content and tone of the commentary accordingly."

[0598] In this way, the server can provide commentary to the user according to their emotions, thereby improving the satisfaction of watching the match.

[0599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0600] Step 1:

[0601] The server collects video and audio data from matches in real time using cameras and microphones installed at the match venue. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data from the match, and the output is the collected video and audio data. This data is stored in a database on the server.

[0602] Step 2:

[0603] The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is data on the analyzed progress of the game and player performance. The AI ​​model analyzes player movements and the progress of the game from the video data, and analyzes commentary and spectator reactions from the audio data.

[0604] Step 3:

[0605] The server generates commentary on the game based on the data analyzed by the AI ​​model. The input is the analyzed data on the progress of the game and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!"

[0606] Step 4:

[0607] The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotional data. This uses Apple's Face ID technology and Amazon's Alexa.

[0608] Step 5:

[0609] The server receives the user's emotion data sent from the device and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional.

[0610] Step 6:

[0611] The server distributes the generated commentary along with the game video. The input is the generated commentary and the game video, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary that matches their emotions.

[0612] (Application example 2)

[0613] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0614] Conventional live basketball broadcasting systems have the problem of providing uniform commentary and being unable to respond flexibly to viewer emotions. Furthermore, they lack game highlights, replay features, and chat functions between viewers, leaving a need for an improved viewing experience. Furthermore, there were limited ways to provide nationwide youth and mini-basketball games to a wide audience, resulting in information disparities between regions.

[0615] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0616] In this invention, the server includes means for AI to provide commentary for live broadcasts of basketball games, means for distributing commentary for youth and mini-basketball games from around the country with AI commentary, means for AI to automatically generate running scores that are created manually for each official and team, means for recognizing viewer emotions and adjusting the content and tone of the commentary according to those emotions, means for providing game highlights and replay functions, and means for providing a chat function between viewers. This enables flexible commentary according to viewer emotions, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

[0617] "A method for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

[0618] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that broadcasts youth and mini-basketball games held all over the country in real time with commentary by artificial intelligence.

[0619] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

[0620] "Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to a technology that recognizes emotions from the viewer's facial expressions and voice, and changes the content and tone of the commentary according to those emotions.

[0621] "Means for providing match highlight and replay functions" refers to technology that provides a highlight function that extracts and plays important scenes from a match, and a replay function that replays a specific scene.

[0622] "Means for providing a chat function between viewers" refers to technology that provides a chat function that enables users watching a match to exchange messages in real time.

[0623] The system for carrying out this invention includes the following main components: a means for AI to provide commentary for live broadcasts of basketball games, a means for providing live commentary of youth and mini-basketball games from around the country with AI, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

[0624] Hardware and software used

[0625] Hardware: Smartphone (camera, microphone, display)

[0626] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

[0627] Data processing and calculation

[0628] 1. Frame Acquisition: The server acquires frames from the live streaming, thereby collecting video data of the match in real time.

[0629] 2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer, thereby obtaining the viewer's emotion data.

[0630] 3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions, which allows commentary to be generated in real time according to the viewer's emotions.

[0631] 4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to watch emotionally relevant commentary in real time.

[0632] 5. Highlights and Replays: The server automatically extracts key scenes from the match and provides highlight and replay features, allowing viewers to rewatch key moments of the match.

[0633] 6. Chat Function: The server provides a chat function that allows viewers to exchange messages with each other in real time, thereby promoting communication between viewers.

[0634] Specific examples

[0635] For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary such as, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary such as, "Don't miss this! Watch the next play!"

[0636] Prompt Sentence Examples

[0637] “Create an AI model that can recognize the emotions of your audience and adjust the content and tone of your commentary accordingly. For example, if your audience is emotional, make sure your commentary reflects that emotion.”

[0638] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0639] Step 1:

[0640] The server acquires frames from the live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

[0641] Step 2:

[0642] The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

[0643] Step 3:

[0644] The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI ​​commentary generation model to generate an appropriate commentary.

[0645] Step 4:

[0646] The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

[0647] Step 5:

[0648] The server automatically extracts important scenes from the game and provides highlight and replay functions. The input is real-time game video data, and the output is highlight and replay videos. Specifically, the server detects events in the game, extracts important scenes, and saves them.

[0649] Step 6:

[0650] The server provides a chat function that allows viewers to exchange messages in real time. The input is message data from viewers, and the output is messages to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

[0651] Example 3

[0652] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0653] In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. Furthermore, there was a lack of a national database of teams and players, and feedback for analyzing player performance and developing players was not provided adequately. Furthermore, the score display did not reflect the user's feelings about the game's score, resulting in low satisfaction when watching the game.

[0654] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0655] In this invention, the server includes means for receiving video data and audio data of a game, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score status and foul status from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display, thereby making it possible to grasp the progress of the game in real time and provide a score display that reflects the user's emotions.

[0656] "Game video data" refers to video data that records the progress of a basketball game.

[0657] "Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

[0658] "Means for receiving" is a function for acquiring video data and audio data of a match from an external source.

[0659] The "analyzing means" is a function for processing received video and audio data and extracting specific information.

[0660] "Score status" is information indicating how many points each team has scored during the game.

[0661] "Foul situation" is information that indicates which player committed what kind of foul during the game.

[0662] A "running score" is a scoreboard that shows the score and foul situation updated in real time as the game progresses.

[0663] The "means for updating and transmitting in real time" is a function for instantly updating the generated running score and transmitting it to the user's terminal.

[0664] The "display means" is a function for visually presenting the transmitted running score to the user.

[0665] "Means for recognizing user emotions" is a function for analyzing and understanding the user's emotions regarding the score of a game, etc.

[0666] The "means for adjusting score display" is a function for changing the display method of the scoreboard according to the recognized user emotion.

[0667] "Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

[0668] "Means for saving to a database" refers to a function for organizing collected match results and stat information and storing it in a database.

[0669] "Means for analyzing performance" is a function for evaluating a player's performance and abilities based on information stored in the database.

[0670] "Means for generating feedback" refers to a function that creates reports based on the analysis results to suggest areas for player development and improvement.

[0671] The "means for transmitting and displaying" is a function for transmitting the generated feedback to the user's terminal and visually presenting it.

[0672] "Visualization" refers to displaying data visually to make it easier to understand.

[0673] "Means for use in development" refers to methods for supporting players' training and development based on data.

[0674] The present invention is a system for automatically generating running scores and analyzing player performance in basketball games. A specific embodiment of this system will be described below.

[0675] System configuration

[0676] Hardware and Software

[0677] 1. Server

[0678] Hardware: A server with a powerful processor and large memory capacity

[0679] Software: video analysis software (e.g., OpenCV), audio analysis software (e.g., Google Cloud Speech-to-Text), database management systems (e.g., MySQL), sentiment analysis engines (e.g., IBM Watson Tone Analyzer)

[0680] 2. Terminal

[0681] Hardware: PCs, tablets, smartphones, etc.

[0682] Software: Web browser, dedicated application

[0683] Program processing

[0684] Receiving and analyzing match data

[0685] The server receives video and audio data from the game. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

[0686] Extraction of scoring and foul situations

[0687] The server combines the results of video and audio analysis to extract the score and foul situations, making it possible to grasp the progress of the game in real time.

[0688] Generate and display a running score

[0689] The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the device using WebSocket or HTTP. The device displays the received running score to the user, allowing the user to check the progress of the game in real time.

[0690] Recognizing user emotions and adjusting score display

[0691] The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. The server adjusts the score display depending on the recognized emotion. For example, if the user is expressing joy at a score, the score display will be emphasized with bright colors and animations. The adjusted score display is sent to the device and displayed to the user.

[0692] Database creation and performance analysis of teams and players nationwide

[0693] The server collects game results and stats for teams and players across the country and stores them in a MySQL database. Based on the stored information, the server analyzes player performance using Python's Pandas library. Based on the analysis results, it generates feedback for player development and sends it to the device. The device then displays the received feedback to the user.

[0694] Specific examples

[0695] When the video data of the game is sent to the server, the server uses OpenCV to detect scoring scenes and Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Please analyze the scoring scenes and foul information of this game and generate a running score."

[0696] The server collects the results of matches from around the country and stores them in a MySQL database. It uses the Python Pandas library to analyze the players' performance and generate feedback. For example, enter the following as a prompt: "Collect the results of matches from around the country, analyze the players' performance, and generate feedback."

[0697] The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer and adjusts the score display to emphasize the joy. For example, the server inputs a prompt statement such as "Please analyze the user's emotions at the scoring scene and adjust the score display." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[0698] Step 1:

[0699] Receiving match data

[0700] The server receives video and audio data from the game. As input, it acquires video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server acquires the data through a network connection and stores it in a specified format.

[0701] Step 2:

[0702] Video data analysis

[0703] The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. Specifically, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

[0704] Step 3:

[0705] Analysis of audio data

[0706] The server analyzes the received audio data. As input, it uses the audio data saved in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data to text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

[0707] Step 4:

[0708] Extraction of scoring and foul situations

[0709] The server integrates the video analysis results and audio analysis results to extract the score and foul situations. The outputs of steps 2 and 3 are used as input. The output is data including the score and foul situations. Specifically, the server integrates the video analysis results and audio analysis results in chronological order to identify the timing of scores and fouls.

[0710] Step 5:

[0711] Generating a running score

[0712] The server generates a running score based on the extracted information. As input, it uses the score and foul situations obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

[0713] Step 6:

[0714] Update and submit running scores

[0715] The server updates the generated running score in real time and sends it to the device. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the device. Specifically, the server sends data using WebSocket or HTTP.

[0716] Step 7:

[0717] Displaying running scores

[0718] The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. Specifically, the terminal displays the scoreboard using a web browser or a dedicated application.

[0719] Step 8:

[0720] Recognizing user emotions

[0721] The server recognizes the user's emotions regarding the score of the game. The input is the user's comments and reaction data. The output is the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

[0722] Step 9:

[0723] Score display adjustments

[0724] The server adjusts the score display method according to the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. Specifically, the server changes the color and animation of the score display.

[0725] Step 10:

[0726] Sending adjusted score display

[0727] The server sends the adjusted score display to the device. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the device. Specifically, the server sends the data using WebSocket or HTTP.

[0728] Step 11:

[0729] Display of adjusted score display

[0730] The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. Specifically, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

[0731] Step 12:

[0732] Database of teams and players nationwide

[0733] The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data provider services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

[0734] Step 13:

[0735] Performance Analysis

[0736] The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

[0737] Step 14:

[0738] Generate feedback

[0739] The server generates feedback for player development based on the analysis results. It uses the analysis results obtained in step 13 as input. The output is a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. Specifically, the server creates the report and provides it to the player and coach.

[0740] Step 15:

[0741] Sending and Viewing Feedback

[0742] The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. Specifically, the server sends data via email or a dedicated application, and the terminal displays it.

[0743] (Application example 3)

[0744] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0745] Traditional live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. Furthermore, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

[0746] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0747] In this invention, the server includes means for AI to provide commentary on live broadcasts of basketball games, means for distributing commentary on youth and mini-basketball games from around the country with AI play-by-play, means for AI to automatically generate running scores that are created manually for each official and team, means for analyzing video and audio data of the game in real time and automatically generating the score and foul situation, means for recognizing the user's emotions and displaying the score according to those emotions, and means for providing feedback for player performance analysis and development using a database of teams and players from around the country. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, thereby improving the efficiency of player performance analysis and development.

[0748] "A method for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

[0749] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" is a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

[0750] "A means of automatically generating running scores using AI, which are currently created by humans for each official and team," is a technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

[0751] "Means for analyzing video and audio data of a match in real time and automatically generating scoring and foul situations" refers to technology that analyzes video and audio data during a match in real time and automatically generates scoring and foul situations.

[0752] "Means for recognizing the user's emotions and displaying a score according to those emotions" refers to a technology that recognizes the user's emotions from their facial expressions and voice, and adjusts the score display according to those emotions.

[0753] "A means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to a technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

[0754] The system for implementing the present invention uses the following hardware and software.

[0755] Hardware and software used

[0756] Hardware: Smartphone, head-mounted display

[0757] Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

[0758] System configuration

[0759] The server includes the following means:

[0760] 1. How AI can provide commentary for live basketball games:

[0761] The server receives video and audio data of the match in real time, analyzes the progress of the match and the details of the play using TensorFlow, and generates commentary.

[0762] 2. How to broadcast live commentary of youth and mini-basketball games across the country with AI:

[0763] The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which is then distributed in real time.

[0764] 3. A way to automatically generate running scores using AI, instead of the running scores currently created by officials and teams:

[0765] The server analyzes the video and audio data of the game and automatically generates score and foul situations.

[0766] 4. A method for analyzing game video and audio data in real time to automatically generate scoring and foul situations:

[0767] The server uses OpenCV to split the video data into frames and extract important scenes, and NLTK to convert the audio data into text and extract important keywords.

[0768] 5. A method to recognize the user's emotions and display a score according to those emotions:

[0769] The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

[0770] 6. Utilizing a national team and player database to provide feedback for player performance analysis and development:

[0771] The server references a database of teams and players from across the country, displays analysis of players' performance, and provides feedback for the next game.

[0772] Specific examples

[0773] When a user using a smartphone sees a goal during a game, the server uses AI to analyze the scene and display the score in real time. If the user shows a joyful expression, the server uses an emotion engine to recognize it and make the score display more colorful. After the game, the server also references a nationwide database to display an analysis of the players' performance and provide feedback for the next game.

[0774] Prompt Sentence Examples

[0775] "Please analyze the video and audio data of the game in real time, and display a running score using an AI model that automatically generates the score and foul situation. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please utilize a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

[0776] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0777] Step 1:

[0778] Acquisition of match video and audio data

[0779] The server receives video and audio data of the game in real time from smartphones and head-mounted displays. The input is the video and audio data of the game, and the output is preprocessed data for analysis. Specifically, the server receives the data stream, divides the video data into frames, and converts the audio data into text.

[0780] Step 2:

[0781] Data Preprocessing

[0782] The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extract important keywords (such as goals and fouls). The input is preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

[0783] Step 3:

[0784] Analysis using AI models

[0785] The server uses TensorFlow to analyze the score and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the score and foul situations. Specifically, the server runs an AI model to automatically determine the score and fouls for each scene.

[0786] Step 4:

[0787] Automatic generation of running scores

[0788] The server automatically generates a running score based on the analysis results. The input is data on the score and foul situation, and the output is a real-time running score. Specifically, the server aggregates the score and foul information and updates the scoreboard.

[0789] Step 5:

[0790] emotion recognition

[0791] The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. Specifically, the server acquires user data from the camera and microphone and analyzes emotions.

[0792] Step 6:

[0793] Emotion-based score display

[0794] The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. Specifically, the server changes the design and effects of the score display based on the user's emotion.

[0795] Step 7:

[0796] Utilizing the database

[0797] The server references a database of teams and players from around the country, displays player performance analysis results, and provides feedback for the next match. The input is match results and player data, and the output is performance analysis results and feedback. Specifically, the server retrieves relevant information from the database and displays the analysis results.

[0798] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0799] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0800] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0801] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0802] [Second embodiment]

[0803] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0804] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0805] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0806] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0807] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0808] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0809] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0810] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0811] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0812] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0813] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0814] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0815] "Example 1"

[0816] A first embodiment of the present invention is a system in which AI provides commentary for live coverage of a basketball game. In this system, AI analyzes video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, it automatically generates commentary for the game and distributes it along with the live coverage.

[0817] "Example 2"

[0818] A second embodiment of the present invention is a system that distributes commentary on youth and mini-basketball games from around the country, accompanied by live coverage by AI. This system collects video and audio data from youth and mini-basketball games held across the country, and, as in the first embodiment, AI analyzes and generates commentary. Then, by distributing this commentary together with the game video, games from around the country can be provided to a wide audience.

[0819] "Example 3"

[0820] A third embodiment of the present invention is a system that uses AI to automatically generate running scores that are currently created manually by officials or teams. In this system, AI analyzes the score and foul situation from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

[0821] "Example 4"

[0822] A fourth embodiment of the present invention is a system that creates a database of teams and players nationwide and utilizes the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats of teams and players nationwide and creates a database. The database is then used to analyze player performance and provide feedback for development.

[0823] The processing flow of each embodiment will be described below.

[0824] "Example 1"

[0825] Step 1: AI collects video and audio data from basketball games.

[0826] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[0827] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[0828] Step 4: The generated commentary is distributed along with the live broadcast.

[0829] "Example 2"

[0830] Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

[0831] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[0832] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[0833] Step 4: The generated commentary is distributed along with the game footage.

[0834] "Example 3"

[0835] Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

[0836] Step 2: Based on the analysis results, the AI ​​automatically generates a running score.

[0837] "Example 4"

[0838] Step 1: Collect information such as game results and stats for teams and players across the country.

[0839] Step 2: Create a database of the collected information.

[0840] Step 3: Use the databased information to analyze player performance and provide feedback for development.

[0841] Example 1

[0842] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0843] Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. Furthermore, providing commentary for youth and mini-basketball games across the country requires a large number of people, which is costly and labor-intensive. Furthermore, running scores created by officials and by each team are done manually, which is time-consuming and can lack accuracy. To solve these issues, an automated system utilizing AI is needed.

[0844] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0845] In this invention, the server includes a means for collecting video and audio data of the game in real time, a means for analyzing the collected video and audio data, a means for generating commentary on the game based on the analysis results, and a means for distributing the generated commentary along with a live broadcast. This enables accurate real-time reporting of detailed game situations and player movements. Furthermore, commentary on youth and mini-basketball games across the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores, which are currently created manually by officials and for each team, using AI can reduce time and improve accuracy.

[0846] "Game video data" refers to digital data that visually records the progress of a basketball game.

[0847] "Audio data" refers to digitally recorded data of sounds occurring during a basketball game.

[0848] "Means of collecting in real time" refers to technology and devices for acquiring video and audio data as the match progresses.

[0849] "Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

[0850] "Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of the players based on the analysis results.

[0851] "Means for distribution along with live coverage" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

[0852] "Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

[0853] "Running score" is data that records the scores and player performances that are updated in real time during the game.

[0854] "AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

[0855] "Means of database creation" refers to the technology and software used to organize collected information and store it in a format that is easy to search and use.

[0856] "Visualization" refers to techniques and methods for visually displaying data to make it easier to understand.

[0857] "Means for use in development" refers to technologies and methods that use data to support the training and development of athletes.

[0858] This invention is a system that uses AI to automatically generate commentary for live broadcasts of basketball games and delivers it in real time. Specific embodiments of this system are described below.

[0859] Server Operation

[0860] 1. Data Collection

[0861] The server collects video and audio data of the match in real time using a camera and microphone to capture the video and audio of the match, with the video data collected at a resolution of 1080p and the audio data collected at a sampling rate of 44.1kHz.

[0862] 2. Data Analysis

[0863] The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g., OpenCV) and voice recognition software (e.g., Google Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the commentary audio at that moment into text.

[0864] 3. Explanation Generation

[0865] The server generates commentary on the game based on the analysis results. A generative AI model (e.g., GPT-4) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, if the information "Player A made a successful three-point shot" is input, the server generates commentary such as "Player A made a brilliant three-point shot."

[0866] 4. Distribution

[0867] The server distributes the generated commentary along with the live broadcast. A streaming server (e.g., Wowza Streaming Engine) is used for distribution, allowing users to watch the game footage and commentary in real time via the Internet.

[0868] Device behavior

[0869] 1. Data Reception

[0870] The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g., VLC Media Player).

[0871] 2.Display

[0872] The device displays the received video and commentary to the user. This is done using hardware such as a display and speakers. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played from the built-in speaker.

[0873] User Actions

[0874] 1. Viewing

[0875] Users can watch game footage and commentary through their devices. Users can receive detailed commentary on the progress of the game and the movements of the players in real time. For example, if a user is watching on a television in their living room, the game footage will be displayed on the TV screen and commentary will be played from the TV speakers.

[0876] Specific examples

[0877] Example 1

[0878] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[0879] Player Action: Home team player A makes a three-point shot.

[0880] Example prompts for generative AI models

[0881] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[0882] Player Action: Home team player A makes a three-point shot.

[0883] Example of generated commentary

[0884] "With five minutes left in the third quarter, the home team is tied at 75 and the away team is tied at 70. Player A on the home team makes a brilliant three-point shot, widening the gap."

[0885] In this way, users can view detailed commentary on the progress of the match and the movements of the players in real time.

[0886] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0887] Step 1:

[0888] The server collects video and audio data of the match in real time.

[0889] Specifically, it uses cameras and microphones to capture video and audio of the game.

[0890] The input is video and audio from a camera and microphone, and the output is video and audio data in digital format.

[0891] Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

[0892] Step 2:

[0893] The server analyzes the collected video and audio data.

[0894] Specifically, image recognition software (e.g., OpenCV) is used to detect player movements from video data, and voice recognition software (e.g., Google Speech-to-Text API) is used to convert the audio data into text to provide commentary.

[0895] The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

[0896] For example, the system detects when player A takes a shot and converts the live audio at that moment into text.

[0897] Step 3:

[0898] The server generates commentary on the match based on the analysis results.

[0899] Specifically, a prompt sentence is input into a generative AI model (e.g., GPT-4) to generate an explanatory sentence.

[0900] The input is the player movement information obtained in step 2 and the commentary converted into text, and the output is the generated commentary.

[0901] For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

[0902] Step 4:

[0903] The server distributes the generated commentary along with the live broadcast.

[0904] Specifically, a streaming server (e.g., Wowza Streaming Engine) is used to stream video and commentary audio in real time.

[0905] The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and audio commentary delivered in real time.

[0906] Step 5:

[0907] The device receives real-time game footage and commentary streamed from the server.

[0908] Specifically, it requires an internet connection and streaming software (e.g., VLC media player).

[0909] The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played back.

[0910] Step 6:

[0911] The terminal displays the received video and commentary to the user.

[0912] Specifically, the system displays video on a screen and plays explanatory audio from a speaker.

[0913] The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user visually and audibly.

[0914] Step 7:

[0915] Users can watch the game footage and commentary through their devices.

[0916] Specifically, the user watches the game footage on the device's display while listening to commentary coming from the speaker.

[0917] The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

[0918] For example, when a user is watching on a television in their living room, the video of the game is displayed on the television screen and commentary audio is played from the television speakers.

[0919] (Application example 1)

[0920] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0921] Conventional basketball game commentary systems lack real-time commentary, post-game highlight generation, and detailed analysis of player performance. Furthermore, they face challenges in providing accurate commentary immediately when users ask questions about specific players or plays.

[0922] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0923] In this invention, the server includes means for AI to provide commentary for live broadcasts of basketball games, means for distributing commentary for youth and mini-basketball games across the country with AI commentary, means for AI to automatically generate running scores that are created manually for each official and team, means for analyzing game video data and generating commentary in real time, means for automatically extracting important scenes after the game ends and generating a highlight video, means for analyzing player movements and performance and providing detailed statistical information, and means for AI to provide commentary when a user asks a question about a specific player or play. This enables real-time commentary, post-game highlight generation, detailed analysis of player performance, and immediate commentary in response to user questions.

[0924] "A means for AI to provide commentary for live broadcasts of basketball games" refers to a technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

[0925] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that uses AI to provide commentary on youth and mini-basketball games from around the country and broadcasts the games in real time with that commentary.

[0926] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses AI to automatically generate scores that are manually recorded by officials and teams during a match.

[0927] "Means for analyzing game video data and generating commentary in real time" refers to technology that analyzes game video data in real time and instantly generates commentary based on the analysis results.

[0928] "Means for automatically extracting important scenes after the end of a match and generating a highlight video" refers to a technology that analyzes video data of a match after it has ended, automatically extracts important scenes, and generates a highlight video.

[0929] "Means for analyzing player movements and performance and providing detailed statistical information" refers to technology for analyzing player movements and performance and providing detailed statistical information based on the results of the analysis.

[0930] "A means for AI to provide commentary when a user asks a question about a specific player or play" is a technology in which AI provides appropriate commentary in response to a question when a user asks a question about a specific player or play.

[0931] To implement the present invention, the following system configuration and processing procedures are required.

[0932] System Configuration

[0933] Hardware

[0934] Server: A high-performance computer for running AI models and analyzing data.

[0935] Smartphone: The device through which users view and interact with the match commentary.

[0936] Camera: A device used to capture footage of the match.

[0937] software

[0938] OpenCV: A library used to read and analyze video data.

[0939] TensorFlow: A framework used to run AI models.

[0940] Transformers: A library that generates explanations using the GPT-2 model.

[0941] Processing flow

[0942] 1. Loading video data

[0943] The server uses OpenCV to read the game video data obtained from the camera frame by frame.

[0944] 2. Frame Analysis

[0945] The server analyzes the loaded frames to detect player movements and important actions, using a deep learning model powered by TensorFlow.

[0946] 3. Generating Explanations

[0947] The server generates an explanatory text based on the analysis results. Specifically, it runs the GPT-2 model using the Transformers library and generates an explanatory text using the prompt text as input.

[0948] 4. Streaming commentary

[0949] The generated commentary is delivered to smartphones in real time, allowing users to view and listen to the commentary on the match.

[0950] 5. Highlight video generation

[0951] After the match ends, the server automatically extracts important scenes and generates a highlight video, which is also delivered to smartphones.

[0952] 6. Player performance analysis

[0953] The server analyzes the players' movements and performance in detail and generates statistical information that users can view on their smartphones.

[0954] 7. User Interaction

[0955] When a user asks a question about a specific player or play, the server generates an appropriate commentary for the question and delivers it to the smartphone.

[0956] Specific examples

[0957] Scoring scene in the game: Detects the moment when Player A makes a three-point shot and generates a commentary such as "Player A scores a three-pointer."

[0958] Defensive highlights: Detects when Player B successfully blocks a shot and generates commentary such as "Player B makes a great block."

[0959] Prompt Sentence Examples

[0960] "Player A scores a three-pointer. Generate a commentary for this action."

[0961] "Player B makes a great block. Generate a commentary for this action."

[0962] In this way, the server can analyze the match video data in real time and automatically provide commentary to users, enabling real-time commentary, post-match highlights, detailed analysis of players' performance, and instant commentary in response to user questions.

[0963] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0964] Step 1:

[0965] The server uses OpenCV to read the video data of the game captured by the camera, frame by frame. The input is the video data from the camera, and the output is individual frame images. Specifically, the server divides the video data into a series of still images and stores each frame in memory.

[0966] Step 2:

[0967] The server analyzes the loaded frames to detect player movements and important actions. This analysis uses a deep learning model using TensorFlow. The input is the frame image obtained in step 1, and the output is information about player movements and actions. Specifically, the server detects the position and movement of players within the frame and identifies important actions such as points and blocks.

[0968] Step 3:

[0969] The server generates commentary based on the analysis results. Specifically, it runs a GPT-2 model using the Transformers library and generates commentary using the prompt as input. The input is the action information and prompt obtained in step 2, and the output is the generated commentary. The server generates a prompt corresponding to the player's action and inputs it into the GPT-2 model to obtain commentary.

[0970] Step 4:

[0971] The server delivers the generated commentary to the smartphone in real time. The input is the commentary generated in step 3, and the output is the commentary displayed on the smartphone. Specifically, the server sends the commentary to the smartphone via the network so that the user can view it.

[0972] Step 5:

[0973] After the match ends, the server automatically extracts important scenes and generates a highlight video. The input is the video data of the entire match, and the output is the highlight video. Specifically, the server selects highlight scenes based on important actions detected during the match and links them together to create a highlight video.

[0974] Step 6:

[0975] The server analyzes the players' movements and performance in detail and generates statistical information. The input is frame images from the game and information about the players' actions, and the output is detailed statistical information. Specifically, the server compiles data such as points, rebounds, and assists for each player and compiles it into statistical information.

[0976] Step 7:

[0977] When a user asks a question about a specific player or play, the server generates an appropriate commentary for that question and delivers it to the smartphone. The input is the user's question and prompt, and the output is the generated commentary. Specifically, the server analyzes the user's question, generates a corresponding prompt, inputs it into the GPT-2 model, obtains the commentary, and sends it to the smartphone.

[0978] Example 2

[0979] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0980] Traditional basketball game commentary requires specialized commentators, and securing commentators for youth and mini-basketball games across the country has been a challenge. It's also difficult to communicate game progress and player movements to viewers in real time, limiting the viewing experience. Furthermore, game data collection and analysis is done manually, resulting in inefficiencies and a lack of data consistency and accuracy.

[0981] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0982] In this invention, the server includes means for collecting video and audio data of games held nationwide, means for storing the collected data in cloud storage, means for transmitting the stored data to a generative AI model to generate game commentary, means for synchronizing the generated commentary with the game video, means for streaming the game video with commentary, means for providing an interface for users to view, means for receiving the game video with commentary streamed from the server, and means for displaying the received video to users. This enables live broadcasts with high-quality commentary by AI, even for youth and mini-basketball games held nationwide, improving the viewing experience. Furthermore, the automation of game data collection and analysis enables efficient and accurate data management.

[0983] "Means for collecting video and audio data from matches held across the country" refers to devices and systems that use cameras and microphones installed at match venues around the country to capture video and audio data from matches in real time and transmit it to a server.

[0984] "Means for storing collected data in cloud storage" refers to devices or systems that convert collected video and audio data into an appropriate format and store the data using cloud services.

[0985] "Means for sending stored data to a generative AI model and generating commentary on a match" refers to a device or system that retrieves data stored in cloud storage, sends prompts to a generative AI model to analyze the data, and generates commentary on a match.

[0986] "Means for synchronizing generated commentary with game footage" refers to a device or system that inserts generated commentary text at specific timestamps in game footage, allowing the footage and commentary to be played in a synchronized state.

[0987] A "means for streaming commentary-enhanced game footage" is a device or system that encodes synchronized video data in real time and transmits it to a streaming platform for distribution to viewers.

[0988] "Means for providing an interface for users to view" refers to a device or system for displaying a viewing interface to users through devices such as smartphones, tablets, and PCs.

[0989] The "means for receiving game footage with commentary streamed from a server" refers to a device or system for receiving game footage with commentary streamed from a server in real time via an Internet connection.

[0990] The "means for displaying the received video to the user" refers to a device or system for decoding the received video data and displaying it on the screen of the user's device.

[0991] This invention is a system that broadcasts youth and mini-basketball matches held across the country with AI commentary. A specific embodiment of this system is described below.

[0992] Server Roles

[0993] The server collects video and audio data from matches held across the country. Cameras and microphones installed at match venues capture video and audio of matches in real time and send this data to the server. The collected data is then stored in cloud storage such as Amazon S3 and Google Cloud Storage.

[0994] The server then sends the stored data to a generative AI model, such as OpenAI's GPT-4 or Google's BERT. The server retrieves the data from cloud storage and sends prompts to the generative AI model. The prompts include instructions for analyzing the progress of the game, player movements, score status, etc.

[0995] The generative AI model analyzes the data based on the prompt text and generates commentary in natural language. The generated commentary is sent back to the server in text format. The server then synchronizes the generated commentary with the game footage. This synchronization process is performed using video editing software such as FFmpeg. The server inserts the commentary text at specific timestamps in the footage, ensuring that the footage and commentary are synchronized when played.

[0996] Finally, the server streams the match footage with commentary to a streaming platform such as YouTube Live or Twitch. The server encodes the synchronized video data in real time and sends it to the streaming platform.

[0997] Device Role

[0998] The terminal provides the user with an interface for viewing. Devices such as smartphones, tablets, and PCs are used here. The terminal displays the viewing interface to the user through a dedicated application or web browser. The interface includes a list of matches and a search function.

[0999] The device receives game video with commentary streamed from the server. This requires an internet connection. The device receives video data from the streaming platform in real time, buffers it, decodes the received video data, and displays it on the user's device screen.

[1000] User Roles

[1001] Users access the system using a terminal. To access the system, they use a dedicated application or a web browser. Users launch a dedicated application on their smartphone or PC, or access the system's URL using a web browser.

[1002] Users select the match they want to watch. They are provided with a list of matches and a search function. Users can scroll through the list of matches on the interface and click on the match they want to watch. They can also use the search function to find a specific match.

[1003] The user watches the commentary video of the selected match. While watching, the user can perform operations such as play, pause, and rewind. The user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed.

[1004] Specific examples

[1005] For example, if a user wants to watch a national youth basketball tournament game on their smartphone, they might take the following steps:

[1006] 1. The user launches the dedicated app on their smartphone.

[1007] 2. Use the in-app search function and type in "National Youth Basketball Tournament."

[1008] 3. Select the match you want to watch from the search results.

[1009] 4. The app streams game footage with commentary from the server.

[1010] 5. Users can watch the game on their smartphone screen and enjoy the commentary.

[1011] An example of a prompt to be input to the generative AI model is, "Analyze the game video and audio data from the national youth basketball tournament and generate commentary that includes the progress of the game, player movements, score status, etc." This prompt is used to have the AI ​​analysis system generate commentary for the game.

[1012] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1013] Step 1:

[1014] The server collects video and audio data from games held across the country. As input, it receives real-time data from cameras and microphones installed at game venues around the country. Specifically, it sends the video and audio captured by these devices to the server via the Internet. As output, it obtains the collected video and audio data.

[1015] Step 2:

[1016] The server stores the collected data in cloud storage. As input, it receives the video and audio data collected in step 1. Specifically, it converts the data into an appropriate format and uploads it to a cloud service such as Amazon S3 or Google Cloud Storage. As output, it obtains the data stored in cloud storage.

[1017] Step 3:

[1018] The server sends the saved data to the generative AI model to generate commentary on the match. As input, it receives video and audio data obtained from cloud storage. Specifically, it sends prompts to the generative AI model to analyze the data. The prompts include instructions for analyzing the progress of the match, player movements, and the score. The output is commentary text returned by the generative AI model.

[1019] Step 4:

[1020] The server synchronizes the generated commentary with the game footage. As input, it receives the commentary text obtained in step 3 and the video data obtained from cloud storage. Specifically, it uses video editing software such as FFmpeg to insert the commentary text at a specific timestamp in the video. As output, it obtains the game footage synchronized with the commentary.

[1021] Step 5:

[1022] The server streams the game video with commentary. As input, it receives the game video synchronized with the commentary obtained in step 4. Specifically, it encodes the video data in real time and sends it to a streaming platform such as YouTube Live or Twitch. As output, it obtains the streaming video that is distributed to viewers.

[1023] Step 6:

[1024] The terminal provides an interface for users to watch games. It receives user operations as input. Specific operations include displaying the viewing interface through a dedicated application or web browser. The interface includes a list of games and a search function. The output provides information that allows users to select the game they want to watch.

[1025] Step 7:

[1026] The device receives game video with commentary streamed from the server. The input is the streaming data from the server. Specifically, the device receives the video data in real time via an Internet connection and buffers it. The output is the received video data.

[1027] Step 8:

[1028] The terminal displays the received video to the user. As input, it receives the video data received in step 7. Specifically, it decodes the video data and displays it on the user's device screen. As output, it obtains the video that the user watches.

[1029] Step 9:

[1030] Users access the system using a terminal. As input, they use a dedicated application or a web browser. Specifically, they launch a dedicated application on a smartphone or PC, or access the system's URL in a web browser. As output, they gain access to the system.

[1031] Step 10:

[1032] The user selects the match they want to watch. The input is from the list of matches or the search function on the viewing interface. The specific action is to scroll through the list of matches and click on the match they want to watch. They can also use the search function to search for a specific match. The output is the selected match.

[1033] Step 11:

[1034] The user watches the commentary video of the selected match. As input, the match selected in step 10 is received. Specifically, the user clicks the play button to start watching the video, and uses the pause and rewind buttons to navigate the video as needed. As output, the video of the match being watched is obtained.

[1035] (Application example 2)

[1036] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1037] Conventional basketball game commentary systems rely on manual broadcasting and commentary, making it difficult to provide a wide audience with information on youth and mini-basketball games held across the country. Furthermore, they lacked game highlights and replay features, as well as post-game statistical data and analysis reports, making it difficult to increase viewer satisfaction. Furthermore, they lacked the ability for users to follow their favorite teams and players, making it difficult to meet individual needs.

[1038] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for AI to provide commentary on live broadcasts of basketball games, means for distributing commentary on youth and mini-basketball games across the country with AI commentary, means for AI to automatically generate running scores that are created manually by officials and for each team, means for collecting video and audio data of the games and analyzing it with AI to generate commentary, means for distributing games in real time and providing commentary by AI through an application installed on a smartphone, means for providing game highlights and replay functions, means for enabling users to follow their favorite teams and players, and means for providing post-game statistical data and analysis reports. This makes it possible to widely provide youth and mini-basketball games held across the country to viewers and increase viewer satisfaction.

[1039] "A means for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to generate commentary in real time during a basketball game and provide it to viewers.

[1040] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that broadcasts youth and mini-basketball games held all over the country in real time with commentary by artificial intelligence.

[1041] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[1042] "Method of collecting video and audio data from the match and analyzing it with AI to generate commentary" refers to a technology that collects video and audio data from the match and analyzes it with artificial intelligence to generate commentary.

[1043] "Means of broadcasting matches in real time and providing commentary by AI through an application installed on a smartphone" refers to a technology that uses a smartphone application to broadcast matches in real time and provide commentary by artificial intelligence.

[1044] "Means for providing match highlight and replay functions" refers to technology that provides a function for compiling important scenes and plays during a match as highlights or playing them as replays.

[1045] "Means for providing a function that enables users to follow their favorite teams and players" refers to technology that provides a function that enables users to select and follow their favorite teams and players and receive information about them on a priority basis.

[1046] "Means for providing post-match statistical data and analytical reports" refers to technology that generates match statistical data and analytical reports after the match has ended and provides them to users.

[1047] The following system configuration will be described as an embodiment of the present invention.

[1048] System Configuration

[1049] The system includes a server, a smartphone, and an AI model. The server collects video and audio data from matches, analyzes it using the AI ​​model, and generates commentary. The smartphone provides an application that allows users to watch matches in real time and receive commentary from the AI.

[1050] Hardware and software used

[1051] Server: A server with high-performance data processing capabilities (e.g., AWS EC2 instance)

[1052] Smartphone: iOS or Android device

[1053] AI model: OpenAI's GPT-4

[1054] Data processing libraries: OpenCV (for image processing), Python (programming language)

[1055] Data processing and calculation

[1056] 1. Collection of match video and audio data:

[1057] The server collects video and audio data from youth and mini-basketball matches held across the country in real time, using input devices such as cameras and microphones.

[1058] 2. AI-based analysis and commentary generation:

[1059] The server analyzes the collected video and audio data using OpenCV to extract important scenes and plays from the match. Based on the extracted information, it generates commentary using OpenAI's GPT-4. Specifically, it uses the following prompt:

[1060] Example prompt sentence:

[1061] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[1062] 3. Real-time streaming and commentary:

[1063] The server delivers the game footage along with the generated commentary in real time to smartphones, where an application allows users to receive AI-generated commentary while watching the game.

[1064] 4. Providing highlight and replay features:

[1065] The server compiles key scenes from the match as highlights and provides a replay function, allowing users to rewatch key moments of the match.

[1066] 5. User following feature:

[1067] The smartphone application allows users to follow their favorite teams and players, which allows them to receive information about those teams and players on a priority basis.

[1068] 6. Providing post-match statistics and analysis reports:

[1069] After the match, the server generates statistical data and analysis reports and provides them to users through a smartphone application, allowing them to obtain detailed analysis information about the match.

[1070] Specific examples

[1071] For example, if a user is watching a youth basketball game through a smartphone application, the server will deliver the game footage and AI commentary in real time. The moment Player A scores a goal during the game, the server will analyze the scene and generate the following prompt:

[1072] Example prompt sentence:

[1073] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[1074] Based on this prompt, the AI ​​model generates commentary and provides it to users. Users can enjoy the AI ​​commentary in real time while watching the game. After the game, they can also get detailed information about the game through statistical data and analysis reports.

[1075] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1076] Step 1:

[1077] The server collects video and audio data from youth and mini-basketball matches held across the country. As input, it receives real-time data from input devices such as cameras and microphones, and as output, it stores this data in storage. Specifically, the server receives video and audio data sent from each match venue, converts it into the appropriate format, and stores it.

[1078] Step 2:

[1079] The server analyzes the collected video and audio data. It uses the video and audio data saved in step 1 as input and extracts information about important scenes and plays from the game as output. Specifically, the server uses OpenCV to analyze the video data frame by frame and identify player movements and the position of the ball. It also extracts events such as goals and fouls from the audio data.

[1080] Step 3:

[1081] The server generates a prompt sentence based on the analysis results. It uses information about important scenes and plays in the game extracted in step 2 as input, and generates a prompt sentence to be passed to the AI ​​model as output. Specifically, the server converts information such as player movements and scoring status into text format and generates a prompt sentence like the one below.

[1082] Example prompt sentence:

[1083] Generate commentary based on the following match situations: Player A has the ball. Player B is defending. Player A scores a goal.

[1084] Step 4:

[1085] The server sends the generated prompt sentence to the AI ​​model to generate an explanation. It uses the prompt sentence generated in step 3 as input and receives the explanation text generated from the AI ​​model as output. Specifically, the server calls OpenAI's GPT-4 API, sends the prompt sentence, and generates an explanation.

[1086] Step 5:

[1087] The server delivers the generated commentary and match video to smartphones in real time. It uses the commentary text generated in step 4 and the video data collected in step 1 as input, and generates data to be delivered to the user's smartphone as output. Specifically, the server overlays the commentary text on the video data and sends it to the smartphone in streaming format.

[1088] Step 6:

[1089] Users watch the game through a smartphone application and receive AI commentary. The input is the video and commentary data delivered from the server, and the output is the display of the game video and commentary to the user. Specifically, the smartphone application decodes the received data and displays it on the screen.

[1090] Step 7:

[1091] The server compiles important scenes from the match as highlights and provides a replay function. It uses the information of important scenes extracted in step 2 as input and generates a highlight video as output. Specifically, the server edits the important scenes and saves them in a format that users can replay.

[1092] Step 8:

[1093] Users follow their favorite teams and players through a smartphone application. The application uses the user's selection information as input and prioritizes displaying information about the followed teams and players as output. Specifically, the application records the user's selection and prioritizes retrieving and displaying relevant information.

[1094] Step 9:

[1095] The server generates statistical data and analysis reports after the match ends and provides them to users. It uses data collected during the match as input and generates statistical data and analysis reports as output. Specifically, the server aggregates the match data, creates a report in a visually easy-to-understand format, and provides it to users through a smartphone application.

[1096] Example 3

[1097] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1098] In traditional basketball games, keeping track of the score and foul situation in real time required the creation of a running score manually, which was time-consuming and labor-intensive. Additionally, there was a lack of a system to effectively collect and analyze data on teams and players nationwide and use it for player development. This meant that there was insufficient visualization of player performance and feedback for development.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1100] In this invention, the server includes means for receiving video data and audio data of a game, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score and foul status from the analysis results, means for generating a running score based on the extracted information, means for updating the generated running score in real time and transmitting it to the terminal, means for displaying the running score transmitted to the terminal, means for collecting information such as game results and stats of teams and players nationwide, means for compiling the collected information into a database, means for analyzing the databased information and evaluating player performance, means for generating feedback for player development based on the analysis results, means for transmitting the generated feedback to the terminal, and means for displaying the feedback transmitted to the terminal. This makes it possible to grasp the progress of the game in real time, and to effectively visualize player performance and provide feedback for development.

[1101] "Game video data" refers to a video file of a basketball game captured on camera.

[1102] "Audio data" refers to audio files recorded by a microphone during a basketball game.

[1103] "Means for receiving" is a function that allows the server to obtain data from the outside.

[1104] The "analyzing means" is a function for processing received data and extracting necessary information.

[1105] "Score status" is information indicating how many points each team has scored during the game.

[1106] "Foul situation" is information that indicates which player committed what kind of foul during the game.

[1107] A "running score" is a record of points and fouls that is updated in real time as the game progresses.

[1108] "Means of updating in real time" refers to a function that instantly reflects data as the game progresses.

[1109] A "terminal" is a device that a user uses to view information.

[1110] "Display means" refers to a function for visually presenting information on a terminal.

[1111] "Match result" is information indicating the final score and outcome after the match has finished.

[1112] "Stats" are statistical data that quantify the performance of a player or team.

[1113] "Means of collection" refers to the function for collecting the necessary data.

[1114] "Means for creating a database" is a function for organizing and storing collected data.

[1115] "Means of analysis" refers to the function of processing databased information and deriving meaningful results.

[1116] "Feedback" refers to advice or evaluation provided based on the analysis results.

[1117] "Visualization" refers to displaying data visually to make it easier to understand.

[1118] "Development" refers to training and instruction to improve a player's abilities.

[1119] This invention is a system for evaluating and developing player performance by grasping the score and foul situation in a basketball game in real time. A specific embodiment of this system will be described below.

[1120] System configuration

[1121] Hardware

[1122] Server: A computer with high-performance computing power that receives, analyzes, generates, and transmits data.

[1123] Terminal: A device on which a user checks information, including a smartphone, tablet, or PC.

[1124] Camera: A device used to capture video data of a match.

[1125] Microphone: A device used to capture audio data from a match.

[1126] software

[1127] Video analysis software: Uses libraries such as OpenCV to analyze video data.

[1128] Audio analysis software: Use a library such as LibROSA to analyze audio data.

[1129] Database management system: Collected data is stored in a database using a system such as MySQL.

[1130] Data analysis tools: Analyze data using tools such as Pandas and NumPy.

[1131] Generative AI models, such as GPT-4, are used to generate running scores and feedback.

[1132] System Operation

[1133] Data reception and analysis

[1134] The server receives video and audio data from the camera and microphone in real time. The video data is analyzed using video analysis software to detect goals and fouls. The audio data is analyzed using audio analysis software to analyze the commentary and audience reactions.

[1135] Information extraction and running score generation

[1136] The server extracts the score and foul situations from the analysis results. Based on the extracted information, it uses a generative AI model to generate a running score. The generated running score is updated in real time and sent to the device.

[1137] Data Display

[1138] The terminal displays the received running score, allowing the user to grasp the progress of the match in real time.

[1139] Data collection and database creation

[1140] The server collects information such as match results and stats for teams and players from around the country, and the collected information is stored in a database using a database management system.

[1141] Data analysis and feedback generation

[1142] The server analyzes the databased information and evaluates the player's performance. Based on the analysis results, a generative AI model is used to generate feedback for player development. The generated feedback is sent to the device, which displays it. This allows the user to check the analysis results of the player's performance and receive advice for development.

[1143] Specific examples

[1144] When a user is watching a basketball game, the server analyzes the video and audio of the game and automatically extracts information about scores and fouls.

[1145] The server generates information such as "Team A has scored" or "Team B has committed a foul" in real time and sends it to the terminal.

[1146] The device displays this information, allowing users to check the progress of the match in real time.

[1147] Prompt Sentence Examples

[1148] "Analyze video and audio data from basketball games, extract the scoring and foul situations, and generate a running score."

[1149] "Analyze the game results and stats of basketball teams and players nationwide, evaluate the performance of the players, and generate feedback for training." The flow of the specific process in the third embodiment will be described with reference to FIG.

[1150] Step 1:

[1151] The server receives the video and audio data of the match.

[1152] Input: Real-time data from camera and microphone

[1153] Specific operation: Receives video and audio transmitted from the camera and microphone via the network.

[1154] Output: Received video and audio data

[1155] Step 2:

[1156] The server analyzes the received video data.

[1157] Input: Received video data

[1158] Specific operation: Using video analysis software (e.g., OpenCV), score and foul scenes are detected from video data.

[1159] Output: Time-stamped data of goals and fouls

[1160] Step 3:

[1161] The server analyzes the received audio data.

[1162] Input: Received audio data

[1163] What it does: Use audio analysis software (e.g., LibROSA) to analyze commentary and audience reactions from audio data.

[1164] Output: Timestamped data of commentary and audience reactions

[1165] Step 4:

[1166] The server extracts the score and foul situations from the analysis results.

[1167] Input: Timestamps of goals and fouls, as well as timestamped commentary and crowd reactions

[1168] Specific operations: Integrate the results of video analysis and audio analysis to extract the occurrence of goals and fouls.

[1169] Output: Scoring and foul data

[1170] Step 5:

[1171] The server generates a running score based on the extracted information.

[1172] Input: Scoring and foul data

[1173] What it does: Uses a generative AI model (e.g., GPT-4) to generate score and foul information in text format.

[1174] Output: Text data of running score

[1175] Step 6:

[1176] The server updates the generated running score in real time and transmits it to the terminal.

[1177] Input: Text data of running score

[1178] Specific operation: Sends the running score to the device using WebSocket or HTTP protocol.

[1179] Output: Running score sent to the device

[1180] Step 7:

[1181] The terminal displays the received running score.

[1182] Input: Running score sent to the device

[1183] Specific operation: Visually display the running score on the device display.

[1184] Output: A running score that can be visually checked by the user

[1185] Step 8:

[1186] The server collects information such as match results and stats for teams and players across the country.

[1187] Input: Data from each team's official website and sports data provider services

[1188] What it does: Collects match results and stats using web scraping and APIs.

[1189] Output: Collected match results and stats data

[1190] Step 9:

[1191] The server stores the collected information in a database.

[1192] Input: Collected match results and stats data

[1193] Specific behavior: Use a database management system (e.g. MySQL) to store data in tables.

[1194] Output: Match results and stats stored in a database

[1195] Step 10:

[1196] The server analyzes the databased information and evaluates the players' performance.

[1197] Input: Match results and stats data stored in a database

[1198] Specific operations: Use data analysis tools (e.g., Pandas, NumPy) to calculate players' shooting percentages, defensive strength, etc.

[1199] Output: Player performance evaluation data

[1200] Step 11:

[1201] The server generates feedback for player development based on the analysis results.

[1202] Input: Player performance evaluation data

[1203] What it does: Uses a generative AI model (e.g., GPT-4) to generate textual feedback for player development.

[1204] Output: Text data of feedback for training

[1205] Step 12:

[1206] The server transmits the generated feedback to the terminal.

[1207] Input: Text data of feedback for training

[1208] Specific behavior: Sends feedback to the device using WebSocket or HTTP protocols.

[1209] Output: Feedback sent to the terminal

[1210] Step 13:

[1211] The device displays the feedback it receives.

[1212] Input: Feedback sent to the device

[1213] Specific behavior: Provide visual feedback on the device display.

[1214] Output: Visual feedback for the user

[1215] (Application example 3)

[1216] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1217] Traditional live broadcasts and commentary of basketball games rely on human commentators, making it difficult to provide accurate information in real time. Commentary on youth and mini-basketball games across the country also relies on human intervention, making it difficult to provide information efficiently. Furthermore, detecting anomalies and improving production efficiency are key issues when monitoring production lines in factories. To solve these issues, an automated system utilizing AI is needed.

[1218] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1219] In this invention, the server includes a means for AI to provide commentary on live broadcasts of basketball games, a means for distributing commentary on youth and mini-basketball games from around the country with AI play-by-play, a means for AI to automatically generate running scores that are created manually by officials and for each team, a means for analyzing camera footage and audio data within the factory to detect production status and abnormalities in real time and provide scores, and a means for storing past production data in a database and providing feedback for improving efficiency. This makes it possible to accurately grasp the progress of games and the status of production lines in real time, enabling efficient information provision and improved production efficiency.

[1220] "AI-based commentary for live broadcasts of basketball games" refers to a technology that uses artificial intelligence to automatically provide commentary on the progress of a basketball game and the plays in real time.

[1221] "A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by AI" refers to a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games nationwide and broadcasts that commentary in real time.

[1222] "A means of automatically generating running scores using AI, which are currently created manually by officials and for each team," is a technology in which artificial intelligence automatically analyzes scores such as points scored and fouls while the game is in progress, and generates scores in real time.

[1223] "A means of analyzing camera footage and audio data within a factory to detect and score production status and abnormalities in real time" is a technology in which artificial intelligence analyzes video and audio data obtained from cameras and microphones installed within a factory, detects production line status and abnormalities in real time, and displays the results as a score.

[1224] "Means of creating a database of past production data and providing feedback to improve efficiency" refers to a technology that stores past production data collected within a factory in a database, analyzes that data, and provides feedback to improve production efficiency.

[1225] The following system configuration will be described as an embodiment of the present invention.

[1226] System Configuration

[1227] Hardware

[1228] Camera: Cameras installed inside the factory capture video data.

[1229] Microphone: A microphone installed in the factory captures voice data.

[1230] Server: A server for analyzing and storing data.

[1231] Device: The device (PC, smartphone, etc.) used by the administrator to check the results.

[1232] software

[1233] OpenCV: A library for acquiring and preprocessing camera footage.

[1234] TensorFlow / Keras: Libraries for loading AI models and making predictions.

[1235] SQLite: A database management system for storing production data.

[1236] Data processing and calculation

[1237] The server captures camera images and audio data from inside the factory in real time and preprocesses them using OpenCV. The preprocessed data is input into an AI model using TensorFlow / Keras to detect production status and abnormalities. The detection results are displayed as scores and stored in an SQLite database.

[1238] Specific examples

[1239] For example, if an abnormality occurs in a factory, it can detect it from camera footage and notify managers in real time. It can also analyze past production data and provide feedback to improve efficiency.

[1240] Prompt Sentence Examples

[1241] An example of a prompt to input to a generative AI model is as follows:

[1242] "Please create an AI model that analyzes camera footage within the factory and detects abnormalities on the production line in real time. Please also add a function that notifies the administrator when an abnormality is detected. Also, please build a system that stores past production data in a database and provides feedback to improve efficiency."

[1243] In this way, the server can monitor the status of the production line in real time, detect abnormalities, and provide feedback for efficiency improvements based on past data.

[1244] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1245] Step 1:

[1246] The server acquires video and audio data in real time from cameras and microphones installed in the factory. The input is raw data from the cameras and microphones, and the output is the acquired video and audio data. Specifically, the server connects to the cameras and microphones and starts the data stream.

[1247] Step 2:

[1248] The server preprocesses the acquired video data using OpenCV. The input is the video data acquired in step 1, and the output is the preprocessed video data. Specifically, the server resizes the video data to 224x224 pixels and normalizes it.

[1249] Step 3:

[1250] The server inputs the preprocessed video data into the AI ​​model using TensorFlow / Keras to make predictions. The input is the video data preprocessed in step 2, and the output is the predicted results of production status and abnormality occurrence. Specifically, the server loads the AI ​​model, inputs the preprocessed data into the model, and makes predictions.

[1251] Step 4:

[1252] The server displays the prediction result as a score and saves it in an SQLite database. The input is the prediction result obtained in step 3, and the output is the displayed score and the data saved in the database. Specifically, the server converts the prediction result into a score, displays it on the screen, and inserts it into the database.

[1253] Step 5:

[1254] The terminal receives the score and anomaly detection notification sent from the server and displays them to the administrator. The input is the score and notification sent from the server, and the output is the display information that the administrator can check. Specifically, the terminal receives data from the server and displays it on the user interface.

[1255] Step 6:

[1256] The server analyzes past production data and generates feedback for efficiency improvement. The input is past production data stored in a database, and the output is feedback information for efficiency improvement. Specifically, the server retrieves past data from the database and generates feedback by applying an analysis algorithm.

[1257] Step 7:

[1258] The terminal receives the feedback information sent from the server and displays it to the administrator. The input is the feedback information sent from the server, and the output is the display information that the administrator can check. In concrete terms, the terminal receives the feedback information from the server and displays it on the user interface.

[1259] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1260] "Example 1"

[1261] In one embodiment of the present invention, an AI system for commentating on live broadcasts of basketball games is combined with an emotion engine that recognizes the user's emotions. Specifically, the emotion engine recognizes the emotions (joy, sadness, surprise, etc.) expressed by the user while watching the game, and adjusts the content and tone of the commentary according to those emotions. For example, if the user is happy with the progress of the game, the commentary will also share that joy. This allows the user to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the game.

[1262] "Example 2"

[1263] The emotion engine is also incorporated into a system that broadcasts live commentary of youth and mini-basketball games from around the country, complete with AI commentary. Specifically, the emotion engine recognizes the emotions of users watching youth and mini-basketball games, and adjusts the content and tone of the commentary accordingly. For example, if a user is moved by their child's performance, the commentary will also share that emotion. This allows users to hear commentary that reflects their own emotions, further increasing their satisfaction with watching the games.

[1264] "Example 3"

[1265] Furthermore, emotion engines are also being incorporated into systems that automatically generate running scores created by AI for each official or team. Specifically, the emotion engine recognizes the user's emotions regarding the score of the game and adjusts the score display method according to those emotions. For example, if the user expresses joy at a score, the score display will also emphasize that joy. This allows users to see a score display that reflects their own emotions, further increasing their satisfaction when watching the game.

[1266] The processing flow of each embodiment will be described below.

[1267] "Example 1"

[1268] Step 1: The AI ​​begins commentating on a live basketball game.

[1269] Step 2: The emotion engine recognizes the user's emotions in real time.

[1270] Step 3: The AI ​​adjusts the content and tone of the commentary based on the emotions recognized by the emotion engine.

[1271] "Example 2"

[1272] Step 1: AI will begin providing live commentary for youth and mini-basketball matches across the country.

[1273] Step 2: The emotion engine recognizes the emotions of users watching the game in real time.

[1274] Step 3: The AI ​​adjusts the content and tone of the commentary based on the emotions recognized by the emotion engine.

[1275] "Example 3"

[1276] Step 1: AI automatically generates a running score based on the score of the match.

[1277] Step 2: The emotion engine recognizes the user's emotions regarding the score of the game in real time.

[1278] Step 3: The AI ​​adjusts the way the score is displayed depending on the emotion recognized by the emotion engine.

[1279] Example 1

[1280] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1281] Conventional basketball game commentary systems have struggled to analyze the game situation and player movements in real time and provide commentary that responds to the user's emotions. Furthermore, there are lack of efficient ways to deliver commentary for youth and mini-basketball games across the country, and a lack of ways to automatically generate running scores that are currently created manually by officials and teams. Furthermore, there is insufficient utilization of emotion recognition technology to improve the user viewing experience.

[1282] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1283] In this invention, the server includes means for providing commentary for live coverage of basketball games using artificial intelligence, means for distributing commentary for youth and mini-basketball games from around the country with live coverage by artificial intelligence, means for automatically generating running scores created by humans for each official and team, means for analyzing game video data, means for analyzing game audio data, means for receiving and analyzing user emotion data, means for automatically generating commentary based on the analysis results and adjusting the content and tone of the commentary according to the user's emotion, and means for distributing the adjusted commentary along with the live coverage. This makes it possible to analyze the game situation and player movements in real time and provide commentary according to the user's emotion.

[1284] "Means for artificial intelligence to provide commentary for live broadcasts of basketball games" refers to a technology that analyzes video and audio data of basketball games, and automatically generates commentary using artificial intelligence based on the results of that analysis, which is then distributed along with the live broadcast.

[1285] "A means of broadcasting commentary on youth and mini-basketball games nationwide with live coverage by artificial intelligence" is a technology that uses artificial intelligence to automatically generate commentary for youth and mini-basketball games nationwide and broadcasts that commentary along with live coverage.

[1286] "A means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[1287] "Means for analyzing game video data" refers to technology that analyzes game video data frame by frame to identify the positions and movements of players, the position of the ball, etc.

[1288] "Means for analyzing audio data of a match" refers to a technology that analyzes audio data of a match and identifies the content of the commentary and the reactions of the audience.

[1289] The "means for receiving and analyzing user emotion data" refers to a technology that captures the user's facial expressions and voice, analyzes the data, and identifies the user's emotion.

[1290] "Means for automatically generating commentary based on analysis results and adjusting the content and tone of the commentary according to the user's emotions" refers to a technology that automatically generates commentary based on the analysis results of video and audio data, and further adjusts the content and tone of the commentary taking into account the user's emotional data.

[1291] The "means for delivering adjusted commentary together with live broadcast" is a technology for delivering adjusted commentary to a user's terminal in real time together with live broadcast.

[1292] MODE FOR CARRYING OUT THE INVENTION

[1293] This invention is a system that uses artificial intelligence to provide commentary for live coverage of basketball games, analyzing video and audio data of the game and providing commentary that responds to the user's emotions. A specific embodiment of this system is described below.

[1294] Hardware and software used

[1295] Hardware: High-performance servers, user devices (PCs, smartphones, tablets, etc.)

[1296] Software: Video analysis software (e.g., OpenCV), audio analysis software (e.g., Google Cloud Speech-to-Text API), emotion engine (e.g., Microsoft Azure Emotion API), generative AI model (e.g., GPT-4)

[1297] System configuration

[1298] 1. The server receives the video and audio data of the match in real time. It uses a high-speed network connection to receive the streaming data sent from the match venue.

[1299] 2. The server uses video analysis software to analyze the received video data frame by frame. Specifically, it uses computer vision techniques, such as OpenCV, to identify the positions and movements of players and the position of the ball.

[1300] 3. The server uses voice analysis software to analyze the received audio data. Specifically, it uses voice recognition technology, such as the Google Cloud Speech-to-Text API, to identify the content of the commentary and the audience's reactions.

[1301] 4. The server automatically generates commentary on the match using a generative AI model based on the results of video and audio analysis. Specifically, it uses a natural language generation model such as GPT-4 to generate sentences that explain the situation of the match and the movements of the players.

[1302] 5. The device captures the user's facial expressions and voice in real time and sends them to the server as emotional data, specifically using a camera and microphone.

[1303] 6. The server uses an emotion engine to analyze the received emotion data. Specifically, it uses emotion recognition technology, such as the Microsoft Azure Emotion API, to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

[1304] 7. The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed by the server. Specifically, if the user is happy, the tone of the commentary will be brighter and more positive expressions will be used.

[1305] 8. The server delivers the adjusted commentary along with the live broadcast to the user's device. Specifically, it delivers it in real time using streaming technology, such as WebRTC.

[1306] Specific examples

[1307] For example, if a user expresses joy while watching a game, the emotion engine recognizes that joy. Based on that information, the server adjusts the tone of the commentary generated by the generative AI model to be more cheerful and share the joy. This allows users to hear commentary that reflects their own emotions, improving their satisfaction watching the game.

[1308] Prompt Sentence Examples

[1309] "Design an AI system to provide commentary for live broadcasts of basketball games. It must analyze video and audio data from the game, understand the game situation and player movements, and incorporate an emotion engine to adjust the content and tone of the commentary based on the user's emotions."

[1310] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1311] Step 1:

[1312] The server receives the video and audio data of the match in real time. The input is streaming data sent from the match venue, and the output is video and audio data stored in the server. Specifically, the server receives the data using a high-speed network connection and stores it in the appropriate format.

[1313] Step 2:

[1314] The server uses video analysis software to analyze the received video data frame by frame. The input is the saved video data, and the output is the analysis results, such as the position of players, their movements, and the position of the ball. Specifically, it uses OpenCV to analyze each frame and identify the positions of players and the ball.

[1315] Step 3:

[1316] The server uses speech analysis software to analyze the received audio data. The input is the stored audio data, and the output is the analysis results, which are a text version of the commentary and audience reactions. Specifically, the server converts the audio data into text using the Google Cloud Speech-to-Text API, and extracts key information.

[1317] Step 4:

[1318] The server automatically generates commentary on the match using a generative AI model based on the results of video and audio analysis. The input is the results of video and audio analysis, and the output is the generated commentary. Specifically, it uses a natural language generation model such as GPT-4 to generate text that explains the situation of the match and the movements of the players.

[1319] Step 5:

[1320] The device captures the user's facial expressions and voice in real time and sends them to the server as emotional data. The input is the user's facial expressions and voice, and the output is the emotional data sent to the server. Specifically, the device uses a camera and microphone to capture the user's facial expressions and voice and sends them to the server in real time.

[1321] Step 6:

[1322] The server uses an emotion engine to analyze the received emotion data. The input is the transmitted emotion data, and the output is the analysis result that identifies the user's emotion. Specifically, it uses the Microsoft Azure Emotion API to identify emotions such as joy, sadness, and surprise from the user's facial expressions and voice.

[1323] Step 7:

[1324] The server adjusts the content and tone of the generated commentary according to the user's emotions analyzed. The input is the result of emotion analysis and the generated commentary, and the output is the adjusted commentary. Specifically, if the user expresses joy, the tone of the commentary becomes brighter and more positive expressions are included.

[1325] Step 8:

[1326] The server delivers the adjusted commentary along with the live broadcast to the user's device. The input is the adjusted commentary and the live broadcast data, and the output is the live broadcast with real-time commentary delivered to the user's device. Specifically, WebRTC is used to deliver the adjusted commentary in real time.

[1327] (Application example 1)

[1328] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1329] Conventional basketball game commentary systems have the problem that commentary is uniform and cannot flexibly respond to viewer emotions. Furthermore, improving the quality of live game coverage and commentary requires a large number of people, which increases costs and effort. Furthermore, there is a lack of efficient means to provide commentary on youth and mini-basketball games across the country.

[1330] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for AI to provide commentary on live broadcasts of basketball games, a means for distributing commentary on youth and mini-basketball games nationwide with AI commentary, a means for AI to automatically generate running scores that are created manually for each official and team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes user emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting prompt sentences. This enables flexible commentary that responds to viewer emotions, improving the quality of live broadcasts and commentary. Furthermore, commentary on youth and mini-basketball games nationwide can be efficiently provided.

[1331] "A method for AI to provide commentary for live broadcasts of basketball games" refers to a method that uses artificial intelligence technology to analyze video and audio data from basketball games, understand the situation of the game and the movements of the players, and automatically generate commentary.

[1332] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" means a means of broadcasting live coverage including commentary on youth and mini-basketball games from around the country with AI providing commentary.

[1333] "A means for AI to automatically generate running scores that are created manually by officials and teams" refers to a means for AI to automatically generate scores that are manually created by officials and teams during a match.

[1334] "Means for adjusting the content and tone of commentary using an emotion engine that recognizes the user's emotions" refers to means that use emotion recognition technology to recognize the user's emotions (joy, sadness, surprise, etc.) and adjust the content and tone of commentary according to those emotions.

[1335] "Means for generating commentary using a generative AI model" refers to means for generating commentary based on the situation of a match using a generative AI model (such as GPT-3).

[1336] "Means for inputting a prompt sentence and generating an explanation" refers to means for inputting a specific prompt sentence into a generative AI model and generating an explanation based on that prompt.

[1337] A system for implementing this invention includes a means for AI to provide commentary for live broadcasts of basketball games, a means for providing live commentary of youth and mini-basketball games across the country with AI, a means for AI to automatically generate running scores that are created manually for each official or team, a means for adjusting the content and tone of the commentary using an emotion engine that recognizes the user's emotions, a means for generating commentary using a generative AI model, and a means for generating commentary by inputting a prompt sentence.

[1338] Hardware and software used

[1339] Hardware: Smartphone camera and microphone

[1340] Software: OpenCV, Keras, Transformers

[1341] Data processing and calculation

[1342] server

[1343] The server receives and analyzes video and audio data from basketball games in real time. It uses OpenCV to acquire the video data and Keras to run an emotion recognition model. The emotion recognition model analyzes the user's facial expressions and recognizes emotions such as joy, sadness, and surprise.

[1344] Terminal

[1345] The device generates commentary using a generative AI model (e.g., GPT-3). A prompt sentence is input into the generative AI model, which generates commentary based on the prompt. Examples of prompt sentences include "The user is happy. Generate a joyful commentary for the basketball game." and "The user is sad. Generate a comforting commentary for the basketball game."

[1346] User

[1347] Users watch the game through their smartphone's camera and microphone, and their emotions are recognized. The emotion engine adjusts the content and tone of the commentary based on the user's emotions. For example, if a user expresses joy at a highlight scene, the commentary might say something like, "Great play! We've been waiting for this moment!" Conversely, if a user expresses sadness at a comeback, the commentary might say, "It's a disappointing result, but there's still a chance. Let's look forward to the next play."

[1348] This allows for flexible commentary that responds to the emotions of viewers, improving the quality of live broadcasts and commentary. It also allows for efficient commentary of youth and mini-basketball games across the country.

[1349] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1350] Step 1:

[1351] The server receives video and audio data from the smartphone's camera and microphone in real time. The input is the video and audio data from the camera and microphone, and the output is a state where this data is ready for analysis.

[1352] Step 2:

[1353] The server processes the video data using OpenCV to detect the user's face. The input is the video data received in step 1, and the output is the detected face area. Specifically, the server converts the video data to grayscale and applies a face detection algorithm.

[1354] Step 3:

[1355] The server runs an emotion recognition model using Keras to recognize the user's emotions. The input is the face region detected in step 2, and the output is the recognized emotion (happiness, sadness, surprise, etc.). Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model.

[1356] Step 4:

[1357] The server generates a prompt sentence to be input to the generative AI model based on the recognized emotion. The input is the emotion recognized in step 3, and the output is the generated prompt sentence. Specifically, the server selects a template sentence according to the emotion and generates the prompt sentence.

[1358] Step 5:

[1359] The server generates an explanation using a generative AI model (e.g., GPT-3). The input is the prompt generated in step 4, and the output is the generated explanation. Specifically, the prompt is input into the generative AI model, and an explanation is generated.

[1360] Step 6:

[1361] The server delivers the generated commentary to the user in real time. The input is the commentary generated in step 5, and the output is the commentary audio delivered to the user's smartphone. Specifically, the commentary is input into a speech synthesis engine, and audio data is generated and delivered.

[1362] Step 7:

[1363] Users watch the game and listen to the commentary through their smartphones. The input is the commentary audio delivered from the server, and the output is to improve the user's viewing experience. Specifically, the commentary audio is played from the smartphone's speaker.

[1364] Example 2

[1365] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1366] Traditional basketball game commentary relied on the commentator's subjective judgment, making it difficult to provide commentary that reflected the viewer's emotions. Furthermore, there was a lack of systems to provide nationwide youth and mini-basketball games to a wide audience, making it difficult to grasp the progress of the game and the performance of the players in real time. Furthermore, manually creating a running score for a game was time-consuming and labor-intensive, requiring efficient management.

[1367] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1368] In this invention, the server includes means for providing commentary on live broadcasts of basketball games using artificial intelligence, means for distributing commentary on youth and mini-basketball games from around the country with live broadcasts by artificial intelligence, means for automatically generating running scores created by humans for each official and team using artificial intelligence, means for collecting game video and audio data, means for analyzing the collected data and generating commentary, means for recognizing viewer emotions and adjusting the content and tone of the commentary according to those emotions, and means for distributing the generated commentary together with the game video. This makes it possible to provide commentary that corresponds to the viewer's emotions and improve the satisfaction of watching the game.

[1369] "Means for AI to provide commentary for live broadcasts of basketball games" refers to a technology in which AI analyzes the progress of a basketball game and the performance of players, and generates commentary in real time.

[1370] "A means of broadcasting commentary on youth and mini-basketball games across the country with live coverage by artificial intelligence" is a technology that uses artificial intelligence to analyze youth and mini-basketball games held across the country, generate commentary, and broadcast it in real time.

[1371] "A means for automatically generating running scores using artificial intelligence that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually created by officials and teams during a match.

[1372] "Means for collecting game video and audio data" refers to technology that uses cameras and microphones installed at the game venue to collect game video and audio data in real time.

[1373] The "means of analyzing collected data and generating commentary" refers to a technology in which artificial intelligence analyzes collected video and audio data and generates commentary based on the progress of the match and the performance of the players.

[1374] "Means for recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to technology that analyzes the viewer's facial expressions and tone of voice and adjusts the content and tone of the commentary according to those emotions.

[1375] "Means for distributing generated commentary together with game footage" refers to technology for distributing commentary generated by artificial intelligence together with game footage via a streaming platform.

[1376] This invention is a system that uses artificial intelligence (AI) to broadcast live basketball games and also provides commentary on youth and mini-basketball games from around the country. Furthermore, it includes a function that automatically generates running scores, which are currently created by humans for each official and team.

[1377] The server collects video and audio data from the match in real time using cameras and microphones installed at the match venue, specifically using hardware such as Sony 4K cameras and Shure high-sensitivity microphones, and stores the collected data in a database on the server.

[1378] The server then inputs the stored video and audio data into an AI model. This AI model uses a generative AI model such as OpenAI's GPT-4 or Google's BERT. The AI ​​model analyzes the input data and generates commentary based on the progress of the game and the performance of the players. For example, it analyzes the moment Player A makes a shot and generates commentary such as, "Player A made a great shot!"

[1379] Furthermore, the device will recognize the emotions of the user watching the game. Specifically, it will use the device's camera and microphone to capture the user's facial expressions and tone of voice. This is done using Apple's Face ID technology or Amazon's Alexa, for example. The device will then send the captured data to a server.

[1380] The server receives the user's emotion data sent from the device and analyzes it using an emotion engine. The emotion engine uses, for example, Affectiva or IBM Watson's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional, generating a commentary such as, "Player A's shot is truly amazing! The crowd is really excited!"

[1381] Finally, the server distributes the generated commentary along with the game footage via streaming platforms such as YouTube and Twitch, allowing users to watch the game and enjoy the emotional commentary.

[1382] As a concrete example, consider a youth basketball game. The server collects video and audio data in real time from a Sony 4K camera and Shure high-sensitivity microphones installed at the game venue. The collected data is then input into OpenAI's GPT-4 to analyze the moment Player A makes a shot. The AI ​​model generates a commentary such as, "Player A made a great shot!"

[1383] Meanwhile, the device uses Apple's Face ID technology to capture the facial expressions of the user watching the game and recognize that they are emotionally moved. The server then uses Affectiva's emotion analysis API to analyze the user's emotions and adjust the tone of the commentary to be more emotional. It generates a commentary such as, "Player A's shot is truly amazing! The crowd is really excited!" and distributes it via YouTube.

[1384] An example of a prompt might be:

[1385] "Analyze video and audio data from youth basketball games to generate commentary on game progress and player performance. Also, recognize viewer emotions and adjust the content and tone of the commentary accordingly."

[1386] In this way, the server can provide commentary to the user according to their emotions, thereby improving the satisfaction of watching the match.

[1387] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1388] Step 1:

[1389] The server collects video and audio data from matches in real time using cameras and microphones installed at the match venue. Specifically, it uses Sony 4K cameras and Shure high-sensitivity microphones. The input is the video and audio data from the match, and the output is the collected video and audio data. This data is stored in a database on the server.

[1390] Step 2:

[1391] The server inputs the stored video and audio data into an AI model. This AI model uses, for example, OpenAI's GPT-4. The input is the collected video and audio data, and the output is data on the analyzed progress of the game and player performance. The AI ​​model analyzes player movements and the progress of the game from the video data, and analyzes commentary and spectator reactions from the audio data.

[1392] Step 3:

[1393] The server generates commentary on the game based on the data analyzed by the AI ​​model. The input is the analyzed data on the progress of the game and the performance of the players, and the output is the generated commentary. For example, the server analyzes the moment when Player A makes a shot and generates commentary such as, "Player A made a great shot!"

[1394] Step 4:

[1395] The device recognizes the emotions of users watching a game. Specifically, it uses the device's camera and microphone to capture the user's facial expressions and tone of voice. The input is data about the user's facial expressions and tone of voice, and the output is the recognized user's emotional data. This uses Apple's Face ID technology and Amazon's Alexa.

[1396] Step 5:

[1397] The server receives the user's emotion data sent from the device and analyzes it using an emotion engine. The input is the user's emotion data, and the output is the analyzed emotion information. The emotion engine uses, for example, Affectiva's emotion analysis API. If the user is emotional, the server adjusts the tone of the commentary to be more emotional.

[1398] Step 6:

[1399] The server distributes the generated commentary along with the game video. The input is the generated commentary and the game video, and the output is the streaming data to be distributed. Streaming platforms such as YouTube and Twitch are used for distribution. Users can watch the game through these platforms and enjoy commentary that matches their emotions.

[1400] (Application example 2)

[1401] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1402] Conventional live basketball broadcasting systems have the problem of providing uniform commentary and being unable to respond flexibly to viewer emotions. Furthermore, they lack game highlights, replay features, and chat functions between viewers, leaving a need for an improved viewing experience. Furthermore, there were limited ways to provide nationwide youth and mini-basketball games to a wide audience, resulting in information disparities between regions.

[1403] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1404] In this invention, the server includes means for AI to provide commentary for live broadcasts of basketball games, means for distributing commentary for youth and mini-basketball games from around the country with AI commentary, means for AI to automatically generate running scores that are created manually for each official and team, means for recognizing viewer emotions and adjusting the content and tone of the commentary according to those emotions, means for providing game highlights and replay functions, and means for providing a chat function between viewers. This enables flexible commentary according to viewer emotions, improving the viewing experience and making it possible to widely provide youth and mini-basketball games from around the country to viewers.

[1405] "A method for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to generate commentary in real time for basketball games and provide it to viewers.

[1406] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" refers to a technology that broadcasts youth and mini-basketball games held all over the country in real time with commentary by artificial intelligence.

[1407] "A means for AI to automatically generate running scores that are created manually by officials and teams" is a technology that uses artificial intelligence to automatically generate scores that are manually recorded by officials and teams during a match.

[1408] "Means of recognizing the viewer's emotions and adjusting the content and tone of the commentary according to those emotions" refers to a technology that recognizes emotions from the viewer's facial expressions and voice, and changes the content and tone of the commentary according to those emotions.

[1409] "Means for providing match highlight and replay functions" refers to technology that provides a highlight function that extracts and plays important scenes from a match, and a replay function that replays a specific scene.

[1410] "Means for providing a chat function between viewers" refers to technology that provides a chat function that enables users watching a match to exchange messages in real time.

[1411] The system for carrying out this invention includes the following main components: a means for AI to provide commentary for live broadcasts of basketball games, a means for providing live commentary of youth and mini-basketball games from around the country with AI, a means for AI to automatically generate running scores that are created manually for each official and team, a means for recognizing the emotions of viewers and adjusting the content and tone of the commentary according to those emotions, a means for providing game highlights and replay functions, and a means for providing a chat function between viewers.

[1412] Hardware and software used

[1413] Hardware: Smartphone (camera, microphone, display)

[1414] Software: OpenCV (image processing library), EmotionRecognizer (emotion recognition model), AICommentary (AI commentary generation model), LiveStreaming (live streaming library)

[1415] Data processing and calculation

[1416] 1. Frame Acquisition: The server acquires frames from the live streaming, thereby collecting video data of the match in real time.

[1417] 2. Emotion Recognition: The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer, thereby obtaining the viewer's emotion data.

[1418] 3. Commentary generation: The server generates commentary using AICommentary based on the recognized emotions, which allows commentary to be generated in real time according to the viewer's emotions.

[1419] 4. Add commentary: The server adds the generated commentary to the live stream, allowing viewers to watch emotionally relevant commentary in real time.

[1420] 5. Highlights and Replays: The server automatically extracts key scenes from the match and provides highlight and replay features, allowing viewers to rewatch key moments of the match.

[1421] 6. Chat Function: The server provides a chat function that allows viewers to exchange messages with each other in real time, thereby promoting communication between viewers.

[1422] Specific examples

[1423] For example, during a youth basketball game, if the viewer is impressed, the server might generate a commentary such as, "Great play! I'll never forget this moment!" Or, if the viewer is excited, the server might generate a commentary such as, "Don't miss this! Watch the next play!"

[1424] Prompt Sentence Examples

[1425] “Create an AI model that can recognize the emotions of your audience and adjust the content and tone of your commentary accordingly. For example, if your audience is emotional, make sure your commentary reflects that emotion.”

[1426] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1427] Step 1:

[1428] The server acquires frames from the live streaming. The input is real-time game video data, and the output is individual frame images. Specifically, the server receives the video data sent from the camera and splits it into frames using OpenCV.

[1429] Step 2:

[1430] The server detects the viewer's face from the frame and recognizes the emotion using EmotionRecognizer. The input is the frame image obtained in step 1, and the output is the viewer's emotion data. Specifically, the server detects the face in the frame and inputs it into the emotion recognition model to classify the emotion.

[1431] Step 3:

[1432] The server generates a commentary using AICommentary based on the recognized emotion. The input is the emotion data obtained in step 2, and the output is commentary text corresponding to the emotion. Specifically, the server inputs the emotion data as a prompt into the AI ​​commentary generation model to generate an appropriate commentary.

[1433] Step 4:

[1434] The server adds the generated commentary to the live stream. The input is the commentary text generated in step 3, and the output is the live video with the commentary added. Specifically, the server synthesizes the commentary text into speech and overlays it on the video.

[1435] Step 5:

[1436] The server automatically extracts important scenes from the game and provides highlight and replay functions. The input is real-time game video data, and the output is highlight and replay videos. Specifically, the server detects events in the game, extracts important scenes, and saves them.

[1437] Step 6:

[1438] The server provides a chat function that allows viewers to exchange messages in real time. The input is message data from viewers, and the output is messages to be sent to other viewers. Specifically, the server receives messages and distributes them to other viewers in the chat room.

[1439] Example 3

[1440] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1441] In conventional basketball games, creating a running score and understanding the progress of the game were done manually, making it difficult to update in real time. Furthermore, there was a lack of a national database of teams and players, and feedback for analyzing player performance and developing players was not provided adequately. Furthermore, the score display did not reflect the user's feelings about the game's score, resulting in low satisfaction when watching the game.

[1442] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1443] In this invention, the server includes means for receiving video data and audio data of a game, means for analyzing the received video data, means for analyzing the received audio data, means for extracting the score status and foul status from the analysis results, means for generating a running score based on the extracted information, means for updating and transmitting the generated running score in real time, means for displaying the transmitted running score, and means for recognizing the user's emotions and adjusting the score display, thereby making it possible to grasp the progress of the game in real time and provide a score display that reflects the user's emotions.

[1444] "Game video data" refers to video data that records the progress of a basketball game.

[1445] "Audio data" refers to audio data that records commentary and commentary during a match, audience reactions, etc.

[1446] "Means for receiving" is a function for acquiring video data and audio data of a match from an external source.

[1447] The "analyzing means" is a function for processing received video and audio data and extracting specific information.

[1448] "Score status" is information indicating how many points each team has scored during the game.

[1449] "Foul situation" is information that indicates which player committed what kind of foul during the game.

[1450] A "running score" is a scoreboard that shows the score and foul situation updated in real time as the game progresses.

[1451] The "means for updating and transmitting in real time" is a function for instantly updating the generated running score and transmitting it to the user's terminal.

[1452] The "display means" is a function for visually presenting the transmitted running score to the user.

[1453] "Means for recognizing user emotions" is a function for analyzing and understanding the user's emotions regarding the score of a game, etc.

[1454] The "means for adjusting score display" is a function for changing the display method of the scoreboard according to the recognized user emotion.

[1455] "Game results and stats information for teams and players nationwide" refers to game results for basketball teams and players nationwide and statistical data for individual players.

[1456] "Means for saving to a database" refers to a function for organizing collected match results and stat information and storing it in a database.

[1457] "Means for analyzing performance" is a function for evaluating a player's performance and abilities based on information stored in the database.

[1458] "Means for generating feedback" refers to a function that creates reports based on the analysis results to suggest areas for player development and improvement.

[1459] The "means for transmitting and displaying" is a function for transmitting the generated feedback to the user's terminal and visually presenting it.

[1460] "Visualization" refers to displaying data visually to make it easier to understand.

[1461] "Means for use in development" refers to methods for supporting players' training and development based on data.

[1462] The present invention is a system for automatically generating running scores and analyzing player performance in basketball games. A specific embodiment of this system will be described below.

[1463] System configuration

[1464] Hardware and Software

[1465] 1. Server

[1466] Hardware: A server with a powerful processor and large memory capacity

[1467] Software: video analysis software (e.g., OpenCV), audio analysis software (e.g., Google Cloud Speech-to-Text), database management systems (e.g., MySQL), sentiment analysis engines (e.g., IBM Watson Tone Analyzer)

[1468] 2. Terminal

[1469] Hardware: PCs, tablets, smartphones, etc.

[1470] Software: Web browser, dedicated application

[1471] Program processing

[1472] Receiving and analyzing match data

[1473] The server receives video and audio data from the game. The video data is obtained from cameras and streaming services, and the audio data includes commentary and narration. The received video data is analyzed using OpenCV to detect scoring and foul scenes. The audio data is converted to text using Google Cloud Speech-to-Text, and information about scores and fouls is extracted from the commentary.

[1474] Extraction of scoring and foul situations

[1475] The server combines the results of video and audio analysis to extract the score and foul situations, making it possible to grasp the progress of the game in real time.

[1476] Generate and display a running score

[1477] The server generates a running score based on the extracted information. The generated running score is updated in real time and sent to the device using WebSocket or HTTP. The device displays the received running score to the user, allowing the user to check the progress of the game in real time.

[1478] Recognizing user emotions and adjusting score display

[1479] The server uses IBM Watson Tone Analyzer to recognize the user's emotions regarding the score of the game. The server adjusts the score display depending on the recognized emotion. For example, if the user is expressing joy at a score, the score display will be emphasized with bright colors and animations. The adjusted score display is sent to the device and displayed to the user.

[1480] Database creation and performance analysis of teams and players nationwide

[1481] The server collects game results and stats for teams and players across the country and stores them in a MySQL database. Based on the stored information, the server analyzes player performance using Python's Pandas library. Based on the analysis results, it generates feedback for player development and sends it to the device. The device then displays the received feedback to the user.

[1482] Specific examples

[1483] When the video data of the game is sent to the server, the server uses OpenCV to detect scoring scenes and Google Cloud Speech-to-Text to extract foul information from the commentary audio. For example, the prompt text can be "Please analyze the scoring scenes and foul information of this game and generate a running score."

[1484] The server collects the results of matches from around the country and stores them in a MySQL database. It uses the Python Pandas library to analyze the players' performance and generate feedback. For example, enter the following as a prompt: "Collect the results of matches from around the country, analyze the players' performance, and generate feedback."

[1485] The server recognizes the user's joy at the scoring scene using IBM Watson Tone Analyzer and adjusts the score display to emphasize the joy. For example, the server inputs a prompt statement such as "Please analyze the user's emotions at the scoring scene and adjust the score display." The flow of the identification process in the third embodiment will be described with reference to FIG. 21.

[1486] Step 1:

[1487] Receiving match data

[1488] The server receives video and audio data from the game. As input, it acquires video and audio data provided in real time from cameras and streaming services. As output, it stores the received video and audio data in its internal memory. Specifically, the server acquires the data through a network connection and stores it in a specified format.

[1489] Step 2:

[1490] Video data analysis

[1491] The server analyzes the received video data. As input, it uses the video data saved in step 1. As output, it obtains the results of detecting scoring and foul scenes. Specifically, the server uses OpenCV to analyze the video data frame by frame and executes an algorithm to detect specific events (scoring and fouls).

[1492] Step 3:

[1493] Analysis of audio data

[1494] The server analyzes the received audio data. As input, it uses the audio data saved in step 1. As output, it obtains text information extracted from the audio data. Specifically, the server converts the audio data to text using Google Cloud Speech-to-Text and extracts information about scores and fouls from the commentary.

[1495] Step 4:

[1496] Extraction of scoring and foul situations

[1497] The server integrates the video analysis results and audio analysis results to extract the score and foul situations. The outputs of steps 2 and 3 are used as input. The output is data including the score and foul situations. Specifically, the server integrates the video analysis results and audio analysis results in chronological order to identify the timing of scores and fouls.

[1498] Step 5:

[1499] Generating a running score

[1500] The server generates a running score based on the extracted information. As input, it uses the score and foul situations obtained in step 4. As output, it obtains data including the running score. Specifically, the server organizes the score and foul information in chronological order and compiles it in a scoreboard format.

[1501] Step 6:

[1502] Update and submit running scores

[1503] The server updates the generated running score in real time and sends it to the device. As input, it uses the running score generated in step 5. As output, it sends the updated running score to the device. Specifically, the server sends data using WebSocket or HTTP.

[1504] Step 7:

[1505] Displaying running scores

[1506] The terminal displays the received running score to the user. As input, it uses the running score sent in step 6. As output, it obtains a scoreboard that is visually presented to the user. Specifically, the terminal displays the scoreboard using a web browser or a dedicated application.

[1507] Step 8:

[1508] Recognizing user emotions

[1509] The server recognizes the user's emotions regarding the score of the game. The input is the user's comments and reaction data. The output is the recognized emotion data. Specifically, the server analyzes the user's emotions using IBM Watson Tone Analyzer.

[1510] Step 9:

[1511] Score display adjustments

[1512] The server adjusts the score display method according to the recognized emotion. As input, it uses the emotion data obtained in step 8 and the running score generated in step 5. As output, it obtains the adjusted score display data. Specifically, the server changes the color and animation of the score display.

[1513] Step 10:

[1514] Sending adjusted score display

[1515] The server sends the adjusted score display to the device. As input, it uses the score display data adjusted in step 9. As output, it sends the adjusted score display to the device. Specifically, the server sends the data using WebSocket or HTTP.

[1516] Step 11:

[1517] Display of adjusted score display

[1518] The terminal displays the adjusted score display to the user. As input, it uses the adjusted score display data sent in step 10. As output, it obtains the adjusted scoreboard that is visually presented to the user. Specifically, the terminal displays the adjusted scoreboard using a web browser or a dedicated application.

[1519] Step 12:

[1520] Database of teams and players nationwide

[1521] The server collects match results and stats information for teams and players across the country and stores it in a database. As input, it uses data obtained from each team's official website and sports data provider services. As output, it obtains match results and stats information stored in the database. Specifically, the server organizes the data and stores it in a MySQL database.

[1522] Step 13:

[1523] Performance Analysis

[1524] The server analyzes the players' performance based on the information stored in the database. As input, it uses the database information stored in step 12. As output, it obtains the analysis results that evaluate the players' performance and abilities. Specifically, the server processes the data using Python's Pandas library and evaluates the players' performance.

[1525] Step 14:

[1526] Generate feedback

[1527] The server generates feedback for player development based on the analysis results. It uses the analysis results obtained in step 13 as input. The output is a feedback report that points out the player's strengths and weaknesses and suggests areas for improvement. Specifically, the server creates the report and provides it to the player and coach.

[1528] Step 15:

[1529] Sending and Viewing Feedback

[1530] The server sends the generated feedback to the terminal, which displays it to the user. As input, it uses the feedback report generated in step 14. As output, it obtains a feedback report that is visually presented to the user. Specifically, the server sends data via email or a dedicated application, and the terminal displays it.

[1531] (Application example 3)

[1532] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1533] Traditional live broadcasts and commentary of basketball games relied on human commentators, making it difficult to grasp the score and foul situation in real time. Furthermore, analysis of player performance and feedback on development were also done manually, which was inefficient. Furthermore, it was not possible to display scores that reflected the user's emotions, making it difficult to improve the viewing experience.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1535] In this invention, the server includes means for AI to provide commentary on live broadcasts of basketball games, means for distributing commentary on youth and mini-basketball games from around the country with AI play-by-play, means for AI to automatically generate running scores that are created manually for each official and team, means for analyzing video and audio data of the game in real time and automatically generating the score and foul situation, means for recognizing the user's emotions and displaying the score according to those emotions, and means for providing feedback for player performance analysis and development using a database of teams and players from around the country. This makes it possible to grasp the progress of the game in real time and display scores that reflect the user's emotions, thereby improving the efficiency of player performance analysis and development.

[1536] "A method for AI to provide commentary for live broadcasts of basketball games" is a technology that uses artificial intelligence to provide real-time commentary on the progress and play of a basketball game.

[1537] "A means of broadcasting commentary on youth and mini-basketball games from around the country with live coverage by AI" is a technology that uses artificial intelligence to provide commentary on youth and mini-basketball games from around the country, and broadcasts live coverage including that commentary.

[1538] "A means of automatically generating running scores using AI, which are currently created by humans for each official and team," is a technology that uses artificial intelligence to automatically generate scores such as points and fouls as the game progresses.

[1539] "Means for analyzing video and audio data of a match in real time and automatically generating scoring and foul situations" refers to technology that analyzes video and audio data during a match in real time and automatically generates scoring and foul situations.

[1540] "Means for recognizing the user's emotions and displaying a score according to those emotions" refers to a technology that recognizes the user's emotions from their facial expressions and voice, and adjusts the score display according to those emotions.

[1541] "A means of utilizing a database of teams and players nationwide to analyze player performance and provide feedback for development" refers to a technology that utilizes a database of basketball teams and players nationwide to analyze player performance and provide feedback for development.

[1542] The system for implementing the present invention uses the following hardware and software.

[1543] Hardware and software used

[1544] Hardware: Smartphone, head-mounted display

[1545] Software: TensorFlow (AI model), OpenCV (video analysis), NLTK (natural language processing), Emotion API (emotion recognition)

[1546] System configuration

[1547] The server includes the following means:

[1548] 1. How AI can provide commentary for live basketball games:

[1549] The server receives video and audio data of the match in real time, analyzes the progress of the match and the details of the play using TensorFlow, and generates commentary.

[1550] 2. How to broadcast live commentary of youth and mini-basketball games across the country with AI:

[1551] The server receives footage of youth and mini-basketball matches from across the country, and similarly uses AI to generate commentary, which is then distributed in real time.

[1552] 3. A way to automatically generate running scores using AI, instead of the running scores currently created by officials and teams:

[1553] The server analyzes the video and audio data of the game and automatically generates score and foul situations.

[1554] 4. A method for analyzing game video and audio data in real time to automatically generate scoring and foul situations:

[1555] The server uses OpenCV to split the video data into frames and extract important scenes, and NLTK to convert the audio data into text and extract important keywords.

[1556] 5. A method to recognize the user's emotions and display a score according to those emotions:

[1557] The server uses the Emotion API to recognize emotions from the user's facial expressions and voice, and adjusts the score display accordingly.

[1558] 6. Utilizing a national team and player database to provide feedback for player performance analysis and development:

[1559] The server references a database of teams and players from across the country, displays analysis of players' performance, and provides feedback for the next game.

[1560] Specific examples

[1561] When a user using a smartphone sees a goal during a game, the server uses AI to analyze the scene and display the score in real time. If the user shows a joyful expression, the server uses an emotion engine to recognize it and make the score display more colorful. After the game, the server also references a nationwide database to display an analysis of the players' performance and provide feedback for the next game.

[1562] Prompt Sentence Examples

[1563] "Please analyze the video and audio data of the game in real time, and display a running score using an AI model that automatically generates the score and foul situation. Also, please recognize the user's emotions and display the score according to those emotions. Furthermore, please utilize a database of teams and players from all over the country to provide feedback for analyzing player performance and development."

[1564] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1565] Step 1:

[1566] Acquisition of match video and audio data

[1567] The server receives video and audio data of the game in real time from smartphones and head-mounted displays. The input is the video and audio data of the game, and the output is preprocessed data for analysis. Specifically, the server receives the data stream, divides the video data into frames, and converts the audio data into text.

[1568] Step 2:

[1569] Data Preprocessing

[1570] The server uses OpenCV to split the video data into frames and extract important scenes. It also uses NLTK to convert the audio data into text and extract important keywords (such as goals and fouls). The input is preprocessed video and audio data, and the output is important scenes and keywords for analysis. Specifically, the server analyzes video frames and identifies scoring and foul scenes.

[1571] Step 3:

[1572] Analysis using AI models

[1573] The server uses TensorFlow to analyze the score and foul situations from video and audio data. The input is important scenes and keywords, and the output is data on the score and foul situations. Specifically, the server runs an AI model to automatically determine the score and fouls for each scene.

[1574] Step 4:

[1575] Automatic generation of running scores

[1576] The server automatically generates a running score based on the analysis results. The input is data on the score and foul situation, and the output is a real-time running score. Specifically, the server aggregates the score and foul information and updates the scoreboard.

[1577] Step 5:

[1578] emotion recognition

[1579] The server uses the Emotion API to recognize emotions from the user's facial expressions and voice. The input is the user's facial expression data and voice data, and the output is the recognized emotion data. Specifically, the server acquires user data from the camera and microphone and analyzes emotions.

[1580] Step 6:

[1581] Emotion-based score display

[1582] The server adjusts the score display according to the recognized emotion. The input is emotion data and the running score, and the output is a score display according to the emotion. Specifically, the server changes the design and effects of the score display based on the user's emotion.

[1583] Step 7:

[1584] Utilizing the database

[1585] The server references a database of teams and players from around the country, displays player performance analysis results, and provides feedback for the next match. The input is match results and player data, and the output is performance analysis results and feedback. Specifically, the server retrieves relevant information from the database and displays the analysis results.

[1586] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1587] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1588] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1589] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1590] [Third embodiment]

[1591] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1592] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1593] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1595] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1597] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1598] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1599] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1600] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1601] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1602] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1603] "Example 1"

[1604] A first embodiment of the present invention is a system in which AI provides commentary for live coverage of a basketball game. In this system, AI analyzes video and audio data of the game to understand the situation of the game and the movements of the players. Then, based on the results of this analysis, it automatically generates commentary for the game and distributes it along with the live coverage.

[1605] "Example 2"

[1606] A second embodiment of the present invention is a system that distributes commentary on youth and mini-basketball games from around the country, accompanied by live coverage by AI. This system collects video and audio data from youth and mini-basketball games held across the country, and, as in the first embodiment, AI analyzes and generates commentary. Then, by distributing this commentary together with the game video, games from around the country can be provided to a wide audience.

[1607] "Example 3"

[1608] A third embodiment of the present invention is a system that uses AI to automatically generate running scores that are currently created manually by officials or teams. In this system, AI analyzes the score and foul situation from video and audio data of the game, and automatically generates a running score. This makes it possible to grasp the progress of the game in real time.

[1609] "Example 4"

[1610] A fourth embodiment of the present invention is a system that creates a database of teams and players nationwide and utilizes the information in the database to visualize and develop registered basketball players in Japan. This system collects information such as game results and stats of teams and players nationwide and creates a database. The database is then used to analyze player performance and provide feedback for development.

[1611] The processing flow of each embodiment will be described below.

[1612] "Example 1"

[1613] Step 1: AI collects video and audio data from basketball games.

[1614] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[1615] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[1616] Step 4: The generated commentary is distributed along with the live broadcast.

[1617] "Example 2"

[1618] Step 1: Collect video and audio data from youth and mini-basketball matches held across the country.

[1619] Step 2: AI analyzes the collected data to understand the situation of the game and the movements of the players.

[1620] Step 3: Based on the analysis results, the AI ​​automatically generates commentary for the match.

[1621] Step 4: The generated commentary is distributed along with the game footage.

[1622] "Example 3"

[1623] Step 1: AI analyzes the scoring situation, foul situation, etc. from video and audio data of the game.

[1624] Step 2: Based on the analysis results, the AI ​​automatically generates a running score.

[1625] "Example 4"

[1626] Step 1: Collect information such as game results and stats for teams and players across the country.

[1627] Step 2: Create a database of the collected information.

[1628] Step 3: Use the databased information to analyze player performance and provide feedback for development.

[1629] Example 1

[1630] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1631] Traditional live broadcasts of basketball games depend on the quality and quantity of commentators, making it difficult to accurately convey the detailed situation of the game and the movements of the players in real time. Furthermore, providing commentary for youth and mini-basketball games across the country requires a large number of people, which is costly and labor-intensive. Furthermore, running scores created by officials and by each team are done manually, which is time-consuming and can lack accuracy. To solve these issues, an automated system utilizing AI is needed.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1633] In this invention, the server includes a means for collecting video and audio data of the game in real time, a means for analyzing the collected video and audio data, a means for generating commentary on the game based on the analysis results, and a means for distributing the generated commentary along with a live broadcast. This enables accurate real-time reporting of detailed game situations and player movements. Furthermore, commentary on youth and mini-basketball games across the country can be distributed with live broadcasts by AI, reducing costs and labor. Furthermore, by automatically generating running scores, which are currently created manually by officials and for each team, using AI can reduce time and improve accuracy.

[1634] "Game video data" refers to digital data that visually records the progress of a basketball game.

[1635] "Audio data" refers to digitally recorded data of sounds occurring during a basketball game.

[1636] "Means of collecting in real time" refers to technology and devices for acquiring video and audio data as the match progresses.

[1637] "Means of analysis" refers to technology and software that processes collected video and audio data to understand the situation of the game and the movements of the players.

[1638] "Means for generating commentary" refers to technology or software that automatically creates text that explains the situation of the game and the movements of the players based on the analysis results.

[1639] "Means for distribution along with live coverage" refers to the technology and devices that deliver the generated commentary to viewers in real time along with the video and audio of the match.

[1640] "Youth and mini-basketball game commentary" is commentary on basketball games aimed at young people and elementary school students.

[1641] "Running score" is data that records the scores and player performances that are updated in real time during the game.

[1642] "AI automatic generation means" refers to technology and software that uses artificial intelligence to automate manual tasks and generate data.

[1643] "Means of database creation" refers to the technology and software used to organize collected information and store it in a format that is easy to search and use.

[1644] "Visualization" refers to techniques and methods for visually displaying data to make it easier to understand.

[1645] "Means for use in development" refers to technologies and methods that use data to support the training and development of athletes.

[1646] This invention is a system that uses AI to automatically generate commentary for live broadcasts of basketball games and delivers it in real time. Specific embodiments of this system are described below.

[1647] Server Operation

[1648] 1. Data Collection

[1649] The server collects video and audio data of the match in real time using a camera and microphone to capture the video and audio of the match, with the video data collected at a resolution of 1080p and the audio data collected at a sampling rate of 44.1kHz.

[1650] 2. Data Analysis

[1651] The server analyzes the collected video and audio data. This analysis uses image recognition software (e.g., OpenCV) and voice recognition software (e.g., Google Speech-to-Text API). This allows the server to understand the situation of the game and the movements of the players. For example, it detects the action of Player A taking a shot and converts the commentary audio at that moment into text.

[1652] 3. Explanation Generation

[1653] The server generates commentary on the game based on the analysis results. A generative AI model (e.g., GPT-4) is used to generate this commentary. The generative AI model receives information about the game situation and player movements as input, and generates commentary in natural language. For example, if the information "Player A made a successful three-point shot" is input, the server generates commentary such as "Player A made a brilliant three-point shot."

[1654] 4. Distribution

[1655] The server distributes the generated commentary along with the live broadcast. A streaming server (e.g., Wowza Streaming Engine) is used for distribution, allowing users to watch the game footage and commentary in real time via the Internet.

[1656] Device behavior

[1657] 1. Data Reception

[1658] The device receives real-time game footage and commentary from the server using an internet connection and streaming software (e.g., VLC Media Player).

[1659] 2.Display

[1660] The device displays the received video and commentary to the user. This is done using hardware such as a display and speakers. For example, if the user is watching on a smartphone, the video will be displayed on the smartphone screen and the commentary audio will be played from the built-in speaker.

[1661] User Actions

[1662] 1. Viewing

[1663] Users can watch game footage and commentary through their devices. Users can receive detailed commentary on the progress of the game and the movements of the players in real time. For example, if a user is watching on a television in their living room, the game footage will be displayed on the TV screen and commentary will be played from the TV speakers.

[1664] Specific examples

[1665] Example 1

[1666] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[1667] Player Action: Home team player A makes a three-point shot.

[1668] Example prompts for generative AI models

[1669] Game Situation: 3rd quarter, 5 minutes remaining, home team 75, away team 70.

[1670] Player Action: Home team player A makes a three-point shot.

[1671] Example of generated commentary

[1672] "With five minutes left in the third quarter, the home team is tied at 75 and the away team is tied at 70. Player A on the home team makes a brilliant three-point shot, widening the gap."

[1673] In this way, users can view detailed commentary on the progress of the match and the movements of the players in real time.

[1674] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1675] Step 1:

[1676] The server collects video and audio data of the match in real time.

[1677] Specifically, it uses cameras and microphones to capture video and audio of the game.

[1678] The input is video and audio from a camera and microphone, and the output is video and audio data in digital format.

[1679] Video data is collected at 1080p resolution, and audio data is collected at a sampling rate of 44.1kHz.

[1680] Step 2:

[1681] The server analyzes the collected video and audio data.

[1682] Specifically, image recognition software (e.g., OpenCV) is used to detect player movements from video data, and voice recognition software (e.g., Google Speech-to-Text API) is used to convert the audio data into text to provide commentary.

[1683] The input is the video and audio data collected in step 1, and the output is the analyzed player movement information and the text commentary.

[1684] For example, the system detects when player A takes a shot and converts the live audio at that moment into text.

[1685] Step 3:

[1686] The server generates commentary on the match based on the analysis results.

[1687] Specifically, a prompt sentence is input into a generative AI model (e.g., GPT-4) to generate an explanatory sentence.

[1688] The input is the player movement information obtained in step 2 and the commentary converted into text, and the output is the generated commentary.

[1689] For example, input information such as "Player A made a successful three-point shot" and generate an explanatory text such as "Player A made a brilliant three-point shot."

[1690] Step 4:

[1691] The server distributes the generated commentary along with the live broadcast.

[1692] Specifically, a streaming server (e.g., Wowza Streaming Engine) is used to stream video and commentary audio in real time.

[1693] The input is the explanatory text generated in step 3 and the video data collected in step 1, and the output is the video and audio commentary delivered in real time.

[1694] Step 5:

[1695] The device receives real-time game footage and commentary streamed from the server.

[1696] Specifically, it requires an internet connection and streaming software (e.g., VLC media player).

[1697] The input is streaming data from the server, and the output is the video displayed on the terminal and the commentary audio played back.

[1698] Step 6:

[1699] The terminal displays the received video and commentary to the user.

[1700] Specifically, the system displays video on a screen and plays explanatory audio from a speaker.

[1701] The input is the video and commentary audio received in step 5, and the output is a live broadcast of the game provided to the user visually and audibly.

[1702] Step 7:

[1703] Users can watch the game footage and commentary through their devices.

[1704] Specifically, the user watches the game footage on the device's display while listening to commentary coming from the speaker.

[1705] The input is the video displayed in step 6 and the commentary audio played, and the output is the user's viewing experience.

[1706] For example, when a user is watching on a television in their living room, the video of the game is displayed on the television screen and commentary audio is played from the television speakers.

[1707] (Application example 1)

[1708] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1709] Conventional basketball game commentary systems lack real-time commentary, post-game highlight generation, and detailed analysis of player performance. Furthermore, they face challenges in providing accurate commentary immediately when users ask questions about specific players or plays.

[1710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1711] In this invention, the server includes means for AI to provide commentary for live broadcasts of basketball games, means for distributing commentary for youth and mini-basketball games across the country with AI commentary, means for AI to automatically generate running scores that are created manually for each official and team, means for analyzing game video data and generating commentary in real time, means for automatically extracting important scenes after the game ends and generating a highlight video, means for analyzing player movements and performance and providing detailed statistical information, and means for AI to provide commentary when a user asks a question about a specific player or play. This enables real-time commentary, post-game highlight generation, detailed analysis of player performance, and immediate commentary in response to user questions.

[1712] "A means for AI to provide commentary for live broadcasts of basketball games" refers to a technology in which AI analyzes video and audio data from basketball games, understands the situation of the game and the movements of the players, and automatically generates commentary.

[1713] "A means of broadcasting commentary on youth and ...

Claims

1. A means for receiving video data and audio data of a match; means for analyzing the received video data; means for analyzing the received voice data; means for extracting a score situation and a foul situation from an analysis result of the video data and the audio data; a means for generating a running score based on the extracted score situation and foul situation; A means for updating and transmitting the generated running score in real time; A means for displaying the transmitted running score; a means for recognizing a user's emotion and adjusting the display of the running score based on the user's emotion; A system including:

2. A means for collecting game results and stats information for teams and players nationwide; A means for storing the collected match results and stats information in a database; A means for analyzing the performance of players based on the information stored in said database; A means for generating feedback for player development based on the analysis results; a means for transmitting and displaying the generated feedback; The system of claim 1 further comprising:

3. The system according to claim 1, further comprising a means for utilizing the information stored in the database for visualizing and developing players.

Images