System

Abstract

A system is provided.SOLUTION: A system for analyzing the actions of players during a game, the system comprising: means for pre-processing the acquired information; means for using the pre-processed information to generate a AI model that predicts the next movements of the players and patterns of high scoring probabilities; means for converting the prediction results into a visually comprehensible format; and means for delivering the prediction results to viewer terminals in real-time.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While viewers currently enjoy watching sports in real time, there is a lack of predictive information about players' next moves and plays with high scoring potential. This makes it difficult for viewers to gain a deep understanding of the game's tactics and players' movements, limiting the depth and enjoyment of the viewing experience. In particular, there is a demand for technology that can provide real-time information to answer questions viewers have during the game, such as "What will the players do next?" and "What moves should I make to increase the chances of scoring." [Means for solving the problem]

[0005] To address these challenges, the present invention provides a system that analyzes player behavior during a game and provides viewers with patterns that have a high probability of scoring. This system includes a means for acquiring video data and sensor data from the game, a means for preprocessing the acquired data, an AI model for predicting players' next moves and patterns with a high probability of scoring using the preprocessed data, a means for converting the prediction results into a visually easy-to-understand format, and a means for delivering the prediction results to viewer devices in real time. The system also includes functions for extracting player and ball position information from the video data and preprocessing position information, speed, and heart rate information acquired from sensors worn by players. This allows viewers to gain a deeper understanding of the flow of the game and enjoy watching sports from a new perspective.

[0006] "Player actions during a match" refers to the actions, changes in position, and tactical moves made by players during a match.

[0007] "Game video data" refers to video information captured by cameras or other photographic equipment during a game.

[0008] "Sensor data" refers to data such as location, speed, and heart rate obtained from sensors attached to athletes.

[0009] "Preprocessing" refers to the process of converting the acquired raw data into a format that is easier to analyze, and includes noise removal, data normalization, and missing value completion.

[0010] An "AI model" is an artificial intelligence that has been trained in advance using large amounts of data, and is a program that predicts a player's next move and the probability of scoring based on the input data.

[0011] "Prediction" refers to estimated information calculated by the AI ​​model about a player's next move and patterns with a high probability of scoring.

[0012] A "visually easy-to-understand format" refers to a format in which the analysis results are visualized to make them easy for users to understand, such as heat maps, graphs, or text data.

[0013] "Real-time" refers to processing and information provision that is carried out almost immediately, with extremely little delay.

[0014] A "viewer device" is a device used by a viewer to watch a game, typically a smartphone, tablet, or computer. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[0037] Overall system configuration

[0038] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user device, allowing users to enjoy the game from a new perspective.

[0039] Server-side processing

[0040] 1. Data Collection

[0041] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0042] 2. Data Preprocessing

[0043] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0044] 3. Analysis using AI models

[0045] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0046] 4. Generating analysis results

[0047] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0048] 5. Distribution of Results

[0049] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0050] Terminal side processing

[0051] 1. Receiving Data

[0052] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0053] 2. Displaying Data

[0054] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0055] User processing

[0056] 1. Verify the information

[0057] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[0058] Specific examples

[0059] For example, consider the case of watching a soccer match.

[0060] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[0061] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0062] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0063] The server transmits this information to the viewer's terminal, which displays it visually.

[0064] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[0065] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.

[0066] The processing flow will be explained below.

[0067] Server-side processing

[0068] Step 1: Data collection

[0069] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[0070] The server also collects real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, and more.

[0071] Step 2: Data Preprocessing

[0072] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[0073] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[0074] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[0075] Step 3: Analysis by AI model

[0076] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[0077] The server's AI model predicts a player's next move or play. For example, if a specific player receives a pass, it calculates the probability of each subsequent move: dribbling, passing, and shooting.

[0078] The server's AI model also calculates the probability of play patterns that lead to goals, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[0079] Step 4: Generate analysis results

[0080] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[0081] The server compiles these analysis results into packets for efficient distribution over the network.

[0082] Step 5: Delivering results

[0083] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[0084] Terminal side processing

[0085] Step 1: Receiving Data

[0086] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0087] Step 2: View the data

[0088] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[0089] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[0090] User processing

[0091] Step 1: Verify the information

[0092] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, and enjoy watching the game.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional sports viewing systems have had difficulty analyzing players' actions and scoring probabilities during a game in real time and providing the analysis to viewers. This has resulted in viewers merely passively watching the video and not gaining a deeper understanding of the game's tactics or players' movements. Furthermore, the delivery of real-time analysis results has been insufficient, preventing viewers from immediately obtaining useful information for predicting the course of the game. The purpose of this invention is to solve these problems.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to a viewer terminal in real time, and means for displaying the received analysis results in real time on the viewer terminal. This allows viewers to understand the actions of players during the game and scoring probabilities in real time, enabling them to enjoy the game from a deeper perspective.

[0098] "Game video data" refers to data obtained from video footage taken during a game, including player movements and ball trajectories.

[0099] "Sensor data" refers to data obtained from sensors worn by players during a match, and includes information such as location, speed, and heart rate.

[0100] "Preprocessing" refers to performing processes such as noise removal, normalization, and missing value completion on the acquired game video data and sensor data.

[0101] An "AI model" is an artificial intelligence model trained on large amounts of match data, which predicts a player's next move and the probability of scoring during a match.

[0102] "Converting into a visually easy-to-understand format" refers to the process of converting the prediction results into a format such as a heat map, graph, or text data so that viewers can intuitively understand it.

[0103] "Real-time delivery" means transmitting data from the server to the viewer's terminal with minimal delay.

[0104] A "viewer terminal" is a device that receives the analysis results and displays them to the viewer, and includes, for example, a smartphone, tablet, or PC.

[0105] "Display in real time" means that the data received is immediately processed and displayed on the viewer terminal.

[0106] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[0107] Overall system configuration

[0108] This system consists of a server, viewer terminals, and a user interface. The processing on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user terminal, allowing users to enjoy the game from a new perspective.

[0109] Server-side processing

[0110] Data collection

[0111] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it acquires game footage from standard high-definition video cameras, capturing player positions and ball trajectories, and also acquires data such as player heart rate and movement speed from standard heart rate monitors.

[0112] Data Preprocessing

[0113] The server preprocesses the collected data, using Python data processing libraries such as Pandas to clean the data, impute missing values, and remove outliers, and NumPy to normalize the data and convert it into a consistent format.

[0114] Analysis using AI models

[0115] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[0116] Generate analysis results

[0117] The server receives the predictions generated by the AI ​​model and converts them into a visually understandable format. This process involves generating heat maps and graphs using Matplotlib and Seaborn, and outputting the results as text data. The analysis results are then compiled into a data packet.

[0118] Results distribution

[0119] The server uses the WebSocket protocol to deliver analysis result data packets to viewer devices in real time, using efficient data transmission techniques to minimize communication delays during this process.

[0120] Terminal side processing

[0121] Receiving data

[0122] The device sequentially receives the analysis result data sent from the server. Since the data is sent in packet format, it is analyzed and processed sequentially. For example, data is received via WebSocket using JavaScript.

[0123] Viewing Data

[0124] The device visually displays the received analysis results in real time using HTML5 and JavaScript, displaying the received heat maps and graphs on a web page for easy user access.

[0125] User processing

[0126] Verify the information

[0127] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which areas are most likely to score. Users can use this information to gain a deeper understanding of the flow of the match and enjoy it even more.

[0128] Specific examples

[0129] For example, consider the case of watching a soccer match.

[0130] Data collection

[0131] The server collects real-time data on player positions and ball movement from high-definition video cameras and heart rate monitors.

[0132] Data Preprocessing

[0133] The server uses Python's Pandas and NumPy to remove noise from the collected data and normalize it.

[0134] Analysis using AI models

[0135] The server inputs preprocessed data into an AI model built with TensorFlow to calculate behavioral predictions and score probabilities.

[0136] Generate analysis results

[0137] The server uses Matplotlib and Seaborn to generate heat maps showing the probability of forward players running towards the goal and the positions where they are likely to score a shot.

[0138] Results distribution

[0139] The server uses WebSocket to send the generated heatmap and prediction results to the device in real time.

[0140] Receiving data

[0141] The device uses JavaScript to sequentially access and analyze data received via WebSocket.

[0142] Viewing Data

[0143] Using HTML5, the received heatmap is rendered into a Canvas element and displayed on the device in an intuitive, easy-to-view format for the user.

[0144] User confirms information

[0145] While watching a live broadcast of a match, users can check the heat map displayed on their device. For example, the locations where a particular player has a high probability of scoring a goal are indicated by color, allowing users to predict how the match will unfold and help them cheer on their team.

[0146] Prompt Sentence Examples

[0147] "During a soccer match, I want to predict which player will receive the ball next. Please create an AI model that uses data such as players' locations, movement speeds, and heart rates to predict their next actions."

[0148] In this way, by creating specific prompts, the AI ​​model can analyze the necessary data and provide predictions.

[0149] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0150] The processing flow of the program in this system is divided into processing steps.

[0151] Step 1: Data collection

[0152] Step 2: Data Preprocessing

[0153] Step 3: Analysis by AI model

[0154] Step 4: Generate analysis results

[0155] Step 5: Delivering results

[0156] Step 6: Receiving Data

[0157] Step 7: View the data

[0158] Step 8: User confirms information

[0159] Each processing step will now be described in detail.

[0160] Step 1: Data collection

[0161] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it captures game footage from high-resolution video cameras and analyzes it frame by frame to capture player positions and ball trajectories. It also collects data such as players' heart rates, movement speeds, and location information from heart rate monitors and GPS sensors.

[0162] Input: Video camera image data, sensor data (heart rate, movement speed, location information)

[0163] Output: Raw data collected

[0164] Step 2: Data Preprocessing

[0165] The server preprocesses the collected data. This process involves using Python's Pandas to clean the data, impute missing values, and remove outliers. It also uses NumPy to normalize the data and convert it into a consistent format. For example, if heart rate data is missing, it is imputed based on the previous data, and outliers are detected and removed to remove noise.

[0166] Input: Raw data collected

[0167] Output: Preprocessed data

[0168] Step 3: Analysis by AI model

[0169] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[0170] Input: Preprocessed data

[0171] Output: Prediction result (next move, score probability, etc.)

[0172] Step 4: Generate analysis results

[0173] The server receives the prediction results from the AI ​​model and converts them into a visually understandable format, using Matplotlib and Seaborn to generate heat maps and graphs, which are then output as text data, allowing viewers to intuitively understand the results.

[0174] Input: Prediction result

[0175] Output: Analysis results in a visually understandable format (heat maps, graphs, text data, etc.)

[0176] Step 5: Delivering results

[0177] The server uses the WebSocket protocol to deliver analysis result data packets to the viewer's device in real time. To minimize communication delays, efficient data transmission techniques are used, such as immediate delivery without buffering.

[0178] Input: Analysis results in a visually easy-to-understand format

[0179] Output: Analysis result data packets delivered in real time

[0180] Step 6: Receiving Data

[0181] The device sequentially receives the analysis result data sent from the server. It uses JavaScript to receive the data via WebSocket, then sequentially analyzes and processes it. It analyzes the received data and converts it into the format required for display.

[0182] Input: Analysis result data packet delivered in real time

[0183] Output: processed data in the terminal

[0184] Step 7: View the data

[0185] The device visually displays the received analysis results in real time using HTML5 and JavaScript, rendering heat maps and graphs onto a Canvas element on a web page in an intuitive, user-friendly format.

[0186] Input: Data processed on the terminal

[0187] Output: Analysis results displayed in real time (heat maps, graphs, etc.)

[0188] Step 8: User confirms information

[0189] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which area is most likely to score. This allows them to gain a deeper understanding of the flow of the match and use it to support their team.

[0190] Input: Analysis results displayed in real time

[0191] Output: Improved user understanding, prediction, and support

[0192] (Application example 1)

[0193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0194] When watching sports, viewers need real-time information such as player actions and scoring probabilities to better understand and enjoy the flow of the game. However, current technology is limited in its means of visually providing players' specific movements during the game, predictions of their next actions, and scoring possibilities in an easy-to-understand manner. This makes it difficult for viewers to understand and enjoy the game from a more tactical and strategic perspective. The purpose of this invention is to solve this problem and provide a system that allows viewers to enjoy the game from a new perspective.

[0195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0196] In this invention, the server includes means for acquiring video data and sensor data of the match, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probability using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for displaying the analysis results in an interactive user interface so that viewers can check which player is most likely to receive the ball next or the position with high scoring probability, and means for delivering the prediction results to viewer terminals in real time, thereby enabling viewers to check detailed analysis information in real time and enjoy the flow of the match more deeply.

[0197] "Game video data" refers to visual digital information captured to record the positions and movements of players and the ball during a game.

[0198] "Sensor data" refers to various physiological and physical data such as location information, speed, and heart rate obtained from sensors attached to athletes.

[0199] "Preprocessing" is a preparatory stage in which acquired data is subjected to processes such as noise removal, normalization, and missing value completion, allowing for more accurate and efficient analysis.

[0200] An "AI model" is an artificial intelligence algorithm that has been trained in advance using large amounts of match data to predict a player's next move and patterns with a high probability of scoring.

[0201] A "visually easy-to-understand format" is a display format in which the prediction results are converted into heat maps, graphs, text data, etc. so that viewers can intuitively understand them.

[0202] An "interactive user interface" is an operating screen that allows users to actively perform operations and make selections, and displays analysis results in combination with actual game footage.

[0203] A "viewer device" is an electronic device such as a smartphone, smart glasses, or head-mounted display that allows viewers to easily view real-time analysis results.

[0204] "Real-time distribution" is a communication method that transmits acquired and analyzed data to viewer terminals in real time without delay.

[0205] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. As an embodiment of the present invention, the following detailed process and specific examples will be described.

[0206] Overall system configuration

[0207] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. The viewer device displays the received analysis results, allowing users to enjoy the game from a new perspective.

[0208] Server-side processing

[0209] 1. Data Collection

[0210] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0211] 2. Data Preprocessing

[0212] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0213] 3. Analysis using AI models

[0214] The server inputs the preprocessed data into an AI model to predict players' next moves and the probability of scoring. This AI model has been trained in advance using TensorFlow and other tools on a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0215] 4. Generating analysis results

[0216] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0217] 5. Distribution of Results

[0218] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0219] Terminal side processing

[0220] 1. Receiving Data

[0221] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0222] 2. Displaying Data

[0223] The device analyzes the received data and displays it in a visually easy-to-understand format via an interactive user interface, allowing users to view the analysis results in real time along with the game footage.

[0224] User processing

[0225] 1. Verify the information

[0226] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[0227] Specific examples

[0228] For example, consider the case of watching a soccer match.

[0229] The server collects player positions and ball movement in real time from cameras and GPS sensors.

[0230] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0231] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0232] The server transmits this information to the viewer's terminal, which displays it visually.

[0233] By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the match and enjoy it more.

[0234] Prompt Sentence Examples

[0235] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[0236] data:

[0237] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[0238] Ball position information: {x: 45, y: 50}

[0239] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[0240] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[0241] Through this specific example, the Sports Analysis Viewer makes watching a game more interactive and enjoyable. By being able to check the analysis results in real time, viewers can gain a deeper understanding of players' movements and strategies, and enjoy the game more.

[0242] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0243] Step 1:

[0244] Data collection

[0245] How it works: The server collects real-time data from cameras installed at the venue and sensors worn by players. The cameras capture footage of the game and acquire data including the positions of players and the ball. The sensors record physiological data such as players' movement speed and heart rate.

[0246] Input: Video and sensor data captured from the match venue.

[0247] Output: Raw data that is sent to subsequent preprocessing steps.

[0248] Step 2:

[0249] Data Preprocessing

[0250] Specific operation: The server preprocesses the collected data. This preprocessing includes removing noise from the video data, normalizing the data, and filling in missing values. Specifically, it removes unnecessary pixel information from the video data and properly fills in missing parts of the sensor data. It also normalizes the data to make it easier to analyze.

[0251] Input: The raw data obtained in step 1.

[0252] Output: Pre-processed data that is sent to subsequent AI analysis steps.

[0253] Step 3:

[0254] Analysis using AI models

[0255] Specific operation: The server inputs the preprocessed data into the AI ​​model. The AI ​​model has been trained in advance with a large amount of match data using TensorFlow and other tools, and uses this data to predict the players' next moves and patterns with high scoring probabilities. Specifically, the model inputs the match situation and outputs player positions and score predictions.

[0256] Input: The preprocessed data from step 2.

[0257] Output: Player action predictions and scoring probability data as analysis results.

[0258] Step 4:

[0259] Generate analysis results

[0260] How it works: The server uses the predictions from the AI ​​model to convert the analysis results into a visually understandable format. Specifically, it generates heat maps, graphs, and text data, and compiles them into data packets. For example, a heat map uses colors to indicate where a player is likely to score.

[0261] Input: Action prediction and score probability data generated in Step 3.

[0262] Output: Visual analysis data packet sent to subsequent delivery steps.

[0263] Step 5:

[0264] Results distribution

[0265] Specific operation: The server delivers analysis results to the viewer's device in real time. To minimize communication delays, an efficient data transmission protocol is used. Specifically, WebSocket or HTTP / 2 is used to transmit data at high speed.

[0266] Input: The visual analysis data packet generated in step 4.

[0267] Output: Real-time analysis results sent to viewer devices.

[0268] Step 6:

[0269] Receiving data

[0270] Specific operation: The terminal receives the analysis result data sent from the server. Since the received data is sent in packet format, it is processed sequentially. Specifically, the terminal receives the data packets using the network library and converts them into a data format for analysis.

[0271] Input: Real-time analytics data delivered in Step 5.

[0272] Output: Analysis data transformed for display.

[0273] Step 7:

[0274] Viewing Data

[0275] How it works: The device analyzes the received data and displays it in an interactive user interface in a visually easy-to-understand format. For example, it can overlay game footage with a heat map showing players' positions and goal probability, allowing viewers to see the game's tactics and player movements in real time.

[0276] Input: The parsed data received and transformed in step 6.

[0277] Output: Visually displayed analysis results and match footage.

[0278] Examples of prompt statements

[0279] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[0280] data:

[0281] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[0282] Ball position information: {x: 45, y: 50}

[0283] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[0284] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] The present invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time, and further combines the function of recognizing viewers' emotions and customizing the display content. Specific embodiments of the present invention are described below.

[0287] Overall system configuration

[0288] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[0289] Server-side processing

[0290] 1. Data Collection

[0291] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0292] 2. Data Preprocessing

[0293] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0294] 3. Analysis using AI models

[0295] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0296] 4. Generating analysis results

[0297] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0298] 5. Distribution of Results

[0299] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0300] Emotion engine processing

[0301] 1. Acquiring Emotion Data

[0302] The emotion engine captures the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time.

[0303] 2. Emotional state determination

[0304] The emotion engine analyzes the captured data and determines the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[0305] 3. Customizing the display content

[0306] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight predictions for the next play, but if the viewer is calm, it will display detailed data analysis results.

[0307] Terminal side processing

[0308] 1. Receiving Data

[0309] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0310] 2. Displaying Data

[0311] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0312] User processing

[0313] 1. Verify the information

[0314] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[0315] Specific examples

[0316] For example, consider the case of watching a soccer match.

[0317] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[0318] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0319] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0320] The server transmits this information to the viewer's terminal, which displays it visually.

[0321] The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with high success rates for shots.

[0322] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[0323] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[0324] The processing flow will be explained below.

[0325] Server-side processing

[0326] Step 1: Data collection

[0327] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[0328] The server also receives real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, etc.

[0329] Step 2: Data Preprocessing

[0330] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[0331] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[0332] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[0333] Step 3: Analysis by AI model

[0334] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[0335] The server's AI model predicts a player's next move or play. For example, if a specific player receives the ball, it calculates the probability of each subsequent move: dribbling, passing, or shooting.

[0336] The server's AI model calculates the probability of a play pattern leading to a goal, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[0337] Step 4: Generate analysis results

[0338] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[0339] The server compiles these analysis results into packets for efficient distribution over the network.

[0340] Step 5: Delivering results

[0341] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[0342] Emotion engine processing

[0343] Step 1: Obtaining emotion data

[0344] The emotion engine in the server acquires emotion data (face recognition data, voice data, biometric data) sent from the viewer terminal.

[0345] Step 2: Determine your emotional state

[0346] The emotion engine analyzes the acquired emotion data to determine the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[0347] Step 3: Customize what's displayed

[0348] The server customizes the display format and content of the analysis results based on the viewer's emotional state as determined by the emotion engine. For example, if the viewer is excited, it will highlight the prediction of the next play, but if the viewer is calm, it will display detailed data analysis results.

[0349] Terminal side processing

[0350] Step 1: Receiving Data

[0351] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0352] Step 2: View the data

[0353] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[0354] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[0355] User processing

[0356] Step 1: Verify the information

[0357] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[0358] Example 2

[0359] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0360] Conventional match viewing systems make it difficult for viewers to visually grasp player behavior and score probability predictions during a match, resulting in a lack of real-time information provision for viewers. Furthermore, they lack the ability to customize information based on each viewer's emotional state, preventing a personalized viewing experience. This makes it difficult for viewers to deeply understand and enjoy the flow and tactics of the match.

[0361] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0362] In this invention, the server includes means for acquiring video data and sensor data of a match, means for preprocessing the acquired data, artificial intelligence model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to viewer terminals in real time, means for acquiring viewer emotional data and analyzing the viewer's emotional state, and means for customizing the display of the prediction results based on the viewer's emotional state. This allows viewers to grasp the analysis results of the match in real time, and the provision of personalized information allows them to gain a deeper understanding of the flow and tactics of the match, making it even more enjoyable.

[0363] "Game video data" refers to real-time video information captured by cameras installed at the game venue, and includes data such as the positions and movements of players and the trajectory of the ball.

[0364] "Sensor data" refers to data such as location information, speed, and biometric information obtained from wearable devices attached to athletes.

[0365] "Preprocessing" refers to processing of collected data, such as noise removal, data normalization, and missing value completion, to improve the accuracy of analysis.

[0366] An "artificial intelligence model" is a system that uses algorithms such as neural networks that have been trained in advance on large amounts of match data to predict a player's next move and the probability of scoring.

[0367] A "visually easy-to-understand format" means converting the analysis results into a format such as a heat map, graph, or text data, making it easy for viewers to understand.

[0368] "Means for delivering to viewer devices in real time" refers to communication protocols and network technologies for instantly transmitting analysis results to viewer devices.

[0369] "Emotion data" is data for analyzing the viewer's emotional state based on the viewer's facial recognition data, voice data, biometric data, etc.

[0370] "Emotional state" refers to the psychological state of the viewer, such as excitement, surprise, or calmness.

[0371] "Means for customization" refers to a mechanism that adjusts the content and format of the analysis results displayed according to the viewer's emotional state, providing optimal information to each individual viewer.

[0372] MODE FOR CARRYING OUT THE INVENTION

[0373] This invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time. It also combines a function to recognize viewers' emotions and customize the display content. Specific embodiments of this system are described below.

[0374] Overall system configuration

[0375] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[0376] Server-side processing

[0377] The server collects data in real time from cameras installed at the venue and sensors worn by players. The video data includes information on the players' positions and the ball's trajectory during the game, while the sensor data records the players' movement speed and heart rate. The collected data is preprocessed to remove noise, normalize the data, and fill in missing values. This preprocessing improves the accuracy of the analysis.

[0378] The preprocessed data is input into an AI model. This AI model uses algorithms such as neural networks that have been trained on large amounts of match data in advance to predict players' next moves and the probability of scoring. Based on these predictions, the server converts the analysis results into a visually understandable format. Specifically, heat maps, graphs, text data, etc. are generated and packaged as data packets. The analysis results are then distributed to viewer devices in real time. An efficient data transmission protocol (e.g., WebSocket) is used to minimize communication latency.

[0379] Emotion engine processing

[0380] The emotion engine acquires the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time. Based on the acquired data, the server determines the viewer's emotional state. For example, it determines whether the viewer is excited, surprised, or calm. Based on this emotional state, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it highlights a prediction of the next play, and if the viewer is calm, it displays detailed data analysis results.

[0381] Terminal side processing

[0382] The device receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially. The device analyzes the received data and displays it in a visually easy-to-understand format. Users can check the analysis results in real time along with the game footage. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[0383] User processing

[0384] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides personalized information, allowing for an optimal viewing experience tailored to the viewer's interests and emotional state.

[0385] Specific examples

[0386] For example, when watching a soccer match, the server collects player positions and ball movement in real time from cameras and GPS sensors. The server preprocesses the data and inputs it into an AI model to calculate action predictions and goal probabilities. As a result, a heat map is generated showing the probability of forward players running toward the goal and the positions where they are likely to score a shot. The server sends this information to the viewer's device, which then displays it visually. The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with a high probability of scoring. By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the game and enjoy it more.

[0387] Examples of prompt statements

[0388] "Please explain in natural language how an AI model can accurately predict the probability of a forward player running towards goal and the success rate of a shot in a soccer match. Also, explain how the model can customize the display of the prediction results in real time depending on whether the viewer is excited or calm."

[0389] This system allows viewers to understand the tactics and movements of players in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[0390] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0391] Specific processing flow of the program

[0392] Step 1: Data collection

[0393] The server collects data in real time from cameras installed at the venue and sensors worn by players.

[0394] Input: Camera video data, sensor data (location, speed, biometric information)

[0395] How it works: Video data includes the positional information of players and the trajectory of the ball during the match, and sensor data such as the player's movement speed and heart rate are recorded in real time. This data is sent to a server in streaming format.

[0396] Output: Collected raw data (video data, sensor data)

[0397] Step 2: Data Preprocessing

[0398] The server performs noise removal, data normalization, and missing value completion on the collected data.

[0399] Input: Raw data collected

[0400] Specific operations: Use a noise reduction algorithm to properly remove high frequency noise, standardize each data point, and apply a Kalman filter or similar to fill in missing data.

[0401] Output: Preprocessed data

[0402] Step 3: Analysis by AI model

[0403] Based on the pre-processed data, the server uses an artificial intelligence model to predict the player's next move and the probability of scoring.

[0404] Input: Preprocessed data

[0405] How it works: Preprocessed data is fed into a neural network (e.g., LSTM or CNN) to predict player movements and shot success rates. The AI ​​model makes highly accurate predictions based on pre-trained patterns.

[0406] Output: Predicted result (next move, score probability)

[0407] Step 4: Generate analysis results

[0408] The server converts the analysis results into a visually easy-to-understand format based on the output of the AI ​​model.

[0409] Input: AI model prediction results

[0410] Specific operations: Generate a heat map, represent the score probability as a bar graph, and add an explanation as text data. Organize the result data into a packet format.

[0411] Output: Visually formatted analysis results (heatmaps, graphs, text data)

[0412] Step 5: Delivering results

[0413] The server delivers the analysis results to the viewer's device in real time.

[0414] Input: Visually formatted analysis results

[0415] Specific operation: Data packets are sent to the viewer device using an efficient data transmission protocol (e.g., WebSocket).

[0416] Output: Analysis result data delivered to the viewer's device

[0417] Step 6: Obtaining emotion data

[0418] The server's emotion engine collects viewers' facial recognition data, voice data, and biometric information.

[0419] Input: Facial recognition data, voice data, biometric data

[0420] Specific operation: Collect data from cameras, microphones, and wearable devices, and analyze the emotional state using facial expression recognition algorithms (e.g., OpenCV).

[0421] Output: Emotion data

[0422] Step 7: Determine your emotional state

[0423] The server's emotion engine determines the viewer's emotional state based on the acquired data.

[0424] Input: Emotion data

[0425] Specific operations: Performs voice tone analysis and heart rate variability analysis to cross-check and determine the viewer's emotional state, such as excitement, surprise, or calmness.

[0426] Output: Determined emotional state

[0427] Step 8: Customize what's displayed

[0428] The server customizes the display of the analysis results based on the viewer's emotional state.

[0429] Input: Determined emotional state, analysis result data delivered to the viewer's device

[0430] Specific behavior: When viewers are excited, the layout and font size are adaptively changed to highlight highlights and next play predictions, and when viewers are calm, to display detailed data analysis results.

[0431] Output: Customized display content

[0432] Step 9: Receiving Data

[0433] The terminal receives the analysis result data sent from the server.

[0434] Input: Data packet sent by the server

[0435] What it does: Receives data packets over a WebSocket connection and stores them in a local memory buffer.

[0436] Output: Analysis result data stored in memory buffer

[0437] Step 10: Displaying the data

[0438] The device analyzes the received data and displays it in a visually easy-to-understand format.

[0439] Input: Analysis result data stored in memory buffer

[0440] What it does: Render heatmaps, graphs, and text data in real time and integrate them into the user interface, providing customized displays based on the results of the sentiment engine.

[0441] Output: Analysis results displayed in the user interface

[0442] (Application example 2)

[0443] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0444] Conventional match analysis systems are limited to predicting player behavior and scoring probability during a match, and are not customized to reflect the viewer's emotional state. This makes it difficult to provide information tailored to each viewer's interests and emotions, resulting in a lack of personalized viewing experiences. To address this issue, a system is needed that can recognize viewer emotions in real time and customize the display format and content of analysis results based on those emotions.

[0445] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to the viewer terminal in real time, means for acquiring face recognition data, voice data, and biometric information data of the viewer, means for determining the viewer's emotional state using the acquired data, and means for customizing the display format and content of the prediction results according to the viewer's emotional state. This makes it possible to provide analysis results suited to the viewer's emotional state, thereby realizing a personalized viewing experience.

[0446] "Game video data" refers to digital video data captured during a game, including the positional information of players and the ball, their movements, and the flow of the game.

[0447] "Sensor data" refers to biometric and environmental data such as location, speed, and heart rate obtained from sensors attached to athletes.

[0448] "Preprocessing" refers to the process of converting acquired data, such as by removing noise, normalizing, and filling in missing values, to prepare the data for analysis.

[0449] An "AI model" refers to artificial intelligence technology that uses pre-trained algorithms to predict player movements and scoring probabilities from input data.

[0450] "Visually easy-to-understand format" refers to formats such as heat maps, graphs, and text data that present analysis results in a way that is easy for users to understand.

[0451] "Real-time" refers to the delivery or processing of data nearly simultaneously with minimal delay.

[0452] A "viewer terminal" is a device used by a viewer to receive data and check the analysis results, and includes smartphones, head-mounted displays, etc.

[0453] "Facial Recognition Data" refers to digital image data used to analyze a viewer's facial expressions.

[0454] "Audio Data" means digital audio data that captures the voice of a viewer and is used for audio analysis.

[0455] "Biometric data" refers to personal physiological data obtained from biometric sensors, such as a viewer's heart rate or skin temperature.

[0456] "Emotional state" refers to the psychological state, such as excitement, calmness, or surprise, that viewers express in real time.

[0457] "Customization" refers to adapting the content and format of a presentation based on the emotional state of each viewer.

[0458] This system analyzes the behavior of players during a match, provides the results to viewers in real time, and customizes the results according to the viewer's emotional state. This system includes the collection, preprocessing, and analysis of match data, the generation and distribution of results, viewer emotion recognition, and the customization of the display content.

[0459] Overall system configuration

[0460] It consists of a server, viewer terminal, emotion engine, and user interface. The server-side processing is divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, it recognizes viewer emotions and reflects this information in the display of analysis results.

[0461] Server-side processing

[0462] 1. Data Collection

[0463] The server collects video and sensor data from the match in real time. Specifically, it acquires data from cameras installed at the match venue and sensors worn by players. The hardware used includes cameras (e.g., Logitech C920) and heart rate monitors.

[0464] 2. Data Preprocessing

[0465] The server preprocesses the collected data. This preprocessing includes noise removal, data normalization, and missing value imputation, making the data ready for analysis. Software libraries used include NumPy and Pandas.

[0466] 3. Analysis using AI models

[0467] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model is trained in advance using a large amount of match data and is able to predict moves and patterns with high accuracy. Frameworks used include TensorFlow and PyTorch.

[0468] 4. Generating analysis results

[0469] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format. Specifically, heat maps, graphs, text data, etc. are generated and compiled into data packets. The software used is a JavaScript library.

[0470] 5. Distribution of Results

[0471] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0472] Emotion engine processing

[0473] 1. Acquiring Emotion Data

[0474] The emotion engine captures viewers' facial recognition data, voice data, and biometric data, and analyzes their emotional state in real time. The hardware used includes cameras, microphones, and biometric sensors.

[0475] 2. Emotional state determination

[0476] The emotion engine analyzes the acquired data and determines the viewer's emotional state. The software used is OpenCV (face recognition) and DeepFace (emotion recognition).

[0477] 3. Customizing the display content

[0478] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight the predictions for the next play, and if the viewer is calm, it will display detailed data analysis results.

[0479] Terminal side processing

[0480] 1. Receiving Data

[0481] The viewer terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0482] 2. Displaying Data

[0483] The viewer device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0484] User processing

[0485] 1. Verify the information

[0486] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game.In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[0487] Examples of specific examples and prompts

[0488] For example, imagine a viewer watching a soccer match on their smartphone. The app displays the positions of players and the movement of the ball in real time, and an emotion recognition engine detects the viewer's excitement. For viewers who are excited, the app highlights scenes with high probability of the next shot, and users can check the analysis results in real time, allowing them to enjoy the match more deeply.

[0489] Example prompts for generative AI models

[0490] You are an AI that generates content for a sports viewing app. Using the following information, you must generate content that displays analysis results based on player behavior predictions and viewer emotions in real time.

[0491] Input information

[0492] Player position data (JSON format)

[0493] Ball trajectory data (JSON format)

[0494] The viewer's emotional state (excited, calm, surprised, etc.)

[0495] Real-time video data of the match

[0496] Example output

[0497] 1. Highlighting high probability shots for excited viewers

[0498] 2. Display detailed data analysis results for a calm audience

[0499] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0500] Step 1:

[0501] Data collection

[0502] The server collects game video data and sensor data (location information, speed, heart rate, etc.) in real time from cameras installed at the venue and sensors worn by players. The input is camera video data and sensor data, and the output is a data packet that compiles these. Specifically, the camera (Logitech C920) and heart rate sensor capture the game situation and player biometric data, which the server receives.

[0503] Step 2:

[0504] Data Preprocessing

[0505] The server denoises the collected video and sensor data, normalizes the data, and fills in missing values ​​to prepare it for analysis. The input is the collected data packet, and the output is the preprocessed data. Specifically, the server cleans the data using NumPy and Pandas.

[0506] Step 3:

[0507] Action prediction and score probability calculation

[0508] The server inputs the preprocessed data into an AI model to predict the player's next move and the pattern with the highest probability of scoring. The input is the preprocessed data, and the output is the prediction result. Specifically, a pre-trained TensorFlow or PyTorch model analyzes the data and makes a prediction.

[0509] Step 4:

[0510] Generate analysis results

[0511] The server uses the prediction results from the AI ​​model to convert them into visually understandable formats such as heat maps, graphs, and text data. The input is the prediction results, and the output is visualized data packets. Specifically, it uses a JavaScript library to graph the data and organize it into packets.

[0512] Step 5:

[0513] Obtaining viewer sentiment data

[0514] The emotion engine acquires the viewer's facial recognition data, audio data, and biometric data. The input is camera footage, microphone audio, and biometric sensor data, and the output is an emotion data packet that combines these. Specifically, it captures data using the camera and microphone, and collects biometric information using sensors.

[0515] Step 6:

[0516] Determining emotional state

[0517] The emotion engine analyzes the acquired data and determines the viewer's emotional state (excitement, calmness, surprise, etc.). The input is an emotion data packet, and the output is the emotional state. Specifically, it uses OpenCV and DeepFace to perform face recognition and emotion analysis.

[0518] Step 7:

[0519] Customizing the display

[0520] The server customizes the display format and content of the prediction results depending on the viewer's emotional state. The input is a data packet of the analysis results and the viewer's emotional state, and the output is a customized data packet of the analysis results. Specifically, if the viewer's emotional state is excited, the shot scene is highlighted, and if the viewer is calm, detailed data is displayed.

[0521] Step 8:

[0522] Data distribution

[0523] The server delivers data packets of customized analysis results to the viewer terminal in real time. The input is the data packets of customized analysis results, and the output is delivery to the viewer terminal. Specifically, the server transmits the data using an efficient data transmission protocol.

[0524] Step 9:

[0525] Receiving and displaying data

[0526] The viewer device receives data packets sent from the server, analyzes them, and displays them in a visually easy-to-understand format. The input is the delivered data packets, and the output is the display data on the device. Specifically, it analyzes the received data and updates the UI displayed on the screen.

[0527] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0528] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0529] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0530] [Second embodiment]

[0531] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0532] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0533] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0534] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0535] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0536] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0537] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0538] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0539] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0540] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0541] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0542] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0543] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[0544] Overall system configuration

[0545] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user device, allowing users to enjoy the game from a new perspective.

[0546] Server-side processing

[0547] 1. Data Collection

[0548] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0549] 2. Data Preprocessing

[0550] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0551] 3. Analysis using AI models

[0552] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0553] 4. Generating analysis results

[0554] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0555] 5. Distribution of Results

[0556] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0557] Terminal side processing

[0558] 1. Receiving Data

[0559] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0560] 2. Displaying Data

[0561] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0562] User processing

[0563] 1. Verify the information

[0564] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[0565] Specific examples

[0566] For example, consider the case of watching a soccer match.

[0567] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[0568] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0569] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0570] The server transmits this information to the viewer's terminal, which displays it visually.

[0571] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[0572] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.

[0573] The processing flow will be explained below.

[0574] Server-side processing

[0575] Step 1: Data collection

[0576] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[0577] The server also collects real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, and more.

[0578] Step 2: Data Preprocessing

[0579] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[0580] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[0581] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[0582] Step 3: Analysis by AI model

[0583] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[0584] The server's AI model predicts a player's next move or play. For example, if a specific player receives a pass, it calculates the probability of each subsequent move: dribbling, passing, and shooting.

[0585] The server's AI model also calculates the probability of play patterns that lead to goals, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[0586] Step 4: Generate analysis results

[0587] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[0588] The server compiles these analysis results into packets for efficient distribution over the network.

[0589] Step 5: Delivering results

[0590] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[0591] Terminal side processing

[0592] Step 1: Receiving Data

[0593] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0594] Step 2: View the data

[0595] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[0596] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[0597] User processing

[0598] Step 1: Verify the information

[0599] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, and enjoy watching the game.

[0600] Example 1

[0601] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] Conventional sports viewing systems have had difficulty analyzing players' actions and scoring probabilities during a game in real time and providing the analysis to viewers. This has resulted in viewers merely passively watching the video and not gaining a deeper understanding of the game's tactics or players' movements. Furthermore, the delivery of real-time analysis results has been insufficient, preventing viewers from immediately obtaining useful information for predicting the course of the game. The purpose of this invention is to solve these problems.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0604] In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to a viewer terminal in real time, and means for displaying the received analysis results in real time on the viewer terminal. This allows viewers to understand the actions of players during the game and scoring probabilities in real time, enabling them to enjoy the game from a deeper perspective.

[0605] "Game video data" refers to data obtained from video footage taken during a game, including player movements and ball trajectories.

[0606] "Sensor data" refers to data obtained from sensors worn by players during a match, and includes information such as location, speed, and heart rate.

[0607] "Preprocessing" refers to performing processes such as noise removal, normalization, and missing value completion on the acquired game video data and sensor data.

[0608] An "AI model" is an artificial intelligence model trained on large amounts of match data, which predicts a player's next move and the probability of scoring during a match.

[0609] "Converting into a visually easy-to-understand format" refers to the process of converting the prediction results into a format such as a heat map, graph, or text data so that viewers can intuitively understand it.

[0610] "Real-time delivery" means transmitting data from the server to the viewer's terminal with minimal delay.

[0611] A "viewer terminal" is a device that receives the analysis results and displays them to the viewer, and includes, for example, a smartphone, tablet, or PC.

[0612] "Display in real time" means that the data received is immediately processed and displayed on the viewer terminal.

[0613] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[0614] Overall system configuration

[0615] This system consists of a server, viewer terminals, and a user interface. The processing on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user terminal, allowing users to enjoy the game from a new perspective.

[0616] Server-side processing

[0617] Data collection

[0618] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it acquires game footage from standard high-definition video cameras, capturing player positions and ball trajectories, and also acquires data such as player heart rate and movement speed from standard heart rate monitors.

[0619] Data Preprocessing

[0620] The server preprocesses the collected data, using Python data processing libraries such as Pandas to clean the data, impute missing values, and remove outliers, and NumPy to normalize the data and convert it into a consistent format.

[0621] Analysis using AI models

[0622] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[0623] Generate analysis results

[0624] The server receives the predictions generated by the AI ​​model and converts them into a visually understandable format. This process involves generating heat maps and graphs using Matplotlib and Seaborn, and outputting the results as text data. The analysis results are then compiled into a data packet.

[0625] Results distribution

[0626] The server uses the WebSocket protocol to deliver analysis result data packets to viewer devices in real time, using efficient data transmission techniques to minimize communication delays during this process.

[0627] Terminal side processing

[0628] Receiving data

[0629] The device sequentially receives the analysis result data sent from the server. Since the data is sent in packet format, it is analyzed and processed sequentially. For example, data is received via WebSocket using JavaScript.

[0630] Viewing Data

[0631] The device visually displays the received analysis results in real time using HTML5 and JavaScript, displaying the received heat maps and graphs on a web page for easy user access.

[0632] User processing

[0633] Verify the information

[0634] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which areas are most likely to score. Users can use this information to gain a deeper understanding of the flow of the match and enjoy it even more.

[0635] Specific examples

[0636] For example, consider the case of watching a soccer match.

[0637] Data collection

[0638] The server collects real-time data on player positions and ball movement from high-definition video cameras and heart rate monitors.

[0639] Data Preprocessing

[0640] The server uses Python's Pandas and NumPy to remove noise from the collected data and normalize it.

[0641] Analysis using AI models

[0642] The server inputs preprocessed data into an AI model built with TensorFlow to calculate behavioral predictions and score probabilities.

[0643] Generate analysis results

[0644] The server uses Matplotlib and Seaborn to generate heat maps showing the probability of forward players running towards the goal and the positions where they are likely to score a shot.

[0645] Results distribution

[0646] The server uses WebSocket to send the generated heatmap and prediction results to the device in real time.

[0647] Receiving data

[0648] The device uses JavaScript to sequentially access and analyze data received via WebSocket.

[0649] Viewing Data

[0650] Using HTML5, the received heatmap is rendered into a Canvas element and displayed on the device in an intuitive, easy-to-view format for the user.

[0651] User confirms information

[0652] While watching a live broadcast of a match, users can check the heat map displayed on their device. For example, the locations where a particular player has a high probability of scoring a goal are indicated by color, allowing users to predict how the match will unfold and help them cheer on their team.

[0653] Prompt Sentence Examples

[0654] "During a soccer match, I want to predict which player will receive the ball next. Please create an AI model that uses data such as players' locations, movement speeds, and heart rates to predict their next actions."

[0655] In this way, by creating specific prompts, the AI ​​model can analyze the necessary data and provide predictions.

[0656] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0657] The processing flow of the program in this system is divided into processing steps.

[0658] Step 1: Data collection

[0659] Step 2: Data Preprocessing

[0660] Step 3: Analysis by AI model

[0661] Step 4: Generate analysis results

[0662] Step 5: Delivering results

[0663] Step 6: Receiving Data

[0664] Step 7: View the data

[0665] Step 8: User confirms information

[0666] Each processing step will now be described in detail.

[0667] Step 1: Data collection

[0668] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it captures game footage from high-resolution video cameras and analyzes it frame by frame to capture player positions and ball trajectories. It also collects data such as players' heart rates, movement speeds, and location information from heart rate monitors and GPS sensors.

[0669] Input: Video camera image data, sensor data (heart rate, movement speed, location information)

[0670] Output: Raw data collected

[0671] Step 2: Data Preprocessing

[0672] The server preprocesses the collected data. This process involves using Python's Pandas to clean the data, impute missing values, and remove outliers. It also uses NumPy to normalize the data and convert it into a consistent format. For example, if heart rate data is missing, it is imputed based on the previous data, and outliers are detected and removed to remove noise.

[0673] Input: Raw data collected

[0674] Output: Preprocessed data

[0675] Step 3: Analysis by AI model

[0676] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[0677] Input: Preprocessed data

[0678] Output: Prediction result (next move, score probability, etc.)

[0679] Step 4: Generate analysis results

[0680] The server receives the prediction results from the AI ​​model and converts them into a visually understandable format, using Matplotlib and Seaborn to generate heat maps and graphs, which are then output as text data, allowing viewers to intuitively understand the results.

[0681] Input: Prediction result

[0682] Output: Analysis results in a visually understandable format (heat maps, graphs, text data, etc.)

[0683] Step 5: Delivering results

[0684] The server uses the WebSocket protocol to deliver analysis result data packets to the viewer's device in real time. To minimize communication delays, efficient data transmission techniques are used, such as immediate delivery without buffering.

[0685] Input: Analysis results in a visually easy-to-understand format

[0686] Output: Analysis result data packets delivered in real time

[0687] Step 6: Receiving Data

[0688] The device sequentially receives the analysis result data sent from the server. It uses JavaScript to receive the data via WebSocket, then sequentially analyzes and processes it. It analyzes the received data and converts it into the format required for display.

[0689] Input: Analysis result data packet delivered in real time

[0690] Output: processed data in the terminal

[0691] Step 7: View the data

[0692] The device visually displays the received analysis results in real time using HTML5 and JavaScript, rendering heat maps and graphs onto a Canvas element on a web page in an intuitive, user-friendly format.

[0693] Input: Data processed on the terminal

[0694] Output: Analysis results displayed in real time (heat maps, graphs, etc.)

[0695] Step 8: User confirms information

[0696] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which area is most likely to score. This allows them to gain a deeper understanding of the flow of the match and use it to support their team.

[0697] Input: Analysis results displayed in real time

[0698] Output: Improved user understanding, prediction, and support

[0699] (Application example 1)

[0700] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0701] When watching sports, viewers need real-time information such as player actions and scoring probabilities to better understand and enjoy the flow of the game. However, current technology is limited in its means of visually providing players' specific movements during the game, predictions of their next actions, and scoring possibilities in an easy-to-understand manner. This makes it difficult for viewers to understand and enjoy the game from a more tactical and strategic perspective. The purpose of this invention is to solve this problem and provide a system that allows viewers to enjoy the game from a new perspective.

[0702] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0703] In this invention, the server includes means for acquiring video data and sensor data of the match, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probability using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for displaying the analysis results in an interactive user interface so that viewers can check which player is most likely to receive the ball next or the position with high scoring probability, and means for delivering the prediction results to viewer terminals in real time, thereby enabling viewers to check detailed analysis information in real time and enjoy the flow of the match more deeply.

[0704] "Game video data" refers to visual digital information captured to record the positions and movements of players and the ball during a game.

[0705] "Sensor data" refers to various physiological and physical data such as location information, speed, and heart rate obtained from sensors attached to athletes.

[0706] "Preprocessing" is a preparatory stage in which acquired data is subjected to processes such as noise removal, normalization, and missing value completion, allowing for more accurate and efficient analysis.

[0707] An "AI model" is an artificial intelligence algorithm that has been trained in advance using large amounts of match data to predict a player's next move and patterns with a high probability of scoring.

[0708] A "visually easy-to-understand format" is a display format in which the prediction results are converted into heat maps, graphs, text data, etc. so that viewers can intuitively understand them.

[0709] An "interactive user interface" is an operating screen that allows users to actively perform operations and make selections, and displays analysis results in combination with actual game footage.

[0710] A "viewer device" is an electronic device such as a smartphone, smart glasses, or head-mounted display that allows viewers to easily view real-time analysis results.

[0711] "Real-time distribution" is a communication method that transmits acquired and analyzed data to viewer terminals in real time without delay.

[0712] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. As an embodiment of the present invention, the following detailed process and specific examples will be described.

[0713] Overall system configuration

[0714] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. The viewer device displays the received analysis results, allowing users to enjoy the game from a new perspective.

[0715] Server-side processing

[0716] 1. Data Collection

[0717] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0718] 2. Data Preprocessing

[0719] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0720] 3. Analysis using AI models

[0721] The server inputs the preprocessed data into an AI model to predict players' next moves and the probability of scoring. This AI model has been trained in advance using TensorFlow and other tools on a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0722] 4. Generating analysis results

[0723] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0724] 5. Distribution of Results

[0725] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0726] Terminal side processing

[0727] 1. Receiving Data

[0728] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0729] 2. Displaying Data

[0730] The device analyzes the received data and displays it in a visually easy-to-understand format via an interactive user interface, allowing users to view the analysis results in real time along with the game footage.

[0731] User processing

[0732] 1. Verify the information

[0733] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[0734] Specific examples

[0735] For example, consider the case of watching a soccer match.

[0736] The server collects player positions and ball movement in real time from cameras and GPS sensors.

[0737] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0738] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0739] The server transmits this information to the viewer's terminal, which displays it visually.

[0740] By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the match and enjoy it more.

[0741] Prompt Sentence Examples

[0742] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[0743] data:

[0744] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[0745] Ball position information: {x: 45, y: 50}

[0746] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[0747] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[0748] Through this specific example, the Sports Analysis Viewer makes watching a game more interactive and enjoyable. By being able to check the analysis results in real time, viewers can gain a deeper understanding of players' movements and strategies, and enjoy the game more.

[0749] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0750] Step 1:

[0751] Data collection

[0752] How it works: The server collects real-time data from cameras installed at the venue and sensors worn by players. The cameras capture footage of the game and acquire data including the positions of players and the ball. The sensors record physiological data such as players' movement speed and heart rate.

[0753] Input: Video and sensor data captured from the match venue.

[0754] Output: Raw data that is sent to subsequent preprocessing steps.

[0755] Step 2:

[0756] Data Preprocessing

[0757] Specific operation: The server preprocesses the collected data. This preprocessing includes removing noise from the video data, normalizing the data, and filling in missing values. Specifically, it removes unnecessary pixel information from the video data and properly fills in missing parts of the sensor data. It also normalizes the data to make it easier to analyze.

[0758] Input: The raw data obtained in step 1.

[0759] Output: Pre-processed data that is sent to subsequent AI analysis steps.

[0760] Step 3:

[0761] Analysis using AI models

[0762] Specific operation: The server inputs the preprocessed data into the AI ​​model. The AI ​​model has been trained in advance with a large amount of match data using TensorFlow and other tools, and uses this data to predict the players' next moves and patterns with high scoring probabilities. Specifically, the model inputs the match situation and outputs player positions and score predictions.

[0763] Input: The preprocessed data from step 2.

[0764] Output: Player action predictions and scoring probability data as analysis results.

[0765] Step 4:

[0766] Generate analysis results

[0767] How it works: The server uses the predictions from the AI ​​model to convert the analysis results into a visually understandable format. Specifically, it generates heat maps, graphs, and text data, and compiles them into data packets. For example, a heat map uses colors to indicate where a player is likely to score.

[0768] Input: Action prediction and score probability data generated in Step 3.

[0769] Output: Visual analysis data packet sent to subsequent delivery steps.

[0770] Step 5:

[0771] Results distribution

[0772] Specific operation: The server delivers analysis results to the viewer's device in real time. To minimize communication delays, an efficient data transmission protocol is used. Specifically, WebSocket or HTTP / 2 is used to transmit data at high speed.

[0773] Input: The visual analysis data packet generated in step 4.

[0774] Output: Real-time analysis results sent to viewer devices.

[0775] Step 6:

[0776] Receiving data

[0777] Specific operation: The terminal receives the analysis result data sent from the server. Since the received data is sent in packet format, it is processed sequentially. Specifically, the terminal receives the data packets using the network library and converts them into a data format for analysis.

[0778] Input: Real-time analytics data delivered in Step 5.

[0779] Output: Analysis data transformed for display.

[0780] Step 7:

[0781] Viewing Data

[0782] How it works: The device analyzes the received data and displays it in an interactive user interface in a visually easy-to-understand format. For example, it can overlay game footage with a heat map showing players' positions and goal probability, allowing viewers to see the game's tactics and player movements in real time.

[0783] Input: The parsed data received and transformed in step 6.

[0784] Output: Visually displayed analysis results and match footage.

[0785] Examples of prompt statements

[0786] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[0787] data:

[0788] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[0789] Ball position information: {x: 45, y: 50}

[0790] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[0791] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[0792] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0793] The present invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time, and further combines the function of recognizing viewers' emotions and customizing the display content. Specific embodiments of the present invention are described below.

[0794] Overall system configuration

[0795] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[0796] Server-side processing

[0797] 1. Data Collection

[0798] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[0799] 2. Data Preprocessing

[0800] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[0801] 3. Analysis using AI models

[0802] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[0803] 4. Generating analysis results

[0804] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[0805] 5. Distribution of Results

[0806] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0807] Emotion engine processing

[0808] 1. Acquiring Emotion Data

[0809] The emotion engine captures the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time.

[0810] 2. Emotional state determination

[0811] The emotion engine analyzes the captured data and determines the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[0812] 3. Customizing the display content

[0813] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight predictions for the next play, but if the viewer is calm, it will display detailed data analysis results.

[0814] Terminal side processing

[0815] 1. Receiving Data

[0816] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0817] 2. Displaying Data

[0818] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0819] User processing

[0820] 1. Verify the information

[0821] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[0822] Specific examples

[0823] For example, consider the case of watching a soccer match.

[0824] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[0825] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[0826] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[0827] The server transmits this information to the viewer's terminal, which displays it visually.

[0828] The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with high success rates for shots.

[0829] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[0830] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[0831] The processing flow will be explained below.

[0832] Server-side processing

[0833] Step 1: Data collection

[0834] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[0835] The server also receives real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, etc.

[0836] Step 2: Data Preprocessing

[0837] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[0838] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[0839] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[0840] Step 3: Analysis by AI model

[0841] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[0842] The server's AI model predicts a player's next move or play. For example, if a specific player receives the ball, it calculates the probability of each subsequent move: dribbling, passing, or shooting.

[0843] The server's AI model calculates the probability of a play pattern leading to a goal, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[0844] Step 4: Generate analysis results

[0845] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[0846] The server compiles these analysis results into packets for efficient distribution over the network.

[0847] Step 5: Delivering results

[0848] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[0849] Emotion engine processing

[0850] Step 1: Obtaining emotion data

[0851] The emotion engine in the server acquires emotion data (face recognition data, voice data, biometric data) sent from the viewer terminal.

[0852] Step 2: Determine your emotional state

[0853] The emotion engine analyzes the acquired emotion data to determine the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[0854] Step 3: Customize what's displayed

[0855] The server customizes the display format and content of the analysis results based on the viewer's emotional state as determined by the emotion engine. For example, if the viewer is excited, it will highlight the prediction of the next play, but if the viewer is calm, it will display detailed data analysis results.

[0856] Terminal side processing

[0857] Step 1: Receiving Data

[0858] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0859] Step 2: View the data

[0860] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[0861] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[0862] User processing

[0863] Step 1: Verify the information

[0864] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[0865] Example 2

[0866] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0867] Conventional match viewing systems make it difficult for viewers to visually grasp player behavior and score probability predictions during a match, resulting in a lack of real-time information provision for viewers. Furthermore, they lack the ability to customize information based on each viewer's emotional state, preventing a personalized viewing experience. This makes it difficult for viewers to deeply understand and enjoy the flow and tactics of the match.

[0868] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0869] In this invention, the server includes means for acquiring video data and sensor data of a match, means for preprocessing the acquired data, artificial intelligence model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to viewer terminals in real time, means for acquiring viewer emotional data and analyzing the viewer's emotional state, and means for customizing the display of the prediction results based on the viewer's emotional state. This allows viewers to grasp the analysis results of the match in real time, and the provision of personalized information allows them to gain a deeper understanding of the flow and tactics of the match, making it even more enjoyable.

[0870] "Game video data" refers to real-time video information captured by cameras installed at the game venue, and includes data such as the positions and movements of players and the trajectory of the ball.

[0871] "Sensor data" refers to data such as location information, speed, and biometric information obtained from wearable devices attached to athletes.

[0872] "Preprocessing" refers to processing of collected data, such as noise removal, data normalization, and missing value completion, to improve the accuracy of analysis.

[0873] An "artificial intelligence model" is a system that uses algorithms such as neural networks that have been trained in advance on large amounts of match data to predict a player's next move and the probability of scoring.

[0874] A "visually easy-to-understand format" means converting the analysis results into a format such as a heat map, graph, or text data, making it easy for viewers to understand.

[0875] "Means for delivering to viewer devices in real time" refers to communication protocols and network technologies for instantly transmitting analysis results to viewer devices.

[0876] "Emotion data" is data for analyzing the viewer's emotional state based on the viewer's facial recognition data, voice data, biometric data, etc.

[0877] "Emotional state" refers to the psychological state of the viewer, such as excitement, surprise, or calmness.

[0878] "Means for customization" refers to a mechanism that adjusts the content and format of the analysis results displayed according to the viewer's emotional state, providing optimal information to each individual viewer.

[0879] MODE FOR CARRYING OUT THE INVENTION

[0880] This invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time. It also combines a function to recognize viewers' emotions and customize the display content. Specific embodiments of this system are described below.

[0881] Overall system configuration

[0882] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[0883] Server-side processing

[0884] The server collects data in real time from cameras installed at the venue and sensors worn by players. The video data includes information on the players' positions and the ball's trajectory during the game, while the sensor data records the players' movement speed and heart rate. The collected data is preprocessed to remove noise, normalize the data, and fill in missing values. This preprocessing improves the accuracy of the analysis.

[0885] The preprocessed data is input into an AI model. This AI model uses algorithms such as neural networks that have been trained on large amounts of match data in advance to predict players' next moves and the probability of scoring. Based on these predictions, the server converts the analysis results into a visually understandable format. Specifically, heat maps, graphs, text data, etc. are generated and packaged as data packets. The analysis results are then distributed to viewer devices in real time. An efficient data transmission protocol (e.g., WebSocket) is used to minimize communication latency.

[0886] Emotion engine processing

[0887] The emotion engine acquires the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time. Based on the acquired data, the server determines the viewer's emotional state. For example, it determines whether the viewer is excited, surprised, or calm. Based on this emotional state, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it highlights a prediction of the next play, and if the viewer is calm, it displays detailed data analysis results.

[0888] Terminal side processing

[0889] The device receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially. The device analyzes the received data and displays it in a visually easy-to-understand format. Users can check the analysis results in real time along with the game footage. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[0890] User processing

[0891] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides personalized information, allowing for an optimal viewing experience tailored to the viewer's interests and emotional state.

[0892] Specific examples

[0893] For example, when watching a soccer match, the server collects player positions and ball movement in real time from cameras and GPS sensors. The server preprocesses the data and inputs it into an AI model to calculate action predictions and goal probabilities. As a result, a heat map is generated showing the probability of forward players running toward the goal and the positions where they are likely to score a shot. The server sends this information to the viewer's device, which then displays it visually. The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with a high probability of scoring. By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the game and enjoy it more.

[0894] Examples of prompt statements

[0895] "Please explain in natural language how an AI model can accurately predict the probability of a forward player running towards goal and the success rate of a shot in a soccer match. Also, explain how the model can customize the display of the prediction results in real time depending on whether the viewer is excited or calm."

[0896] This system allows viewers to understand the tactics and movements of players in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[0897] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0898] Specific processing flow of the program

[0899] Step 1: Data collection

[0900] The server collects data in real time from cameras installed at the venue and sensors worn by players.

[0901] Input: Camera video data, sensor data (location, speed, biometric information)

[0902] How it works: Video data includes the positional information of players and the trajectory of the ball during the match, and sensor data such as the player's movement speed and heart rate are recorded in real time. This data is sent to a server in streaming format.

[0903] Output: Collected raw data (video data, sensor data)

[0904] Step 2: Data Preprocessing

[0905] The server performs noise removal, data normalization, and missing value completion on the collected data.

[0906] Input: Raw data collected

[0907] Specific operations: Use a noise reduction algorithm to properly remove high frequency noise, standardize each data point, and apply a Kalman filter or similar to fill in missing data.

[0908] Output: Preprocessed data

[0909] Step 3: Analysis by AI model

[0910] Based on the pre-processed data, the server uses an artificial intelligence model to predict the player's next move and the probability of scoring.

[0911] Input: Preprocessed data

[0912] How it works: Preprocessed data is fed into a neural network (e.g., LSTM or CNN) to predict player movements and shot success rates. The AI ​​model makes highly accurate predictions based on pre-trained patterns.

[0913] Output: Predicted result (next move, score probability)

[0914] Step 4: Generate analysis results

[0915] The server converts the analysis results into a visually easy-to-understand format based on the output of the AI ​​model.

[0916] Input: AI model prediction results

[0917] Specific operations: Generate a heat map, represent the score probability as a bar graph, and add an explanation as text data. Organize the result data into a packet format.

[0918] Output: Visually formatted analysis results (heatmaps, graphs, text data)

[0919] Step 5: Delivering results

[0920] The server delivers the analysis results to the viewer's device in real time.

[0921] Input: Visually formatted analysis results

[0922] Specific operation: Data packets are sent to the viewer device using an efficient data transmission protocol (e.g., WebSocket).

[0923] Output: Analysis result data delivered to the viewer's device

[0924] Step 6: Obtaining emotion data

[0925] The server's emotion engine collects viewers' facial recognition data, voice data, and biometric information.

[0926] Input: Facial recognition data, voice data, biometric data

[0927] Specific operation: Collect data from cameras, microphones, and wearable devices, and analyze the emotional state using facial expression recognition algorithms (e.g., OpenCV).

[0928] Output: Emotion data

[0929] Step 7: Determine your emotional state

[0930] The server's emotion engine determines the viewer's emotional state based on the acquired data.

[0931] Input: Emotion data

[0932] Specific operations: Performs voice tone analysis and heart rate variability analysis to cross-check and determine the viewer's emotional state, such as excitement, surprise, or calmness.

[0933] Output: Determined emotional state

[0934] Step 8: Customize what's displayed

[0935] The server customizes the display of the analysis results based on the viewer's emotional state.

[0936] Input: Determined emotional state, analysis result data delivered to the viewer's device

[0937] Specific behavior: When viewers are excited, the layout and font size are adaptively changed to highlight highlights and next play predictions, and when viewers are calm, to display detailed data analysis results.

[0938] Output: Customized display content

[0939] Step 9: Receiving Data

[0940] The terminal receives the analysis result data sent from the server.

[0941] Input: Data packet sent by the server

[0942] What it does: Receives data packets over a WebSocket connection and stores them in a local memory buffer.

[0943] Output: Analysis result data stored in memory buffer

[0944] Step 10: Displaying the data

[0945] The device analyzes the received data and displays it in a visually easy-to-understand format.

[0946] Input: Analysis result data stored in memory buffer

[0947] What it does: Render heatmaps, graphs, and text data in real time and integrate them into the user interface, providing customized displays based on the results of the sentiment engine.

[0948] Output: Analysis results displayed in the user interface

[0949] (Application example 2)

[0950] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0951] Conventional match analysis systems are limited to predicting player behavior and scoring probability during a match, and are not customized to reflect the viewer's emotional state. This makes it difficult to provide information tailored to each viewer's interests and emotions, resulting in a lack of personalized viewing experiences. To address this issue, a system is needed that can recognize viewer emotions in real time and customize the display format and content of analysis results based on those emotions.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to the viewer terminal in real time, means for acquiring face recognition data, voice data, and biometric information data of the viewer, means for determining the viewer's emotional state using the acquired data, and means for customizing the display format and content of the prediction results according to the viewer's emotional state. This makes it possible to provide analysis results suited to the viewer's emotional state, thereby realizing a personalized viewing experience.

[0953] "Game video data" refers to digital video data captured during a game, including the positional information of players and the ball, their movements, and the flow of the game.

[0954] "Sensor data" refers to biometric and environmental data such as location, speed, and heart rate obtained from sensors attached to athletes.

[0955] "Preprocessing" refers to the process of converting acquired data, such as by removing noise, normalizing, and filling in missing values, to prepare the data for analysis.

[0956] An "AI model" refers to artificial intelligence technology that uses pre-trained algorithms to predict player movements and scoring probabilities from input data.

[0957] "Visually easy-to-understand format" refers to formats such as heat maps, graphs, and text data that present analysis results in a way that is easy for users to understand.

[0958] "Real-time" refers to the delivery or processing of data nearly simultaneously with minimal delay.

[0959] A "viewer terminal" is a device used by a viewer to receive data and check the analysis results, and includes smartphones, head-mounted displays, etc.

[0960] "Facial Recognition Data" refers to digital image data used to analyze a viewer's facial expressions.

[0961] "Audio Data" means digital audio data that captures the voice of a viewer and is used for audio analysis.

[0962] "Biometric data" refers to personal physiological data obtained from biometric sensors, such as a viewer's heart rate or skin temperature.

[0963] "Emotional state" refers to the psychological state, such as excitement, calmness, or surprise, that viewers express in real time.

[0964] "Customization" refers to adapting the content and format of a presentation based on the emotional state of each viewer.

[0965] This system analyzes the behavior of players during a match, provides the results to viewers in real time, and customizes the results according to the viewer's emotional state. This system includes the collection, preprocessing, and analysis of match data, the generation and distribution of results, viewer emotion recognition, and the customization of the display content.

[0966] Overall system configuration

[0967] It consists of a server, viewer terminal, emotion engine, and user interface. The server-side processing is divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, it recognizes viewer emotions and reflects this information in the display of analysis results.

[0968] Server-side processing

[0969] 1. Data Collection

[0970] The server collects video and sensor data from the match in real time. Specifically, it acquires data from cameras installed at the match venue and sensors worn by players. The hardware used includes cameras (e.g., Logitech C920) and heart rate monitors.

[0971] 2. Data Preprocessing

[0972] The server preprocesses the collected data. This preprocessing includes noise removal, data normalization, and missing value imputation, making the data ready for analysis. Software libraries used include NumPy and Pandas.

[0973] 3. Analysis using AI models

[0974] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model is trained in advance using a large amount of match data and is able to predict moves and patterns with high accuracy. Frameworks used include TensorFlow and PyTorch.

[0975] 4. Generating analysis results

[0976] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format. Specifically, heat maps, graphs, text data, etc. are generated and compiled into data packets. The software used is a JavaScript library.

[0977] 5. Distribution of Results

[0978] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[0979] Emotion engine processing

[0980] 1. Acquiring Emotion Data

[0981] The emotion engine captures viewers' facial recognition data, voice data, and biometric data, and analyzes their emotional state in real time. The hardware used includes cameras, microphones, and biometric sensors.

[0982] 2. Emotional state determination

[0983] The emotion engine analyzes the acquired data and determines the viewer's emotional state. The software used is OpenCV (face recognition) and DeepFace (emotion recognition).

[0984] 3. Customizing the display content

[0985] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight the predictions for the next play, and if the viewer is calm, it will display detailed data analysis results.

[0986] Terminal side processing

[0987] 1. Receiving Data

[0988] The viewer terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[0989] 2. Displaying Data

[0990] The viewer device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[0991] User processing

[0992] 1. Verify the information

[0993] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game.In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[0994] Examples of specific examples and prompts

[0995] For example, imagine a viewer watching a soccer match on their smartphone. The app displays the positions of players and the movement of the ball in real time, and an emotion recognition engine detects the viewer's excitement. For viewers who are excited, the app highlights scenes with high probability of the next shot, and users can check the analysis results in real time, allowing them to enjoy the match more deeply.

[0996] Example prompts for generative AI models

[0997] You are an AI that generates content for a sports viewing app. Using the following information, you must generate content that displays analysis results based on player behavior predictions and viewer emotions in real time.

[0998] Input information

[0999] Player position data (JSON format)

[1000] Ball trajectory data (JSON format)

[1001] The viewer's emotional state (excited, calm, surprised, etc.)

[1002] Real-time video data of the match

[1003] Example output

[1004] 1. Highlighting high probability shots for excited viewers

[1005] 2. Display detailed data analysis results for a calm audience

[1006] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1007] Step 1:

[1008] Data collection

[1009] The server collects game video data and sensor data (location information, speed, heart rate, etc.) in real time from cameras installed at the venue and sensors worn by players. The input is camera video data and sensor data, and the output is a data packet that compiles these. Specifically, the camera (Logitech C920) and heart rate sensor capture the game situation and player biometric data, which the server receives.

[1010] Step 2:

[1011] Data Preprocessing

[1012] The server denoises the collected video and sensor data, normalizes the data, and fills in missing values ​​to prepare it for analysis. The input is the collected data packet, and the output is the preprocessed data. Specifically, the server cleans the data using NumPy and Pandas.

[1013] Step 3:

[1014] Action prediction and score probability calculation

[1015] The server inputs the preprocessed data into an AI model to predict the player's next move and the pattern with the highest probability of scoring. The input is the preprocessed data, and the output is the prediction result. Specifically, a pre-trained TensorFlow or PyTorch model analyzes the data and makes a prediction.

[1016] Step 4:

[1017] Generate analysis results

[1018] The server uses the prediction results from the AI ​​model to convert them into visually understandable formats such as heat maps, graphs, and text data. The input is the prediction results, and the output is visualized data packets. Specifically, it uses a JavaScript library to graph the data and organize it into packets.

[1019] Step 5:

[1020] Obtaining viewer sentiment data

[1021] The emotion engine acquires the viewer's facial recognition data, audio data, and biometric data. The input is camera footage, microphone audio, and biometric sensor data, and the output is an emotion data packet that combines these. Specifically, it captures data using the camera and microphone, and collects biometric information using sensors.

[1022] Step 6:

[1023] Determining emotional state

[1024] The emotion engine analyzes the acquired data and determines the viewer's emotional state (excitement, calmness, surprise, etc.). The input is an emotion data packet, and the output is the emotional state. Specifically, it uses OpenCV and DeepFace to perform face recognition and emotion analysis.

[1025] Step 7:

[1026] Customizing the display

[1027] The server customizes the display format and content of the prediction results depending on the viewer's emotional state. The input is a data packet of the analysis results and the viewer's emotional state, and the output is a customized data packet of the analysis results. Specifically, if the viewer's emotional state is excited, the shot scene is highlighted, and if the viewer is calm, detailed data is displayed.

[1028] Step 8:

[1029] Data distribution

[1030] The server delivers data packets of customized analysis results to the viewer terminal in real time. The input is the data packets of customized analysis results, and the output is delivery to the viewer terminal. Specifically, the server transmits the data using an efficient data transmission protocol.

[1031] Step 9:

[1032] Receiving and displaying data

[1033] The viewer device receives data packets sent from the server, analyzes them, and displays them in a visually easy-to-understand format. The input is the delivered data packets, and the output is the display data on the device. Specifically, it analyzes the received data and updates the UI displayed on the screen.

[1034] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1035] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1036] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1037] [Third embodiment]

[1038] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1039] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1040] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1041] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1042] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1043] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1044] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1045] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1046] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1047] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1048] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1049] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1050] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[1051] Overall system configuration

[1052] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user device, allowing users to enjoy the game from a new perspective.

[1053] Server-side processing

[1054] 1. Data Collection

[1055] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1056] 2. Data Preprocessing

[1057] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1058] 3. Analysis using AI models

[1059] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1060] 4. Generating analysis results

[1061] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1062] 5. Distribution of Results

[1063] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1064] Terminal side processing

[1065] 1. Receiving Data

[1066] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1067] 2. Displaying Data

[1068] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[1069] User processing

[1070] 1. Verify the information

[1071] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[1072] Specific examples

[1073] For example, consider the case of watching a soccer match.

[1074] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[1075] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1076] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1077] The server transmits this information to the viewer's terminal, which displays it visually.

[1078] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[1079] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.

[1080] The processing flow will be explained below.

[1081] Server-side processing

[1082] Step 1: Data collection

[1083] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[1084] The server also collects real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, and more.

[1085] Step 2: Data Preprocessing

[1086] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[1087] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[1088] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[1089] Step 3: Analysis by AI model

[1090] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[1091] The server's AI model predicts a player's next move or play. For example, if a specific player receives a pass, it calculates the probability of each subsequent move: dribbling, passing, and shooting.

[1092] The server's AI model also calculates the probability of play patterns that lead to goals, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[1093] Step 4: Generate analysis results

[1094] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[1095] The server compiles these analysis results into packets for efficient distribution over the network.

[1096] Step 5: Delivering results

[1097] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[1098] Terminal side processing

[1099] Step 1: Receiving Data

[1100] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1101] Step 2: View the data

[1102] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[1103] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[1104] User processing

[1105] Step 1: Verify the information

[1106] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, and enjoy watching the game.

[1107] Example 1

[1108] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1109] Conventional sports viewing systems have had difficulty analyzing players' actions and scoring probabilities during a game in real time and providing the analysis to viewers. This has resulted in viewers merely passively watching the video and not gaining a deeper understanding of the game's tactics or players' movements. Furthermore, the delivery of real-time analysis results has been insufficient, preventing viewers from immediately obtaining useful information for predicting the course of the game. The purpose of this invention is to solve these problems.

[1110] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1111] In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to a viewer terminal in real time, and means for displaying the received analysis results in real time on the viewer terminal. This allows viewers to understand the actions of players during the game and scoring probabilities in real time, enabling them to enjoy the game from a deeper perspective.

[1112] "Game video data" refers to data obtained from video footage taken during a game, including player movements and ball trajectories.

[1113] "Sensor data" refers to data obtained from sensors worn by players during a match, and includes information such as location, speed, and heart rate.

[1114] "Preprocessing" refers to performing processes such as noise removal, normalization, and missing value completion on the acquired game video data and sensor data.

[1115] An "AI model" is an artificial intelligence model trained on large amounts of match data, which predicts a player's next move and the probability of scoring during a match.

[1116] "Converting into a visually easy-to-understand format" refers to the process of converting the prediction results into a format such as a heat map, graph, or text data so that viewers can intuitively understand it.

[1117] "Real-time delivery" means transmitting data from the server to the viewer's terminal with minimal delay.

[1118] A "viewer terminal" is a device that receives the analysis results and displays them to the viewer, and includes, for example, a smartphone, tablet, or PC.

[1119] "Display in real time" means that the data received is immediately processed and displayed on the viewer terminal.

[1120] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[1121] Overall system configuration

[1122] This system consists of a server, viewer terminals, and a user interface. The processing on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user terminal, allowing users to enjoy the game from a new perspective.

[1123] Server-side processing

[1124] Data collection

[1125] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it acquires game footage from standard high-definition video cameras, capturing player positions and ball trajectories, and also acquires data such as player heart rate and movement speed from standard heart rate monitors.

[1126] Data Preprocessing

[1127] The server preprocesses the collected data, using Python data processing libraries such as Pandas to clean the data, impute missing values, and remove outliers, and NumPy to normalize the data and convert it into a consistent format.

[1128] Analysis using AI models

[1129] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[1130] Generate analysis results

[1131] The server receives the predictions generated by the AI ​​model and converts them into a visually understandable format. This process involves generating heat maps and graphs using Matplotlib and Seaborn, and outputting the results as text data. The analysis results are then compiled into a data packet.

[1132] Results distribution

[1133] The server uses the WebSocket protocol to deliver analysis result data packets to viewer devices in real time, using efficient data transmission techniques to minimize communication delays during this process.

[1134] Terminal side processing

[1135] Receiving data

[1136] The device sequentially receives the analysis result data sent from the server. Since the data is sent in packet format, it is analyzed and processed sequentially. For example, data is received via WebSocket using JavaScript.

[1137] Viewing Data

[1138] The device visually displays the received analysis results in real time using HTML5 and JavaScript, displaying the received heat maps and graphs on a web page for easy user access.

[1139] User processing

[1140] Verify the information

[1141] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which areas are most likely to score. Users can use this information to gain a deeper understanding of the flow of the match and enjoy it even more.

[1142] Specific examples

[1143] For example, consider the case of watching a soccer match.

[1144] Data collection

[1145] The server collects real-time data on player positions and ball movement from high-definition video cameras and heart rate monitors.

[1146] Data Preprocessing

[1147] The server uses Python's Pandas and NumPy to remove noise from the collected data and normalize it.

[1148] Analysis using AI models

[1149] The server inputs preprocessed data into an AI model built with TensorFlow to calculate behavioral predictions and score probabilities.

[1150] Generate analysis results

[1151] The server uses Matplotlib and Seaborn to generate heat maps showing the probability of forward players running towards the goal and the positions where they are likely to score a shot.

[1152] Results distribution

[1153] The server uses WebSocket to send the generated heatmap and prediction results to the device in real time.

[1154] Receiving data

[1155] The device uses JavaScript to sequentially access and analyze data received via WebSocket.

[1156] Viewing Data

[1157] Using HTML5, the received heatmap is rendered into a Canvas element and displayed on the device in an intuitive, easy-to-view format for the user.

[1158] User confirms information

[1159] While watching a live broadcast of a match, users can check the heat map displayed on their device. For example, the locations where a particular player has a high probability of scoring a goal are indicated by color, allowing users to predict how the match will unfold and help them cheer on their team.

[1160] Prompt Sentence Examples

[1161] "During a soccer match, I want to predict which player will receive the ball next. Please create an AI model that uses data such as players' locations, movement speeds, and heart rates to predict their next actions."

[1162] In this way, by creating specific prompts, the AI ​​model can analyze the necessary data and provide predictions.

[1163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1164] The processing flow of the program in this system is divided into processing steps.

[1165] Step 1: Data collection

[1166] Step 2: Data Preprocessing

[1167] Step 3: Analysis by AI model

[1168] Step 4: Generate analysis results

[1169] Step 5: Delivering results

[1170] Step 6: Receiving Data

[1171] Step 7: View the data

[1172] Step 8: User confirms information

[1173] Each processing step will now be described in detail.

[1174] Step 1: Data collection

[1175] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it captures game footage from high-resolution video cameras and analyzes it frame by frame to capture player positions and ball trajectories. It also collects data such as players' heart rates, movement speeds, and location information from heart rate monitors and GPS sensors.

[1176] Input: Video camera image data, sensor data (heart rate, movement speed, location information)

[1177] Output: Raw data collected

[1178] Step 2: Data Preprocessing

[1179] The server preprocesses the collected data. This process involves using Python's Pandas to clean the data, impute missing values, and remove outliers. It also uses NumPy to normalize the data and convert it into a consistent format. For example, if heart rate data is missing, it is imputed based on the previous data, and outliers are detected and removed to remove noise.

[1180] Input: Raw data collected

[1181] Output: Preprocessed data

[1182] Step 3: Analysis by AI model

[1183] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[1184] Input: Preprocessed data

[1185] Output: Prediction result (next move, score probability, etc.)

[1186] Step 4: Generate analysis results

[1187] The server receives the prediction results from the AI ​​model and converts them into a visually understandable format, using Matplotlib and Seaborn to generate heat maps and graphs, which are then output as text data, allowing viewers to intuitively understand the results.

[1188] Input: Prediction result

[1189] Output: Analysis results in a visually understandable format (heat maps, graphs, text data, etc.)

[1190] Step 5: Delivering results

[1191] The server uses the WebSocket protocol to deliver analysis result data packets to the viewer's device in real time. To minimize communication delays, efficient data transmission techniques are used, such as immediate delivery without buffering.

[1192] Input: Analysis results in a visually easy-to-understand format

[1193] Output: Analysis result data packets delivered in real time

[1194] Step 6: Receiving Data

[1195] The device sequentially receives the analysis result data sent from the server. It uses JavaScript to receive the data via WebSocket, then sequentially analyzes and processes it. It analyzes the received data and converts it into the format required for display.

[1196] Input: Analysis result data packet delivered in real time

[1197] Output: processed data in the terminal

[1198] Step 7: View the data

[1199] The device visually displays the received analysis results in real time using HTML5 and JavaScript, rendering heat maps and graphs onto a Canvas element on a web page in an intuitive, user-friendly format.

[1200] Input: Data processed on the terminal

[1201] Output: Analysis results displayed in real time (heat maps, graphs, etc.)

[1202] Step 8: User confirms information

[1203] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which area is most likely to score. This allows them to gain a deeper understanding of the flow of the match and use it to support their team.

[1204] Input: Analysis results displayed in real time

[1205] Output: Improved user understanding, prediction, and support

[1206] (Application example 1)

[1207] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1208] When watching sports, viewers need real-time information such as player actions and scoring probabilities to better understand and enjoy the flow of the game. However, current technology is limited in its means of visually providing players' specific movements during the game, predictions of their next actions, and scoring possibilities in an easy-to-understand manner. This makes it difficult for viewers to understand and enjoy the game from a more tactical and strategic perspective. The purpose of this invention is to solve this problem and provide a system that allows viewers to enjoy the game from a new perspective.

[1209] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1210] In this invention, the server includes means for acquiring video data and sensor data of the match, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probability using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for displaying the analysis results in an interactive user interface so that viewers can check which player is most likely to receive the ball next or the position with high scoring probability, and means for delivering the prediction results to viewer terminals in real time, thereby enabling viewers to check detailed analysis information in real time and enjoy the flow of the match more deeply.

[1211] "Game video data" refers to visual digital information captured to record the positions and movements of players and the ball during a game.

[1212] "Sensor data" refers to various physiological and physical data such as location information, speed, and heart rate obtained from sensors attached to athletes.

[1213] "Preprocessing" is a preparatory stage in which acquired data is subjected to processes such as noise removal, normalization, and missing value completion, allowing for more accurate and efficient analysis.

[1214] An "AI model" is an artificial intelligence algorithm that has been trained in advance using large amounts of match data to predict a player's next move and patterns with a high probability of scoring.

[1215] A "visually easy-to-understand format" is a display format in which the prediction results are converted into heat maps, graphs, text data, etc. so that viewers can intuitively understand them.

[1216] An "interactive user interface" is an operating screen that allows users to actively perform operations and make selections, and displays analysis results in combination with actual game footage.

[1217] A "viewer device" is an electronic device such as a smartphone, smart glasses, or head-mounted display that allows viewers to easily view real-time analysis results.

[1218] "Real-time distribution" is a communication method that transmits acquired and analyzed data to viewer terminals in real time without delay.

[1219] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. As an embodiment of the present invention, the following detailed process and specific examples will be described.

[1220] Overall system configuration

[1221] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. The viewer device displays the received analysis results, allowing users to enjoy the game from a new perspective.

[1222] Server-side processing

[1223] 1. Data Collection

[1224] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1225] 2. Data Preprocessing

[1226] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1227] 3. Analysis using AI models

[1228] The server inputs the preprocessed data into an AI model to predict players' next moves and the probability of scoring. This AI model has been trained in advance using TensorFlow and other tools on a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1229] 4. Generating analysis results

[1230] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1231] 5. Distribution of Results

[1232] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1233] Terminal side processing

[1234] 1. Receiving Data

[1235] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1236] 2. Displaying Data

[1237] The device analyzes the received data and displays it in a visually easy-to-understand format via an interactive user interface, allowing users to view the analysis results in real time along with the game footage.

[1238] User processing

[1239] 1. Verify the information

[1240] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[1241] Specific examples

[1242] For example, consider the case of watching a soccer match.

[1243] The server collects player positions and ball movement in real time from cameras and GPS sensors.

[1244] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1245] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1246] The server transmits this information to the viewer's terminal, which displays it visually.

[1247] By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the match and enjoy it more.

[1248] Prompt Sentence Examples

[1249] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[1250] data:

[1251] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[1252] Ball position information: {x: 45, y: 50}

[1253] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[1254] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[1255] Through this specific example, the Sports Analysis Viewer makes watching a game more interactive and enjoyable. By being able to check the analysis results in real time, viewers can gain a deeper understanding of players' movements and strategies, and enjoy the game more.

[1256] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1257] Step 1:

[1258] Data collection

[1259] How it works: The server collects real-time data from cameras installed at the venue and sensors worn by players. The cameras capture footage of the game and acquire data including the positions of players and the ball. The sensors record physiological data such as players' movement speed and heart rate.

[1260] Input: Video and sensor data captured from the match venue.

[1261] Output: Raw data that is sent to subsequent preprocessing steps.

[1262] Step 2:

[1263] Data Preprocessing

[1264] Specific operation: The server preprocesses the collected data. This preprocessing includes removing noise from the video data, normalizing the data, and filling in missing values. Specifically, it removes unnecessary pixel information from the video data and properly fills in missing parts of the sensor data. It also normalizes the data to make it easier to analyze.

[1265] Input: The raw data obtained in step 1.

[1266] Output: Pre-processed data that is sent to subsequent AI analysis steps.

[1267] Step 3:

[1268] Analysis using AI models

[1269] Specific operation: The server inputs the preprocessed data into the AI ​​model. The AI ​​model has been trained in advance with a large amount of match data using TensorFlow and other tools, and uses this data to predict the players' next moves and patterns with high scoring probabilities. Specifically, the model inputs the match situation and outputs player positions and score predictions.

[1270] Input: The preprocessed data from step 2.

[1271] Output: Player action predictions and scoring probability data as analysis results.

[1272] Step 4:

[1273] Generate analysis results

[1274] How it works: The server uses the predictions from the AI ​​model to convert the analysis results into a visually understandable format. Specifically, it generates heat maps, graphs, and text data, and compiles them into data packets. For example, a heat map uses colors to indicate where a player is likely to score.

[1275] Input: Action prediction and score probability data generated in Step 3.

[1276] Output: Visual analysis data packet sent to subsequent delivery steps.

[1277] Step 5:

[1278] Results distribution

[1279] Specific operation: The server delivers analysis results to the viewer's device in real time. To minimize communication delays, an efficient data transmission protocol is used. Specifically, WebSocket or HTTP / 2 is used to transmit data at high speed.

[1280] Input: The visual analysis data packet generated in step 4.

[1281] Output: Real-time analysis results sent to viewer devices.

[1282] Step 6:

[1283] Receiving data

[1284] Specific operation: The terminal receives the analysis result data sent from the server. Since the received data is sent in packet format, it is processed sequentially. Specifically, the terminal receives the data packets using the network library and converts them into a data format for analysis.

[1285] Input: Real-time analytics data delivered in Step 5.

[1286] Output: Analysis data transformed for display.

[1287] Step 7:

[1288] Viewing Data

[1289] How it works: The device analyzes the received data and displays it in an interactive user interface in a visually easy-to-understand format. For example, it can overlay game footage with a heat map showing players' positions and goal probability, allowing viewers to see the game's tactics and player movements in real time.

[1290] Input: The parsed data received and transformed in step 6.

[1291] Output: Visually displayed analysis results and match footage.

[1292] Examples of prompt statements

[1293] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[1294] data:

[1295] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[1296] Ball position information: {x: 45, y: 50}

[1297] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[1298] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[1299] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1300] The present invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time, and further combines the function of recognizing viewers' emotions and customizing the display content. Specific embodiments of the present invention are described below.

[1301] Overall system configuration

[1302] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[1303] Server-side processing

[1304] 1. Data Collection

[1305] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1306] 2. Data Preprocessing

[1307] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1308] 3. Analysis using AI models

[1309] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1310] 4. Generating analysis results

[1311] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1312] 5. Distribution of Results

[1313] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1314] Emotion engine processing

[1315] 1. Acquiring Emotion Data

[1316] The emotion engine captures the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time.

[1317] 2. Emotional state determination

[1318] The emotion engine analyzes the captured data and determines the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[1319] 3. Customizing the display content

[1320] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight predictions for the next play, but if the viewer is calm, it will display detailed data analysis results.

[1321] Terminal side processing

[1322] 1. Receiving Data

[1323] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1324] 2. Displaying Data

[1325] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[1326] User processing

[1327] 1. Verify the information

[1328] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[1329] Specific examples

[1330] For example, consider the case of watching a soccer match.

[1331] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[1332] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1333] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1334] The server transmits this information to the viewer's terminal, which displays it visually.

[1335] The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with high success rates for shots.

[1336] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[1337] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[1338] The processing flow will be explained below.

[1339] Server-side processing

[1340] Step 1: Data collection

[1341] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[1342] The server also receives real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, etc.

[1343] Step 2: Data Preprocessing

[1344] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[1345] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[1346] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[1347] Step 3: Analysis by AI model

[1348] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[1349] The server's AI model predicts a player's next move or play. For example, if a specific player receives the ball, it calculates the probability of each subsequent move: dribbling, passing, or shooting.

[1350] The server's AI model calculates the probability of a play pattern leading to a goal, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[1351] Step 4: Generate analysis results

[1352] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[1353] The server compiles these analysis results into packets for efficient distribution over the network.

[1354] Step 5: Delivering results

[1355] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[1356] Emotion engine processing

[1357] Step 1: Obtaining emotion data

[1358] The emotion engine in the server acquires emotion data (face recognition data, voice data, biometric data) sent from the viewer terminal.

[1359] Step 2: Determine your emotional state

[1360] The emotion engine analyzes the acquired emotion data to determine the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[1361] Step 3: Customize what's displayed

[1362] The server customizes the display format and content of the analysis results based on the viewer's emotional state as determined by the emotion engine. For example, if the viewer is excited, it will highlight the prediction of the next play, but if the viewer is calm, it will display detailed data analysis results.

[1363] Terminal side processing

[1364] Step 1: Receiving Data

[1365] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1366] Step 2: View the data

[1367] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[1368] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[1369] User processing

[1370] Step 1: Verify the information

[1371] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[1372] Example 2

[1373] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1374] Conventional match viewing systems make it difficult for viewers to visually grasp player behavior and score probability predictions during a match, resulting in a lack of real-time information provision for viewers. Furthermore, they lack the ability to customize information based on each viewer's emotional state, preventing a personalized viewing experience. This makes it difficult for viewers to deeply understand and enjoy the flow and tactics of the match.

[1375] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1376] In this invention, the server includes means for acquiring video data and sensor data of a match, means for preprocessing the acquired data, artificial intelligence model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to viewer terminals in real time, means for acquiring viewer emotional data and analyzing the viewer's emotional state, and means for customizing the display of the prediction results based on the viewer's emotional state. This allows viewers to grasp the analysis results of the match in real time, and the provision of personalized information allows them to gain a deeper understanding of the flow and tactics of the match, making it even more enjoyable.

[1377] "Game video data" refers to real-time video information captured by cameras installed at the game venue, and includes data such as the positions and movements of players and the trajectory of the ball.

[1378] "Sensor data" refers to data such as location information, speed, and biometric information obtained from wearable devices attached to athletes.

[1379] "Preprocessing" refers to processing of collected data, such as noise removal, data normalization, and missing value completion, to improve the accuracy of analysis.

[1380] An "artificial intelligence model" is a system that uses algorithms such as neural networks that have been trained in advance on large amounts of match data to predict a player's next move and the probability of scoring.

[1381] A "visually easy-to-understand format" means converting the analysis results into a format such as a heat map, graph, or text data, making it easy for viewers to understand.

[1382] "Means for delivering to viewer devices in real time" refers to communication protocols and network technologies for instantly transmitting analysis results to viewer devices.

[1383] "Emotion data" is data for analyzing the viewer's emotional state based on the viewer's facial recognition data, voice data, biometric data, etc.

[1384] "Emotional state" refers to the psychological state of the viewer, such as excitement, surprise, or calmness.

[1385] "Means for customization" refers to a mechanism that adjusts the content and format of the analysis results displayed according to the viewer's emotional state, providing optimal information to each individual viewer.

[1386] MODE FOR CARRYING OUT THE INVENTION

[1387] This invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time. It also combines a function to recognize viewers' emotions and customize the display content. Specific embodiments of this system are described below.

[1388] Overall system configuration

[1389] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[1390] Server-side processing

[1391] The server collects data in real time from cameras installed at the venue and sensors worn by players. The video data includes information on the players' positions and the ball's trajectory during the game, while the sensor data records the players' movement speed and heart rate. The collected data is preprocessed to remove noise, normalize the data, and fill in missing values. This preprocessing improves the accuracy of the analysis.

[1392] The preprocessed data is input into an AI model. This AI model uses algorithms such as neural networks that have been trained on large amounts of match data in advance to predict players' next moves and the probability of scoring. Based on these predictions, the server converts the analysis results into a visually understandable format. Specifically, heat maps, graphs, text data, etc. are generated and packaged as data packets. The analysis results are then distributed to viewer devices in real time. An efficient data transmission protocol (e.g., WebSocket) is used to minimize communication latency.

[1393] Emotion engine processing

[1394] The emotion engine acquires the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time. Based on the acquired data, the server determines the viewer's emotional state. For example, it determines whether the viewer is excited, surprised, or calm. Based on this emotional state, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it highlights a prediction of the next play, and if the viewer is calm, it displays detailed data analysis results.

[1395] Terminal side processing

[1396] The device receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially. The device analyzes the received data and displays it in a visually easy-to-understand format. Users can check the analysis results in real time along with the game footage. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[1397] User processing

[1398] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides personalized information, allowing for an optimal viewing experience tailored to the viewer's interests and emotional state.

[1399] Specific examples

[1400] For example, when watching a soccer match, the server collects player positions and ball movement in real time from cameras and GPS sensors. The server preprocesses the data and inputs it into an AI model to calculate action predictions and goal probabilities. As a result, a heat map is generated showing the probability of forward players running toward the goal and the positions where they are likely to score a shot. The server sends this information to the viewer's device, which then displays it visually. The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with a high probability of scoring. By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the game and enjoy it more.

[1401] Examples of prompt statements

[1402] "Please explain in natural language how an AI model can accurately predict the probability of a forward player running towards goal and the success rate of a shot in a soccer match. Also, explain how the model can customize the display of the prediction results in real time depending on whether the viewer is excited or calm."

[1403] This system allows viewers to understand the tactics and movements of players in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[1404] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1405] Specific processing flow of the program

[1406] Step 1: Data collection

[1407] The server collects data in real time from cameras installed at the venue and sensors worn by players.

[1408] Input: Camera video data, sensor data (location, speed, biometric information)

[1409] How it works: Video data includes the positional information of players and the trajectory of the ball during the match, and sensor data such as the player's movement speed and heart rate are recorded in real time. This data is sent to a server in streaming format.

[1410] Output: Collected raw data (video data, sensor data)

[1411] Step 2: Data Preprocessing

[1412] The server performs noise removal, data normalization, and missing value completion on the collected data.

[1413] Input: Raw data collected

[1414] Specific operations: Use a noise reduction algorithm to properly remove high frequency noise, standardize each data point, and apply a Kalman filter or similar to fill in missing data.

[1415] Output: Preprocessed data

[1416] Step 3: Analysis by AI model

[1417] Based on the pre-processed data, the server uses an artificial intelligence model to predict the player's next move and the probability of scoring.

[1418] Input: Preprocessed data

[1419] How it works: Preprocessed data is fed into a neural network (e.g., LSTM or CNN) to predict player movements and shot success rates. The AI ​​model makes highly accurate predictions based on pre-trained patterns.

[1420] Output: Predicted result (next move, score probability)

[1421] Step 4: Generate analysis results

[1422] The server converts the analysis results into a visually easy-to-understand format based on the output of the AI ​​model.

[1423] Input: AI model prediction results

[1424] Specific operations: Generate a heat map, represent the score probability as a bar graph, and add an explanation as text data. Organize the result data into a packet format.

[1425] Output: Visually formatted analysis results (heatmaps, graphs, text data)

[1426] Step 5: Delivering results

[1427] The server delivers the analysis results to the viewer's device in real time.

[1428] Input: Visually formatted analysis results

[1429] Specific operation: Data packets are sent to the viewer device using an efficient data transmission protocol (e.g., WebSocket).

[1430] Output: Analysis result data delivered to the viewer's device

[1431] Step 6: Obtaining emotion data

[1432] The server's emotion engine collects viewers' facial recognition data, voice data, and biometric information.

[1433] Input: Facial recognition data, voice data, biometric data

[1434] Specific operation: Collect data from cameras, microphones, and wearable devices, and analyze the emotional state using facial expression recognition algorithms (e.g., OpenCV).

[1435] Output: Emotion data

[1436] Step 7: Determine your emotional state

[1437] The server's emotion engine determines the viewer's emotional state based on the acquired data.

[1438] Input: Emotion data

[1439] Specific operations: Performs voice tone analysis and heart rate variability analysis to cross-check and determine the viewer's emotional state, such as excitement, surprise, or calmness.

[1440] Output: Determined emotional state

[1441] Step 8: Customize what's displayed

[1442] The server customizes the display of the analysis results based on the viewer's emotional state.

[1443] Input: Determined emotional state, analysis result data delivered to the viewer's device

[1444] Specific behavior: When viewers are excited, the layout and font size are adaptively changed to highlight highlights and next play predictions, and when viewers are calm, to display detailed data analysis results.

[1445] Output: Customized display content

[1446] Step 9: Receiving Data

[1447] The terminal receives the analysis result data sent from the server.

[1448] Input: Data packet sent by the server

[1449] What it does: Receives data packets over a WebSocket connection and stores them in a local memory buffer.

[1450] Output: Analysis result data stored in memory buffer

[1451] Step 10: Displaying the data

[1452] The device analyzes the received data and displays it in a visually easy-to-understand format.

[1453] Input: Analysis result data stored in memory buffer

[1454] What it does: Render heatmaps, graphs, and text data in real time and integrate them into the user interface, providing customized displays based on the results of the sentiment engine.

[1455] Output: Analysis results displayed in the user interface

[1456] (Application example 2)

[1457] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1458] Conventional match analysis systems are limited to predicting player behavior and scoring probability during a match, and are not customized to reflect the viewer's emotional state. This makes it difficult to provide information tailored to each viewer's interests and emotions, resulting in a lack of personalized viewing experiences. To address this issue, a system is needed that can recognize viewer emotions in real time and customize the display format and content of analysis results based on those emotions.

[1459] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to the viewer terminal in real time, means for acquiring face recognition data, voice data, and biometric information data of the viewer, means for determining the viewer's emotional state using the acquired data, and means for customizing the display format and content of the prediction results according to the viewer's emotional state. This makes it possible to provide analysis results suited to the viewer's emotional state, thereby realizing a personalized viewing experience.

[1460] "Game video data" refers to digital video data captured during a game, including the positional information of players and the ball, their movements, and the flow of the game.

[1461] "Sensor data" refers to biometric and environmental data such as location, speed, and heart rate obtained from sensors attached to athletes.

[1462] "Preprocessing" refers to the process of converting acquired data, such as by removing noise, normalizing, and filling in missing values, to prepare the data for analysis.

[1463] An "AI model" refers to artificial intelligence technology that uses pre-trained algorithms to predict player movements and scoring probabilities from input data.

[1464] "Visually easy-to-understand format" refers to formats such as heat maps, graphs, and text data that present analysis results in a way that is easy for users to understand.

[1465] "Real-time" refers to the delivery or processing of data nearly simultaneously with minimal delay.

[1466] A "viewer terminal" is a device used by a viewer to receive data and check the analysis results, and includes smartphones, head-mounted displays, etc.

[1467] "Facial Recognition Data" refers to digital image data used to analyze a viewer's facial expressions.

[1468] "Audio Data" means digital audio data that captures the voice of a viewer and is used for audio analysis.

[1469] "Biometric data" refers to personal physiological data obtained from biometric sensors, such as a viewer's heart rate or skin temperature.

[1470] "Emotional state" refers to the psychological state, such as excitement, calmness, or surprise, that viewers express in real time.

[1471] "Customization" refers to adapting the content and format of a presentation based on the emotional state of each viewer.

[1472] This system analyzes the behavior of players during a match, provides the results to viewers in real time, and customizes the results according to the viewer's emotional state. This system includes the collection, preprocessing, and analysis of match data, the generation and distribution of results, viewer emotion recognition, and the customization of the display content.

[1473] Overall system configuration

[1474] It consists of a server, viewer terminal, emotion engine, and user interface. The server-side processing is divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, it recognizes viewer emotions and reflects this information in the display of analysis results.

[1475] Server-side processing

[1476] 1. Data Collection

[1477] The server collects video and sensor data from the match in real time. Specifically, it acquires data from cameras installed at the match venue and sensors worn by players. The hardware used includes cameras (e.g., Logitech C920) and heart rate monitors.

[1478] 2. Data Preprocessing

[1479] The server preprocesses the collected data. This preprocessing includes noise removal, data normalization, and missing value imputation, making the data ready for analysis. Software libraries used include NumPy and Pandas.

[1480] 3. Analysis using AI models

[1481] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model is trained in advance using a large amount of match data and is able to predict moves and patterns with high accuracy. Frameworks used include TensorFlow and PyTorch.

[1482] 4. Generating analysis results

[1483] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format. Specifically, heat maps, graphs, text data, etc. are generated and compiled into data packets. The software used is a JavaScript library.

[1484] 5. Distribution of Results

[1485] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1486] Emotion engine processing

[1487] 1. Acquiring Emotion Data

[1488] The emotion engine captures viewers' facial recognition data, voice data, and biometric data, and analyzes their emotional state in real time. The hardware used includes cameras, microphones, and biometric sensors.

[1489] 2. Emotional state determination

[1490] The emotion engine analyzes the acquired data and determines the viewer's emotional state. The software used is OpenCV (face recognition) and DeepFace (emotion recognition).

[1491] 3. Customizing the display content

[1492] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight the predictions for the next play, and if the viewer is calm, it will display detailed data analysis results.

[1493] Terminal side processing

[1494] 1. Receiving Data

[1495] The viewer terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1496] 2. Displaying Data

[1497] The viewer device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[1498] User processing

[1499] 1. Verify the information

[1500] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game.In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[1501] Examples of specific examples and prompts

[1502] For example, imagine a viewer watching a soccer match on their smartphone. The app displays the positions of players and the movement of the ball in real time, and an emotion recognition engine detects the viewer's excitement. For viewers who are excited, the app highlights scenes with high probability of the next shot, and users can check the analysis results in real time, allowing them to enjoy the match more deeply.

[1503] Example prompts for generative AI models

[1504] You are an AI that generates content for a sports viewing app. Using the following information, you must generate content that displays analysis results based on player behavior predictions and viewer emotions in real time.

[1505] Input information

[1506] Player position data (JSON format)

[1507] Ball trajectory data (JSON format)

[1508] The viewer's emotional state (excited, calm, surprised, etc.)

[1509] Real-time video data of the match

[1510] Example output

[1511] 1. Highlighting high probability shots for excited viewers

[1512] 2. Display detailed data analysis results for a calm audience

[1513] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1514] Step 1:

[1515] Data collection

[1516] The server collects game video data and sensor data (location information, speed, heart rate, etc.) in real time from cameras installed at the venue and sensors worn by players. The input is camera video data and sensor data, and the output is a data packet that compiles these. Specifically, the camera (Logitech C920) and heart rate sensor capture the game situation and player biometric data, which the server receives.

[1517] Step 2:

[1518] Data Preprocessing

[1519] The server denoises the collected video and sensor data, normalizes the data, and fills in missing values ​​to prepare it for analysis. The input is the collected data packet, and the output is the preprocessed data. Specifically, the server cleans the data using NumPy and Pandas.

[1520] Step 3:

[1521] Action prediction and score probability calculation

[1522] The server inputs the preprocessed data into an AI model to predict the player's next move and the pattern with the highest probability of scoring. The input is the preprocessed data, and the output is the prediction result. Specifically, a pre-trained TensorFlow or PyTorch model analyzes the data and makes a prediction.

[1523] Step 4:

[1524] Generate analysis results

[1525] The server uses the prediction results from the AI ​​model to convert them into visually understandable formats such as heat maps, graphs, and text data. The input is the prediction results, and the output is visualized data packets. Specifically, it uses a JavaScript library to graph the data and organize it into packets.

[1526] Step 5:

[1527] Obtaining viewer sentiment data

[1528] The emotion engine acquires the viewer's facial recognition data, audio data, and biometric data. The input is camera footage, microphone audio, and biometric sensor data, and the output is an emotion data packet that combines these. Specifically, it captures data using the camera and microphone, and collects biometric information using sensors.

[1529] Step 6:

[1530] Determining emotional state

[1531] The emotion engine analyzes the acquired data and determines the viewer's emotional state (excitement, calmness, surprise, etc.). The input is an emotion data packet, and the output is the emotional state. Specifically, it uses OpenCV and DeepFace to perform face recognition and emotion analysis.

[1532] Step 7:

[1533] Customizing the display

[1534] The server customizes the display format and content of the prediction results depending on the viewer's emotional state. The input is a data packet of the analysis results and the viewer's emotional state, and the output is a customized data packet of the analysis results. Specifically, if the viewer's emotional state is excited, the shot scene is highlighted, and if the viewer is calm, detailed data is displayed.

[1535] Step 8:

[1536] Data distribution

[1537] The server delivers data packets of customized analysis results to the viewer terminal in real time. The input is the data packets of customized analysis results, and the output is delivery to the viewer terminal. Specifically, the server transmits the data using an efficient data transmission protocol.

[1538] Step 9:

[1539] Receiving and displaying data

[1540] The viewer device receives data packets sent from the server, analyzes them, and displays them in a visually easy-to-understand format. The input is the delivered data packets, and the output is the display data on the device. Specifically, it analyzes the received data and updates the UI displayed on the screen.

[1541] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1542] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1543] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1544] [Fourth embodiment]

[1545] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1546] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1547] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1548] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1549] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1551] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1552] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1553] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1554] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1555] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1556] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1557] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1558] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[1559] Overall system configuration

[1560] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user device, allowing users to enjoy the game from a new perspective.

[1561] Server-side processing

[1562] 1. Data Collection

[1563] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1564] 2. Data Preprocessing

[1565] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1566] 3. Analysis using AI models

[1567] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1568] 4. Generating analysis results

[1569] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1570] 5. Distribution of Results

[1571] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1572] Terminal side processing

[1573] 1. Receiving Data

[1574] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1575] 2. Displaying Data

[1576] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[1577] User processing

[1578] 1. Verify the information

[1579] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[1580] Specific examples

[1581] For example, consider the case of watching a soccer match.

[1582] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[1583] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1584] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1585] The server transmits this information to the viewer's terminal, which displays it visually.

[1586] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[1587] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.

[1588] The processing flow will be explained below.

[1589] Server-side processing

[1590] Step 1: Data collection

[1591] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[1592] The server also collects real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, and more.

[1593] Step 2: Data Preprocessing

[1594] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[1595] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[1596] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[1597] Step 3: Analysis by AI model

[1598] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[1599] The server's AI model predicts a player's next move or play. For example, if a specific player receives a pass, it calculates the probability of each subsequent move: dribbling, passing, and shooting.

[1600] The server's AI model also calculates the probability of play patterns that lead to goals, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[1601] Step 4: Generate analysis results

[1602] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[1603] The server compiles these analysis results into packets for efficient distribution over the network.

[1604] Step 5: Delivering results

[1605] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[1606] Terminal side processing

[1607] Step 1: Receiving Data

[1608] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1609] Step 2: View the data

[1610] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[1611] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[1612] User processing

[1613] Step 1: Verify the information

[1614] Users can check the analysis results displayed on their device while watching the game, which allows them to understand predictions for the next play and the probability of scoring, and enjoy watching the game.

[1615] Example 1

[1616] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1617] Conventional sports viewing systems have had difficulty analyzing players' actions and scoring probabilities during a game in real time and providing the analysis to viewers. This has resulted in viewers merely passively watching the video and not gaining a deeper understanding of the game's tactics or players' movements. Furthermore, the delivery of real-time analysis results has been insufficient, preventing viewers from immediately obtaining useful information for predicting the course of the game. The purpose of this invention is to solve these problems.

[1618] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1619] In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to a viewer terminal in real time, and means for displaying the received analysis results in real time on the viewer terminal. This allows viewers to understand the actions of players during the game and scoring probabilities in real time, enabling them to enjoy the game from a deeper perspective.

[1620] "Game video data" refers to data obtained from video footage taken during a game, including player movements and ball trajectories.

[1621] "Sensor data" refers to data obtained from sensors worn by players during a match, and includes information such as location, speed, and heart rate.

[1622] "Preprocessing" refers to performing processes such as noise removal, normalization, and missing value completion on the acquired game video data and sensor data.

[1623] An "AI model" is an artificial intelligence model trained on large amounts of match data, which predicts a player's next move and the probability of scoring during a match.

[1624] "Converting into a visually easy-to-understand format" refers to the process of converting the prediction results into a format such as a heat map, graph, or text data so that viewers can intuitively understand it.

[1625] "Real-time delivery" means transmitting data from the server to the viewer's terminal with minimal delay.

[1626] A "viewer terminal" is a device that receives the analysis results and displays them to the viewer, and includes, for example, a smartphone, tablet, or PC.

[1627] "Display in real time" means that the data received is immediately processed and displayed on the viewer terminal.

[1628] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. Specific embodiments of the system will be described below.

[1629] Overall system configuration

[1630] This system consists of a server, viewer terminals, and a user interface. The processing on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, the received analysis results are displayed on the user terminal, allowing users to enjoy the game from a new perspective.

[1631] Server-side processing

[1632] Data collection

[1633] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it acquires game footage from standard high-definition video cameras, capturing player positions and ball trajectories, and also acquires data such as player heart rate and movement speed from standard heart rate monitors.

[1634] Data Preprocessing

[1635] The server preprocesses the collected data, using Python data processing libraries such as Pandas to clean the data, impute missing values, and remove outliers, and NumPy to normalize the data and convert it into a consistent format.

[1636] Analysis using AI models

[1637] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[1638] Generate analysis results

[1639] The server receives the predictions generated by the AI ​​model and converts them into a visually understandable format. This process involves generating heat maps and graphs using Matplotlib and Seaborn, and outputting the results as text data. The analysis results are then compiled into a data packet.

[1640] Results distribution

[1641] The server uses the WebSocket protocol to deliver analysis result data packets to viewer devices in real time, using efficient data transmission techniques to minimize communication delays during this process.

[1642] Terminal side processing

[1643] Receiving data

[1644] The device sequentially receives the analysis result data sent from the server. Since the data is sent in packet format, it is analyzed and processed sequentially. For example, data is received via WebSocket using JavaScript.

[1645] Viewing Data

[1646] The device visually displays the received analysis results in real time using HTML5 and JavaScript, displaying the received heat maps and graphs on a web page for easy user access.

[1647] User processing

[1648] Verify the information

[1649] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which areas are most likely to score. Users can use this information to gain a deeper understanding of the flow of the match and enjoy it even more.

[1650] Specific examples

[1651] For example, consider the case of watching a soccer match.

[1652] Data collection

[1653] The server collects real-time data on player positions and ball movement from high-definition video cameras and heart rate monitors.

[1654] Data Preprocessing

[1655] The server uses Python's Pandas and NumPy to remove noise from the collected data and normalize it.

[1656] Analysis using AI models

[1657] The server inputs preprocessed data into an AI model built with TensorFlow to calculate behavioral predictions and score probabilities.

[1658] Generate analysis results

[1659] The server uses Matplotlib and Seaborn to generate heat maps showing the probability of forward players running towards the goal and the positions where they are likely to score a shot.

[1660] Results distribution

[1661] The server uses WebSocket to send the generated heatmap and prediction results to the device in real time.

[1662] Receiving data

[1663] The device uses JavaScript to sequentially access and analyze data received via WebSocket.

[1664] Viewing Data

[1665] Using HTML5, the received heatmap is rendered into a Canvas element and displayed on the device in an intuitive, easy-to-view format for the user.

[1666] User confirms information

[1667] While watching a live broadcast of a match, users can check the heat map displayed on their device. For example, the locations where a particular player has a high probability of scoring a goal are indicated by color, allowing users to predict how the match will unfold and help them cheer on their team.

[1668] Prompt Sentence Examples

[1669] "During a soccer match, I want to predict which player will receive the ball next. Please create an AI model that uses data such as players' locations, movement speeds, and heart rates to predict their next actions."

[1670] In this way, by creating specific prompts, the AI ​​model can analyze the necessary data and provide predictions.

[1671] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1672] The processing flow of the program in this system is divided into processing steps.

[1673] Step 1: Data collection

[1674] Step 2: Data Preprocessing

[1675] Step 3: Analysis by AI model

[1676] Step 4: Generate analysis results

[1677] Step 5: Delivering results

[1678] Step 6: Receiving Data

[1679] Step 7: View the data

[1680] Step 8: User confirms information

[1681] Each processing step will now be described in detail.

[1682] Step 1: Data collection

[1683] The server collects real-time data from cameras installed at the venue and sensors worn by players. Specifically, it captures game footage from high-resolution video cameras and analyzes it frame by frame to capture player positions and ball trajectories. It also collects data such as players' heart rates, movement speeds, and location information from heart rate monitors and GPS sensors.

[1684] Input: Video camera image data, sensor data (heart rate, movement speed, location information)

[1685] Output: Raw data collected

[1686] Step 2: Data Preprocessing

[1687] The server preprocesses the collected data. This process involves using Python's Pandas to clean the data, impute missing values, and remove outliers. It also uses NumPy to normalize the data and convert it into a consistent format. For example, if heart rate data is missing, it is imputed based on the previous data, and outliers are detected and removed to remove noise.

[1688] Input: Raw data collected

[1689] Output: Preprocessed data

[1690] Step 3: Analysis by AI model

[1691] The server inputs the preprocessed data into an AI model built with TensorFlow to predict the players' next moves and scoring probabilities during the game. The model is trained in advance using a large amount of past game data, enabling highly accurate predictions. Specifically, the data is passed through a neural network to analyze the players' behavioral patterns.

[1692] Input: Preprocessed data

[1693] Output: Prediction result (next move, score probability, etc.)

[1694] Step 4: Generate analysis results

[1695] The server receives the prediction results from the AI ​​model and converts them into a visually understandable format, using Matplotlib and Seaborn to generate heat maps and graphs, which are then output as text data, allowing viewers to intuitively understand the results.

[1696] Input: Prediction result

[1697] Output: Analysis results in a visually understandable format (heat maps, graphs, text data, etc.)

[1698] Step 5: Delivering results

[1699] The server uses the WebSocket protocol to deliver analysis result data packets to the viewer's device in real time. To minimize communication delays, efficient data transmission techniques are used, such as immediate delivery without buffering.

[1700] Input: Analysis results in a visually easy-to-understand format

[1701] Output: Analysis result data packets delivered in real time

[1702] Step 6: Receiving Data

[1703] The device sequentially receives the analysis result data sent from the server. It uses JavaScript to receive the data via WebSocket, then sequentially analyzes and processes it. It analyzes the received data and converts it into the format required for display.

[1704] Input: Analysis result data packet delivered in real time

[1705] Output: processed data in the terminal

[1706] Step 7: View the data

[1707] The device visually displays the received analysis results in real time using HTML5 and JavaScript, rendering heat maps and graphs onto a Canvas element on a web page in an intuitive, user-friendly format.

[1708] Input: Data processed on the terminal

[1709] Output: Analysis results displayed in real time (heat maps, graphs, etc.)

[1710] Step 8: User confirms information

[1711] While watching a match, users can check the analysis results displayed on their device in real time. For example, they can see which player is most likely to receive the ball next, or which area is most likely to score. This allows them to gain a deeper understanding of the flow of the match and use it to support their team.

[1712] Input: Analysis results displayed in real time

[1713] Output: Improved user understanding, prediction, and support

[1714] (Application example 1)

[1715] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1716] When watching sports, viewers need real-time information such as player actions and scoring probabilities to better understand and enjoy the flow of the game. However, current technology is limited in its means of visually providing players' specific movements during the game, predictions of their next actions, and scoring possibilities in an easy-to-understand manner. This makes it difficult for viewers to understand and enjoy the game from a more tactical and strategic perspective. The purpose of this invention is to solve this problem and provide a system that allows viewers to enjoy the game from a new perspective.

[1717] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1718] In this invention, the server includes means for acquiring video data and sensor data of the match, means for preprocessing the acquired data, AI model means for predicting the next moves of players and patterns with high scoring probability using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for displaying the analysis results in an interactive user interface so that viewers can check which player is most likely to receive the ball next or the position with high scoring probability, and means for delivering the prediction results to viewer terminals in real time, thereby enabling viewers to check detailed analysis information in real time and enjoy the flow of the match more deeply.

[1719] "Game video data" refers to visual digital information captured to record the positions and movements of players and the ball during a game.

[1720] "Sensor data" refers to various physiological and physical data such as location information, speed, and heart rate obtained from sensors attached to athletes.

[1721] "Preprocessing" is a preparatory stage in which acquired data is subjected to processes such as noise removal, normalization, and missing value completion, allowing for more accurate and efficient analysis.

[1722] An "AI model" is an artificial intelligence algorithm that has been trained in advance using large amounts of match data to predict a player's next move and patterns with a high probability of scoring.

[1723] A "visually easy-to-understand format" is a display format in which the prediction results are converted into heat maps, graphs, text data, etc. so that viewers can intuitively understand them.

[1724] An "interactive user interface" is an operating screen that allows users to actively perform operations and make selections, and displays analysis results in combination with actual game footage.

[1725] A "viewer device" is an electronic device such as a smartphone, smart glasses, or head-mounted display that allows viewers to easily view real-time analysis results.

[1726] "Real-time distribution" is a communication method that transmits acquired and analyzed data to viewer terminals in real time without delay.

[1727] The present invention relates to a system for analyzing the actions of players during a game and providing the analysis results to viewers in real time. As an embodiment of the present invention, the following detailed process and specific examples will be described.

[1728] Overall system configuration

[1729] This system consists of a server-side component, a viewer device-side component, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. The viewer device displays the received analysis results, allowing users to enjoy the game from a new perspective.

[1730] Server-side processing

[1731] 1. Data Collection

[1732] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1733] 2. Data Preprocessing

[1734] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1735] 3. Analysis using AI models

[1736] The server inputs the preprocessed data into an AI model to predict players' next moves and the probability of scoring. This AI model has been trained in advance using TensorFlow and other tools on a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1737] 4. Generating analysis results

[1738] Based on the predictions obtained from the AI ​​model, the server converts the analysis results into a visually understandable format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1739] 5. Distribution of Results

[1740] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1741] Terminal side processing

[1742] 1. Receiving Data

[1743] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1744] 2. Displaying Data

[1745] The device analyzes the received data and displays it in a visually easy-to-understand format via an interactive user interface, allowing users to view the analysis results in real time along with the game footage.

[1746] User processing

[1747] 1. Verify the information

[1748] While watching the game, users can check the analysis results displayed on their device, for example, which player is most likely to receive the ball next, or which areas are most likely to score.

[1749] Specific examples

[1750] For example, consider the case of watching a soccer match.

[1751] The server collects player positions and ball movement in real time from cameras and GPS sensors.

[1752] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1753] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1754] The server transmits this information to the viewer's terminal, which displays it visually.

[1755] By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the match and enjoy it more.

[1756] Prompt Sentence Examples

[1757] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[1758] data:

[1759] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[1760] Ball position information: {x: 45, y: 50}

[1761] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[1762] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[1763] Through this specific example, the Sports Analysis Viewer makes watching a game more interactive and enjoyable. By being able to check the analysis results in real time, viewers can gain a deeper understanding of players' movements and strategies, and enjoy the game more.

[1764] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1765] Step 1:

[1766] Data collection

[1767] How it works: The server collects real-time data from cameras installed at the venue and sensors worn by players. The cameras capture footage of the game and acquire data including the positions of players and the ball. The sensors record physiological data such as players' movement speed and heart rate.

[1768] Input: Video and sensor data captured from the match venue.

[1769] Output: Raw data that is sent to subsequent preprocessing steps.

[1770] Step 2:

[1771] Data Preprocessing

[1772] Specific operation: The server preprocesses the collected data. This preprocessing includes removing noise from the video data, normalizing the data, and filling in missing values. Specifically, it removes unnecessary pixel information from the video data and properly fills in missing parts of the sensor data. It also normalizes the data to make it easier to analyze.

[1773] Input: The raw data obtained in step 1.

[1774] Output: Pre-processed data that is sent to subsequent AI analysis steps.

[1775] Step 3:

[1776] Analysis using AI models

[1777] Specific operation: The server inputs the preprocessed data into the AI ​​model. The AI ​​model has been trained in advance with a large amount of match data using TensorFlow and other tools, and uses this data to predict the players' next moves and patterns with high scoring probabilities. Specifically, the model inputs the match situation and outputs player positions and score predictions.

[1778] Input: The preprocessed data from step 2.

[1779] Output: Player action predictions and scoring probability data as analysis results.

[1780] Step 4:

[1781] Generate analysis results

[1782] How it works: The server uses the predictions from the AI ​​model to convert the analysis results into a visually understandable format. Specifically, it generates heat maps, graphs, and text data, and compiles them into data packets. For example, a heat map uses colors to indicate where a player is likely to score.

[1783] Input: Action prediction and score probability data generated in Step 3.

[1784] Output: Visual analysis data packet sent to subsequent delivery steps.

[1785] Step 5:

[1786] Results distribution

[1787] Specific operation: The server delivers analysis results to the viewer's device in real time. To minimize communication delays, an efficient data transmission protocol is used. Specifically, WebSocket or HTTP / 2 is used to transmit data at high speed.

[1788] Input: The visual analysis data packet generated in step 4.

[1789] Output: Real-time analysis results sent to viewer devices.

[1790] Step 6:

[1791] Receiving data

[1792] Specific operation: The terminal receives the analysis result data sent from the server. Since the received data is sent in packet format, it is processed sequentially. Specifically, the terminal receives the data packets using the network library and converts them into a data format for analysis.

[1793] Input: Real-time analytics data delivered in Step 5.

[1794] Output: Analysis data transformed for display.

[1795] Step 7:

[1796] Viewing Data

[1797] How it works: The device analyzes the received data and displays it in an interactive user interface in a visually easy-to-understand format. For example, it can overlay game footage with a heat map showing players' positions and goal probability, allowing viewers to see the game's tactics and player movements in real time.

[1798] Input: The parsed data received and transformed in step 6.

[1799] Output: Visually displayed analysis results and match footage.

[1800] Examples of prompt statements

[1801] Based on the data below, predict where forward players are most likely to run towards goal during a soccer match.

[1802] data:

[1803] Player location: [{player_id: 1, x: 30, y: 40}, {player_id: 2, x: 50, y: 60}, ...]

[1804] Ball position information: {x: 45, y: 50}

[1805] Player Speed: [{player_id: 1, speed: 5.6}, {player_id: 2, speed: 6.1}, ...]

[1806] Player Heart Rate: [{player_id: 1, heart_rate: 120}, {player_id: 2, heart_rate: 115}, ...]

[1807] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1808] The present invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time, and further combines the function of recognizing viewers' emotions and customizing the display content. Specific embodiments of the present invention are described below.

[1809] Overall system configuration

[1810] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[1811] Server-side processing

[1812] 1. Data Collection

[1813] The server collects real-time data from cameras installed at the venue and sensors worn by players. The video data includes player position information and ball trajectory during the game, while the sensor data records players' movement speed, heart rate, etc.

[1814] 2. Data Preprocessing

[1815] The server preprocesses the collected data, which includes noise removal, data normalization, and missing value imputation, allowing for more accurate analysis.

[1816] 3. Analysis using AI models

[1817] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model has been trained in advance using a large amount of match data, allowing it to predict moves and patterns with high accuracy.

[1818] 4. Generating analysis results

[1819] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format, such as generating heat maps, graphs, and text data, and then compiles them into data packets.

[1820] 5. Distribution of Results

[1821] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1822] Emotion engine processing

[1823] 1. Acquiring Emotion Data

[1824] The emotion engine captures the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time.

[1825] 2. Emotional state determination

[1826] The emotion engine analyzes the captured data and determines the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[1827] 3. Customizing the display content

[1828] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it will highlight predictions for the next play, but if the viewer is calm, it will display detailed data analysis results.

[1829] Terminal side processing

[1830] 1. Receiving Data

[1831] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1832] 2. Displaying Data

[1833] The device analyzes the received data and displays it in a visually easy-to-understand format, allowing users to check the analysis results in real time along with the game footage.

[1834] User processing

[1835] 1. Verify the information

[1836] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[1837] Specific examples

[1838] For example, consider the case of watching a soccer match.

[1839] The server collects real-time data on players' positions and ball movement from cameras and GPS sensors.

[1840] The server preprocesses the data and inputs it into an AI model to calculate behavioral predictions and score probabilities.

[1841] The result is a heat map that shows the probability of a forward player making a run towards goal and where they are most likely to take a shot.

[1842] The server transmits this information to the viewer's terminal, which displays it visually.

[1843] The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with high success rates for shots.

[1844] Users can check the analysis results in real time while watching the game, allowing them to gain a deeper understanding of the flow of the game and enjoy it more.

[1845] The system of the present invention allows viewers to understand the game tactics and player movements in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[1846] The processing flow will be explained below.

[1847] Server-side processing

[1848] Step 1: Data collection

[1849] The server collects real-time video data from cameras installed at the venue and uses a video analysis algorithm to extract the positional information of players and the ball.

[1850] The server also receives real-time data from the GPS sensors and heart rate monitors worn by the athletes, collecting information on each athlete's location, speed, heart rate, etc.

[1851] Step 2: Data Preprocessing

[1852] The server removes noise from the acquired video data and sensor data. For example, if the GPS location information fluctuates significantly from moment to moment, the data is filtered out as an error.

[1853] The server performs data normalization, converting data collected in different formats into a unified format that makes it easier to analyze.

[1854] The server imputes missing data, using historical data and averages to estimate missing values ​​and complete the dataset.

[1855] Step 3: Analysis by AI model

[1856] The server inputs the pre-processed data into an AI model, which has been trained in advance on a large amount of match data.

[1857] The server's AI model predicts a player's next move or play. For example, if a specific player receives the ball, it calculates the probability of each subsequent move: dribbling, passing, or shooting.

[1858] The server's AI model calculates the probability of a play pattern leading to a goal, such as the probability of a successful shot in front of the goal or the probability of a specific passing trajectory leading to a goal.

[1859] Step 4: Generate analysis results

[1860] The server then formats the analysis results based on the results obtained from the AI ​​model into a visually easy-to-understand format, such as heat maps, graphs, and text data.

[1861] The server compiles these analysis results into packets for efficient distribution over the network.

[1862] Step 5: Delivering results

[1863] The server transmits the analysis results to the user terminal in real time, using the optimal data transmission protocol to minimize communication delays.

[1864] Emotion engine processing

[1865] Step 1: Obtaining emotion data

[1866] The emotion engine in the server acquires emotion data (face recognition data, voice data, biometric data) sent from the viewer terminal.

[1867] Step 2: Determine your emotional state

[1868] The emotion engine analyzes the acquired emotion data to determine the viewer's emotional state, for example, whether the viewer is excited, surprised, or calm.

[1869] Step 3: Customize what's displayed

[1870] The server customizes the display format and content of the analysis results based on the viewer's emotional state as determined by the emotion engine. For example, if the viewer is excited, it will highlight the prediction of the next play, but if the viewer is calm, it will display detailed data analysis results.

[1871] Terminal side processing

[1872] Step 1: Receiving Data

[1873] The terminal receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially.

[1874] Step 2: View the data

[1875] The device analyzes the received data and converts it into a display format, such as coordinate data and probability values ​​to generate a heat map.

[1876] The device displays the analysis results on the screen in real time, allowing viewers to check the results in parallel with the progress of the match.

[1877] User processing

[1878] Step 1: Verify the information

[1879] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing them to enjoy a more personalized viewing experience.

[1880] Example 2

[1881] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1882] Conventional match viewing systems make it difficult for viewers to visually grasp player behavior and score probability predictions during a match, resulting in a lack of real-time information provision for viewers. Furthermore, they lack the ability to customize information based on each viewer's emotional state, preventing a personalized viewing experience. This makes it difficult for viewers to deeply understand and enjoy the flow and tactics of the match.

[1883] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1884] In this invention, the server includes means for acquiring video data and sensor data of a match, means for preprocessing the acquired data, artificial intelligence model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to viewer terminals in real time, means for acquiring viewer emotional data and analyzing the viewer's emotional state, and means for customizing the display of the prediction results based on the viewer's emotional state. This allows viewers to grasp the analysis results of the match in real time, and the provision of personalized information allows them to gain a deeper understanding of the flow and tactics of the match, making it even more enjoyable.

[1885] "Game video data" refers to real-time video information captured by cameras installed at the game venue, and includes data such as the positions and movements of players and the trajectory of the ball.

[1886] "Sensor data" refers to data such as location information, speed, and biometric information obtained from wearable devices attached to athletes.

[1887] "Preprocessing" refers to processing of collected data, such as noise removal, data normalization, and missing value completion, to improve the accuracy of analysis.

[1888] An "artificial intelligence model" is a system that uses algorithms such as neural networks that have been trained in advance on large amounts of match data to predict a player's next move and the probability of scoring.

[1889] A "visually easy-to-understand format" means converting the analysis results into a format such as a heat map, graph, or text data, making it easy for viewers to understand.

[1890] "Means for delivering to viewer devices in real time" refers to communication protocols and network technologies for instantly transmitting analysis results to viewer devices.

[1891] "Emotion data" is data for analyzing the viewer's emotional state based on the viewer's facial recognition data, voice data, biometric data, etc.

[1892] "Emotional state" refers to the psychological state of the viewer, such as excitement, surprise, or calmness.

[1893] "Means for customization" refers to a mechanism that adjusts the content and format of the analysis results displayed according to the viewer's emotional state, providing optimal information to each individual viewer.

[1894] MODE FOR CARRYING OUT THE INVENTION

[1895] This invention relates to a system that analyzes the actions of players during a match and provides the analysis results to viewers in real time. It also combines a function to recognize viewers' emotions and customize the display content. Specific embodiments of this system are described below.

[1896] Overall system configuration

[1897] This system consists of a server-side component, a viewer device-side component, an emotion engine, and a user interface. The processing performed on the server side is mainly divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, a process is added to recognize viewer emotions and reflect that information in the display of analysis results.

[1898] Server-side processing

[1899] The server collects data in real time from cameras installed at the venue and sensors worn by players. The video data includes information on the players' positions and the ball's trajectory during the game, while the sensor data records the players' movement speed and heart rate. The collected data is preprocessed to remove noise, normalize the data, and fill in missing values. This preprocessing improves the accuracy of the analysis.

[1900] The preprocessed data is input into an AI model. This AI model uses algorithms such as neural networks that have been trained on large amounts of match data in advance to predict players' next moves and the probability of scoring. Based on these predictions, the server converts the analysis results into a visually understandable format. Specifically, heat maps, graphs, text data, etc. are generated and packaged as data packets. The analysis results are then distributed to viewer devices in real time. An efficient data transmission protocol (e.g., WebSocket) is used to minimize communication latency.

[1901] Emotion engine processing

[1902] The emotion engine acquires the viewer's facial recognition data, voice data, and biometric data, which allows it to analyze the viewer's emotional state in real time. Based on the acquired data, the server determines the viewer's emotional state. For example, it determines whether the viewer is excited, surprised, or calm. Based on this emotional state, the server customizes the display format and content of the analysis results. For example, if the viewer is excited, it highlights a prediction of the next play, and if the viewer is calm, it displays detailed data analysis results.

[1903] Terminal side processing

[1904] The device receives the analysis result data sent from the server. The received data is sent in packet format, so it is processed sequentially. The device analyzes the received data and displays it in a visually easy-to-understand format. Users can check the analysis results in real time along with the game footage. In addition, the emotion engine provides information appropriate to the user's emotional state, allowing for a more personalized viewing experience.

[1905] User processing

[1906] While watching the game, users can check the analysis results displayed on their device. This allows them to understand predictions for the next play and the probability of scoring, allowing them to enjoy the game. In addition, the emotion engine provides personalized information, allowing for an optimal viewing experience tailored to the viewer's interests and emotional state.

[1907] Specific examples

[1908] For example, when watching a soccer match, the server collects player positions and ball movement in real time from cameras and GPS sensors. The server preprocesses the data and inputs it into an AI model to calculate action predictions and goal probabilities. As a result, a heat map is generated showing the probability of forward players running toward the goal and the positions where they are likely to score a shot. The server sends this information to the viewer's device, which then displays it visually. The emotion engine analyzes the viewer's facial expressions, voice, and biometric information, and if it determines that the viewer is excited, it highlights scenes with a high probability of scoring. By checking the analysis results in real time while watching the match, users can gain a deeper understanding of the flow of the game and enjoy it more.

[1909] Examples of prompt statements

[1910] "Please explain in natural language how an AI model can accurately predict the probability of a forward player running towards goal and the success rate of a shot in a soccer match. Also, explain how the model can customize the display of the prediction results in real time depending on whether the viewer is excited or calm."

[1911] This system allows viewers to understand the tactics and movements of players in real time, enabling them to enjoy watching sports from a new perspective.In addition, the emotional engine provides personalized information, providing an optimal viewing experience tailored to the viewer's interests and emotional state.

[1912] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1913] Specific processing flow of the program

[1914] Step 1: Data collection

[1915] The server collects data in real time from cameras installed at the venue and sensors worn by players.

[1916] Input: Camera video data, sensor data (location, speed, biometric information)

[1917] How it works: Video data includes the positional information of players and the trajectory of the ball during the match, and sensor data such as the player's movement speed and heart rate are recorded in real time. This data is sent to a server in streaming format.

[1918] Output: Collected raw data (video data, sensor data)

[1919] Step 2: Data Preprocessing

[1920] The server performs noise removal, data normalization, and missing value completion on the collected data.

[1921] Input: Raw data collected

[1922] Specific operations: Use a noise reduction algorithm to properly remove high frequency noise, standardize each data point, and apply a Kalman filter or similar to fill in missing data.

[1923] Output: Preprocessed data

[1924] Step 3: Analysis by AI model

[1925] Based on the pre-processed data, the server uses an artificial intelligence model to predict the player's next move and the probability of scoring.

[1926] Input: Preprocessed data

[1927] How it works: Preprocessed data is fed into a neural network (e.g., LSTM or CNN) to predict player movements and shot success rates. The AI ​​model makes highly accurate predictions based on pre-trained patterns.

[1928] Output: Predicted result (next move, score probability)

[1929] Step 4: Generate analysis results

[1930] The server converts the analysis results into a visually easy-to-understand format based on the output of the AI ​​model.

[1931] Input: AI model prediction results

[1932] Specific operations: Generate a heat map, represent the score probability as a bar graph, and add an explanation as text data. Organize the result data into a packet format.

[1933] Output: Visually formatted analysis results (heatmaps, graphs, text data)

[1934] Step 5: Delivering results

[1935] The server delivers the analysis results to the viewer's device in real time.

[1936] Input: Visually formatted analysis results

[1937] Specific operation: Data packets are sent to the viewer device using an efficient data transmission protocol (e.g., WebSocket).

[1938] Output: Analysis result data delivered to the viewer's device

[1939] Step 6: Obtaining emotion data

[1940] The server's emotion engine collects viewers' facial recognition data, voice data, and biometric information.

[1941] Input: Facial recognition data, voice data, biometric data

[1942] Specific operation: Collect data from cameras, microphones, and wearable devices, and analyze the emotional state using facial expression recognition algorithms (e.g., OpenCV).

[1943] Output: Emotion data

[1944] Step 7: Determine your emotional state

[1945] The server's emotion engine determines the viewer's emotional state based on the acquired data.

[1946] Input: Emotion data

[1947] Specific operations: Performs voice tone analysis and heart rate variability analysis to cross-check and determine the viewer's emotional state, such as excitement, surprise, or calmness.

[1948] Output: Determined emotional state

[1949] Step 8: Customize what's displayed

[1950] The server customizes the display of the analysis results based on the viewer's emotional state.

[1951] Input: Determined emotional state, analysis result data delivered to the viewer's device

[1952] Specific behavior: When viewers are excited, the layout and font size are adaptively changed to highlight highlights and next play predictions, and when viewers are calm, to display detailed data analysis results.

[1953] Output: Customized display content

[1954] Step 9: Receiving Data

[1955] The terminal receives the analysis result data sent from the server.

[1956] Input: Data packet sent by the server

[1957] What it does: Receives data packets over a WebSocket connection and stores them in a local memory buffer.

[1958] Output: Analysis result data stored in memory buffer

[1959] Step 10: Displaying the data

[1960] The device analyzes the received data and displays it in a visually easy-to-understand format.

[1961] Input: Analysis result data stored in memory buffer

[1962] What it does: Render heatmaps, graphs, and text data in real time and integrate them into the user interface, providing customized displays based on the results of the sentiment engine.

[1963] Output: Analysis results displayed in the user interface

[1964] (Application example 2)

[1965] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1966] Conventional match analysis systems are limited to predicting player behavior and scoring probability during a match, and are not customized to reflect the viewer's emotional state. This makes it difficult to provide information tailored to each viewer's interests and emotions, resulting in a lack of personalized viewing experiences. To address this issue, a system is needed that can recognize viewer emotions in real time and customize the display format and content of analysis results based on those emotions.

[1967] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring video data and sensor data of the game, means for preprocessing the acquired data, AI model means for predicting players' next moves and patterns with high scoring probabilities using the preprocessed data, means for converting the prediction results into a visually easy-to-understand format, means for delivering the prediction results to the viewer terminal in real time, means for acquiring face recognition data, voice data, and biometric information data of the viewer, means for determining the viewer's emotional state using the acquired data, and means for customizing the display format and content of the prediction results according to the viewer's emotional state. This makes it possible to provide analysis results suited to the viewer's emotional state, thereby realizing a personalized viewing experience.

[1968] "Game video data" refers to digital video data captured during a game, including the positional information of players and the ball, their movements, and the flow of the game.

[1969] "Sensor data" refers to biometric and environmental data such as location, speed, and heart rate obtained from sensors attached to athletes.

[1970] "Preprocessing" refers to the process of converting acquired data, such as by removing noise, normalizing, and filling in missing values, to prepare the data for analysis.

[1971] An "AI model" refers to artificial intelligence technology that uses pre-trained algorithms to predict player movements and scoring probabilities from input data.

[1972] "Visually easy-to-understand format" refers to formats such as heat maps, graphs, and text data that present analysis results in a way that is easy for users to understand.

[1973] "Real-time" refers to the delivery or processing of data nearly simultaneously with minimal delay.

[1974] A "viewer terminal" is a device used by a viewer to receive data and check the analysis results, and includes smartphones, head-mounted displays, etc.

[1975] "Facial Recognition Data" refers to digital image data used to analyze a viewer's facial expressions.

[1976] "Audio Data" means digital audio data that captures the voice of a viewer and is used for audio analysis.

[1977] "Biometric data" refers to personal physiological data obtained from biometric sensors, such as a viewer's heart rate or skin temperature.

[1978] "Emotional state" refers to the psychological state, such as excitement, calmness, or surprise, that viewers express in real time.

[1979] "Customization" refers to adapting the content and format of a presentation based on the emotional state of each viewer.

[1980] This system analyzes the behavior of players during a match, provides the results to viewers in real time, and customizes the results according to the viewer's emotional state. This system includes the collection, preprocessing, and analysis of match data, the generation and distribution of results, viewer emotion recognition, and the customization of the display content.

[1981] Overall system configuration

[1982] It consists of a server, viewer terminal, emotion engine, and user interface. The server-side processing is divided into the steps of data collection, preprocessing, analysis, result generation, and result distribution. In addition, it recognizes viewer emotions and reflects this information in the display of analysis results.

[1983] Server-side processing

[1984] 1. Data Collection

[1985] The server collects video and sensor data from the match in real time. Specifically, it acquires data from cameras installed at the match venue and sensors worn by players. The hardware used includes cameras (e.g., Logitech C920) and heart rate monitors.

[1986] 2. Data Preprocessing

[1987] The server preprocesses the collected data. This preprocessing includes noise removal, data normalization, and missing value imputation, making the data ready for analysis. Software libraries used include NumPy and Pandas.

[1988] 3. Analysis using AI models

[1989] The server inputs the preprocessed data into an AI model to predict the player's next move and the probability of scoring. This AI model is trained in advance using a large amount of match data and is able to predict moves and patterns with high accuracy. Frameworks used include TensorFlow and PyTorch.

[1990] 4. Generating analysis results

[1991] Based on the predictions obtained from the AI ​​model, the server formats the analysis results into a visually easy-to-understand format. Specifically, heat maps, graphs, text data, etc. are generated and compiled into data packets. The software used is a JavaScript library.

[1992] 5. Distribution of Results

[1993] The server delivers the analysis results to the viewer's device in real time, using an efficient data transmission protocol to minimize communication delays.

[1994] Emotion engine processing

[1995] 1. Acquiring Emotion Data

[1996] The emotion engine captures viewers' facial recognition data, voice data, and biometric data, and analyzes their emotional state in real time. The hardware used includes cameras, microphones, and biometric sensors.

[1997] 2. Emotional state determination

[1998] The emotion engine analyzes the acquired data and determines the viewer's emotional state. The software used is OpenCV (face recognition) and DeepFace (emotion recognition).

[1999] 3. Customizing the display content

[2000] Based on the viewer's emotional state as determined by the emotion engine, the server customizes the display format and content of the analysis results. For example, if the viewer is excit...

Claims

1. To analyze the players' behavior during the game, A means for acquiring video data and sensor data of a match; means for pre-processing the acquired data; An AI model means for predicting the next movement of a player and a pattern with a high probability of scoring using pre-processed data; A means of converting the prediction results into a visually understandable format; A means for delivering the prediction results to the viewer terminal in real time; A system including:

2. 2. The system according to claim 1, wherein position information of players and a ball is extracted from video data of a match.

3. The system of claim 1 , wherein the system pre-processes position information, speed, and heart rate information obtained from sensors worn by the athlete.

Images