System

Abstract

A system is provided.SOLUTION: A system including means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting feature amounts based on the data and training a machine learning model, means for executing a simulation in which teams of different eras virtually compete against each other, means for visualizing a simulation result, means for distributing the visualized result to a virtual environment, and means for allowing a user to watch a game in the virtual environment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional sports viewing systems are unable to simulate matches between teams or players from different eras, making it difficult for fans to enjoy their dream matches. Furthermore, the low accuracy of simulations and data analysis poses challenges to the reliability of match results. Furthermore, the visual experience of the match is limited, making it difficult to enhance the user's sense of immersion. There is a need for a system that can solve these issues and accurately simulate matches between teams and players from different eras, allowing for visual enjoyment. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A system is constructed that includes means for acquiring game data, means for collecting player and team performance data, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the results of the simulation, means for delivering the visualized results to a virtual environment, and means for users to watch the game in the virtual environment. This allows sports fans to watch their dream match as a highly accurate simulated game and enjoy a realistic visual experience.

[0006] "Match data" refers to data that includes information such as scores from past sports matches, individual player performances, and team tactics.

[0007] "Performance data" refers to data that indicates the actions and achievements of individual players or teams during a game, such as a player's shooting percentage, number of assists, number of rebounds, and turnovers.

[0008] "Data shaping" is the process of converting collected data into a consistent format that makes it easier to compare and analyze.

[0009] "Cleansing" is a data purification process that removes outliers and duplicate data from collected data to improve accuracy.

[0010] "Features" are important attributes or characteristic values ​​of data that are extracted to train machine learning models.

[0011] A "machine learning model" is a type of algorithm that is trained to make predictions or classifications based on data.

[0012] "Simulation" is the process of running a virtual match on a computer to replicate a real match.

[0013] "Visualization" is the process of displaying simulation results in visual form, such as graphs or animations.

[0014] A "virtual environment" is a computer-generated interactive environment that a user can experience through virtual reality (VR) or augmented reality (AR).

[0015] "Match setting" is setting information that specifies the combination of teams and players that will compete in the simulation.

[0016] "Feature extraction" is the process of extracting important information or attributes from data in data analysis.

[0017] "Regression" is a type of machine learning algorithm for predicting continuous values.

[0018] "Classification" is a type of machine learning algorithm for predicting category labels.

[0019] A "neural network" is an algorithm that mimics the neural circuits of living organisms, and is a machine learning model used to analyze and predict complex data.

[0020] "User experience" refers to the series of interactive processes that a user experiences when using a system.

[0021] An "avatar" is a digital character that represents a user within a virtual environment. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] This invention relates to a system that uses big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually within the metaverse. This system consists of a server, terminals, and users.

[0044] System Overview

[0045] Data Collection Component (Server)

[0046] The server retrieves game data from a large database containing data from past sports games, such as game scores and player performance data from past official databases, including each player's shooting percentage, assists, rebounds, and defensive skills.

[0047] Data preprocessing and cleansing (server)

[0048] The server processes the collected data, converting it into a consistent format, and also performs data cleansing to remove outliers and duplicate data to improve data quality.

[0049] Feature extraction and model training (server)

[0050] The server extracts features for each player and team from the preprocessed data and trains a machine learning model using a combination of machine learning algorithms such as regression, classification, and neural networks.

[0051] Running the simulation (server)

[0052] The server simulates a match between teams from different eras specified by the user, such as a match between a famous team A from the 1990s and a famous team B from the 2010s. The simulation is run multiple times to obtain statistically significant results.

[0053] Result visualization and metaverse distribution (server)

[0054] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are distributed to the Metaverse platform, where users can view them within the virtual environment.

[0055] Metaverse experience (device, user)

[0056] Users access the metaverse using a device (e.g., a VR headset or PC) and watch the game in a virtual stadium. They can move around the stadium using their avatar, switch the perspective of the game, and focus on specific players. They can also watch replays and highlights of the game, providing a highly immersive experience.

[0057] Specific examples

[0058] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, calculating the probability of winning or losing and the contribution of each player. The results are distributed to the metaverse platform, allowing users to watch the matches in real time within a virtual environment.

[0059] In this way, the system of the present invention simulates matches between historically important sports teams and players, providing a new, visually enjoyable entertainment experience.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] Start the data collection component (server)

[0063] The server retrieves historical game data from a sports game database, including the score for each game and player performance metrics (shooting percentage, assists, rebounds, turnovers, etc.), and stores this data in a local database.

[0064] Step 2:

[0065] Data preprocessing and cleansing (server)

[0066] The server then formats the collected match data into a consistent format, sorting it by date and time, detecting and correcting or removing outliers and missing data, and detecting and removing duplicate entries within the data.

[0067] Step 3:

[0068] Feature extraction (server)

[0069] The server extracts features from the cleansed data. This is the step where important data points that indicate the characteristics of each player or team are extracted. For example, a player's shooting percentage, defensive strength, or a team's offensive tactics are extracted as features.

[0070] Step 4:

[0071] Machine learning model training (server)

[0072] The server uses the extracted features to train a machine learning model, using algorithms such as regression, classification, and neural networks. To train the model, cross-validation is performed using past match data to find the optimal parameters.

[0073] Step 5:

[0074] Enter match settings (user)

[0075] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0076] Step 6:

[0077] Running the simulation (server)

[0078] The server runs virtual matches based on the match settings specified by the user. These matches are run as multiple simulations, and the results and statistics for each match are generated. For example, Team A from 1996 plays Team B from 2017, and the results and player performances for each simulation are recorded.

[0079] Step 7:

[0080] Aggregation and analysis of simulation results (server)

[0081] The server aggregates all simulation results, calculates the probabilities of win / loss outcomes and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0082] Step 8:

[0083] Visualization of results (server)

[0084] The server visualizes the aggregated results of the simulation as graphs and animations, including the progress of the match and key plays, and sends this visualization data to the Metaverse platform.

[0085] Step 9:

[0086] Distribution and viewing to the metaverse (devices, users)

[0087] Users access the metaverse through their devices and watch simulated matches in a virtual stadium. They can control their avatars to move around the stadium and freely switch the viewpoint of the match. They can also watch replays and highlights of the match, enriching the viewing experience.

[0088] Through the above steps, the system of the present invention can simulate competitions between sports teams and players from different eras, providing users with a visually immersive viewing experience.

[0089] Example 1

[0090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0091] It is difficult to accurately compare and simulate the performance of sports teams and players from different eras, and there is a need for a way to visually enjoy this. Conventional systems do not fully integrate the technologies required to collect, format, and simulate past data, preventing users from visually enjoying the experience in an immersive virtual environment. This necessitates the development of a new entertainment system that can visualize the match results of teams and players from different eras with high accuracy and in real time.

[0092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0093] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping the collected data and converting it into a consistent format, means for cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the results of the simulation, means for delivering the visualized results to a virtual environment, and means for users to watch the game in the virtual environment using a terminal. This makes it possible to simulate matches between sports teams and players from different eras with high accuracy and enjoy them visually.

[0094] "Match data" refers to scores related to past sports matches and various statistical information recorded during those matches.

[0095] "Performance data" refers to data that quantifies the specific abilities and achievements of a player or team, such as shooting percentage, number of assists, and number of rebounds.

[0096] "Data transformation" refers to the process of converting collected data into a consistent format. For example, converting data stored in different formats into a unified format (such as CSV).

[0097] "Data cleansing" refers to the process of removing outliers and duplicates from data and improving data quality.

[0098] "Feature extraction" refers to the process of extracting important data points (features) needed to train a machine learning model, such as a player's average points scored or rebound percentage.

[0099] A "machine learning model" is a set of algorithms that learn patterns from data and make predictions or classifications. The model is trained on collected data.

[0100] "Simulation" refers to the process of virtually pitting teams and players from different eras against each other, recreating these matches in a digital environment and running them multiple times to ensure statistically significant results.

[0101] "Visualization" refers to the display of simulation results in a user-friendly format, including graphs and animations.

[0102] A "virtual environment" refers to a digital space where users can have an immersive experience, in which they can control an avatar and move freely.

[0103] "Device" refers to the device through which a user accesses a virtual environment, including a VR headset or PC.

[0104] This invention is a system that utilizes big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually in a virtual environment (metaverse). Components include a server, terminals, and users.

[0105] Data Collection Component (Server)

[0106] The server has a means to retrieve past sports game data. This can be done using official database APIs or open data. For example, the server can use the official basketball website API to retrieve detailed data such as game scores, players' shooting percentages, assists, and rebounds. The collected data is then stored in a consistent format in a NoSQL database (e.g., MongoDB).

[0107] Data preprocessing and cleansing (server)

[0108] The server has the means to format the collected data and convert it into a consistent format. For example, it can standardize the collected data into CSV files or JSON format. It also uses data cleansing methods to remove outliers and duplicates and improve data quality. A specific example is imputing missing values ​​using the Python Pandas library.

[0109] Feature extraction and model training (server)

[0110] The server has a means to extract player and team features from the preprocessed data. Specific software used is Python's Pandas and NumPy. The extracted features (e.g., average shooting percentage, rebound rate) are used to train machine learning models. Here, the server uses Scikit-learn and TENSORFLOW (registered trademark) to train regression models and neural network models.

[0111] Running the simulation (server)

[0112] The server has a means to simulate teams from different eras specified by the user. The user inputs a prompt to the generated AI model to specify the teams to play against. For example, the prompt might say, "Please simulate a match between Team A from 1996 and Team B from 2017." The server runs the simulation multiple times and obtains statistically significant results.

[0113] Result visualization and metaverse distribution (server)

[0114] The server has a means to visualize the simulation results, using libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations, and distributes the visualized results to the Metaverse platform, where users can access them via VR headsets or PCs.

[0115] Metaverse experience (device, user)

[0116] Users access the Metaverse platform using a device (e.g., an Oculus VR headset) and watch the game in a virtual stadium. They can control an avatar, change the perspective of the game, and focus on specific players. For example, users can view replays of a specific player's shots in detail. This provides a highly immersive experience.

[0117] As described above, the system of the present invention analyzes past sports game data with high accuracy and simulates matches between teams and players from different eras, providing a new, visually enjoyable entertainment experience.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] The server retrieves past sports game data using official database APIs and open data. Specifically, it retrieves detailed data such as game scores, players' shooting percentages, number of assists, and number of rebounds from the official basketball website API. The input at this stage is an API request, and the output is response data in JSON or XML format containing the game data.

[0121] Step 2:

[0122] The server formats the received match data into a consistent format. Specifically, it converts JSON-formatted data into CSV format. This process uses the Python Pandas library. The input is JSON-formatted match data, and the output is formatted CSV data.

[0123] Step 3:

[0124] The server cleanses the formatted match data. Specific operations include filling in missing values, detecting and deleting outliers, and removing duplicate data. The goal of this stage is to obtain high-quality data after cleansing. The input is formatted CSV data, and the output is cleansed CSV data.

[0125] Step 4:

[0126] The server extracts features from the preprocessed data. The specific operation here is to calculate statistical data for each player and team (e.g., average shooting percentage, rebound percentage). At this stage, Python's NumPy and Pandas libraries are used. The input is the cleansed CSV data, and the output is a new dataset containing the features.

[0127] Step 5:

[0128] The server trains a machine learning model based on the extracted features. Specifically, it uses Scikit-learn or TensorFlow to build a regression model or neural network model and trains it with the training data. At this stage, the input is the feature dataset, and the output is a trained machine learning model.

[0129] Step 6:

[0130] The server runs a simulation in which teams from different eras specified by the user play against each other. The user provides a prompt, which the server analyzes and runs the simulation. At this stage, the model is used to run multiple simulations to obtain statistically significant results. The input is the user's prompt and the trained model, and the output is aggregated data from each simulation.

[0131] Step 7:

[0132] The server visualizes the simulation results. Specifically, it uses libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations. The input at this stage is the aggregated data of the simulation results, and the output is the visualized result data.

[0133] Step 8:

[0134] The server delivers the visualized results to the Metaverse platform. Users receive this data using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input here is the visualized results data, and the output is the display on the Metaverse platform.

[0135] Step 9:

[0136] Users access the metaverse platform using their devices and watch the game in a virtual stadium. They can control an avatar and freely switch the viewpoint of the game. Specific actions include moving the avatar, focusing on a specific player, and using the replay function. The input is the user's actions, and the output is the game-watching experience in the virtual environment.

[0137] (Application example 1)

[0138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0139] In modern sports entertainment, there is a growing demand for virtual simulations in which teams and players from different eras compete against each other. However, conventional simulation technologies have struggled to efficiently and accurately analyze past data, visualize the results in real time, and provide an immersive environment for users. In particular, there has been a lack of technology that allows users to switch between different viewing viewpoints within a virtual environment, making it difficult to provide a truly immersive experience. The present invention addresses these challenges.

[0140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0141] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual reality environment, and means for a user to watch the game in a virtual stadium, switch viewpoints using avatars, and focus on specific players. This allows users to not only experience a highly accurate simulation of sports teams and players from different eras competing, but also enables a deeper, immersive entertainment experience by visually watching the game in real time and freely switching viewpoints.

[0142] "Match data" refers to information about past sports matches, such as scores, player data, match dates and times, and venues.

[0143] "Player Data" refers to detailed data about each player's performance, including shooting percentage, number of assists, number of rebounds, defensive skills, etc.

[0144] "Team data" refers to data about the performance and movements of the entire team, including information about team strategy, playing style, and member composition.

[0145] "Shaping and cleansing" refers to the process of converting data into a consistent format and removing outliers and duplicate data.

[0146] "Feature extraction" refers to the process of selecting important variables and parameters from a dataset for use in training a predictive model.

[0147] "Training a machine learning model" refers to the process of using collected data to build a predictive model through algorithms such as regression, classification, and neural networks.

[0148] "Simulation" refers to the process of virtually pitting teams from different eras against each other and simulating the results multiple times.

[0149] "Visualization" refers to the display of simulation results in the form of graphs, animations, virtual reality, etc.

[0150] "Delivery to a virtual reality environment" refers to delivering visualized simulation results to a virtual space accessible to the user.

[0151] A "virtual stadium" is a digital space where users can watch a game in a virtual reality environment, and refers to a virtual space shaped like a stadium.

[0152] "Avatar" refers to a digital character that represents a user within a virtual reality environment.

[0153] "Switching viewpoints" refers to the user freely changing between different camera angles and viewing positions within the virtual environment.

[0154] "Focusing on a specific player" refers to a user focusing on a specific player in the virtual environment and observing their movements in detail.

[0155] This invention relates to a system that uses big data analysis and generative AI models to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy the game visually in a virtual reality environment. This system consists of a server, a terminal, and a user element.

[0156] Data Collection Component (Server)

[0157] The server retrieves game data from a database of past sports games, including details on each player's shooting percentage, assists, rebounds, defensive skills, etc. This data is collected from official databases and reliable public data sources.

[0158] Data preprocessing and cleansing (server)

[0159] The server formats the collected data, converting it into a consistent format, and performs data cleansing to remove outliers and duplicates to improve data quality, using common data analysis tools such as Python, Pandas, and NumPy.

[0160] Feature extraction and model training (server)

[0161] The server extracts features of each player and team from the preprocessed data and trains a machine learning model using a combination of algorithms such as regression, classification, and neural networks. Machine learning frameworks such as Scikit-learn and TensorFlow are used.

[0162] Running the simulation (server)

[0163] The server virtually runs matches between teams from different eras specified by the user. This simulation is run multiple times to obtain statistically significant results. The accuracy of the simulation results is improved by using a generative AI model.

[0164] Visualization of results and distribution to virtual reality environments (server, terminal)

[0165] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are delivered to a virtual reality environment where users can view them using a VR headset or smartphone.

[0166] Virtual stadium experience (device, user)

[0167] Users access the virtual reality environment using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can move around the stadium using their own avatar, freely switch the viewpoint of the game, and focus on specific players. This provides a highly immersive experience.

[0168] Specific examples

[0169] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, and the probability of winning or losing and the contribution of each player are calculated. The results are then streamed to a virtual reality environment, allowing users to watch the matches in real time.

[0170] Prompt Sentence Examples

[0171] "We have data from the 1990 Basketball Finals and data from the 2010 Basketball Finals. Using the shooting percentages, assists, rebounds, defensive skills, etc. from each data set, you can simulate matches between teams from different eras and recreate the games in a virtual environment."

[0172] This configuration allows users to not only experience highly accurate simulations of sports teams and players from different eras competing, but also to visually watch the game in real time and freely switch viewpoints, providing a deeper, more immersive entertainment experience.

[0173] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0174] Step 1:

[0175] The server retrieves past match data. It uses data such as match scores, player data, match dates and times, and venues obtained from official databases or trusted data sources as input. The output is match data obtained in raw data format. Specific operations include issuing database queries and downloading match data.

[0176] Step 2:

[0177] The server collects player and team performance data. As input, it uses detailed performance data such as shooting percentage, assists, rebounds, and defensive skills obtained from the official database. The output is the collected player and team performance data. Specifically, it retrieves the data through the API and stores it in an internal database.

[0178] Step 3:

[0179] The server formats and cleanses the collected data. It uses raw match and performance data as input. The output is clean data formatted in a consistent format. Specifically, it uses the Python Pandas library to format the data and remove outliers and duplicates.

[0180] Step 4:

[0181] The server extracts features from the formatted data and trains a machine learning model. Using the formatted data as input, features are extracted using algorithms such as regression, classification, and neural networks. The output is a trained machine learning model. Specifically, the machine learning model is trained using Scikit-learn and TensorFlow.

[0182] Step 5:

[0183] The server runs a simulation in which teams from different eras virtually compete against each other. It uses a trained machine learning model and data on the competing teams as input. The output is the simulation results. Specifically, it uses a generative AI model to run multiple simulations and obtain statistically significant results.

[0184] Step 6:

[0185] The server visualizes the simulation results. It uses the simulation results as input and visualizes the match highlights and detailed reports in graphs and animations as output. Specifically, it uses Matplotlib and Plotly for visualization.

[0186] Step 7:

[0187] The server distributes the visualized results to the virtual reality environment. It uses the visualized simulation results as input and generates data that is sent to the virtual reality environment as output. Specifically, it converts the data into a format suitable for VR headsets and smartphones and distributes it over the network.

[0188] Step 8:

[0189] Users use their devices to watch the game in a virtual stadium. The input is the simulation results delivered to the virtual reality environment, and the output is the user's viewing experience. Specific actions include operating a VR headset or avatar to switch viewpoints or focus on specific players while watching the game.

[0190] Through these steps, users can not only experience highly accurate simulations of sports teams and players from different eras competing, but also visually watch the game in real time and freely switch perspectives, providing a deeper, immersive entertainment experience.

[0191] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0192] This invention combines a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0193] System Overview

[0194] 1. Data collection component (server)

[0195] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0196] 2. Data preprocessing and cleansing (server)

[0197] The server formats the collected data consistently, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality.

[0198] 3. Feature extraction and model training (server)

[0199] The server extracts features from the organized data and uses them to train machine learning models, using algorithms such as regression, classification, and neural networks.

[0200] 4. Running the simulation (server)

[0201] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras and generating results and statistics for each match.

[0202] 5. Emotion engine operation (server, terminal)

[0203] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone and extracts emotional indicators.

[0204] 6. Visualization of results and virtual environment distribution (server)

[0205] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0206] 7. Providing a Metaverse experience (devices, users)

[0207] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0208] Specific examples

[0209] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This simulates numerous virtual matches between Team A and Team B, and calculates the probability of winning or losing and the contribution of each player.

[0210] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0211] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0212] The processing flow will be explained below.

[0213] Step 1:

[0214] Start the data collection component (server)

[0215] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (e.g., shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0216] Step 2:

[0217] Data preprocessing and cleansing (server)

[0218] The server then formats the collected data into a consistent format, detecting and correcting or removing outliers and missing data, as well as detecting and removing duplicate entries within the data.

[0219] Step 3:

[0220] Feature extraction (server)

[0221] The server extracts features from the cleansed data, such as a player's shooting percentage, defensive strength, and team offensive tactics.

[0222] Step 4:

[0223] Machine learning model training (server)

[0224] The server uses the extracted features to train a machine learning model using algorithms such as regression, classification, and neural networks. The model is trained by cross-validating the data to find the optimal parameters.

[0225] Step 5:

[0226] Enter match settings (user)

[0227] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0228] Step 6:

[0229] Running the simulation (server)

[0230] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras, and generates results and statistics for each match.

[0231] Step 7:

[0232] Aggregation and analysis of simulation results (server)

[0233] The server aggregates all simulation results, calculates the probability of winning or losing, and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0234] Step 8:

[0235] Data acquisition by emotion engine (terminal)

[0236] The device collects emotional data by sensing the user's facial expressions, tone of voice, body movements, etc. This is done using the device's built-in camera and microphone.

[0237] Step 9:

[0238] Emotion data analysis (server)

[0239] The server analyzes the user's emotional data sent from the terminal, for example, to identify the user's emotional state, such as excitement, joy, or surprise.

[0240] Step 10:

[0241] Visualization of results and dynamic adjustment of user experience (server)

[0242] The server visualizes the simulation results based on the collected emotional data and dynamically adjusts the display content. For example, if the user is surprised, the server will highlight that scene as a replay. It will also display more scenes from the perspective of the player who excited the user.

[0243] Step 11:

[0244] Distribution and viewing to the metaverse (devices, users)

[0245] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely switch the viewpoint of the game. The system also dynamically adjusts the viewpoint and display based on emotional data, providing users with a richer, more immersive experience.

[0246] Through the above steps, the system of the present invention can simulate matches between historical sports teams and players, providing users with a visually and emotionally rich viewing experience.

[0247] Example 2

[0248] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] In modern times, simulations of matches between sports teams and players from different eras often lack accuracy. In addition, the use of emotional data to enrich the user's viewing experience is insufficient, resulting in a low sense of immersion and an inability to reflect real-time reactions.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0251] In this invention, the server includes means for acquiring game data, means for collecting player and team performance data, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for acquiring and analyzing user emotion data in real time, means for dynamically adjusting the visualization data based on the emotion data, and means for the user to watch the game in the virtual environment. This enables a more immersive viewing experience by simulating matches between sports teams and players from different eras with high accuracy and dynamically adjusting the visualization data of the match according to the user's emotional state.

[0252] "Means for obtaining game data" refers to means for obtaining information about past sports games from a database or API.

[0253] "Means for collecting performance data" are means for collecting information about specific plays and performance of players and teams.

[0254] "Forming and cleansing means" refers to means for converting acquired data into a consistent format and correcting or removing outliers and missing data.

[0255] "Means for extracting features and training machine learning models" refers to means for extracting important patterns and features from formatted data and using machine learning algorithms to train a model based on that data.

[0256] "Means for performing simulations" means means for virtually playing matches between sports teams and players from different eras using trained machine learning models.

[0257] A "visualization means" is a means for displaying the results of a simulation in a visual format such as a graph or animation.

[0258] "Means for delivery to a virtual environment" means means for transmitting the visualization results to the metaverse or other virtual reality platforms.

[0259] "Means for acquiring and analyzing emotional data in real time" refers to a means for collecting a user's facial expressions, voice, body movements, etc. using sensors, and analyzing this to identify their emotional state.

[0260] The "means for dynamically adjusting visualization data" refers to a means for optimizing visualization data in real time in accordance with the user's emotions based on the analyzed emotion data.

[0261] A "means for viewing a game" is a means by which a user can view and interact with a simulated game within a virtual environment.

[0262] This invention combines a system that can accurately simulate matches between sports teams and players from different eras and visually enjoy them in the metaverse with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0263] System Overview

[0264] Data Collection Component (Server)

[0265] The server retrieves game data from a database of past sports games. Specifically, it collects game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.) and stores them in a local database. For example, it manages this data using an SQL database.

[0266] Data preprocessing and cleansing (server)

[0267] The server cleans the collected data into a consistent format, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality. One example is data cleansing using the Python Pandas library.

[0268] Feature extraction and model training (server)

[0269] The server extracts important features from the formatted data and uses them to train a machine learning model using algorithms such as regression, classification, and neural networks. Specific examples include model training using Scikit-learn and TensorFlow.

[0270] Running the simulation (server)

[0271] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between teams from different eras and generates results and statistics for each match. Cloud computing services (e.g., AWS® Lambda) can be used.

[0272] Emotion engine operation (server, terminal)

[0273] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone to extract emotional indicators. One example is the use of facial expression recognition using OpenCV and Dlib, or an emotion analysis API.

[0274] Visualization of results and virtual environment distribution (server)

[0275] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments. This visualization data is sent to a metaverse platform (e.g., Unity) so that the user can view it in the virtual environment.

[0276] Providing a metaverse experience (devices, users)

[0277] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can control their avatar to move around the stadium and freely switch the viewpoint of the game. Based on emotion data, the system dynamically adjusts the viewpoint and the information displayed.

[0278] Specific examples

[0279] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects Team B's data from 2017 and runs it through a machine learning model to simulate a hypothetical match between Team A and Team B. The probability of winning or losing and the contribution of each player are calculated.

[0280] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0281] Example prompts for generative AI models

[0282] 1. "Simulate a hypothetical match between Team A from 1996 and Team B from 2017 and visualize the outcome of the match."

[0283] 2. "Based on the user's emotional data, please replay scenes that emphasize when the user is in an excited state."

[0284] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0285] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0286] Step 1: Data collection

[0287] The server accesses a database of past sports games to retrieve game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.). The input is the database address of a specific sports game database via an API or SQL query. The output is the retrieved game data. Specifically, for example, the server retrieves data from an endpoint such as "https: / / sportsdata.example.com / api / games / 1996" via a REST API and stores it in a local SQL database.

[0288] Step 2: Data preprocessing and cleansing

[0289] The server formats the data retrieved from the local database into a consistent format. The input is the retrieved match data. The output is the formatted and cleansed match data. The server uses the Python Pandas library to load the data into a data frame and perform missing value imputation and duplicate data removal. For example, the "fillna" method is used to impute missing values ​​and the "drop_duplicates" method is used to remove duplicate data.

[0290] Step 3: Feature extraction and model training

[0291] The server extracts important features from the formatted data and trains a machine learning model based on them. The input is the formatted and cleansed match data. The output is a trained machine learning model. The server extracts features using the Scikit-learn library and trains a regression model using, for example, the "LinearRegression" class. The data is split into training and test data, and the model is fitted.

[0292] Step 4: Run the simulation

[0293] The server runs a virtual match based on the match settings specified by the user. The input is the match settings specified by the user (for example, data on Team A and Team B). The output is statistical data on the simulation results. The server runs multiple simulations using the saved machine learning model and stores the results in a database. The statistical data on the simulation results is output as win / loss probabilities and player contributions.

[0294] Step 5: Acquire and analyze emotion data

[0295] The device senses the user's facial expressions, tone of voice, and body movements using a camera and microphone, and sends the data to a server. The input is the user's real-time emotional data (facial expressions, tone of voice, and body movements). The output is analyzed emotional indicators. The server analyzes this data using OpenCV and an emotion analysis API to extract emotional indicators.

[0296] Step 6: Visualizing the results and delivering the virtual environment

[0297] The server integrates the simulation results and the user's emotional data to generate visualized results. The input is the statistical data of the simulation results and the analyzed emotional index. The output is the visualized match results. The server sends this visualized data to a metaverse platform such as Unity. The data is distributed in real time so that users can view it within the metaverse.

[0298] Step 7: Deliver the Metaverse experience

[0299] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input is the visualized game results and real-time updated emotional data. The output is the user's immersive experience. Users can control their avatar to move freely around the stadium and switch the perspective of the game. Based on the emotional data, the system dynamically adjusts the perspective and the information displayed.

[0300] (Application example 2)

[0301] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0302] Conventional sports viewing experiences have struggled to reflect real-time emotional changes, limiting the dynamic visual effects and information provided based on user emotions. Furthermore, there has been a lack of technology to virtually simulate and visualize matches between sports teams and players from different eras with high accuracy. As a result, the user's sense of immersion and the quality of the viewing experience have declined, making it difficult to improve user engagement.

[0303] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0304] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for a user to watch the game in the virtual environment, means for analyzing user emotional data and dynamically adjusting the visualization results, and means for collecting user emotional data using a smart device and changing the viewpoint and display information in the virtual environment based on the results. This enables personalized visual effects and information to be provided based on the user's emotional state, resulting in a more immersive and richer viewing experience.

[0305] "Match data" refers to various data collected from past sports matches, such as scores, play records, and player performance indicators.

[0306] "Performance data" is data that quantitatively indicates the skills and results that players and teams demonstrate during a game, and includes shooting percentage, number of assists, number of rebounds, turnovers, etc.

[0307] "Forming and cleansing procedures" refers to the process of transforming collected data into a consistent format and correcting or removing outliers, missing data, and duplicate entries.

[0308] "Features" are important data points or metrics used to train machine learning models, allowing them to make more accurate predictions.

[0309] A "machine learning model" is a set of mathematical algorithms that learns from data, finds patterns, and makes subsequent predictions or decisions.

[0310] "Means for performing simulations" refers to the process of using a machine learning model to conduct a large number of virtual matches under set conditions and generate the results.

[0311] "Visualization means" refers to technology that graphically displays data obtained through simulation, allowing users to intuitively understand it.

[0312] "Means of delivery to a virtual environment" refers to a method of sending visualized simulation results to a virtual space such as a metaverse platform, allowing users who access the space to experience it.

[0313] "Means for analyzing emotional data" refers to technology that detects a user's emotional state in real time based on their facial expressions, tone of voice, and body movements.

[0314] A "smart device" is a device that can connect to the Internet and collect and process real-time data using sensors, cameras, microphones, etc., and includes smartphones and head-mounted displays.

[0315] The present invention is a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, and also combines it with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0316] 1. Data Collection and Preprocessing

[0317] The server retrieves past sports game data and collects player and team performance data. This data is then stored in a local database. The collected data is then formatted, cleansed, and converted into a consistent format. Outliers, missing data, and duplicate entries are also corrected or removed. Software used includes OpenCV and DeepFace.

[0318] 2. Feature extraction and model training

[0319] Features are extracted from the organized data and used to train machine learning models using algorithms such as regression, classification, and neural networks. This process is carried out using machine learning frameworks such as TensorFlow.

[0320] 3. Running and Visualizing the Simulation

[0321] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between organizations from different eras, generating results and statistical data for each match. These simulation results are then visualized and displayed in a way that is intuitively understandable to the user.

[0322] 4. Operation of the Emotion Engine

[0323] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the user's emotional state to be grasped in real time. This emotion engine analyzes input from the user's webcam and microphone and extracts emotional indicators. DeepFace is used for this processing. Smart devices such as smartphones and head-mounted displays (HMDs) are also used.

[0324] 5. Results visualization and virtual environment delivery

[0325] The simulation results are integrated with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0326] 6. Providing a Metaverse Experience

[0327] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0328] Examples of concrete examples and prompts

[0329] For example, to set up a virtual match between Team A in 1996 and Team B in 2017, the server retrieves all the match data from each team and analyzes the performance data of each player in detail. The system then inputs the following prompts into the generative AI model:

[0330] "Provide simulation results of Team A (1996) vs Team B (2017) and highlight key plays when viewer is excited."

[0331] This allows the system to visualize the results of each match and important play scenes, providing dynamic visual effects and information that respond to the user's emotions.

[0332] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0333] Step 1:

[0334] The server retrieves past sports game data and collects performance data of individuals or groups, including detailed data such as scores, play records, shooting success rates, and assist numbers, thereby obtaining the necessary game data.

[0335] Input: Past match database

[0336] Output: Collected match data

[0337] Step 2:

[0338] The server formats and cleanses the collected data, correcting or removing outliers, missing data, and duplicate entries, and converting it into a consistent format, improving data quality and facilitating subsequent processing.

[0339] Input: Collected match data

[0340] Output: Cleansed match data

[0341] Step 3:

[0342] The server extracts features from the organized data and uses them to train a machine learning model. It uses algorithms such as regression, classification, and neural networks to create highly accurate predictive models. It uses machine learning frameworks such as TensorFlow.

[0343] Input: Cleansed match data

[0344] Output: A trained machine learning model

[0345] Step 4:

[0346] The server executes virtual matches based on the match settings specified by the user. In this process, matches between organizations from different eras are simulated, and results and statistical data for each match are generated, resulting in the results of the virtual matches.

[0347] Input: Trained machine learning model, user match settings

[0348] Output: Simulation results of a virtual match

[0349] Step 5:

[0350] The server and device detect the user's facial expressions, tone of voice, and body movements to recognize emotions in real time. Data acquired from the user's webcam and microphone is analyzed to extract emotional indicators. This allows the user's emotional data to be acquired.

[0351] Input: User's facial expression data, voice data

[0352] Output: User emotion data

[0353] Step 6:

[0354] The server integrates the simulation results with the user's emotional data to generate visualized results, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0355] Input: Simulation results of virtual matches, user emotion data

[0356] Output: Visualized match results data

[0357] Step 7:

[0358] The device displays the visualized results in a virtual environment, allowing users to watch the game in a metaverse stadium. Based on the user's emotional data, the device dynamically adjusts the viewpoint and display information, enhancing the user's sense of immersion and providing a personalized viewing experience.

[0359] Input: Visualized match results data

[0360] Output: The match viewing experience shown to the user

[0361] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0362] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search<url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0363] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0364] [Second embodiment]

[0365] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0366] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0367] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0368] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0369] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0370] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0371] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0372] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0373] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0374] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0375] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0376] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0377] This invention relates to a system that uses big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually within the metaverse. This system consists of a server, terminals, and users.

[0378] System Overview

[0379] Data Collection Component (Server)

[0380] The server retrieves game data from a large database containing data from past sports games, such as game scores and player performance data from past official databases, including each player's shooting percentage, assists, rebounds, and defensive skills.

[0381] Data preprocessing and cleansing (server)

[0382] The server processes the collected data, converting it into a consistent format, and also performs data cleansing to remove outliers and duplicate data to improve data quality.

[0383] Feature extraction and model training (server)

[0384] The server extracts features for each player and team from the preprocessed data and trains a machine learning model using a combination of machine learning algorithms such as regression, classification, and neural networks.

[0385] Running the simulation (server)

[0386] The server simulates a match between teams from different eras specified by the user, such as a match between a famous team A from the 1990s and a famous team B from the 2010s. The simulation is run multiple times to obtain statistically significant results.

[0387] Result visualization and metaverse distribution (server)

[0388] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are distributed to the Metaverse platform, where users can view them within the virtual environment.

[0389] Metaverse experience (device, user)

[0390] Users access the metaverse using a device (e.g., a VR headset or PC) and watch the game in a virtual stadium. They can move around the stadium using their avatar, switch the perspective of the game, and focus on specific players. They can also watch replays and highlights of the game, providing a highly immersive experience.

[0391] Specific examples

[0392] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, calculating the probability of winning or losing and the contribution of each player. The results are distributed to the metaverse platform, allowing users to watch the matches in real time within a virtual environment.

[0393] In this way, the system of the present invention simulates matches between historically important sports teams and players, providing a new, visually enjoyable entertainment experience.

[0394] The processing flow will be explained below.

[0395] Step 1:

[0396] Start the data collection component (server)

[0397] The server retrieves historical game data from a sports game database, including the score for each game and player performance metrics (shooting percentage, assists, rebounds, turnovers, etc.), and stores this data in a local database.

[0398] Step 2:

[0399] Data preprocessing and cleansing (server)

[0400] The server then formats the collected match data into a consistent format, sorting it by date and time, detecting and correcting or removing outliers and missing data, and detecting and removing duplicate entries within the data.

[0401] Step 3:

[0402] Feature extraction (server)

[0403] The server extracts features from the cleansed data. This is the step where important data points that indicate the characteristics of each player or team are extracted. For example, a player's shooting percentage, defensive strength, or a team's offensive tactics are extracted as features.

[0404] Step 4:

[0405] Machine learning model training (server)

[0406] The server uses the extracted features to train a machine learning model, using algorithms such as regression, classification, and neural networks. To train the model, cross-validation is performed using past match data to find the optimal parameters.

[0407] Step 5:

[0408] Enter match settings (user)

[0409] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0410] Step 6:

[0411] Running the simulation (server)

[0412] The server runs virtual matches based on the match settings specified by the user. These matches are run as multiple simulations, and the results and statistics for each match are generated. For example, Team A from 1996 plays Team B from 2017, and the results and player performances for each simulation are recorded.

[0413] Step 7:

[0414] Aggregation and analysis of simulation results (server)

[0415] The server aggregates all simulation results, calculates the probabilities of win / loss outcomes and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0416] Step 8:

[0417] Visualization of results (server)

[0418] The server visualizes the aggregated results of the simulation as graphs and animations, including the progress of the match and key plays, and sends this visualization data to the Metaverse platform.

[0419] Step 9:

[0420] Distribution and viewing to the metaverse (devices, users)

[0421] Users access the metaverse through their devices and watch simulated matches in a virtual stadium. They can control their avatars to move around the stadium and freely switch the viewpoint of the match. They can also watch replays and highlights of the match, enriching the viewing experience.

[0422] Through the above steps, the system of the present invention can simulate competitions between sports teams and players from different eras, providing users with a visually immersive viewing experience.

[0423] Example 1

[0424] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0425] It is difficult to accurately compare and simulate the performance of sports teams and players from different eras, and there is a need for a way to visually enjoy this. Conventional systems do not fully integrate the technologies required to collect, format, and simulate past data, preventing users from visually enjoying the experience in an immersive virtual environment. This necessitates the development of a new entertainment system that can visualize the match results of teams and players from different eras with high accuracy and in real time.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0427] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping the collected data and converting it into a consistent format, means for cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the results of the simulation, means for delivering the visualized results to a virtual environment, and means for users to watch the game in the virtual environment using a terminal. This makes it possible to simulate matches between sports teams and players from different eras with high accuracy and enjoy them visually.

[0428] "Match data" refers to scores related to past sports matches and various statistical information recorded during those matches.

[0429] "Performance data" refers to data that quantifies the specific abilities and achievements of a player or team, such as shooting percentage, number of assists, and number of rebounds.

[0430] "Data transformation" refers to the process of converting collected data into a consistent format. For example, converting data stored in different formats into a unified format (such as CSV).

[0431] "Data cleansing" refers to the process of removing outliers and duplicates from data and improving data quality.

[0432] "Feature extraction" refers to the process of extracting important data points (features) needed to train a machine learning model, such as a player's average points scored or rebound percentage.

[0433] A "machine learning model" is a set of algorithms that learn patterns from data and make predictions or classifications. The model is trained on collected data.

[0434] "Simulation" refers to the process of virtually pitting teams and players from different eras against each other, recreating these matches in a digital environment and running them multiple times to ensure statistically significant results.

[0435] "Visualization" refers to the display of simulation results in a user-friendly format, including graphs and animations.

[0436] A "virtual environment" refers to a digital space where users can have an immersive experience, in which they can control an avatar and move freely.

[0437] "Device" refers to the device through which a user accesses a virtual environment, including a VR headset or PC.

[0438] This invention is a system that utilizes big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually in a virtual environment (metaverse). Components include a server, terminals, and users.

[0439] Data Collection Component (Server)

[0440] The server has a means to retrieve past sports game data. This can be done using official database APIs or open data. For example, the server can use the official basketball website API to retrieve detailed data such as game scores, players' shooting percentages, assists, and rebounds. The collected data is then stored in a consistent format in a NoSQL database (e.g., MongoDB).

[0441] Data preprocessing and cleansing (server)

[0442] The server has the means to format the collected data and convert it into a consistent format. For example, it can standardize the collected data into CSV files or JSON format. It also uses data cleansing methods to remove outliers and duplicates and improve data quality. A specific example is imputing missing values ​​using the Python Pandas library.

[0443] Feature extraction and model training (server)

[0444] The server has a means to extract player and team features from the preprocessed data. Specific software used is Python's Pandas and NumPy. The extracted features (e.g., average shooting percentage, rebound rate) are used to train machine learning models. Here, the server uses Scikit-learn and TensorFlow to train regression and neural network models.

[0445] Running the simulation (server)

[0446] The server has a means to simulate teams from different eras specified by the user. The user inputs a prompt to the generated AI model to specify the teams to play against. For example, the prompt might say, "Please simulate a match between Team A from 1996 and Team B from 2017." The server runs the simulation multiple times and obtains statistically significant results.

[0447] Result visualization and metaverse distribution (server)

[0448] The server has a means to visualize the simulation results, using libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations, and distributes the visualized results to the Metaverse platform, where users can access them via VR headsets or PCs.

[0449] Metaverse experience (device, user)

[0450] Users access the Metaverse platform using a device (e.g., an Oculus VR headset) and watch the game in a virtual stadium. They can control an avatar, change the perspective of the game, and focus on specific players. For example, users can view replays of a specific player's shots in detail. This provides a highly immersive experience.

[0451] As described above, the system of the present invention analyzes past sports game data with high accuracy and simulates matches between teams and players from different eras, providing a new, visually enjoyable entertainment experience.

[0452] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0453] Step 1:

[0454] The server retrieves past sports game data using official database APIs and open data. Specifically, it retrieves detailed data such as game scores, players' shooting percentages, number of assists, and number of rebounds from the official basketball website API. The input at this stage is an API request, and the output is response data in JSON or XML format containing the game data.

[0455] Step 2:

[0456] The server formats the received match data into a consistent format. Specifically, it converts JSON-formatted data into CSV format. This process uses the Python Pandas library. The input is JSON-formatted match data, and the output is formatted CSV data.

[0457] Step 3:

[0458] The server cleanses the formatted match data. Specific operations include filling in missing values, detecting and deleting outliers, and removing duplicate data. The goal of this stage is to obtain high-quality data after cleansing. The input is formatted CSV data, and the output is cleansed CSV data.

[0459] Step 4:

[0460] The server extracts features from the preprocessed data. The specific operation here is to calculate statistical data for each player and team (e.g., average shooting percentage, rebound percentage). At this stage, Python's NumPy and Pandas libraries are used. The input is the cleansed CSV data, and the output is a new dataset containing the features.

[0461] Step 5:

[0462] The server trains a machine learning model based on the extracted features. Specifically, it uses Scikit-learn or TensorFlow to build a regression model or neural network model and trains it with the training data. At this stage, the input is the feature dataset, and the output is a trained machine learning model.

[0463] Step 6:

[0464] The server runs a simulation in which teams from different eras specified by the user play against each other. The user provides a prompt, which the server analyzes and runs the simulation. At this stage, the model is used to run multiple simulations to obtain statistically significant results. The input is the user's prompt and the trained model, and the output is aggregated data from each simulation.

[0465] Step 7:

[0466] The server visualizes the simulation results. Specifically, it uses libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations. The input at this stage is the aggregated data of the simulation results, and the output is the visualized result data.

[0467] Step 8:

[0468] The server delivers the visualized results to the Metaverse platform. Users receive this data using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input here is the visualized results data, and the output is the display on the Metaverse platform.

[0469] Step 9:

[0470] Users access the metaverse platform using their devices and watch the game in a virtual stadium. They can control an avatar and freely switch the viewpoint of the game. Specific actions include moving the avatar, focusing on a specific player, and using the replay function. The input is the user's actions, and the output is the game-watching experience in the virtual environment.

[0471] (Application example 1)

[0472] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0473] In modern sports entertainment, there is a growing demand for virtual simulations in which teams and players from different eras compete against each other. However, conventional simulation technologies have struggled to efficiently and accurately analyze past data, visualize the results in real time, and provide an immersive environment for users. In particular, there has been a lack of technology that allows users to switch between different viewing viewpoints within a virtual environment, making it difficult to provide a truly immersive experience. The present invention addresses these challenges.

[0474] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0475] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual reality environment, and means for a user to watch the game in a virtual stadium, switch viewpoints using avatars, and focus on specific players. This allows users to not only experience a highly accurate simulation of sports teams and players from different eras competing, but also enables a deeper, immersive entertainment experience by visually watching the game in real time and freely switching viewpoints.

[0476] "Match data" refers to information about past sports matches, such as scores, player data, match dates and times, and venues.

[0477] "Player Data" refers to detailed data about each player's performance, including shooting percentage, number of assists, number of rebounds, defensive skills, etc.

[0478] "Team data" refers to data about the performance and movements of the entire team, including information about team strategy, playing style, and member composition.

[0479] "Shaping and cleansing" refers to the process of converting data into a consistent format and removing outliers and duplicate data.

[0480] "Feature extraction" refers to the process of selecting important variables and parameters from a dataset for use in training a predictive model.

[0481] "Training a machine learning model" refers to the process of using collected data to build a predictive model through algorithms such as regression, classification, and neural networks.

[0482] "Simulation" refers to the process of virtually pitting teams from different eras against each other and simulating the results multiple times.

[0483] "Visualization" refers to the display of simulation results in the form of graphs, animations, virtual reality, etc.

[0484] "Delivery to a virtual reality environment" refers to delivering visualized simulation results to a virtual space accessible to the user.

[0485] A "virtual stadium" is a digital space where users can watch a game in a virtual reality environment, and refers to a virtual space shaped like a stadium.

[0486] "Avatar" refers to a digital character that represents a user within a virtual reality environment.

[0487] "Switching viewpoints" refers to the user freely changing between different camera angles and viewing positions within the virtual environment.

[0488] "Focusing on a specific player" refers to a user focusing on a specific player in the virtual environment and observing their movements in detail.

[0489] This invention relates to a system that uses big data analysis and generative AI models to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy the game visually in a virtual reality environment. This system consists of a server, a terminal, and a user element.

[0490] Data Collection Component (Server)

[0491] The server retrieves game data from a database of past sports games, including details on each player's shooting percentage, assists, rebounds, defensive skills, etc. This data is collected from official databases and reliable public data sources.

[0492] Data preprocessing and cleansing (server)

[0493] The server formats the collected data, converting it into a consistent format, and performs data cleansing to remove outliers and duplicates to improve data quality, using common data analysis tools such as Python, Pandas, and NumPy.

[0494] Feature extraction and model training (server)

[0495] The server extracts features of each player and team from the preprocessed data and trains a machine learning model using a combination of algorithms such as regression, classification, and neural networks. Machine learning frameworks such as Scikit-learn and TensorFlow are used.

[0496] Running the simulation (server)

[0497] The server virtually runs matches between teams from different eras specified by the user. This simulation is run multiple times to obtain statistically significant results. The accuracy of the simulation results is improved by using a generative AI model.

[0498] Visualization of results and distribution to virtual reality environments (server, terminal)

[0499] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are delivered to a virtual reality environment where users can view them using a VR headset or smartphone.

[0500] Virtual stadium experience (device, user)

[0501] Users access the virtual reality environment using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can move around the stadium using their own avatar, freely switch the viewpoint of the game, and focus on specific players. This provides a highly immersive experience.

[0502] Specific examples

[0503] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, and the probability of winning or losing and the contribution of each player are calculated. The results are then streamed to a virtual reality environment, allowing users to watch the matches in real time.

[0504] Prompt Sentence Examples

[0505] "We have data from the 1990 Basketball Finals and data from the 2010 Basketball Finals. Using the shooting percentages, assists, rebounds, defensive skills, etc. from each data set, you can simulate matches between teams from different eras and recreate the games in a virtual environment."

[0506] This configuration allows users to not only experience highly accurate simulations of sports teams and players from different eras competing, but also to visually watch the game in real time and freely switch viewpoints, providing a deeper, more immersive entertainment experience.

[0507] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0508] Step 1:

[0509] The server retrieves past match data. It uses data such as match scores, player data, match dates and times, and venues obtained from official databases or trusted data sources as input. The output is match data obtained in raw data format. Specific operations include issuing database queries and downloading match data.

[0510] Step 2:

[0511] The server collects player and team performance data. As input, it uses detailed performance data such as shooting percentage, assists, rebounds, and defensive skills obtained from the official database. The output is the collected player and team performance data. Specifically, it retrieves the data through the API and stores it in an internal database.

[0512] Step 3:

[0513] The server formats and cleanses the collected data. It uses raw match and performance data as input. The output is clean data formatted in a consistent format. Specifically, it uses the Python Pandas library to format the data and remove outliers and duplicates.

[0514] Step 4:

[0515] The server extracts features from the formatted data and trains a machine learning model. Using the formatted data as input, features are extracted using algorithms such as regression, classification, and neural networks. The output is a trained machine learning model. Specifically, the machine learning model is trained using Scikit-learn and TensorFlow.

[0516] Step 5:

[0517] The server runs a simulation in which teams from different eras virtually compete against each other. It uses a trained machine learning model and data on the competing teams as input. The output is the simulation results. Specifically, it uses a generative AI model to run multiple simulations and obtain statistically significant results.

[0518] Step 6:

[0519] The server visualizes the simulation results. It uses the simulation results as input and visualizes the match highlights and detailed reports in graphs and animations as output. Specifically, it uses Matplotlib and Plotly for visualization.

[0520] Step 7:

[0521] The server distributes the visualized results to the virtual reality environment. It uses the visualized simulation results as input and generates data that is sent to the virtual reality environment as output. Specifically, it converts the data into a format suitable for VR headsets and smartphones and distributes it over the network.

[0522] Step 8:

[0523] Users use their devices to watch the game in a virtual stadium. The input is the simulation results delivered to the virtual reality environment, and the output is the user's viewing experience. Specific actions include operating a VR headset or avatar to switch viewpoints or focus on specific players while watching the game.

[0524] Through these steps, users can not only experience highly accurate simulations of sports teams and players from different eras competing, but also visually watch the game in real time and freely switch perspectives, providing a deeper, immersive entertainment experience.

[0525] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0526] This invention combines a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0527] System Overview

[0528] 1. Data collection component (server)

[0529] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0530] 2. Data preprocessing and cleansing (server)

[0531] The server formats the collected data consistently, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality.

[0532] 3. Feature extraction and model training (server)

[0533] The server extracts features from the organized data and uses them to train machine learning models, using algorithms such as regression, classification, and neural networks.

[0534] 4. Running the simulation (server)

[0535] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras and generating results and statistics for each match.

[0536] 5. Emotion engine operation (server, terminal)

[0537] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone and extracts emotional indicators.

[0538] 6. Visualization of results and virtual environment distribution (server)

[0539] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0540] 7. Providing a Metaverse experience (devices, users)

[0541] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0542] Specific examples

[0543] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This simulates numerous virtual matches between Team A and Team B, and calculates the probability of winning or losing and the contribution of each player.

[0544] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0545] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0546] The processing flow will be explained below.

[0547] Step 1:

[0548] Start the data collection component (server)

[0549] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (e.g., shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0550] Step 2:

[0551] Data preprocessing and cleansing (server)

[0552] The server then formats the collected data into a consistent format, detecting and correcting or removing outliers and missing data, as well as detecting and removing duplicate entries within the data.

[0553] Step 3:

[0554] Feature extraction (server)

[0555] The server extracts features from the cleansed data, such as a player's shooting percentage, defensive strength, and team offensive tactics.

[0556] Step 4:

[0557] Machine learning model training (server)

[0558] The server uses the extracted features to train a machine learning model using algorithms such as regression, classification, and neural networks. The model is trained by cross-validating the data to find the optimal parameters.

[0559] Step 5:

[0560] Enter match settings (user)

[0561] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0562] Step 6:

[0563] Running the simulation (server)

[0564] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras, and generates results and statistics for each match.

[0565] Step 7:

[0566] Aggregation and analysis of simulation results (server)

[0567] The server aggregates all simulation results, calculates the probability of winning or losing, and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0568] Step 8:

[0569] Data acquisition by emotion engine (terminal)

[0570] The device collects emotional data by sensing the user's facial expressions, tone of voice, body movements, etc. This is done using the device's built-in camera and microphone.

[0571] Step 9:

[0572] Emotion data analysis (server)

[0573] The server analyzes the user's emotional data sent from the terminal, for example, to identify the user's emotional state, such as excitement, joy, or surprise.

[0574] Step 10:

[0575] Visualization of results and dynamic adjustment of user experience (server)

[0576] The server visualizes the simulation results based on the collected emotional data and dynamically adjusts the display content. For example, if the user is surprised, the server will highlight that scene as a replay. It will also display more scenes from the perspective of the player who excited the user.

[0577] Step 11:

[0578] Distribution and viewing to the metaverse (devices, users)

[0579] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely switch the viewpoint of the game. The system also dynamically adjusts the viewpoint and display based on emotional data, providing users with a richer, more immersive experience.

[0580] Through the above steps, the system of the present invention can simulate matches between historical sports teams and players, providing users with a visually and emotionally rich viewing experience.

[0581] Example 2

[0582] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0583] In modern times, simulations of matches between sports teams and players from different eras often lack accuracy. In addition, the use of emotional data to enrich the user's viewing experience is insufficient, resulting in a low sense of immersion and an inability to reflect real-time reactions.

[0584] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0585] In this invention, the server includes means for acquiring game data, means for collecting player and team performance data, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for acquiring and analyzing user emotion data in real time, means for dynamically adjusting the visualization data based on the emotion data, and means for the user to watch the game in the virtual environment. This enables a more immersive viewing experience by simulating matches between sports teams and players from different eras with high accuracy and dynamically adjusting the visualization data of the match according to the user's emotional state.

[0586] "Means for obtaining game data" refers to means for obtaining information about past sports games from a database or API.

[0587] "Means for collecting performance data" are means for collecting information about specific plays and performance of players and teams.

[0588] "Forming and cleansing means" refers to means for converting acquired data into a consistent format and correcting or removing outliers and missing data.

[0589] "Means for extracting features and training machine learning models" refers to means for extracting important patterns and features from formatted data and using machine learning algorithms to train a model based on that data.

[0590] "Means for performing simulations" means means for virtually playing matches between sports teams and players from different eras using trained machine learning models.

[0591] A "visualization means" is a means for displaying the results of a simulation in a visual format such as a graph or animation.

[0592] "Means for delivery to a virtual environment" means means for transmitting the visualization results to the metaverse or other virtual reality platforms.

[0593] "Means for acquiring and analyzing emotional data in real time" refers to a means for collecting a user's facial expressions, voice, body movements, etc. using sensors, and analyzing this to identify their emotional state.

[0594] The "means for dynamically adjusting visualization data" refers to a means for optimizing visualization data in real time in accordance with the user's emotions based on the analyzed emotion data.

[0595] A "means for viewing a game" is a means by which a user can view and interact with a simulated game within a virtual environment.

[0596] This invention combines a system that can accurately simulate matches between sports teams and players from different eras and visually enjoy them in the metaverse with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0597] System Overview

[0598] Data Collection Component (Server)

[0599] The server retrieves game data from a database of past sports games. Specifically, it collects game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.) and stores them in a local database. For example, it manages this data using an SQL database.

[0600] Data preprocessing and cleansing (server)

[0601] The server cleans the collected data into a consistent format, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality. One example is data cleansing using the Python Pandas library.

[0602] Feature extraction and model training (server)

[0603] The server extracts important features from the formatted data and uses them to train a machine learning model using algorithms such as regression, classification, and neural networks. Specific examples include model training using Scikit-learn and TensorFlow.

[0604] Running the simulation (server)

[0605] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between teams from different eras and generates results and statistics for each match. It can use cloud computing services (e.g., AWS Lambda).

[0606] Emotion engine operation (server, terminal)

[0607] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone to extract emotional indicators. One example is the use of facial expression recognition using OpenCV and Dlib, or an emotion analysis API.

[0608] Visualization of results and virtual environment distribution (server)

[0609] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments. This visualization data is sent to a metaverse platform (e.g., Unity) so that the user can view it in the virtual environment.

[0610] Providing a metaverse experience (devices, users)

[0611] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can control their avatar to move around the stadium and freely switch the viewpoint of the game. Based on emotion data, the system dynamically adjusts the viewpoint and the information displayed.

[0612] Specific examples

[0613] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects Team B's data from 2017 and runs it through a machine learning model to simulate a hypothetical match between Team A and Team B. The probability of winning or losing and the contribution of each player are calculated.

[0614] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0615] Example prompts for generative AI models

[0616] 1. "Simulate a hypothetical match between Team A from 1996 and Team B from 2017 and visualize the outcome of the match."

[0617] 2. "Based on the user's emotional data, please replay scenes that emphasize when the user is in an excited state."

[0618] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0619] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0620] Step 1: Data collection

[0621] The server accesses a database of past sports games to retrieve game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.). The input is the database address of a specific sports game database via an API or SQL query. The output is the retrieved game data. Specifically, for example, the server retrieves data from an endpoint such as "https: / / sportsdata.example.com / api / games / 1996" via a REST API and stores it in a local SQL database.

[0622] Step 2: Data preprocessing and cleansing

[0623] The server formats the data retrieved from the local database into a consistent format. The input is the retrieved match data. The output is the formatted and cleansed match data. The server uses the Python Pandas library to load the data into a data frame and perform missing value imputation and duplicate data removal. For example, the "fillna" method is used to impute missing values ​​and the "drop_duplicates" method is used to remove duplicate data.

[0624] Step 3: Feature extraction and model training

[0625] The server extracts important features from the formatted data and trains a machine learning model based on them. The input is the formatted and cleansed match data. The output is a trained machine learning model. The server extracts features using the Scikit-learn library and trains a regression model using, for example, the "LinearRegression" class. The data is split into training and test data, and the model is fitted.

[0626] Step 4: Run the simulation

[0627] The server runs a virtual match based on the match settings specified by the user. The input is the match settings specified by the user (for example, data on Team A and Team B). The output is statistical data on the simulation results. The server runs multiple simulations using the saved machine learning model and stores the results in a database. The statistical data on the simulation results is output as win / loss probabilities and player contributions.

[0628] Step 5: Acquire and analyze emotion data

[0629] The device senses the user's facial expressions, tone of voice, and body movements using a camera and microphone, and sends the data to a server. The input is the user's real-time emotional data (facial expressions, tone of voice, and body movements). The output is analyzed emotional indicators. The server analyzes this data using OpenCV and an emotion analysis API to extract emotional indicators.

[0630] Step 6: Visualizing the results and delivering the virtual environment

[0631] The server integrates the simulation results and the user's emotional data to generate visualized results. The input is the statistical data of the simulation results and the analyzed emotional index. The output is the visualized match results. The server sends this visualized data to a metaverse platform such as Unity. The data is distributed in real time so that users can view it within the metaverse.

[0632] Step 7: Deliver the Metaverse experience

[0633] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input is the visualized game results and real-time updated emotional data. The output is the user's immersive experience. Users can control their avatar to move freely around the stadium and switch the perspective of the game. Based on the emotional data, the system dynamically adjusts the perspective and the information displayed.

[0634] (Application example 2)

[0635] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0636] Conventional sports viewing experiences have struggled to reflect real-time emotional changes, limiting the dynamic visual effects and information provided based on user emotions. Furthermore, there has been a lack of technology to virtually simulate and visualize matches between sports teams and players from different eras with high accuracy. As a result, the user's sense of immersion and the quality of the viewing experience have declined, making it difficult to improve user engagement.

[0637] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0638] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for a user to watch the game in the virtual environment, means for analyzing user emotional data and dynamically adjusting the visualization results, and means for collecting user emotional data using a smart device and changing the viewpoint and display information in the virtual environment based on the results. This enables personalized visual effects and information to be provided based on the user's emotional state, resulting in a more immersive and richer viewing experience.

[0639] "Match data" refers to various data collected from past sports matches, such as scores, play records, and player performance indicators.

[0640] "Performance data" is data that quantitatively indicates the skills and results that players and teams demonstrate during a game, and includes shooting percentage, number of assists, number of rebounds, turnovers, etc.

[0641] "Forming and cleansing procedures" refers to the process of transforming collected data into a consistent format and correcting or removing outliers, missing data, and duplicate entries.

[0642] "Features" are important data points or metrics used to train machine learning models, allowing them to make more accurate predictions.

[0643] A "machine learning model" is a set of mathematical algorithms that learns from data, finds patterns, and makes subsequent predictions or decisions.

[0644] "Means for performing simulations" refers to the process of using a machine learning model to conduct a large number of virtual matches under set conditions and generate the results.

[0645] "Visualization means" refers to technology that graphically displays data obtained through simulation, allowing users to intuitively understand it.

[0646] "Means of delivery to a virtual environment" refers to a method of sending visualized simulation results to a virtual space such as a metaverse platform, allowing users who access the space to experience it.

[0647] "Means for analyzing emotional data" refers to technology that detects a user's emotional state in real time based on their facial expressions, tone of voice, and body movements.

[0648] A "smart device" is a device that can connect to the Internet and collect and process real-time data using sensors, cameras, microphones, etc., and includes smartphones and head-mounted displays.

[0649] The present invention is a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, and also combines it with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0650] 1. Data Collection and Preprocessing

[0651] The server retrieves past sports game data and collects player and team performance data. This data is then stored in a local database. The collected data is then formatted, cleansed, and converted into a consistent format. Outliers, missing data, and duplicate entries are also corrected or removed. Software used includes OpenCV and DeepFace.

[0652] 2. Feature extraction and model training

[0653] Features are extracted from the organized data and used to train machine learning models using algorithms such as regression, classification, and neural networks. This process is carried out using machine learning frameworks such as TensorFlow.

[0654] 3. Running and Visualizing the Simulation

[0655] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between organizations from different eras, generating results and statistical data for each match. These simulation results are then visualized and displayed in a way that is intuitively understandable to the user.

[0656] 4. Operation of the Emotion Engine

[0657] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the user's emotional state to be grasped in real time. This emotion engine analyzes input from the user's webcam and microphone and extracts emotional indicators. DeepFace is used for this processing. Smart devices such as smartphones and head-mounted displays (HMDs) are also used.

[0658] 5. Results visualization and virtual environment delivery

[0659] The simulation results are integrated with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0660] 6. Providing a Metaverse Experience

[0661] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0662] Examples of concrete examples and prompts

[0663] For example, to set up a virtual match between Team A in 1996 and Team B in 2017, the server retrieves all the match data from each team and analyzes the performance data of each player in detail. The system then inputs the following prompts into the generative AI model:

[0664] "Provide simulation results of Team A (1996) vs Team B (2017) and highlight key plays when viewer is excited."

[0665] This allows the system to visualize the results of each match and important play scenes, providing dynamic visual effects and information that respond to the user's emotions.

[0666] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0667] Step 1:

[0668] The server retrieves past sports game data and collects performance data of individuals or groups, including detailed data such as scores, play records, shooting success rates, and assist numbers, thereby obtaining the necessary game data.

[0669] Input: Past match database

[0670] Output: Collected match data

[0671] Step 2:

[0672] The server formats and cleanses the collected data, correcting or removing outliers, missing data, and duplicate entries, and converting it into a consistent format, improving data quality and facilitating subsequent processing.

[0673] Input: Collected match data

[0674] Output: Cleansed match data

[0675] Step 3:

[0676] The server extracts features from the organized data and uses them to train a machine learning model. It uses algorithms such as regression, classification, and neural networks to create highly accurate predictive models. It uses machine learning frameworks such as TensorFlow.

[0677] Input: Cleansed match data

[0678] Output: A trained machine learning model

[0679] Step 4:

[0680] The server executes virtual matches based on the match settings specified by the user. In this process, matches between organizations from different eras are simulated, and results and statistical data for each match are generated, resulting in the results of the virtual matches.

[0681] Input: Trained machine learning model, user match settings

[0682] Output: Simulation results of a virtual match

[0683] Step 5:

[0684] The server and device detect the user's facial expressions, tone of voice, and body movements to recognize emotions in real time. Data acquired from the user's webcam and microphone is analyzed to extract emotional indicators. This allows the user's emotional data to be acquired.

[0685] Input: User's facial expression data, voice data

[0686] Output: User emotion data

[0687] Step 6:

[0688] The server integrates the simulation results with the user's emotional data to generate visualized results, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0689] Input: Simulation results of virtual matches, user emotion data

[0690] Output: Visualized match results data

[0691] Step 7:

[0692] The device displays the visualized results in a virtual environment, allowing users to watch the game in a metaverse stadium. Based on the user's emotional data, the device dynamically adjusts the viewpoint and display information, enhancing the user's sense of immersion and providing a personalized viewing experience.

[0693] Input: Visualized match results data

[0694] Output: The match viewing experience shown to the user

[0695] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0696] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0697] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0698] [Third embodiment]

[0699] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0700] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0701] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0702] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0703] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0704] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0705] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0706] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0707] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0708] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0709] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0710] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0711] This invention relates to a system that uses big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually within the metaverse. This system consists of a server, terminals, and users.

[0712] System Overview

[0713] Data Collection Component (Server)

[0714] The server retrieves game data from a large database containing data from past sports games, such as game scores and player performance data from past official databases, including each player's shooting percentage, assists, rebounds, and defensive skills.

[0715] Data preprocessing and cleansing (server)

[0716] The server processes the collected data, converting it into a consistent format, and also performs data cleansing to remove outliers and duplicate data to improve data quality.

[0717] Feature extraction and model training (server)

[0718] The server extracts features for each player and team from the preprocessed data and trains a machine learning model using a combination of machine learning algorithms such as regression, classification, and neural networks.

[0719] Running the simulation (server)

[0720] The server simulates a match between teams from different eras specified by the user, such as a match between a famous team A from the 1990s and a famous team B from the 2010s. The simulation is run multiple times to obtain statistically significant results.

[0721] Result visualization and metaverse distribution (server)

[0722] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are distributed to the Metaverse platform, where users can view them within the virtual environment.

[0723] Metaverse experience (device, user)

[0724] Users access the metaverse using a device (e.g., a VR headset or PC) and watch the game in a virtual stadium. They can move around the stadium using their avatar, switch the perspective of the game, and focus on specific players. They can also watch replays and highlights of the game, providing a highly immersive experience.

[0725] Specific examples

[0726] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, calculating the probability of winning or losing and the contribution of each player. The results are distributed to the metaverse platform, allowing users to watch the matches in real time within a virtual environment.

[0727] In this way, the system of the present invention simulates matches between historically important sports teams and players, providing a new, visually enjoyable entertainment experience.

[0728] The processing flow will be explained below.

[0729] Step 1:

[0730] Start the data collection component (server)

[0731] The server retrieves historical game data from a sports game database, including the score for each game and player performance metrics (shooting percentage, assists, rebounds, turnovers, etc.), and stores this data in a local database.

[0732] Step 2:

[0733] Data preprocessing and cleansing (server)

[0734] The server then formats the collected match data into a consistent format, sorting it by date and time, detecting and correcting or removing outliers and missing data, and detecting and removing duplicate entries within the data.

[0735] Step 3:

[0736] Feature extraction (server)

[0737] The server extracts features from the cleansed data. This is the step where important data points that indicate the characteristics of each player or team are extracted. For example, a player's shooting percentage, defensive strength, or a team's offensive tactics are extracted as features.

[0738] Step 4:

[0739] Machine learning model training (server)

[0740] The server uses the extracted features to train a machine learning model, using algorithms such as regression, classification, and neural networks. To train the model, cross-validation is performed using past match data to find the optimal parameters.

[0741] Step 5:

[0742] Enter match settings (user)

[0743] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0744] Step 6:

[0745] Running the simulation (server)

[0746] The server runs virtual matches based on the match settings specified by the user. These matches are run as multiple simulations, and the results and statistics for each match are generated. For example, Team A from 1996 plays Team B from 2017, and the results and player performances for each simulation are recorded.

[0747] Step 7:

[0748] Aggregation and analysis of simulation results (server)

[0749] The server aggregates all simulation results, calculates the probabilities of win / loss outcomes and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0750] Step 8:

[0751] Visualization of results (server)

[0752] The server visualizes the aggregated results of the simulation as graphs and animations, including the progress of the match and key plays, and sends this visualization data to the Metaverse platform.

[0753] Step 9:

[0754] Distribution and viewing to the metaverse (devices, users)

[0755] Users access the metaverse through their devices and watch simulated matches in a virtual stadium. They can control their avatars to move around the stadium and freely switch the viewpoint of the match. They can also watch replays and highlights of the match, enriching the viewing experience.

[0756] Through the above steps, the system of the present invention can simulate competitions between sports teams and players from different eras, providing users with a visually immersive viewing experience.

[0757] Example 1

[0758] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0759] It is difficult to accurately compare and simulate the performance of sports teams and players from different eras, and there is a need for a way to visually enjoy this. Conventional systems do not fully integrate the technologies required to collect, format, and simulate past data, preventing users from visually enjoying the experience in an immersive virtual environment. This necessitates the development of a new entertainment system that can visualize the match results of teams and players from different eras with high accuracy and in real time.

[0760] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0761] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping the collected data and converting it into a consistent format, means for cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the results of the simulation, means for delivering the visualized results to a virtual environment, and means for users to watch the game in the virtual environment using a terminal. This makes it possible to simulate matches between sports teams and players from different eras with high accuracy and enjoy them visually.

[0762] "Match data" refers to scores related to past sports matches and various statistical information recorded during those matches.

[0763] "Performance data" refers to data that quantifies the specific abilities and achievements of a player or team, such as shooting percentage, number of assists, and number of rebounds.

[0764] "Data transformation" refers to the process of converting collected data into a consistent format. For example, converting data stored in different formats into a unified format (such as CSV).

[0765] "Data cleansing" refers to the process of removing outliers and duplicates from data and improving data quality.

[0766] "Feature extraction" refers to the process of extracting important data points (features) needed to train a machine learning model, such as a player's average points scored or rebound percentage.

[0767] A "machine learning model" is a set of algorithms that learn patterns from data and make predictions or classifications. The model is trained on collected data.

[0768] "Simulation" refers to the process of virtually pitting teams and players from different eras against each other, recreating these matches in a digital environment and running them multiple times to ensure statistically significant results.

[0769] "Visualization" refers to the display of simulation results in a user-friendly format, including graphs and animations.

[0770] A "virtual environment" refers to a digital space where users can have an immersive experience, in which they can control an avatar and move freely.

[0771] "Device" refers to the device through which a user accesses a virtual environment, including a VR headset or PC.

[0772] This invention is a system that utilizes big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually in a virtual environment (metaverse). Components include a server, terminals, and users.

[0773] Data Collection Component (Server)

[0774] The server has a means to retrieve past sports game data. This can be done using official database APIs or open data. For example, the server can use the official basketball website API to retrieve detailed data such as game scores, players' shooting percentages, assists, and rebounds. The collected data is then stored in a consistent format in a NoSQL database (e.g., MongoDB).

[0775] Data preprocessing and cleansing (server)

[0776] The server has the means to format the collected data and convert it into a consistent format. For example, it can standardize the collected data into CSV files or JSON format. It also uses data cleansing methods to remove outliers and duplicates and improve data quality. A specific example is imputing missing values ​​using the Python Pandas library.

[0777] Feature extraction and model training (server)

[0778] The server has a means to extract player and team features from the preprocessed data. Specific software used is Python's Pandas and NumPy. The extracted features (e.g., average shooting percentage, rebound rate) are used to train machine learning models. Here, the server uses Scikit-learn and TensorFlow to train regression and neural network models.

[0779] Running the simulation (server)

[0780] The server has a means to simulate teams from different eras specified by the user. The user inputs a prompt to the generated AI model to specify the teams to play against. For example, the prompt might say, "Please simulate a match between Team A from 1996 and Team B from 2017." The server runs the simulation multiple times and obtains statistically significant results.

[0781] Result visualization and metaverse distribution (server)

[0782] The server has a means to visualize the simulation results, using libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations, and distributes the visualized results to the Metaverse platform, where users can access them via VR headsets or PCs.

[0783] Metaverse experience (device, user)

[0784] Users access the Metaverse platform using a device (e.g., an Oculus VR headset) and watch the game in a virtual stadium. They can control an avatar, change the perspective of the game, and focus on specific players. For example, users can view replays of a specific player's shots in detail. This provides a highly immersive experience.

[0785] As described above, the system of the present invention analyzes past sports game data with high accuracy and simulates matches between teams and players from different eras, providing a new, visually enjoyable entertainment experience.

[0786] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0787] Step 1:

[0788] The server retrieves past sports game data using official database APIs and open data. Specifically, it retrieves detailed data such as game scores, players' shooting percentages, number of assists, and number of rebounds from the official basketball website API. The input at this stage is an API request, and the output is response data in JSON or XML format containing the game data.

[0789] Step 2:

[0790] The server formats the received match data into a consistent format. Specifically, it converts JSON-formatted data into CSV format. This process uses the Python Pandas library. The input is JSON-formatted match data, and the output is formatted CSV data.

[0791] Step 3:

[0792] The server cleanses the formatted match data. Specific operations include filling in missing values, detecting and deleting outliers, and removing duplicate data. The goal of this stage is to obtain high-quality data after cleansing. The input is formatted CSV data, and the output is cleansed CSV data.

[0793] Step 4:

[0794] The server extracts features from the preprocessed data. The specific operation here is to calculate statistical data for each player and team (e.g., average shooting percentage, rebound percentage). At this stage, Python's NumPy and Pandas libraries are used. The input is the cleansed CSV data, and the output is a new dataset containing the features.

[0795] Step 5:

[0796] The server trains a machine learning model based on the extracted features. Specifically, it uses Scikit-learn or TensorFlow to build a regression model or neural network model and trains it with the training data. At this stage, the input is the feature dataset, and the output is a trained machine learning model.

[0797] Step 6:

[0798] The server runs a simulation in which teams from different eras specified by the user play against each other. The user provides a prompt, which the server analyzes and runs the simulation. At this stage, the model is used to run multiple simulations to obtain statistically significant results. The input is the user's prompt and the trained model, and the output is aggregated data from each simulation.

[0799] Step 7:

[0800] The server visualizes the simulation results. Specifically, it uses libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations. The input at this stage is the aggregated data of the simulation results, and the output is the visualized result data.

[0801] Step 8:

[0802] The server delivers the visualized results to the Metaverse platform. Users receive this data using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input here is the visualized results data, and the output is the display on the Metaverse platform.

[0803] Step 9:

[0804] Users access the metaverse platform using their devices and watch the game in a virtual stadium. They can control an avatar and freely switch the viewpoint of the game. Specific actions include moving the avatar, focusing on a specific player, and using the replay function. The input is the user's actions, and the output is the game-watching experience in the virtual environment.

[0805] (Application example 1)

[0806] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0807] In modern sports entertainment, there is a growing demand for virtual simulations in which teams and players from different eras compete against each other. However, conventional simulation technologies have struggled to efficiently and accurately analyze past data, visualize the results in real time, and provide an immersive environment for users. In particular, there has been a lack of technology that allows users to switch between different viewing viewpoints within a virtual environment, making it difficult to provide a truly immersive experience. The present invention addresses these challenges.

[0808] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0809] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual reality environment, and means for a user to watch the game in a virtual stadium, switch viewpoints using avatars, and focus on specific players. This allows users to not only experience a highly accurate simulation of sports teams and players from different eras competing, but also enables a deeper, immersive entertainment experience by visually watching the game in real time and freely switching viewpoints.

[0810] "Match data" refers to information about past sports matches, such as scores, player data, match dates and times, and venues.

[0811] "Player Data" refers to detailed data about each player's performance, including shooting percentage, number of assists, number of rebounds, defensive skills, etc.

[0812] "Team data" refers to data about the performance and movements of the entire team, including information about team strategy, playing style, and member composition.

[0813] "Shaping and cleansing" refers to the process of converting data into a consistent format and removing outliers and duplicate data.

[0814] "Feature extraction" refers to the process of selecting important variables and parameters from a dataset for use in training a predictive model.

[0815] "Training a machine learning model" refers to the process of using collected data to build a predictive model through algorithms such as regression, classification, and neural networks.

[0816] "Simulation" refers to the process of virtually pitting teams from different eras against each other and simulating the results multiple times.

[0817] "Visualization" refers to the display of simulation results in the form of graphs, animations, virtual reality, etc.

[0818] "Delivery to a virtual reality environment" refers to delivering visualized simulation results to a virtual space accessible to the user.

[0819] A "virtual stadium" is a digital space where users can watch a game in a virtual reality environment, and refers to a virtual space shaped like a stadium.

[0820] "Avatar" refers to a digital character that represents a user within a virtual reality environment.

[0821] "Switching viewpoints" refers to the user freely changing between different camera angles and viewing positions within the virtual environment.

[0822] "Focusing on a specific player" refers to a user focusing on a specific player in the virtual environment and observing their movements in detail.

[0823] This invention relates to a system that uses big data analysis and generative AI models to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy the game visually in a virtual reality environment. This system consists of a server, a terminal, and a user element.

[0824] Data Collection Component (Server)

[0825] The server retrieves game data from a database of past sports games, including details on each player's shooting percentage, assists, rebounds, defensive skills, etc. This data is collected from official databases and reliable public data sources.

[0826] Data preprocessing and cleansing (server)

[0827] The server formats the collected data, converting it into a consistent format, and performs data cleansing to remove outliers and duplicates to improve data quality, using common data analysis tools such as Python, Pandas, and NumPy.

[0828] Feature extraction and model training (server)

[0829] The server extracts features of each player and team from the preprocessed data and trains a machine learning model using a combination of algorithms such as regression, classification, and neural networks. Machine learning frameworks such as Scikit-learn and TensorFlow are used.

[0830] Running the simulation (server)

[0831] The server virtually runs matches between teams from different eras specified by the user. This simulation is run multiple times to obtain statistically significant results. The accuracy of the simulation results is improved by using a generative AI model.

[0832] Visualization of results and distribution to virtual reality environments (server, terminal)

[0833] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are delivered to a virtual reality environment where users can view them using a VR headset or smartphone.

[0834] Virtual stadium experience (device, user)

[0835] Users access the virtual reality environment using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can move around the stadium using their own avatar, freely switch the viewpoint of the game, and focus on specific players. This provides a highly immersive experience.

[0836] Specific examples

[0837] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, and the probability of winning or losing and the contribution of each player are calculated. The results are then streamed to a virtual reality environment, allowing users to watch the matches in real time.

[0838] Prompt Sentence Examples

[0839] "We have data from the 1990 Basketball Finals and data from the 2010 Basketball Finals. Using the shooting percentages, assists, rebounds, defensive skills, etc. from each data set, you can simulate matches between teams from different eras and recreate the games in a virtual environment."

[0840] This configuration allows users to not only experience highly accurate simulations of sports teams and players from different eras competing, but also to visually watch the game in real time and freely switch viewpoints, providing a deeper, more immersive entertainment experience.

[0841] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0842] Step 1:

[0843] The server retrieves past match data. It uses data such as match scores, player data, match dates and times, and venues obtained from official databases or trusted data sources as input. The output is match data obtained in raw data format. Specific operations include issuing database queries and downloading match data.

[0844] Step 2:

[0845] The server collects player and team performance data. As input, it uses detailed performance data such as shooting percentage, assists, rebounds, and defensive skills obtained from the official database. The output is the collected player and team performance data. Specifically, it retrieves the data through the API and stores it in an internal database.

[0846] Step 3:

[0847] The server formats and cleanses the collected data. It uses raw match and performance data as input. The output is clean data formatted in a consistent format. Specifically, it uses the Python Pandas library to format the data and remove outliers and duplicates.

[0848] Step 4:

[0849] The server extracts features from the formatted data and trains a machine learning model. Using the formatted data as input, features are extracted using algorithms such as regression, classification, and neural networks. The output is a trained machine learning model. Specifically, the machine learning model is trained using Scikit-learn and TensorFlow.

[0850] Step 5:

[0851] The server runs a simulation in which teams from different eras virtually compete against each other. It uses a trained machine learning model and data on the competing teams as input. The output is the simulation results. Specifically, it uses a generative AI model to run multiple simulations and obtain statistically significant results.

[0852] Step 6:

[0853] The server visualizes the simulation results. It uses the simulation results as input and visualizes the match highlights and detailed reports in graphs and animations as output. Specifically, it uses Matplotlib and Plotly for visualization.

[0854] Step 7:

[0855] The server distributes the visualized results to the virtual reality environment. It uses the visualized simulation results as input and generates data that is sent to the virtual reality environment as output. Specifically, it converts the data into a format suitable for VR headsets and smartphones and distributes it over the network.

[0856] Step 8:

[0857] Users use their devices to watch the game in a virtual stadium. The input is the simulation results delivered to the virtual reality environment, and the output is the user's viewing experience. Specific actions include operating a VR headset or avatar to switch viewpoints or focus on specific players while watching the game.

[0858] Through these steps, users can not only experience highly accurate simulations of sports teams and players from different eras competing, but also visually watch the game in real time and freely switch perspectives, providing a deeper, immersive entertainment experience.

[0859] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0860] This invention combines a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0861] System Overview

[0862] 1. Data collection component (server)

[0863] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0864] 2. Data preprocessing and cleansing (server)

[0865] The server formats the collected data consistently, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality.

[0866] 3. Feature extraction and model training (server)

[0867] The server extracts features from the organized data and uses them to train machine learning models, using algorithms such as regression, classification, and neural networks.

[0868] 4. Running the simulation (server)

[0869] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras and generating results and statistics for each match.

[0870] 5. Emotion engine operation (server, terminal)

[0871] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone and extracts emotional indicators.

[0872] 6. Visualization of results and virtual environment distribution (server)

[0873] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0874] 7. Providing a Metaverse experience (devices, users)

[0875] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0876] Specific examples

[0877] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This simulates numerous virtual matches between Team A and Team B, and calculates the probability of winning or losing and the contribution of each player.

[0878] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0879] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0880] The processing flow will be explained below.

[0881] Step 1:

[0882] Start the data collection component (server)

[0883] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (e.g., shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[0884] Step 2:

[0885] Data preprocessing and cleansing (server)

[0886] The server then formats the collected data into a consistent format, detecting and correcting or removing outliers and missing data, as well as detecting and removing duplicate entries within the data.

[0887] Step 3:

[0888] Feature extraction (server)

[0889] The server extracts features from the cleansed data, such as a player's shooting percentage, defensive strength, and team offensive tactics.

[0890] Step 4:

[0891] Machine learning model training (server)

[0892] The server uses the extracted features to train a machine learning model using algorithms such as regression, classification, and neural networks. The model is trained by cross-validating the data to find the optimal parameters.

[0893] Step 5:

[0894] Enter match settings (user)

[0895] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[0896] Step 6:

[0897] Running the simulation (server)

[0898] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras, and generates results and statistics for each match.

[0899] Step 7:

[0900] Aggregation and analysis of simulation results (server)

[0901] The server aggregates all simulation results, calculates the probability of winning or losing, and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[0902] Step 8:

[0903] Data acquisition by emotion engine (terminal)

[0904] The device collects emotional data by sensing the user's facial expressions, tone of voice, body movements, etc. This is done using the device's built-in camera and microphone.

[0905] Step 9:

[0906] Emotion data analysis (server)

[0907] The server analyzes the user's emotional data sent from the terminal, for example, to identify the user's emotional state, such as excitement, joy, or surprise.

[0908] Step 10:

[0909] Visualization of results and dynamic adjustment of user experience (server)

[0910] The server visualizes the simulation results based on the collected emotional data and dynamically adjusts the display content. For example, if the user is surprised, the server will highlight that scene as a replay. It will also display more scenes from the perspective of the player who excited the user.

[0911] Step 11:

[0912] Distribution and viewing to the metaverse (devices, users)

[0913] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely switch the viewpoint of the game. The system also dynamically adjusts the viewpoint and display based on emotional data, providing users with a richer, more immersive experience.

[0914] Through the above steps, the system of the present invention can simulate matches between historical sports teams and players, providing users with a visually and emotionally rich viewing experience.

[0915] Example 2

[0916] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0917] In modern times, simulations of matches between sports teams and players from different eras often lack accuracy. In addition, the use of emotional data to enrich the user's viewing experience is insufficient, resulting in a low sense of immersion and an inability to reflect real-time reactions.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0919] In this invention, the server includes means for acquiring game data, means for collecting player and team performance data, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for acquiring and analyzing user emotion data in real time, means for dynamically adjusting the visualization data based on the emotion data, and means for the user to watch the game in the virtual environment. This enables a more immersive viewing experience by simulating matches between sports teams and players from different eras with high accuracy and dynamically adjusting the visualization data of the match according to the user's emotional state.

[0920] "Means for obtaining game data" refers to means for obtaining information about past sports games from a database or API.

[0921] "Means for collecting performance data" are means for collecting information about specific plays and performance of players and teams.

[0922] "Forming and cleansing means" refers to means for converting acquired data into a consistent format and correcting or removing outliers and missing data.

[0923] "Means for extracting features and training machine learning models" refers to means for extracting important patterns and features from formatted data and using machine learning algorithms to train a model based on that data.

[0924] "Means for performing simulations" means means for virtually playing matches between sports teams and players from different eras using trained machine learning models.

[0925] A "visualization means" is a means for displaying the results of a simulation in a visual format such as a graph or animation.

[0926] "Means for delivery to a virtual environment" means means for transmitting the visualization results to the metaverse or other virtual reality platforms.

[0927] "Means for acquiring and analyzing emotional data in real time" refers to a means for collecting a user's facial expressions, voice, body movements, etc. using sensors, and analyzing this to identify their emotional state.

[0928] The "means for dynamically adjusting visualization data" refers to a means for optimizing visualization data in real time in accordance with the user's emotions based on the analyzed emotion data.

[0929] A "means for viewing a game" is a means by which a user can view and interact with a simulated game within a virtual environment.

[0930] This invention combines a system that can accurately simulate matches between sports teams and players from different eras and visually enjoy them in the metaverse with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[0931] System Overview

[0932] Data Collection Component (Server)

[0933] The server retrieves game data from a database of past sports games. Specifically, it collects game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.) and stores them in a local database. For example, it manages this data using an SQL database.

[0934] Data preprocessing and cleansing (server)

[0935] The server cleans the collected data into a consistent format, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality. One example is data cleansing using the Python Pandas library.

[0936] Feature extraction and model training (server)

[0937] The server extracts important features from the formatted data and uses them to train a machine learning model using algorithms such as regression, classification, and neural networks. Specific examples include model training using Scikit-learn and TensorFlow.

[0938] Running the simulation (server)

[0939] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between teams from different eras and generates results and statistics for each match. It can use cloud computing services (e.g., AWS Lambda).

[0940] Emotion engine operation (server, terminal)

[0941] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone to extract emotional indicators. One example is the use of facial expression recognition using OpenCV and Dlib, or an emotion analysis API.

[0942] Visualization of results and virtual environment distribution (server)

[0943] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments. This visualization data is sent to a metaverse platform (e.g., Unity) so that the user can view it in the virtual environment.

[0944] Providing a metaverse experience (devices, users)

[0945] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can control their avatar to move around the stadium and freely switch the viewpoint of the game. Based on emotion data, the system dynamically adjusts the viewpoint and the information displayed.

[0946] Specific examples

[0947] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects Team B's data from 2017 and runs it through a machine learning model to simulate a hypothetical match between Team A and Team B. The probability of winning or losing and the contribution of each player are calculated.

[0948] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[0949] Example prompts for generative AI models

[0950] 1. "Simulate a hypothetical match between Team A from 1996 and Team B from 2017 and visualize the outcome of the match."

[0951] 2. "Based on the user's emotional data, please replay scenes that emphasize when the user is in an excited state."

[0952] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[0953] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0954] Step 1: Data collection

[0955] The server accesses a database of past sports games to retrieve game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.). The input is the database address of a specific sports game database via an API or SQL query. The output is the retrieved game data. Specifically, for example, the server retrieves data from an endpoint such as "https: / / sportsdata.example.com / api / games / 1996" via a REST API and stores it in a local SQL database.

[0956] Step 2: Data preprocessing and cleansing

[0957] The server formats the data retrieved from the local database into a consistent format. The input is the retrieved match data. The output is the formatted and cleansed match data. The server uses the Python Pandas library to load the data into a data frame and perform missing value imputation and duplicate data removal. For example, the "fillna" method is used to impute missing values ​​and the "drop_duplicates" method is used to remove duplicate data.

[0958] Step 3: Feature extraction and model training

[0959] The server extracts important features from the formatted data and trains a machine learning model based on them. The input is the formatted and cleansed match data. The output is a trained machine learning model. The server extracts features using the Scikit-learn library and trains a regression model using, for example, the "LinearRegression" class. The data is split into training and test data, and the model is fitted.

[0960] Step 4: Run the simulation

[0961] The server runs a virtual match based on the match settings specified by the user. The input is the match settings specified by the user (for example, data on Team A and Team B). The output is statistical data on the simulation results. The server runs multiple simulations using the saved machine learning model and stores the results in a database. The statistical data on the simulation results is output as win / loss probabilities and player contributions.

[0962] Step 5: Acquire and analyze emotion data

[0963] The device senses the user's facial expressions, tone of voice, and body movements using a camera and microphone, and sends the data to a server. The input is the user's real-time emotional data (facial expressions, tone of voice, and body movements). The output is analyzed emotional indicators. The server analyzes this data using OpenCV and an emotion analysis API to extract emotional indicators.

[0964] Step 6: Visualizing the results and delivering the virtual environment

[0965] The server integrates the simulation results and the user's emotional data to generate visualized results. The input is the statistical data of the simulation results and the analyzed emotional index. The output is the visualized match results. The server sends this visualized data to a metaverse platform such as Unity. The data is distributed in real time so that users can view it within the metaverse.

[0966] Step 7: Deliver the Metaverse experience

[0967] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input is the visualized game results and real-time updated emotional data. The output is the user's immersive experience. Users can control their avatar to move freely around the stadium and switch the perspective of the game. Based on the emotional data, the system dynamically adjusts the perspective and the information displayed.

[0968] (Application example 2)

[0969] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0970] Conventional sports viewing experiences have struggled to reflect real-time emotional changes, limiting the dynamic visual effects and information provided based on user emotions. Furthermore, there has been a lack of technology to virtually simulate and visualize matches between sports teams and players from different eras with high accuracy. As a result, the user's sense of immersion and the quality of the viewing experience have declined, making it difficult to improve user engagement.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0972] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for a user to watch the game in the virtual environment, means for analyzing user emotional data and dynamically adjusting the visualization results, and means for collecting user emotional data using a smart device and changing the viewpoint and display information in the virtual environment based on the results. This enables personalized visual effects and information to be provided based on the user's emotional state, resulting in a more immersive and richer viewing experience.

[0973] "Match data" refers to various data collected from past sports matches, such as scores, play records, and player performance indicators.

[0974] "Performance data" is data that quantitatively indicates the skills and results that players and teams demonstrate during a game, and includes shooting percentage, number of assists, number of rebounds, turnovers, etc.

[0975] "Forming and cleansing procedures" refers to the process of transforming collected data into a consistent format and correcting or removing outliers, missing data, and duplicate entries.

[0976] "Features" are important data points or metrics used to train machine learning models, allowing them to make more accurate predictions.

[0977] A "machine learning model" is a set of mathematical algorithms that learns from data, finds patterns, and makes subsequent predictions or decisions.

[0978] "Means for performing simulations" refers to the process of using a machine learning model to conduct a large number of virtual matches under set conditions and generate the results.

[0979] "Visualization means" refers to technology that graphically displays data obtained through simulation, allowing users to intuitively understand it.

[0980] "Means of delivery to a virtual environment" refers to a method of sending visualized simulation results to a virtual space such as a metaverse platform, allowing users who access the space to experience it.

[0981] "Means for analyzing emotional data" refers to technology that detects a user's emotional state in real time based on their facial expressions, tone of voice, and body movements.

[0982] A "smart device" is a device that can connect to the Internet and collect and process real-time data using sensors, cameras, microphones, etc., and includes smartphones and head-mounted displays.

[0983] The present invention is a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, and also combines it with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[0984] 1. Data Collection and Preprocessing

[0985] The server retrieves past sports game data and collects player and team performance data. This data is then stored in a local database. The collected data is then formatted, cleansed, and converted into a consistent format. Outliers, missing data, and duplicate entries are also corrected or removed. Software used includes OpenCV and DeepFace.

[0986] 2. Feature extraction and model training

[0987] Features are extracted from the organized data and used to train machine learning models using algorithms such as regression, classification, and neural networks. This process is carried out using machine learning frameworks such as TensorFlow.

[0988] 3. Running and Visualizing the Simulation

[0989] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between organizations from different eras, generating results and statistical data for each match. These simulation results are then visualized and displayed in a way that is intuitively understandable to the user.

[0990] 4. Operation of the Emotion Engine

[0991] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the user's emotional state to be grasped in real time. This emotion engine analyzes input from the user's webcam and microphone and extracts emotional indicators. DeepFace is used for this processing. Smart devices such as smartphones and head-mounted displays (HMDs) are also used.

[0992] 5. Results visualization and virtual environment delivery

[0993] The simulation results are integrated with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[0994] 6. Providing a Metaverse Experience

[0995] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[0996] Examples of concrete examples and prompts

[0997] For example, to set up a virtual match between Team A in 1996 and Team B in 2017, the server retrieves all the match data from each team and analyzes the performance data of each player in detail. The system then inputs the following prompts into the generative AI model:

[0998] "Provide simulation results of Team A (1996) vs Team B (2017) and highlight key plays when viewer is excited."

[0999] This allows the system to visualize the results of each match and important play scenes, providing dynamic visual effects and information that respond to the user's emotions.

[1000] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1001] Step 1:

[1002] The server retrieves past sports game data and collects performance data of individuals or groups, including detailed data such as scores, play records, shooting success rates, and assist numbers, thereby obtaining the necessary game data.

[1003] Input: Past match database

[1004] Output: Collected match data

[1005] Step 2:

[1006] The server formats and cleanses the collected data, correcting or removing outliers, missing data, and duplicate entries, and converting it into a consistent format, improving data quality and facilitating subsequent processing.

[1007] Input: Collected match data

[1008] Output: Cleansed match data

[1009] Step 3:

[1010] The server extracts features from the organized data and uses them to train a machine learning model. It uses algorithms such as regression, classification, and neural networks to create highly accurate predictive models. It uses machine learning frameworks such as TensorFlow.

[1011] Input: Cleansed match data

[1012] Output: A trained machine learning model

[1013] Step 4:

[1014] The server executes virtual matches based on the match settings specified by the user. In this process, matches between organizations from different eras are simulated, and results and statistical data for each match are generated, resulting in the results of the virtual matches.

[1015] Input: Trained machine learning model, user match settings

[1016] Output: Simulation results of a virtual match

[1017] Step 5:

[1018] The server and device detect the user's facial expressions, tone of voice, and body movements to recognize emotions in real time. Data acquired from the user's webcam and microphone is analyzed to extract emotional indicators. This allows the user's emotional data to be acquired.

[1019] Input: User's facial expression data, voice data

[1020] Output: User emotion data

[1021] Step 6:

[1022] The server integrates the simulation results with the user's emotional data to generate visualized results, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[1023] Input: Simulation results of virtual matches, user emotion data

[1024] Output: Visualized match results data

[1025] Step 7:

[1026] The device displays the visualized results in a virtual environment, allowing users to watch the game in a metaverse stadium. Based on the user's emotional data, the device dynamically adjusts the viewpoint and display information, enhancing the user's sense of immersion and providing a personalized viewing experience.

[1027] Input: Visualized match results data

[1028] Output: The match viewing experience shown to the user

[1029] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1030] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1031] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1032] [Fourth embodiment]

[1033] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1034] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1035] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1036] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1037] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1038] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1039] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1040] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1041] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1042] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1044] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1046] This invention relates to a system that uses big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually within the metaverse. This system consists of a server, terminals, and users.

[1047] System Overview

[1048] Data Collection Component (Server)

[1049] The server retrieves game data from a large database containing data from past sports games, such as game scores and player performance data from past official databases, including each player's shooting percentage, assists, rebounds, and defensive skills.

[1050] Data preprocessing and cleansing (server)

[1051] The server processes the collected data, converting it into a consistent format, and also performs data cleansing to remove outliers and duplicate data to improve data quality.

[1052] Feature extraction and model training (server)

[1053] The server extracts features for each player and team from the preprocessed data and trains a machine learning model using a combination of machine learning algorithms such as regression, classification, and neural networks.

[1054] Running the simulation (server)

[1055] The server simulates a match between teams from different eras specified by the user, such as a match between a famous team A from the 1990s and a famous team B from the 2010s. The simulation is run multiple times to obtain statistically significant results.

[1056] Result visualization and metaverse distribution (server)

[1057] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are distributed to the Metaverse platform, where users can view them within the virtual environment.

[1058] Metaverse experience (device, user)

[1059] Users access the metaverse using a device (e.g., a VR headset or PC) and watch the game in a virtual stadium. They can move around the stadium using their avatar, switch the perspective of the game, and focus on specific players. They can also watch replays and highlights of the game, providing a highly immersive experience.

[1060] Specific examples

[1061] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, calculating the probability of winning or losing and the contribution of each player. The results are distributed to the metaverse platform, allowing users to watch the matches in real time within a virtual environment.

[1062] In this way, the system of the present invention simulates matches between historically important sports teams and players, providing a new, visually enjoyable entertainment experience.

[1063] The processing flow will be explained below.

[1064] Step 1:

[1065] Start the data collection component (server)

[1066] The server retrieves historical game data from a sports game database, including the score for each game and player performance metrics (shooting percentage, assists, rebounds, turnovers, etc.), and stores this data in a local database.

[1067] Step 2:

[1068] Data preprocessing and cleansing (server)

[1069] The server then formats the collected match data into a consistent format, sorting it by date and time, detecting and correcting or removing outliers and missing data, and detecting and removing duplicate entries within the data.

[1070] Step 3:

[1071] Feature extraction (server)

[1072] The server extracts features from the cleansed data. This is the step where important data points that indicate the characteristics of each player or team are extracted. For example, a player's shooting percentage, defensive strength, or a team's offensive tactics are extracted as features.

[1073] Step 4:

[1074] Machine learning model training (server)

[1075] The server uses the extracted features to train a machine learning model, using algorithms such as regression, classification, and neural networks. To train the model, cross-validation is performed using past match data to find the optimal parameters.

[1076] Step 5:

[1077] Enter match settings (user)

[1078] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[1079] Step 6:

[1080] Running the simulation (server)

[1081] The server runs virtual matches based on the match settings specified by the user. These matches are run as multiple simulations, and the results and statistics for each match are generated. For example, Team A from 1996 plays Team B from 2017, and the results and player performances for each simulation are recorded.

[1082] Step 7:

[1083] Aggregation and analysis of simulation results (server)

[1084] The server aggregates all simulation results, calculates the probabilities of win / loss outcomes and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[1085] Step 8:

[1086] Visualization of results (server)

[1087] The server visualizes the aggregated results of the simulation as graphs and animations, including the progress of the match and key plays, and sends this visualization data to the Metaverse platform.

[1088] Step 9:

[1089] Distribution and viewing to the metaverse (devices, users)

[1090] Users access the metaverse through their devices and watch simulated matches in a virtual stadium. They can control their avatars to move around the stadium and freely switch the viewpoint of the match. They can also watch replays and highlights of the match, enriching the viewing experience.

[1091] Through the above steps, the system of the present invention can simulate competitions between sports teams and players from different eras, providing users with a visually immersive viewing experience.

[1092] Example 1

[1093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1094] It is difficult to accurately compare and simulate the performance of sports teams and players from different eras, and there is a need for a way to visually enjoy this. Conventional systems do not fully integrate the technologies required to collect, format, and simulate past data, preventing users from visually enjoying the experience in an immersive virtual environment. This necessitates the development of a new entertainment system that can visualize the match results of teams and players from different eras with high accuracy and in real time.

[1095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1096] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping the collected data and converting it into a consistent format, means for cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the results of the simulation, means for delivering the visualized results to a virtual environment, and means for users to watch the game in the virtual environment using a terminal. This makes it possible to simulate matches between sports teams and players from different eras with high accuracy and enjoy them visually.

[1097] "Match data" refers to scores related to past sports matches and various statistical information recorded during those matches.

[1098] "Performance data" refers to data that quantifies the specific abilities and achievements of a player or team, such as shooting percentage, number of assists, and number of rebounds.

[1099] "Data transformation" refers to the process of converting collected data into a consistent format. For example, converting data stored in different formats into a unified format (such as CSV).

[1100] "Data cleansing" refers to the process of removing outliers and duplicates from data and improving data quality.

[1101] "Feature extraction" refers to the process of extracting important data points (features) needed to train a machine learning model, such as a player's average points scored or rebound percentage.

[1102] A "machine learning model" is a set of algorithms that learn patterns from data and make predictions or classifications. The model is trained on collected data.

[1103] "Simulation" refers to the process of virtually pitting teams and players from different eras against each other, recreating these matches in a digital environment and running them multiple times to ensure statistically significant results.

[1104] "Visualization" refers to the display of simulation results in a user-friendly format, including graphs and animations.

[1105] A "virtual environment" refers to a digital space where users can have an immersive experience, in which they can control an avatar and move freely.

[1106] "Device" refers to the device through which a user accesses a virtual environment, including a VR headset or PC.

[1107] This invention is a system that utilizes big data analysis and AI simulation to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy them visually in a virtual environment (metaverse). Components include a server, terminals, and users.

[1108] Data Collection Component (Server)

[1109] The server has a means to retrieve past sports game data. This can be done using official database APIs or open data. For example, the server can use the official basketball website API to retrieve detailed data such as game scores, players' shooting percentages, assists, and rebounds. The collected data is then stored in a consistent format in a NoSQL database (e.g., MongoDB).

[1110] Data preprocessing and cleansing (server)

[1111] The server has the means to format the collected data and convert it into a consistent format. For example, it can standardize the collected data into CSV files or JSON format. It also uses data cleansing methods to remove outliers and duplicates and improve data quality. A specific example is imputing missing values ​​using the Python Pandas library.

[1112] Feature extraction and model training (server)

[1113] The server has a means to extract player and team features from the preprocessed data. Specific software used is Python's Pandas and NumPy. The extracted features (e.g., average shooting percentage, rebound rate) are used to train machine learning models. Here, the server uses Scikit-learn and TensorFlow to train regression and neural network models.

[1114] Running the simulation (server)

[1115] The server has a means to simulate teams from different eras specified by the user. The user inputs a prompt to the generated AI model to specify the teams to play against. For example, the prompt might say, "Please simulate a match between Team A from 1996 and Team B from 2017." The server runs the simulation multiple times and obtains statistically significant results.

[1116] Result visualization and metaverse distribution (server)

[1117] The server has a means to visualize the simulation results, using libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations, and distributes the visualized results to the Metaverse platform, where users can access them via VR headsets or PCs.

[1118] Metaverse experience (device, user)

[1119] Users access the Metaverse platform using a device (e.g., an Oculus VR headset) and watch the game in a virtual stadium. They can control an avatar, change the perspective of the game, and focus on specific players. For example, users can view replays of a specific player's shots in detail. This provides a highly immersive experience.

[1120] As described above, the system of the present invention analyzes past sports game data with high accuracy and simulates matches between teams and players from different eras, providing a new, visually enjoyable entertainment experience.

[1121] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1122] Step 1:

[1123] The server retrieves past sports game data using official database APIs and open data. Specifically, it retrieves detailed data such as game scores, players' shooting percentages, number of assists, and number of rebounds from the official basketball website API. The input at this stage is an API request, and the output is response data in JSON or XML format containing the game data.

[1124] Step 2:

[1125] The server formats the received match data into a consistent format. Specifically, it converts JSON-formatted data into CSV format. This process uses the Python Pandas library. The input is JSON-formatted match data, and the output is formatted CSV data.

[1126] Step 3:

[1127] The server cleanses the formatted match data. Specific operations include filling in missing values, detecting and deleting outliers, and removing duplicate data. The goal of this stage is to obtain high-quality data after cleansing. The input is formatted CSV data, and the output is cleansed CSV data.

[1128] Step 4:

[1129] The server extracts features from the preprocessed data. The specific operation here is to calculate statistical data for each player and team (e.g., average shooting percentage, rebound percentage). At this stage, Python's NumPy and Pandas libraries are used. The input is the cleansed CSV data, and the output is a new dataset containing the features.

[1130] Step 5:

[1131] The server trains a machine learning model based on the extracted features. Specifically, it uses Scikit-learn or TensorFlow to build a regression model or neural network model and trains it with the training data. At this stage, the input is the feature dataset, and the output is a trained machine learning model.

[1132] Step 6:

[1133] The server runs a simulation in which teams from different eras specified by the user play against each other. The user provides a prompt, which the server analyzes and runs the simulation. At this stage, the model is used to run multiple simulations to obtain statistically significant results. The input is the user's prompt and the trained model, and the output is aggregated data from each simulation.

[1134] Step 7:

[1135] The server visualizes the simulation results. Specifically, it uses libraries such as Matplotlib and D3.js to display the results in the form of graphs and animations. The input at this stage is the aggregated data of the simulation results, and the output is the visualized result data.

[1136] Step 8:

[1137] The server delivers the visualized results to the Metaverse platform. Users receive this data using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input here is the visualized results data, and the output is the display on the Metaverse platform.

[1138] Step 9:

[1139] Users access the metaverse platform using their devices and watch the game in a virtual stadium. They can control an avatar and freely switch the viewpoint of the game. Specific actions include moving the avatar, focusing on a specific player, and using the replay function. The input is the user's actions, and the output is the game-watching experience in the virtual environment.

[1140] (Application example 1)

[1141] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1142] In modern sports entertainment, there is a growing demand for virtual simulations in which teams and players from different eras compete against each other. However, conventional simulation technologies have struggled to efficiently and accurately analyze past data, visualize the results in real time, and provide an immersive environment for users. In particular, there has been a lack of technology that allows users to switch between different viewing viewpoints within a virtual environment, making it difficult to provide a truly immersive experience. The present invention addresses these challenges.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1144] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual reality environment, and means for a user to watch the game in a virtual stadium, switch viewpoints using avatars, and focus on specific players. This allows users to not only experience a highly accurate simulation of sports teams and players from different eras competing, but also enables a deeper, immersive entertainment experience by visually watching the game in real time and freely switching viewpoints.

[1145] "Match data" refers to information about past sports matches, such as scores, player data, match dates and times, and venues.

[1146] "Player Data" refers to detailed data about each player's performance, including shooting percentage, number of assists, number of rebounds, defensive skills, etc.

[1147] "Team data" refers to data about the performance and movements of the entire team, including information about team strategy, playing style, and member composition.

[1148] "Shaping and cleansing" refers to the process of converting data into a consistent format and removing outliers and duplicate data.

[1149] "Feature extraction" refers to the process of selecting important variables and parameters from a dataset for use in training a predictive model.

[1150] "Training a machine learning model" refers to the process of using collected data to build a predictive model through algorithms such as regression, classification, and neural networks.

[1151] "Simulation" refers to the process of virtually pitting teams from different eras against each other and simulating the results multiple times.

[1152] "Visualization" refers to the display of simulation results in the form of graphs, animations, virtual reality, etc.

[1153] "Delivery to a virtual reality environment" refers to delivering visualized simulation results to a virtual space accessible to the user.

[1154] A "virtual stadium" is a digital space where users can watch a game in a virtual reality environment, and refers to a virtual space shaped like a stadium.

[1155] "Avatar" refers to a digital character that represents a user within a virtual reality environment.

[1156] "Switching viewpoints" refers to the user freely changing between different camera angles and viewing positions within the virtual environment.

[1157] "Focusing on a specific player" refers to a user focusing on a specific player in the virtual environment and observing their movements in detail.

[1158] This invention relates to a system that uses big data analysis and generative AI models to accurately recreate matches between sports teams and players from different eras, allowing users to enjoy the game visually in a virtual reality environment. This system consists of a server, a terminal, and a user element.

[1159] Data Collection Component (Server)

[1160] The server retrieves game data from a database of past sports games, including details on each player's shooting percentage, assists, rebounds, defensive skills, etc. This data is collected from official databases and reliable public data sources.

[1161] Data preprocessing and cleansing (server)

[1162] The server formats the collected data, converting it into a consistent format, and performs data cleansing to remove outliers and duplicates to improve data quality, using common data analysis tools such as Python, Pandas, and NumPy.

[1163] Feature extraction and model training (server)

[1164] The server extracts features of each player and team from the preprocessed data and trains a machine learning model using a combination of algorithms such as regression, classification, and neural networks. Machine learning frameworks such as Scikit-learn and TensorFlow are used.

[1165] Running the simulation (server)

[1166] The server virtually runs matches between teams from different eras specified by the user. This simulation is run multiple times to obtain statistically significant results. The accuracy of the simulation results is improved by using a generative AI model.

[1167] Visualization of results and distribution to virtual reality environments (server, terminal)

[1168] The server generates match highlights and detailed reports based on the simulation results and displays them in visual formats (graphs, animations, etc.) The generated visualization results are delivered to a virtual reality environment where users can view them using a VR headset or smartphone.

[1169] Virtual stadium experience (device, user)

[1170] Users access the virtual reality environment using a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can move around the stadium using their own avatar, freely switch the viewpoint of the game, and focus on specific players. This provides a highly immersive experience.

[1171] Specific examples

[1172] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This allows multiple virtual matches between Team A and Team B to be simulated, and the probability of winning or losing and the contribution of each player are calculated. The results are then streamed to a virtual reality environment, allowing users to watch the matches in real time.

[1173] Prompt Sentence Examples

[1174] "We have data from the 1990 Basketball Finals and data from the 2010 Basketball Finals. Using the shooting percentages, assists, rebounds, defensive skills, etc. from each data set, you can simulate matches between teams from different eras and recreate the games in a virtual environment."

[1175] This configuration allows users to not only experience highly accurate simulations of sports teams and players from different eras competing, but also to visually watch the game in real time and freely switch viewpoints, providing a deeper, more immersive entertainment experience.

[1176] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1177] Step 1:

[1178] The server retrieves past match data. It uses data such as match scores, player data, match dates and times, and venues obtained from official databases or trusted data sources as input. The output is match data obtained in raw data format. Specific operations include issuing database queries and downloading match data.

[1179] Step 2:

[1180] The server collects player and team performance data. As input, it uses detailed performance data such as shooting percentage, assists, rebounds, and defensive skills obtained from the official database. The output is the collected player and team performance data. Specifically, it retrieves the data through the API and stores it in an internal database.

[1181] Step 3:

[1182] The server formats and cleanses the collected data. It uses raw match and performance data as input. The output is clean data formatted in a consistent format. Specifically, it uses the Python Pandas library to format the data and remove outliers and duplicates.

[1183] Step 4:

[1184] The server extracts features from the formatted data and trains a machine learning model. Using the formatted data as input, features are extracted using algorithms such as regression, classification, and neural networks. The output is a trained machine learning model. Specifically, the machine learning model is trained using Scikit-learn and TensorFlow.

[1185] Step 5:

[1186] The server runs a simulation in which teams from different eras virtually compete against each other. It uses a trained machine learning model and data on the competing teams as input. The output is the simulation results. Specifically, it uses a generative AI model to run multiple simulations and obtain statistically significant results.

[1187] Step 6:

[1188] The server visualizes the simulation results. It uses the simulation results as input and visualizes the match highlights and detailed reports in graphs and animations as output. Specifically, it uses Matplotlib and Plotly for visualization.

[1189] Step 7:

[1190] The server distributes the visualized results to the virtual reality environment. It uses the visualized simulation results as input and generates data that is sent to the virtual reality environment as output. Specifically, it converts the data into a format suitable for VR headsets and smartphones and distributes it over the network.

[1191] Step 8:

[1192] Users use their devices to watch the game in a virtual stadium. The input is the simulation results delivered to the virtual reality environment, and the output is the user's viewing experience. Specific actions include operating a VR headset or avatar to switch viewpoints or focus on specific players while watching the game.

[1193] Through these steps, users can not only experience highly accurate simulations of sports teams and players from different eras competing, but also visually watch the game in real time and freely switch perspectives, providing a deeper, immersive entertainment experience.

[1194] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1195] This invention combines a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[1196] System Overview

[1197] 1. Data collection component (server)

[1198] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[1199] 2. Data preprocessing and cleansing (server)

[1200] The server formats the collected data consistently, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality.

[1201] 3. Feature extraction and model training (server)

[1202] The server extracts features from the organized data and uses them to train machine learning models, using algorithms such as regression, classification, and neural networks.

[1203] 4. Running the simulation (server)

[1204] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras and generating results and statistics for each match.

[1205] 5. Emotion engine operation (server, terminal)

[1206] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone and extracts emotional indicators.

[1207] 6. Visualization of results and virtual environment distribution (server)

[1208] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[1209] 7. Providing a Metaverse experience (devices, users)

[1210] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[1211] Specific examples

[1212] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects data from Team B from 2017 and trains it through a machine learning model. This simulates numerous virtual matches between Team A and Team B, and calculates the probability of winning or losing and the contribution of each player.

[1213] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[1214] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[1215] The processing flow will be explained below.

[1216] Step 1:

[1217] Start the data collection component (server)

[1218] The server retrieves game data from a database of past sports games, including game scores and player performance indicators (e.g., shooting percentage, assists, rebounds, turnovers, etc.), and stores them in a local database.

[1219] Step 2:

[1220] Data preprocessing and cleansing (server)

[1221] The server then formats the collected data into a consistent format, detecting and correcting or removing outliers and missing data, as well as detecting and removing duplicate entries within the data.

[1222] Step 3:

[1223] Feature extraction (server)

[1224] The server extracts features from the cleansed data, such as a player's shooting percentage, defensive strength, and team offensive tactics.

[1225] Step 4:

[1226] Machine learning model training (server)

[1227] The server uses the extracted features to train a machine learning model using algorithms such as regression, classification, and neural networks. The model is trained by cross-validating the data to find the optimal parameters.

[1228] Step 5:

[1229] Enter match settings (user)

[1230] The user inputs the team and player combinations they want to simulate through their device. For example, they can set up a match between "Team A from 1996" and "Team B from 2017."

[1231] Step 6:

[1232] Running the simulation (server)

[1233] The server runs virtual matches based on user-specified match settings, simulating multiple matches between teams from different eras, and generates results and statistics for each match.

[1234] Step 7:

[1235] Aggregation and analysis of simulation results (server)

[1236] The server aggregates all simulation results, calculates the probability of winning or losing, and the contribution of each player, and generates detailed reports and highlights based on this aggregated data.

[1237] Step 8:

[1238] Data acquisition by emotion engine (terminal)

[1239] The device collects emotional data by sensing the user's facial expressions, tone of voice, body movements, etc. This is done using the device's built-in camera and microphone.

[1240] Step 9:

[1241] Emotion data analysis (server)

[1242] The server analyzes the user's emotional data sent from the terminal, for example, to identify the user's emotional state, such as excitement, joy, or surprise.

[1243] Step 10:

[1244] Visualization of results and dynamic adjustment of user experience (server)

[1245] The server visualizes the simulation results based on the collected emotional data and dynamically adjusts the display content. For example, if the user is surprised, the server will highlight that scene as a replay. It will also display more scenes from the perspective of the player who excited the user.

[1246] Step 11:

[1247] Distribution and viewing to the metaverse (devices, users)

[1248] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely switch the viewpoint of the game. The system also dynamically adjusts the viewpoint and display based on emotional data, providing users with a richer, more immersive experience.

[1249] Through the above steps, the system of the present invention can simulate matches between historical sports teams and players, providing users with a visually and emotionally rich viewing experience.

[1250] Example 2

[1251] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1252] In modern times, simulations of matches between sports teams and players from different eras often lack accuracy. In addition, the use of emotional data to enrich the user's viewing experience is insufficient, resulting in a low sense of immersion and an inability to reflect real-time reactions.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1254] In this invention, the server includes means for acquiring game data, means for collecting player and team performance data, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for acquiring and analyzing user emotion data in real time, means for dynamically adjusting the visualization data based on the emotion data, and means for the user to watch the game in the virtual environment. This enables a more immersive viewing experience by simulating matches between sports teams and players from different eras with high accuracy and dynamically adjusting the visualization data of the match according to the user's emotional state.

[1255] "Means for obtaining game data" refers to means for obtaining information about past sports games from a database or API.

[1256] "Means for collecting performance data" are means for collecting information about specific plays and performance of players and teams.

[1257] "Forming and cleansing means" refers to means for converting acquired data into a consistent format and correcting or removing outliers and missing data.

[1258] "Means for extracting features and training machine learning models" refers to means for extracting important patterns and features from formatted data and using machine learning algorithms to train a model based on that data.

[1259] "Means for performing simulations" means means for virtually playing matches between sports teams and players from different eras using trained machine learning models.

[1260] A "visualization means" is a means for displaying the results of a simulation in a visual format such as a graph or animation.

[1261] "Means for delivery to a virtual environment" means means for transmitting the visualization results to the metaverse or other virtual reality platforms.

[1262] "Means for acquiring and analyzing emotional data in real time" refers to a means for collecting a user's facial expressions, voice, body movements, etc. using sensors, and analyzing this to identify their emotional state.

[1263] The "means for dynamically adjusting visualization data" refers to a means for optimizing visualization data in real time in accordance with the user's emotions based on the analyzed emotion data.

[1264] A "means for viewing a game" is a means by which a user can view and interact with a simulated game within a virtual environment.

[1265] This invention combines a system that can accurately simulate matches between sports teams and players from different eras and visually enjoy them in the metaverse with an emotion engine that recognizes the user's emotions. This system consists of a server, a terminal, and a user.

[1266] System Overview

[1267] Data Collection Component (Server)

[1268] The server retrieves game data from a database of past sports games. Specifically, it collects game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.) and stores them in a local database. For example, it manages this data using an SQL database.

[1269] Data preprocessing and cleansing (server)

[1270] The server cleans the collected data into a consistent format, corrects or removes outliers and missing data, and detects and removes duplicate entries to improve data quality. One example is data cleansing using the Python Pandas library.

[1271] Feature extraction and model training (server)

[1272] The server extracts important features from the formatted data and uses them to train a machine learning model using algorithms such as regression, classification, and neural networks. Specific examples include model training using Scikit-learn and TensorFlow.

[1273] Running the simulation (server)

[1274] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between teams from different eras and generates results and statistics for each match. It can use cloud computing services (e.g., AWS Lambda).

[1275] Emotion engine operation (server, terminal)

[1276] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the system to grasp the user's real-time emotional state. The emotion engine analyzes input from the user's camera and microphone to extract emotional indicators. One example is the use of facial expression recognition using OpenCV and Dlib, or an emotion analysis API.

[1277] Visualization of results and virtual environment distribution (server)

[1278] The server integrates the simulation results with the user's emotional data to generate visualizations, including the progress of the match and key play moments. This visualization data is sent to a metaverse platform (e.g., Unity) so that the user can view it in the virtual environment.

[1279] Providing a metaverse experience (devices, users)

[1280] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. Users can control their avatar to move around the stadium and freely switch the viewpoint of the game. Based on emotion data, the system dynamically adjusts the viewpoint and the information displayed.

[1281] Specific examples

[1282] For example, the server retrieves all of Team A's match data from 1996 and performs detailed analysis of each player's performance data. Similarly, it collects Team B's data from 2017 and runs it through a machine learning model to simulate a hypothetical match between Team A and Team B. The probability of winning or losing and the contribution of each player are calculated.

[1283] While a user is watching a game in the metaverse, the emotion engine analyzes the user's facial expressions and tone of voice, highlighting scenes of excitement. When a player of the user's particular interest scores, the engine automatically plays a replay from that player's perspective.

[1284] Example prompts for generative AI models

[1285] 1. "Simulate a hypothetical match between Team A from 1996 and Team B from 2017 and visualize the outcome of the match."

[1286] 2. "Based on the user's emotional data, please replay scenes that emphasize when the user is in an excited state."

[1287] In this way, the system of the present invention can simulate historically significant sports teams and player matches, providing users with a visually and emotionally rich viewing experience.

[1288] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1289] Step 1: Data collection

[1290] The server accesses a database of past sports games to retrieve game scores and player performance indicators (shooting percentage, assists, rebounds, turnovers, etc.). The input is the database address of a specific sports game database via an API or SQL query. The output is the retrieved game data. Specifically, for example, the server retrieves data from an endpoint such as "https: / / sportsdata.example.com / api / games / 1996" via a REST API and stores it in a local SQL database.

[1291] Step 2: Data preprocessing and cleansing

[1292] The server formats the data retrieved from the local database into a consistent format. The input is the retrieved match data. The output is the formatted and cleansed match data. The server uses the Python Pandas library to load the data into a data frame and perform missing value imputation and duplicate data removal. For example, the "fillna" method is used to impute missing values ​​and the "drop_duplicates" method is used to remove duplicate data.

[1293] Step 3: Feature extraction and model training

[1294] The server extracts important features from the formatted data and trains a machine learning model based on them. The input is the formatted and cleansed match data. The output is a trained machine learning model. The server extracts features using the Scikit-learn library and trains a regression model using, for example, the "LinearRegression" class. The data is split into training and test data, and the model is fitted.

[1295] Step 4: Run the simulation

[1296] The server runs a virtual match based on the match settings specified by the user. The input is the match settings specified by the user (for example, data on Team A and Team B). The output is statistical data on the simulation results. The server runs multiple simulations using the saved machine learning model and stores the results in a database. The statistical data on the simulation results is output as win / loss probabilities and player contributions.

[1297] Step 5: Acquire and analyze emotion data

[1298] The device senses the user's facial expressions, tone of voice, and body movements using a camera and microphone, and sends the data to a server. The input is the user's real-time emotional data (facial expressions, tone of voice, and body movements). The output is analyzed emotional indicators. The server analyzes this data using OpenCV and an emotion analysis API to extract emotional indicators.

[1299] Step 6: Visualizing the results and delivering the virtual environment

[1300] The server integrates the simulation results and the user's emotional data to generate visualized results. The input is the statistical data of the simulation results and the analyzed emotional index. The output is the visualized match results. The server sends this visualized data to a metaverse platform such as Unity. The data is distributed in real time so that users can view it within the metaverse.

[1301] Step 7: Deliver the Metaverse experience

[1302] Users access the metaverse through a device (e.g., a VR headset or a PC) and watch the game in a virtual stadium. The input is the visualized game results and real-time updated emotional data. The output is the user's immersive experience. Users can control their avatar to move freely around the stadium and switch the perspective of the game. Based on the emotional data, the system dynamically adjusts the perspective and the information displayed.

[1303] (Application example 2)

[1304] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1305] Conventional sports viewing experiences have struggled to reflect real-time emotional changes, limiting the dynamic visual effects and information provided based on user emotions. Furthermore, there has been a lack of technology to virtually simulate and visualize matches between sports teams and players from different eras with high accuracy. As a result, the user's sense of immersion and the quality of the viewing experience have declined, making it difficult to improve user engagement.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1307] In this invention, the server includes means for acquiring game data, means for collecting performance data of players and teams, means for shaping and cleansing the collected data, means for extracting features based on the data and training a machine learning model, means for running a simulation in which teams from different eras virtually compete against each other, means for visualizing the simulation results, means for delivering the visualized results to a virtual environment, means for a user to watch the game in the virtual environment, means for analyzing user emotional data and dynamically adjusting the visualization results, and means for collecting user emotional data using a smart device and changing the viewpoint and display information in the virtual environment based on the results. This enables personalized visual effects and information to be provided based on the user's emotional state, resulting in a more immersive and richer viewing experience.

[1308] "Match data" refers to various data collected from past sports matches, such as scores, play records, and player performance indicators.

[1309] "Performance data" is data that quantitatively indicates the skills and results that players and teams demonstrate during a game, and includes shooting percentage, number of assists, number of rebounds, turnovers, etc.

[1310] "Forming and cleansing procedures" refers to the process of transforming collected data into a consistent format and correcting or removing outliers, missing data, and duplicate entries.

[1311] "Features" are important data points or metrics used to train machine learning models, allowing them to make more accurate predictions.

[1312] A "machine learning model" is a set of mathematical algorithms that learns from data, finds patterns, and makes subsequent predictions or decisions.

[1313] "Means for performing simulations" refers to the process of using a machine learning model to conduct a large number of virtual matches under set conditions and generate the results.

[1314] "Visualization means" refers to technology that graphically displays data obtained through simulation, allowing users to intuitively understand it.

[1315] "Means of delivery to a virtual environment" refers to a method of sending visualized simulation results to a virtual space such as a metaverse platform, allowing users who access the space to experience it.

[1316] "Means for analyzing emotional data" refers to technology that detects a user's emotional state in real time based on their facial expressions, tone of voice, and body movements.

[1317] A "smart device" is a device that can connect to the Internet and collect and process real-time data using sensors, cameras, microphones, etc., and includes smartphones and head-mounted displays.

[1318] The present invention is a system that simulates matches between sports teams and players from different eras with high accuracy, allowing users to enjoy the matches visually within the metaverse, and also combines it with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described in detail below.

[1319] 1. Data Collection and Preprocessing

[1320] The server retrieves past sports game data and collects player and team performance data. This data is then stored in a local database. The collected data is then formatted, cleansed, and converted into a consistent format. Outliers, missing data, and duplicate entries are also corrected or removed. Software used includes OpenCV and DeepFace.

[1321] 2. Feature extraction and model training

[1322] Features are extracted from the organized data and used to train machine learning models using algorithms such as regression, classification, and neural networks. This process is carried out using machine learning frameworks such as TensorFlow.

[1323] 3. Running and Visualizing the Simulation

[1324] The server runs virtual matches based on the match settings specified by the user. It simulates multiple matches between organizations from different eras, generating results and statistical data for each match. These simulation results are then visualized and displayed in a way that is intuitively understandable to the user.

[1325] 4. Operation of the Emotion Engine

[1326] The server and device recognize emotions by sensing the user's facial expressions, tone of voice, and body movements. This allows the user's emotional state to be grasped in real time. This emotion engine analyzes input from the user's webcam and microphone and extracts emotional indicators. DeepFace is used for this processing. Smart devices such as smartphones and head-mounted displays (HMDs) are also used.

[1327] 5. Results visualization and virtual environment delivery

[1328] The simulation results are integrated with the user's emotional data to generate visualizations, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[1329] 6. Providing a Metaverse Experience

[1330] Users access the metaverse through their devices and watch the game in a virtual stadium. They can control their avatar to move around the stadium and freely change the viewpoint of the game. Based on the user's emotional data, the system dynamically adjusts the viewpoint and displayed information. This enhances the user's sense of immersion and enables a richer viewing experience.

[1331] Examples of concrete examples and prompts

[1332] For example, to set up a virtual match between Team A in 1996 and Team B in 2017, the server retrieves all the match data from each team and analyzes the performance data of each player in detail. The system then inputs the following prompts into the generative AI model:

[1333] "Provide simulation results of Team A (1996) vs Team B (2017) and highlight key plays when viewer is excited."

[1334] This allows the system to visualize the results of each match and important play scenes, providing dynamic visual effects and information that respond to the user's emotions.

[1335] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1336] Step 1:

[1337] The server retrieves past sports game data and collects performance data of individuals or groups, including detailed data such as scores, play records, shooting success rates, and assist numbers, thereby obtaining the necessary game data.

[1338] Input: Past match database

[1339] Output: Collected match data

[1340] Step 2:

[1341] The server formats and cleanses the collected data, correcting or removing outliers, missing data, and duplicate entries, and converting it into a consistent format, improving data quality and facilitating subsequent processing.

[1342] Input: Collected match data

[1343] Output: Cleansed match data

[1344] Step 3:

[1345] The server extracts features from the organized data and uses them to train a machine learning model. It uses algorithms such as regression, classification, and neural networks to create highly accurate predictive models. It uses machine learning frameworks such as TensorFlow.

[1346] Input: Cleansed match data

[1347] Output: A trained machine learning model

[1348] Step 4:

[1349] The server executes virtual matches based on the match settings specified by the user. In this process, matches between organizations from different eras are simulated, and results and statistical data for each match are generated, resulting in the results of the virtual matches.

[1350] Input: Trained machine learning model, user match settings

[1351] Output: Simulation results of a virtual match

[1352] Step 5:

[1353] The server and device detect the user's facial expressions, tone of voice, and body movements to recognize emotions in real time. Data acquired from the user's webcam and microphone is analyzed to extract emotional indicators. This allows the user's emotional data to be acquired.

[1354] Input: User's facial expression data, voice data

[1355] Output: User emotion data

[1356] Step 6:

[1357] The server integrates the simulation results with the user's emotional data to generate visualized results, including the progress of the match and key play moments, which are then sent to the Metaverse platform for users to view within the virtual environment.

[1358] Input: Simulation results of virtual matches, user emotion data

[1359] Output: Visualized match results data

[1360] Step 7:

[1361] The device displays the visualized results in a virtual environment, allowing users to watch the game in a metaverse stadium. Based on the user's emotional data, the device dynamically adjusts the viewpoint and display information, enhancing the user's sense of immersion and providing a personalized viewing experience.

[1362] Input: Visualized match results data

[1363] Output: The match viewing experience shown to the user

[1364] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1365] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1366] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1367] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1368] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1369] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1370] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1371] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1372] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1373] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1374] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1375] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1376] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1377] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1378] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1379] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1380] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1381] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1382] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1383] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1384] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1385] The following is further disclosed regarding the above embodiment.

[1386] (Claim 1)

[1387] a means for obtaining match data;

[1388] a means for collecting player and team performance data;

[1389] a means for formatting and cleansing the collected data;

[1390] A means for extracting features based on the data and training a machine learning model;

[1391] A means to run simulations that pit teams from different eras against each other virtually;

[1392] a means for visualizing the simulation results;

[1393] means for delivering said visualized results to a virtual environment;

[1394] means for a user to watch a match in said virtual environment;

[1395] A system including:

[1396] (Claim 2)

[1397] 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

[1398] (Claim 3)

[1399] 10. The system of claim 1, further comprising means for using regression, classification, and neural networks in training the machine learning model.

[1400] "Example 1"

[1401] (Claim 1)

[1402] a means for obtaining match data;

[1403] a means for collecting player and team performance data;

[1404] A means of formatting the collected data and converting it into a consistent format;

[1405] a means for cleansing the collected data; and

[1406] A means for extracting features based on the data and training a machine learning model;

[1407] A means to run simulations that pit teams from different eras against each other virtually;

[1408] a means for visualizing the simulation results;

[1409] means for delivering said visualized results to a virtual environment;

[1410] means for a user to use a terminal to watch a match in a virtual environment;

[1411] A system including:

[1412] (Claim 2)

[1413] 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

[1414] (Claim 3)

[1415] 10. The system of claim 1, further comprising means for using regression, classification, and neural networks in training the machine learning model.

[1416] "Application Example 1"

[1417] (Claim 1)

[1418] a means for obtaining match data;

[1419] a means for collecting player and team performance data;

[1420] a means for formatting and cleansing the collected data;

[1421] A means for extracting features based on the data and training a machine learning model;

[1422] A means to run simulations that pit teams from different eras against each other virtually;

[1423] a means for visualizing the simulation results;

[1424] means for delivering said visualization results to a virtual reality environment;

[1425] A way for users to watch the game in a virtual stadium and use avatars to switch perspectives and focus on specific players;

[1426] A system including:

[1427] (Claim 2)

[1428] 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

[1429] (Claim 3)

[1430] 10. The system of claim 1, further comprising means for using regression, classification, and neural networks in training the machine learning model.

[1431] "Example 2: Combining Emotion Engines"

[1432] (Claim 1)

[1433] a means for obtaining match data;

[1434] a means for collecting player and team performance data;

[1435] a means for formatting and cleansing the collected data;

[1436] A means for extracting features based on the data and training a machine learning model;

[1437] A means to run simulations that pit teams from different eras against each other virtually;

[1438] a means for visualizing the simulation results;

[1439] means for delivering said visualized results to a virtual environment;

[1440] A means for acquiring and analyzing user emotion data in real time;

[1441] means for dynamically adjusting visualization data based on the emotion data;

[1442] means for a user to watch a match in said virtual environment;

[1443] A system including:

[1444] (Claim 2)

[1445] 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

[1446] (Claim 3)

[1447] 10. The system of claim 1, further comprising means for using regression, classification, and neural networks in training the machine learning model.

[1448] "Application example 2 when combining emotion engines"

[1449] (Claim 1)

[1450] a means for obtaining match data;

[1451] a means of collecting athlete and team performance data;

[1452] a means for formatting and cleansing the collected data;

[1453] A means for extracting features based on the data and training a machine learning model;

[1454] A means of running a simulation in which organizations from different eras virtually compete against each other;

[1455] a means for visualizing the simulation results;

[1456] means for delivering said visualized results to a virtual environment;

[1457] means for a user to watch a match in said virtual environment;

[1458] means for analyzing user emotion data and dynamically adjusting the visualization results;

[1459] A means for collecting emotional data from a user using a smart device and changing the viewpoint and display information in the virtual environment based on the collected data;

[1460] A system including:

[1461] (Claim 2)

[1462] 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

[1463] (Claim 3)

[1464] 10. The system of claim 1, further comprising means for using regression, classification, and neural networks in training the machine learning model. [Explanation of symbols]

[1465] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for obtaining match data; a means for collecting player and team performance data; a means for formatting and cleansing the collected data; A means for extracting features based on the data and training a machine learning model; A means to run simulations that pit teams from different eras against each other virtually; a means for visualizing the simulation results; means for delivering said visualized results to a virtual environment; means for a user to watch a match in said virtual environment; A system including:

2. 10. The system of claim 1, further comprising means for converting the game data, player data, and team data into a consistent format.

3. The system of claim 1 , further comprising means for using regression, classification, and neural networks in training the machine learning model.

Images