System

Abstract

A system is provided.SOLUTION: The system includes a means for acquiring information from a data source, a means for analyzing the acquired information and extracting important information related to a player and a team, a means for analyzing a question in natural language from a user and generating an answer based on the extracted information, and a means for providing the generated answer to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional sports broadcasts have the problem of lacking information tailored to individual users' needs and preferences. It has been difficult to obtain detailed information on specific players or teams in real time and provide customized commentary to users. Furthermore, there has been a lack of effective means to combine information posted by individuals on blogs and social media with official data, making it difficult to improve the user experience. [Means for solving the problem]

[0005] To solve this problem, the present invention provides the following means. First, a means for acquiring information from data sources is used to collect data from official websites, local information sites, blogs, social networking services, etc. Next, a means is provided for analyzing the acquired information and extracting important information about players and teams. Next, a means is provided for analyzing questions from users in natural language and generating answers based on the extracted information. Finally, a means is provided for providing the generated answers to users, and these means are used to build a system that provides commentary customized to the user's interests and preferences in real time. In this way, it is possible to provide information that meets the individual needs of each user, improving the sports broadcast experience.

[0006] "Data source" refers to an external website or service from which information is obtained, such as an official website, a local information site, a blog, or a social networking service.

[0007] "Means of obtaining information" refers to mechanisms or algorithms for automatically collecting the required information from data sources.

[0008] "Means for analyzing information" refers to algorithms and mechanisms for analyzing acquired information and extracting and organizing necessary data.

[0009] "Means of extraction" refers to algorithms or mechanisms for selecting and extracting important data about specific players or teams from the results of information analysis.

[0010] "Means for analyzing natural language questions from users" refers to natural language processing technology that analyzes the natural language text entered by the user, understands their intent, and retrieves appropriate information.

[0011] "Means for generating answers" refers to generative AI or text generation algorithms that generate appropriate explanations and answers for users based on analyzed data.

[0012] "Means for providing answers to a user" refers to means for providing communication and an interface for displaying generated answers to a user.

[0013] "Real-time" refers to generating and providing answers to user questions instantly, with minimal delay.

[0014] "Customized commentary" refers to the provision of information specifically tailored to a user's individual needs and preferences. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention relates to a system for providing real-time commentary of live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[0037] Server Processing

[0038] 1. Data Collection:

[0039] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0040] 2. Data Analysis:

[0041] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[0042] 3. Update the database:

[0043] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[0044] 4. Question analysis:

[0045] It analyzes natural language questions from users and understands their intent. For example, if it receives a question like, "How is player X performing recently?", it generates a query to retrieve that player's recent match data.

[0046] 5. Information generation:

[0047] Based on the analyzed data, answers to user questions are generated. Generative AI is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, the generated explanation would be, "Player X has averaged 15.2 points in his last five games."

[0048] 6. Sending a response:

[0049] The generated answer is provided to the user, and the response is sent to the terminal in JSON format and displayed in the user interface (UI).

[0050] Terminal handling

[0051] 1. User Interface (UI):

[0052] The device displays a live sports broadcast screen and a question input field. The UI is designed to be intuitive, allowing users to easily input questions.

[0053] 2. Submit your request:

[0054] The question entered by the user is converted into JSON format and sent to the server.

[0055] 3. Receive and display the response:

[0056] It receives the response from the server, parses it, and converts it into a displayable format. For example, it displays the answer "Player X's average points this season is 15.2 points."

[0057] User processing

[0058] 1. Enter your question:

[0059] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0060] 2. Verify the information:

[0061] The server response will be displayed on your device, and you can enter additional questions if necessary.

[0062] 3. Feedback:

[0063] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[0064] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of users, providing a significantly improved user experience compared to traditional broadcasting.

[0065] The processing flow will be explained below.

[0066] Server Processing

[0067] Step 1: Data collection

[0068] 1. Configure the data source:

[0069] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[0070] 2. Send API request:

[0071] The server periodically sends requests to the API of the configured data source.

[0072] 3. Data Acquisition:

[0073] The server receives the returned data in JSON or XML format.

[0074] Example: Get the latest match results from the official website.

[0075] Step 2: Data analysis

[0076] 1. Data Parsing:

[0077] The server parses the retrieved data and converts it into structured data.

[0078] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[0079] 2. Data Formatting:

[0080] The server formats the parsed data and filters out the necessary information.

[0081] Example: Extracting positive comments about a specific player from social media comments.

[0082] 3. Update the database:

[0083] The server stores and updates the formatted data in the database.

[0084] Example: Updating player performance data with the latest match results.

[0085] Step 3: Analyze user questions

[0086] 1. User request received:

[0087] The server receives a user's question request sent from the terminal.

[0088] Example: You receive a question such as, "What is player X's average score this season?"

[0089] 2. Natural language analysis:

[0090] The server analyzes the received question using natural language processing technology and identifies the required information.

[0091] Example: Extract keywords to identify "X player's average points this season."

[0092] Step 4: Generate and execute a data query

[0093] 1. Query Generation:

[0094] The server generates a query to a database based on the question.

[0095] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0096] 2. Query execution:

[0097] The server executes the generated query against the database to obtain the required information.

[0098] Example: Retrieve player X's average points this season from the database.

[0099] Step 5: Information generation and transmission

[0100] 1. Information generation:

[0101] The server generates answers to the user's questions based on the acquired data.

[0102] Example: Generates the text "Player X's average points this season is 15.2 points."

[0103] 2. Send response:

[0104] The server sends the generated answer to the terminal as a response.

[0105] Terminal handling

[0106] Step 1: User Interface (UI)

[0107] 1. UI initialization:

[0108] The device displays a live sports broadcast screen and a question input field.

[0109] 2. User gesture detection:

[0110] The device detects when the user enters text into the input field.

[0111] Step 2: Submitting a request

[0112] 1. Enter your question:

[0113] The user types in a specific question.

[0114] For example, enter "What is player X's average points this season?"

[0115] 2. Request Generation:

[0116] The device converts the user's question into JSON format and generates a request.

[0117] 3. Submit your request:

[0118] The terminal sends the generated request to the server.

[0119] Step 3: Receive and display the response

[0120] 1. Response received:

[0121] The terminal receives the response from the server.

[0122] 2. Response parsing:

[0123] The terminal parses the response and converts it into a displayable format.

[0124] 3. Information display:

[0125] The terminal displays the analyzed information on the screen.

[0126] Example: Display "Player X's average points this season is 15.2 points."

[0127] User Action

[0128] Step 1: Enter your question

[0129] 1. Enter your question:

[0130] While watching the game, users can enter questions about players or teams they are interested in.

[0131] For example, enter "How many points did player X score in yesterday's game?"

[0132] Step 2: Verify the information

[0133] 1. Verify the information:

[0134] The user checks the response from the server displayed on the terminal.

[0135] 2. Enter additional questions:

[0136] If desired, the user can enter further questions.

[0137] Step 3: Feedback

[0138] 1. Providing Feedback:

[0139] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[0140] Example 1

[0141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0142] In conventional sports broadcasts, it was difficult for users to obtain detailed information about the players or teams they were interested in in real time. Furthermore, there was a lack of a way to quickly and appropriately answer specific questions from users. Therefore, in order to increase user satisfaction, there is a need for more accurate information provision and faster responses.

[0143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0144] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question in natural language from a user and understanding the intent of the question, means for generating an answer using a generative AI model based on the understood intent, and means for providing the generated answer to the user. This makes it possible to provide detailed information based on the individual needs of the user in real time and respond quickly and appropriately.

[0145] "Data sources" are sources of information such as official websites, local information sites, blogs, and social networking services for collecting information.

[0146] "Means of obtaining information" refers to technologies such as APIs and RSS feeds that automatically collect the necessary information from data sources.

[0147] "Means for analyzing information" refers to natural language processing and statistical analysis techniques used to analyze collected data and extract important information about players and teams.

[0148] "Means for analyzing user questions" refers to natural language processing technology for understanding questions in natural language from users and analyzing their intent.

[0149] A "generative AI model" is an artificial intelligence model that generates appropriate answers to user questions, and specifically includes generative AI models such as GPT-3 (registered trademark).

[0150] "Means for generating answers" refers to technology that uses a generative AI model to generate answers to users' questions.

[0151] The "means for providing an answer" refers to the communication means and user interface for conveying the generated answer to the user.

[0152] The present invention relates to a system for providing real-time commentary on live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[0153] Server Processing

[0154] The server provides the technical infrastructure for retrieving information from a variety of data sources, analyzing the retrieved information, and responding to user queries. The server includes the following functions:

[0155] 1. Data Collection:

[0156] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services. For example, it retrieves the latest match results from the official website and collects fan comments from social media.

[0157] 2. Data Analysis:

[0158] The collected data is parsed and key information about players and teams is extracted using natural language processing techniques. For example, it can extract only positive opinions about a specific player from social media comments, or derive statistical data such as a player's score and performance from game results.

[0159] 3. Update the database:

[0160] The extracted information is stored and updated in a database, which allows the latest information to be maintained. Examples of databases that can be used include MySQL (registered trademark) and PostgreSQL.

[0161] 4. Question analysis:

[0162] It uses natural language processing models such as BERT and GPT-3 to analyze natural language questions from users and understand the intent of the question. For example, it analyzes the question "How is player X's recent performance?" and generates a query to retrieve the player's recent game data.

[0163] 5. Information generation:

[0164] Based on the analyzed data, answers to user questions are generated. Generative AI models such as GPT-3 are used for generation, and explanations are provided in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "player X has averaged 15.2 points in his last five games" is generated.

[0165] 6. Sending a response:

[0166] The generated answer is sent to the device in JSON format, and the response is displayed in the user interface (UI).

[0167] Terminal handling

[0168] The terminal provides an interface for users to enter questions and display answers from the server. The terminal includes the following features:

[0169] 1. User Interface (UI):

[0170] The device displays a live sports broadcast screen and a question input field. The UI is built using React.js and other tools, and is designed to make it easy for users to enter questions.

[0171] 2. Submit your request:

[0172] The question entered by the user is converted into JSON format and sent to the server. For example, if the user enters a question such as "How is player X's recent performance?", this question is sent to the server using the axios.post method.

[0173] 3. Receive and display the response:

[0174] Receives the response from the server and converts it into a displayable format. For example, if the server returns an answer such as "Player X's average score this season is 15.2 points," it displays it on the screen.

[0175] User processing

[0176] Users can watch live sports broadcasts through their devices and ask questions about information they are interested in. The following functions are included in the user-side processing:

[0177] 1. Enter your question:

[0178] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0179] 2. Verify the information:

[0180] The user sees the response from the server displayed on the device and can enter additional questions if needed.

[0181] 3. Feedback:

[0182] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[0183] Examples and prompts

[0184] For example, if a user types a question like, "How is player X performing recently?", the server collects and analyzes past game data and social media comments to generate an answer such as, "X player has averaged 15.2 points in his last five games."

[0185] Example prompt sentence:

[0186] "Please tell me about player XX's recent performance."

[0187] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of the user.

[0188] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0189] Step 1: Data collection

[0190] The server collects information from official websites, local information sites, blogs, and social networking services. As input, it provides API endpoints and RSS feed URLs, which it uses to retrieve data. As output, it obtains data in JSON or XML format. Specifically, it sends a "GET" request to the official website's API to retrieve match results and player information. It also uses the Twitter API to collect tweets containing specific hashtags.

[0191] Step 2: Data analysis

[0192] The server parses the collected data and extracts important information about players and teams. The JSON or XML data collected in Step 1 is used as input. The output is an analysis of players' performance and social media comments. Specifically, the server parses the data using the json.loads() function and uses natural language processing technology to classify comments as positive or negative. It also analyzes match results to extract players' points and assists.

[0193] Step 3: Update the database

[0194] The server stores and updates the parsed information in the database. The parsed results from step 2 are used as input. The output is to update the database with the latest information. Specifically, the SQLAlchemy library is used to execute a query to insert the latest player statistics into the database. For example, an SQL statement such as INSERT INTO player_stats (player_id, match_id, points) VALUES (...) is generated and executed.

[0195] Step 4: Question Analysis

[0196] The server analyzes natural language questions from users and understands their intent. The input is a text question from the user. The output is a query that reflects the intent of the question. Specifically, the BERT model is used to analyze the question "How is player X's recent performance?" and generate a query that identifies "player X" and asks for statistical information related to his or her "performance."

[0197] Step 5: Information Generation

[0198] The server generates an answer to the user's question based on the analyzed data. The query generated in step 4 and information obtained from the database are used as input. The output is an answer expressed in natural language. Specifically, the query result is input into a generative AI model such as GPT-3, and an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[0199] Step 6: Sending a response

[0200] The server sends the generated answer in JSON format to the terminal. The answer generated in step 5 is used as input. The output is a JSON response to be sent to the terminal. Specifically, a response is generated to return the generated answer in JSON format to the client side through Flask's API endpoint.

[0201] Step 7: User Interface (UI) Display

[0202] The device displays a sports broadcast screen and a question input field. HTML and JavaScript files are used as input. As output, an interface where users can enter questions is displayed on the screen. Specifically, React.js is used to build the question input field and broadcast screen, providing a UI that users can operate intuitively.

[0203] Step 8: Submitting the request

[0204] The device converts the question entered by the user into JSON format and sends it to the server. The text question entered by the user is used as input. An HTTP request to the server is generated as output. Specifically, the question "How has player X performed recently?" is sent to the server using the axios.post method.

[0205] Step 9: Receive and display the response

[0206] The terminal receives the response from the server and converts it into a displayable format. The JSON response returned by the server is used as input. The output is a text answer that is displayed on the screen. Specifically, the response received by axios is analyzed and the text "Player X's average score this season is 15.2 points" is displayed on the screen.

[0207] Step 10: Feedback

[0208] Users provide feedback on the information provided. The text of the feedback provided by the user is used as input. The feedback content is sent to the server as output, and is used to improve the system. Specifically, a function is provided to enter feedback such as "The information was accurate" or "The explanation was easy to understand" in an input field and send it.

[0209] (Application example 1)

[0210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] Traditional sports broadcasting systems often only provided general information, making it difficult for users to obtain real-time commentary tailored to their individual needs. Furthermore, users lacked the means to quickly obtain detailed information about specific players or teams while watching a game. This limited the user experience and reduced the enjoyment of viewing.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0213] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about people and organizations, means for analyzing a question entered in natural language by a user and generating an answer based on the extracted information, means for providing the generated answer to the user, means for providing real-time commentary about live sports broadcasts using a mobile communication device, means for converting a question entered by a user into JSON format and sending the JSON format to the server, and means for receiving, analyzing, and converting a response from the server into a displayable format, thereby enabling users to quickly obtain detailed, real-time commentary tailored to their individual needs while watching a game.

[0214] "Data sources" are information sources for automatically obtaining information from official websites, local information sites, blogs, social networking services, etc.

[0215] "Acquired information" refers to information such as match results, player and team performance data, and comments on social media collected from data sources.

[0216] "Analysis" is the act of automatically processing and analyzing acquired information and extracting necessary information and patterns.

[0217] "Person" is a term that refers to a specific individual, such as a sports player or a related person.

[0218] "Organization" is a term that refers to an organization or group, such as a sports team or association.

[0219] "Users" refer to sports spectators and viewers who use this system.

[0220] "Natural language" refers to the language that users use in their daily lives, in the form of input before it is converted into a machine-interpretable format.

[0221] A "question" is something a user inputs in natural language about the information they want to know.

[0222] "JSON format" is a type of data exchange format and is an abbreviation for JavaScript Object Notation.

[0223] "Server" means a central processing unit that collects and analyzes data and generates and provides answers to user questions.

[0224] "Mobile communication device" refers to a mobile communication device such as a smartphone or tablet.

[0225] "Real-time commentary" refers to detailed information and commentary that is available instantly while watching live sports broadcasts.

[0226] A "response" is an answer that a server generates to a user's question.

[0227] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs. The system has the function of acquiring information from data sources, analyzing the acquired information, and generating answers to user questions.

[0228] Server Processing

[0229] The server retrieves information from data sources such as the official website, local information sites, blogs, and social networking services. Information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0230] The collected information is analyzed to extract important information about individuals and organizations. Natural language processing technology is used for the analysis, making it possible to extract only positive opinions about a specific individual from social media comments, for example. Statistical data such as player scores and performance can also be extracted from match results.

[0231] The extracted information is stored and updated in a database, and the latest information is always kept in real time. For example, player performance data is updated after every game.

[0232] Questions from users are sent to the server in natural language. The server analyzes the question and understands the user's intent. For example, if the server receives a question like "How has player X performed recently?", it generates a query to retrieve the player's recent game data.

[0233] Based on the analyzed data, answers to user questions are generated. A generative AI model is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[0234] The generated answer is sent to the user's device in JSON format.

[0235] Terminal handling

[0236] The device displays a live sports broadcast screen and a question input field. The user interface (UI) is designed to be intuitive, allowing users to easily input questions.

[0237] The question entered by the user is converted into JSON format and sent to the server. The response from the server is received, parsed, and converted into a displayable format. For example, the answer displayed is "Player X's average points this season is 15.2 points."

[0238] User Actions

[0239] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?". They can also provide feedback on the information provided, which helps improve the accuracy of the system.

[0240] Examples of concrete examples and prompts

[0241] As a concrete example, if a user uses a smartphone app while watching a game to enter a question such as "How many points did player X score in yesterday's game?", the following prompt sentence will be generated:

[0242] Prompt Sentence Examples

[0243] "How many points did player X score in yesterday's game?"

[0244] Using this system, users can quickly obtain detailed, real-time commentary tailored to their individual needs while watching a sporting event, making live sports broadcasts even more enjoyable.

[0245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0246] Processing step flow

[0247] Step 1:

[0248] The server obtains information from data sources such as official websites, local information sites, blogs, and social networking services (SNS). Information is collected automatically through APIs and RSS feeds. The input is various information provided by the data sources, and the output is raw data stored in temporary data storage on the server.

[0249] Step 2:

[0250] The server analyzes the collected information and uses natural language processing technology to extract important information about individuals and organizations. For example, it can extract positive opinions about a specific individual from social media comments, or statistical data such as a player's score and performance from game results. The input is the acquired raw data, and the output is the analyzed important information.

[0251] Step 3:

[0252] The server stores and updates the extracted information in the database, which keeps the database up-to-date in real time. The input is the analyzed important information, and the output is the updated database information.

[0253] Step 4:

[0254] The device displays a live sports broadcast screen and a question input field through a user interface (UI). The user inputs a question in natural language. The input is the text question entered by the user, and the output is the question data converted to JSON format.

[0255] Step 5:

[0256] The terminal converts the input question into JSON format and sends it to the server. The converted question is sent via the API. The input is text-formatted question data, and the output is JSON-formatted question data sent to the server.

[0257] Step 6:

[0258] The server receives a question in JSON format sent by the user and uses natural language processing technology to analyze the question and understand the user's intent. The input is the question data in JSON format, and the output is the analyzed intent information.

[0259] Step 7:

[0260] The server extracts relevant data from the database based on the analyzed user intent and generates commentary in natural language using a generative AI model. For example, in response to the question "How is player X's recent performance?", it generates commentary such as "Player X has averaged 15.2 points in the last five games" based on the latest match data. The input is the analyzed intent information and related data from the database, and the output is commentary generated in natural language.

[0261] Step 8:

[0262] The server sends the generated explanation to the terminal in JSON format. The input is the generated natural language explanation, and the output is JSON format data.

[0263] Step 9:

[0264] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input is the response data in JSON format, and the output is a displayable text description.

[0265] Step 10:

[0266] The user checks the explanation displayed on the terminal and inputs additional questions as necessary. They can also provide feedback on the information provided. This feedback information is sent to the server and used to improve the accuracy of the system. The input is the displayed explanation information and feedback, and the output is increased user satisfaction and improved system performance.

[0267] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0268] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[0269] Server Processing

[0270] 1. Data Collection:

[0271] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0272] 2. Data Analysis:

[0273] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[0274] 3. Update the database:

[0275] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[0276] 4. Emotion analysis:

[0277] The server receives questions from users and facial expression data displayed on the device, and analyzes them using an emotion engine. Emotion analysis uses facial recognition and natural language processing technologies to identify emotional states from the text and facial expressions entered by the user.

[0278] Example: If a user types, "Player X was not good today either...", the emotion engine will detect "disappointment."

[0279] 5. Question analysis:

[0280] The server analyzes the user's question request received from the device, and uses natural language processing technology to understand the intent of the question and identify the required information.

[0281] Example: If a question is received such as "What is player X's average points this season?", a query will be generated to retrieve that player's game data this season.

[0282] 6. Data query generation and execution:

[0283] The server generates a query to a database based on the question to retrieve the required information.

[0284] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0285] 7. Information generation:

[0286] Based on the acquired data and the analysis results of the emotion engine, answers to user questions are generated using generative AI, which adjusts the answers according to the user's emotional state.

[0287] For example, if a user is detected as "disappointed," an encouraging commentary will be provided, such as, "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[0288] 8. Sending a response:

[0289] The generated response is formatted in JSON and sent to the terminal.

[0290] Terminal handling

[0291] 1. User Interface (UI):

[0292] The device displays a live sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[0293] 2. Submit your request:

[0294] The question entered by the user and the emotion-recognized facial expression data are converted into JSON format and sent to the server.

[0295] 3. Receive and display the response:

[0296] Receives responses from the server, analyzes them, and converts them into a displayable format. Displays answers based on the analyzed information and emotions on the screen.

[0297] User Action

[0298] 1. Enter your question:

[0299] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0300] 2. Facial expression recognition:

[0301] The user's facial expression data is automatically collected through the camera and analyzed by the emotion engine.

[0302] 3. Verify the information:

[0303] Check the response from the server displayed on the device. For example, it may say, "Player X's average points this season is 15.2 points."

[0304] 4. Feedback:

[0305] Users can provide feedback on the information they provide, which helps improve the system's accuracy and user experience.

[0306] In this way, the present invention can recognize user emotions and provide personalized real-time sports commentary, providing a significantly improved user experience compared to traditional broadcasts.

[0307] The processing flow will be explained below.

[0308] Server Processing

[0309] Step 1: Data collection

[0310] 1. Configure the data source:

[0311] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[0312] 2. Send API request:

[0313] The server periodically sends requests to the API of the configured data source.

[0314] 3. Data Acquisition:

[0315] The server receives the returned data in JSON or XML format.

[0316] Example: Get the latest match results from the official website.

[0317] Step 2: Data analysis

[0318] 1. Data Parsing:

[0319] The server parses the retrieved data and converts it into structured data.

[0320] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[0321] 2. Data Formatting:

[0322] The server formats the parsed data and filters out the necessary information.

[0323] Example: Extracting positive comments about a specific player from social media comments.

[0324] 3. Update the database:

[0325] The server stores and updates the formatted data in the database.

[0326] Example: Updating player performance data with the latest match results.

[0327] Step 3: Sentiment Analysis

[0328] 1. Receiving questions and facial expression data:

[0329] The server receives the user's questions and facial expression data sent from the terminal.

[0330] For example, if a user types in "How many points did player X score in yesterday's game?", the collected facial expression data will be received along with the text.

[0331] 2. Natural Language and Facial Expression Analysis:

[0332] The server analyzes the received question text and facial expression data to identify the user's emotions.

[0333] Example: Analyzing emotions such as "surprise" and "disappointment" from question text and facial expression data.

[0334] Step 4: Question Analysis

[0335] 1. Question analysis:

[0336] The server uses natural language processing technology to understand the intent of the user's question.

[0337] Example: Analyze the question "What is player X's average points this season?" and identify relevant information.

[0338] Step 5: Generate and execute a data query

[0339] 1. Query Generation:

[0340] The server generates a query to a database based on the question.

[0341] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0342] 2. Query execution:

[0343] The server executes the generated query against the database to obtain the required information.

[0344] Example: Retrieve player X's average points this season from the database.

[0345] Step 6: Information Generation

[0346] 1. Information generation:

[0347] The server generates answers to the user's questions based on the acquired data and the analysis results of the emotion engine.

[0348] For example, if a user is detected as "disappointed," an encouraging commentary will be generated, such as "Player X hasn't been in good form recently, but his average score this season is 15.2 points, which is still the highest on the team."

[0349] Step 7: Sending a response

[0350] 1. Response formatting:

[0351] The server formats the generated response in JSON format.

[0352] 2. Send response:

[0353] The server sends the formatted response to the terminal.

[0354] Terminal handling

[0355] Step 1: User Interface (UI)

[0356] 1. UI initialization:

[0357] The device displays a sports broadcast screen, a question input field, and a camera for facial recognition.

[0358] 2. User gesture detection:

[0359] The device uses a camera to detect the user's movements and facial expressions as they enter text into an input field.

[0360] Step 2: Submitting a request

[0361] 1. Enter your question:

[0362] The user types in a specific question.

[0363] For example, enter "What is player X's average points this season?"

[0364] 2. Acquiring facial expression data:

[0365] The device collects facial expression data from the user through the camera.

[0366] 3. Request Generation:

[0367] The device converts the user's question and facial expression data into JSON format to generate a request.

[0368] 4. Submit your request:

[0369] The terminal sends the generated request to the server.

[0370] Step 3: Receive and display the response

[0371] 1. Response received:

[0372] The terminal receives the response from the server.

[0373] 2. Response parsing:

[0374] The terminal parses the response and converts it into a displayable format.

[0375] 3. Information display:

[0376] The terminal displays the analyzed information on the screen.

[0377] Example: Display "Player X's average points this season is 15.2 points."

[0378] User Action

[0379] Step 1: Enter your question

[0380] 1. Enter your question:

[0381] While watching the game, users can enter questions about players or teams they are interested in.

[0382] For example, enter "How many points did player X score in yesterday's game?"

[0383] Step 2: Facial Expression Recognition

[0384] 1. Facial expression recognition:

[0385] The user's facial expression data is automatically collected through the camera.

[0386] Step 3: Verify the information

[0387] 1. Verify the information:

[0388] The user checks the response from the server displayed on the terminal.

[0389] 2. Enter additional questions:

[0390] If desired, the user can enter further questions.

[0391] Step 4: Feedback

[0392] 1. Providing Feedback:

[0393] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[0394] Example 2

[0395] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0396] Conventional sports broadcasting systems have difficulty providing real-time commentary that responds to the individual needs and emotional state of users. This leads to a poor user experience and difficulty in improving satisfaction. Another issue is that answers to questions entered by users are standardized, making it difficult to provide personalized responses.

[0397] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for identifying an emotional state by analyzing facial expression data of a user, means for analyzing a question from the user in natural language and generating an answer based on the extracted information and the identified emotional state, and means for providing the generated answer to the user. This makes it possible to provide real-time sports commentary that meets the individual needs and emotional state of each user.

[0398] "Data sources" are information sources for obtaining information from official websites, local information sites, blogs, social networking services, etc.

[0399] A "means of obtaining information" is a mechanism for automatically gathering information from data sources, such as through an API or RSS feed.

[0400] "Means of analyzing information" is the process of parsing the collected information and extracting key data about players and teams.

[0401] "Facial expression data" is image data used to capture a user's facial expression, and is basic data for analyzing the user's emotions.

[0402] "Emotional state" refers to the user's emotional state, such as disappointment or excitement, analyzed based on the user's facial expression data and natural language input.

[0403] "Means for question analysis" refers to technology for understanding natural language questions from users and identifying their intent.

[0404] The "answer generation means" is the process for creating an answer to the user's question based on the extracted information and the identified emotional state.

[0405] A "means for providing an answer" is a mechanism for communicating and displaying the generated answer to the user.

[0406] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[0407] Server Processing

[0408] Data collection:

[0409] The server collects information from data sources such as the official website, local information sites, blogs, and social networking services (SNS). This process uses APIs and RSS feeds. For example, the latest match results are obtained from the official website, and fan comments are collected from SNS.

[0410] Data Analysis:

[0411] The server analyzes the collected data and extracts important information about players and teams, using natural language processing technologies such as Apache OpenNLP and Google Cloud Natural Language API to extract positive opinions and statistical data from social media comments.

[0412] Database update:

[0413] The server stores the extracted information in a database and updates it in real time. For example, player performance data is updated after each game. The database uses MySQL or PostgreSQL.

[0414] Emotion analysis:

[0415] The server receives the questions and facial expression data sent by the user and analyzes them with the emotion engine. Using Azure® Face API and Amazon Rekognition, the server identifies the user's emotional state using facial recognition and natural language processing technology. For example, if a user types, "Player X, you're not doing so well today...", the emotion engine will detect "disappointment."

[0416] Question analysis:

[0417] The server uses natural language processing technology to analyze the user's question. For example, if it receives a question like "What is player X's average score this season?", it converts the question into a database query.

[0418] Data query generation and execution:

[0419] The server generates a query to the database based on the question and retrieves the information. Specifically, it generates and executes the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0420] Information generation:

[0421] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The answers are adjusted according to the user's emotional state. For example, if the user is disappointed, an encouraging answer such as, "It's true that player X hasn't been playing well recently, but his average score of 15.2 points this season is still the highest on the team" will be generated.

[0422] Response sent:

[0423] The server formats the generated response in JSON format and sends it to the terminal.

[0424] Terminal handling

[0425] User Interface (UI):

[0426] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[0427] Submit a request:

[0428] The device converts the question entered by the user and the emotion-recognized facial expression data into JSON format and sends it to the server.

[0429] Response received and displayed:

[0430] The device receives the response from the server, analyzes it, and converts it into a displayable format. It then displays an answer based on the analyzed information and emotions on the screen.

[0431] User Action

[0432] Enter your question:

[0433] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0434] Facial expression recognition:

[0435] The device's camera captures the user's face and sends the image data to the emotion engine in real time.

[0436] Verify the information:

[0437] The user checks the response from the server displayed on the device, for example, "Player X's average points this season is 15.2 points."

[0438] feedback:

[0439] Users can provide feedback on the information provided, such as whether it was helpful or not, which improves the accuracy of the system and the user experience.

[0440] Examples and prompts

[0441] For example, if a user asks, "When is player X's next game scheduled?", the server retrieves the date of the next game and information about the opposing team from the database and generates a positive response through the emotion engine.

[0442] Example prompt sentence:

[0443] "Provide information about a player's next game schedule and opposing team based on the user's question and facial expression. Example question: 'When is player X's next game scheduled?' Facial expression: Excitement"

[0444] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0445] Step 1: Data collection

[0446] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services (SNS). For example, the server accesses endpoints such as "http: / / officialsite.com / api / results" and "http: / / sns.com / api / comments" to retrieve match results and fan comments. The input data is in XML or JSON format, and the output data is saved in the server's storage.

[0447] Step 2: Data analysis

[0448] The server analyzes the collected data and extracts important information related to players and teams. It uses Apache OpenNLP and Google Cloud Natural Language API to parse the text data and extract positive opinions and statistical data. The input data is the raw data collected in the previous step, and the output data is the parsed structured data. For example, from a comment such as "Player X played well today," the server extracts the name of "Player X" and the sentiment "positive."

[0449] Step 3: Update the database

[0450] The server stores the analyzed data in a database such as MySQL or PostgreSQL. Based on the analysis results, it executes INSERT and UPDATE statements to update information such as player performance and fan comments. The input data is the analysis result of the previous step, and the output data is the updated database. For example, the data is stored by executing the SQL statement "INSERT INTO performance (player, date, points) VALUES ('〇〇', '2023-10-01', 25)".

[0451] Step 4: Sentiment Analysis

[0452] The server receives questions and facial expression data sent by users and analyzes them using an emotion engine. Azure Face API and Amazon Rekognition are used for facial recognition technology, and natural language processing technology is used to identify emotions from the user's text input. The input data is the user's question and facial expression image data, and the output data is the analyzed emotional state. For example, an input such as "Player X is doing poorly again today..." is identified as the emotion of "disappointment."

[0453] Step 5: Question Analysis

[0454] The server uses natural language processing technology to analyze natural language questions from users. The input data is the user's question text, and the output data is structured data that identifies the intent of the question. For example, the server analyzes the question "What is player X's average score this season?" and extracts the information "player='X' AND season='current'".

[0455] Step 6: Generate and execute a data query

[0456] The server generates a query to the database based on the question and retrieves the information. The input data is the result of question analysis, and the output data is the information retrieved from the database. For example, execute the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'" to retrieve the average score.

[0457] Step 7: Information Generation

[0458] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The input data is information from the database and the results of emotion analysis, and the output data is the generated answer text. For example, in response to the emotion detected as "disappointment," the server generates an answer such as "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[0459] Step 8: Sending a response

[0460] The server formats the generated answer in JSON format and sends it to the terminal. The input data is the generated answer text, and the output data is the sent JSON response.

[0461] Step 9: User Interface (UI)

[0462] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. It provides an interface that allows users to easily input questions. The input data are user actions, and the output data are the displayed UI elements.

[0463] Step 10: Submitting the request

[0464] The device converts the user's input question and emotion-recognized facial expression data into JSON format and sends it to the server. The input data is the user's question and facial expression data, and the output data is the request sent to the server.

[0465] Step 11: Receive and display the response

[0466] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input data is the JSON-formatted response from the server, and the output data is the answer text that is displayed on the screen.

[0467] Step 12: Enter your question

[0468] While watching a game, a user can input a question about a player or team that they are interested in. For example, they might input, "How many points did player X score in yesterday's game?" The input data is the user's question text, and the output data is the question data sent to the device.

[0469] Step 13: Facial Expression Recognition

[0470] The device camera captures the user's face and sends the image data to the emotion engine in real time. The input data is the user's facial expression image data, and the output data is the analyzed emotional state.

[0471] Step 14: Verify the information

[0472] The user confirms the response from the server displayed on the device. For example, "Player X's average points this season is 15.2 points." The input data is the text displayed on the screen, and the output data is the information confirmed by the user.

[0473] Step 15: Feedback

[0474] Users provide feedback on the information provided, such as "it was helpful" or "it wasn't." This allows the system to learn and improve the accuracy of answers from the next time onwards. The input data is the user's feedback, and the output data is the system's learning data.

[0475] (Application example 2)

[0476] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0477] The problem that this invention aims to solve is that conventional sports broadcast commentary systems have difficulty in providing information tailored to individual user needs, and in particular, they are unable to respond based on real-time emotion recognition. Furthermore, because they lack the functionality of an entertainment system in an autonomous vehicle, users lack a means to effectively obtain sports information while driving.

[0478] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question from a user in natural language and generating an answer based on the extracted information, means for recognizing the user's emotional state, means for adjusting and providing the generated answer according to the user's emotional state, means for capturing facial expression data of passengers in real time using an in-vehicle camera and analyzing their emotional state, and means for providing the generated answer to the user in the autonomous vehicle in voice and text. This makes it possible to provide sports information in real time that meets individual needs while taking user emotions into consideration.

[0479] "Data sources" refer to various sources of information, such as official websites, local information sites, blogs, and social networking services.

[0480] "Means of obtaining information" refers to the methods and techniques used to collect the required information from the data source.

[0481] "Means for analyzing information" refers to the mechanism for processing the acquired information and extracting important data about players and teams.

[0482] "Important information" refers to statistical data, evaluations, performance data, etc. about players and teams obtained through analysis.

[0483] A "natural language question" is a question posed by a user via free-form text or voice.

[0484] "User emotional state" refers to the user's emotions identified using facial recognition and natural language processing technologies.

[0485] "Means for recognizing emotional states" refers to technology that analyzes a user's facial expression data and text input to identify their emotions.

[0486] The "answer generation means" refers to a mechanism that generates an answer to a user's question based on the analyzed information and the perceived emotional state.

[0487] "Means for adjusting and providing" refers to methods and techniques for appropriately changing the generated answer according to the user's emotions and communicating it to the user.

[0488] An "in-car camera" is a camera device installed inside an autonomous vehicle to capture passengers' facial expressions in real time.

[0489] "Facial expression data" refers to information captured as image or video data of a user's facial expressions.

[0490] An "autonomous vehicle" is a vehicle that can drive autonomously by utilizing artificial intelligence and sensor technology.

[0491] "Voice and text delivery means" refers to methods and technologies for communicating generated answers to users using speech synthesis and display technologies.

[0492] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and also combines it with an emotion engine that recognizes user emotions. It is particularly applicable to entertainment systems that allow users to comfortably receive sports information while driving in autonomous vehicles.

[0493] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. Information collected from data sources is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0494] The server then analyzes the collected data to extract key information about players and teams. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. It also extracts statistical data such as players' scores and performance from game results. This is done using technologies such as Hugging Face's Transformers and the Google Natural Language API.

[0495] The analyzed information is stored and updated in a database. Then, questions in natural language from users are analyzed. The server analyzes the user's question request received from the device, identifies the required information, and understands the intent of the question. Natural language processing technology is also used in this process.

[0496] Furthermore, the server recognizes the user's emotional state. It uses an in-car camera to capture facial expression data of passengers in the autonomous vehicle in real time and analyzes their emotions using facial recognition technology. Based on the analysis results, it generates customized answers using a generative AI model (e.g., OpenAI's GPT-4).

[0497] The generated answers are adjusted appropriately according to the user's emotional state and are provided to the user in the autonomous vehicle via voice and text. By using speech synthesis and display technologies, real-time sports commentary is realized.

[0498] This system takes into consideration the user's emotions and makes it possible to provide real-time sports information tailored to individual needs. For example, if a user inputs a question such as "How many points did player X score in the current game?" and the system recognizes the user's emotional state as disappointment, it will provide commentary that includes encouragement, such as "Player X has currently scored 15 points in the game. He may not be in good form right now, but let's look forward to his future performances!"

[0499] Specific examples of prompt sentences are as follows:

[0500] example:

[0501] text

[0502] Question: How many points did player X score in the game?

[0503] Emotion: disappointment

[0504] Answer: Player XX is currently scoring 15 points per game. He may not be in good form right now, but let's look forward to his future performance!

[0505] In this way, the present invention can improve the user experience in an autonomous vehicle and provide comfortable live sports commentary.

[0506] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0507] Step 1:

[0508] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. The collected information is automatically retrieved through APIs and RSS feeds. For example, the latest match results can be obtained from the official website, and fan comments can be collected from social media. The input is data from the data sources, and the output is the collected raw data.

[0509] Step 2:

[0510] The server analyzes the collected raw data and extracts important information about players and teams. The analysis is performed using natural language processing techniques such as Hugging Face's Transformers and Google Natural Language API. The input is the collected raw data, and the output is the analyzed important information (e.g., player scores and performance).

[0511] Step 3:

[0512] The server stores and updates the analyzed information in a database, ensuring that the latest information is always available in real time. The input is the analyzed important information, and the output is an updated database.

[0513] Step 4:

[0514] The user inputs a question in natural language or asks it by voice. For example, a question like, "How many points did player X score in the current game?" The input is the user's question in natural language, and the output is the text data of the question.

[0515] Step 5:

[0516] The terminal captures the user's facial expression data in real time using an in-car camera. The captured image data is sent to a server. The input is the captured facial expression data, and the output is the facial expression image data.

[0517] Step 6:

[0518] The server analyzes the user's question request and facial expression image data received from the device. Facial expression image data is analyzed using facial recognition technology, and natural language processing technology is used to understand the intent of the question. For example, in response to the question "How many points did player X score in the current game?", a query is generated to obtain the score information of a specific player. The input is the question request and facial expression image data, and the output is the analysis result and the query.

[0519] Step 7:

[0520] The server retrieves the necessary information from the database based on the generated query, for example, the latest scores and performance data of a player. The input is the query, and the output is the retrieved information.

[0521] Step 8:

[0522] The server generates answers to the user's questions based on the emotion engine's analysis results and the acquired information. Generative AI such as OpenAI's GPT-4 is used for generation. If the user is disappointed, an answer containing encouragement is generated. The input is the emotion analysis results and the acquired information, and the output is the generated answer.

[0523] Step 9:

[0524] The server formats the generated answer in JSON format and sends it to the terminal. The terminal parses the received JSON format data, converts it into a displayable format, and provides it to the user in voice and text. The input is the generated answer, and the output is the voice and text format answer to the user.

[0525] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0526] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0527] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0528] [Second embodiment]

[0529] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0530] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0531] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0532] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0533] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0534] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0535] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0536] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0537] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0538] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0539] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0540] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0541] The present invention relates to a system for providing real-time commentary of live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[0542] Server Processing

[0543] 1. Data Collection:

[0544] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0545] 2. Data Analysis:

[0546] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[0547] 3. Update the database:

[0548] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[0549] 4. Question analysis:

[0550] It analyzes natural language questions from users and understands their intent. For example, if it receives a question like, "How is player X performing recently?", it generates a query to retrieve that player's recent match data.

[0551] 5. Information generation:

[0552] Based on the analyzed data, answers to user questions are generated. Generative AI is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, the generated explanation would be, "Player X has averaged 15.2 points in his last five games."

[0553] 6. Sending a response:

[0554] The generated answer is provided to the user, and the response is sent to the terminal in JSON format and displayed in the user interface (UI).

[0555] Terminal handling

[0556] 1. User Interface (UI):

[0557] The device displays a live sports broadcast screen and a question input field. The UI is designed to be intuitive, allowing users to easily input questions.

[0558] 2. Submit your request:

[0559] The question entered by the user is converted into JSON format and sent to the server.

[0560] 3. Receive and display the response:

[0561] It receives the response from the server, parses it, and converts it into a displayable format. For example, it displays the answer "Player X's average points this season is 15.2 points."

[0562] User processing

[0563] 1. Enter your question:

[0564] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0565] 2. Verify the information:

[0566] The server response will be displayed on your device, and you can enter additional questions if necessary.

[0567] 3. Feedback:

[0568] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[0569] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of users, providing a significantly improved user experience compared to traditional broadcasting.

[0570] The processing flow will be explained below.

[0571] Server Processing

[0572] Step 1: Data collection

[0573] 1. Configure the data source:

[0574] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[0575] 2. Send API request:

[0576] The server periodically sends requests to the API of the configured data source.

[0577] 3. Data Acquisition:

[0578] The server receives the returned data in JSON or XML format.

[0579] Example: Get the latest match results from the official website.

[0580] Step 2: Data analysis

[0581] 1. Data Parsing:

[0582] The server parses the retrieved data and converts it into structured data.

[0583] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[0584] 2. Data Formatting:

[0585] The server formats the parsed data and filters out the necessary information.

[0586] Example: Extracting positive comments about a specific player from social media comments.

[0587] 3. Update the database:

[0588] The server stores and updates the formatted data in the database.

[0589] Example: Updating player performance data with the latest match results.

[0590] Step 3: Analyze user questions

[0591] 1. User request received:

[0592] The server receives a user's question request sent from the terminal.

[0593] Example: You receive a question such as, "What is player X's average score this season?"

[0594] 2. Natural language analysis:

[0595] The server analyzes the received question using natural language processing technology and identifies the required information.

[0596] Example: Extract keywords to identify "X player's average points this season."

[0597] Step 4: Generate and execute a data query

[0598] 1. Query Generation:

[0599] The server generates a query to a database based on the question.

[0600] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0601] 2. Query execution:

[0602] The server executes the generated query against the database to obtain the required information.

[0603] Example: Retrieve player X's average points this season from the database.

[0604] Step 5: Information generation and transmission

[0605] 1. Information generation:

[0606] The server generates answers to the user's questions based on the acquired data.

[0607] Example: Generates the text "Player X's average points this season is 15.2 points."

[0608] 2. Send response:

[0609] The server sends the generated answer to the terminal as a response.

[0610] Terminal handling

[0611] Step 1: User Interface (UI)

[0612] 1. UI initialization:

[0613] The device displays a live sports broadcast screen and a question input field.

[0614] 2. User gesture detection:

[0615] The device detects when the user enters text into the input field.

[0616] Step 2: Submitting a request

[0617] 1. Enter your question:

[0618] The user types in a specific question.

[0619] For example, enter "What is player X's average points this season?"

[0620] 2. Request Generation:

[0621] The device converts the user's question into JSON format and generates a request.

[0622] 3. Submit your request:

[0623] The terminal sends the generated request to the server.

[0624] Step 3: Receive and display the response

[0625] 1. Response received:

[0626] The terminal receives the response from the server.

[0627] 2. Response parsing:

[0628] The terminal parses the response and converts it into a displayable format.

[0629] 3. Information display:

[0630] The terminal displays the analyzed information on the screen.

[0631] Example: Display "Player X's average points this season is 15.2 points."

[0632] User Action

[0633] Step 1: Enter your question

[0634] 1. Enter your question:

[0635] While watching the game, users can enter questions about players or teams they are interested in.

[0636] For example, enter "How many points did player X score in yesterday's game?"

[0637] Step 2: Verify the information

[0638] 1. Verify the information:

[0639] The user checks the response from the server displayed on the terminal.

[0640] 2. Enter additional questions:

[0641] If desired, the user can enter further questions.

[0642] Step 3: Feedback

[0643] 1. Providing Feedback:

[0644] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[0645] Example 1

[0646] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0647] In conventional sports broadcasts, it was difficult for users to obtain detailed information about the players or teams they were interested in in real time. Furthermore, there was a lack of a way to quickly and appropriately answer specific questions from users. Therefore, in order to increase user satisfaction, there is a need for more accurate information provision and faster responses.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0649] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question in natural language from a user and understanding the intent of the question, means for generating an answer using a generative AI model based on the understood intent, and means for providing the generated answer to the user. This makes it possible to provide detailed information based on the individual needs of the user in real time and respond quickly and appropriately.

[0650] "Data sources" are sources of information such as official websites, local information sites, blogs, and social networking services for collecting information.

[0651] "Means of obtaining information" refers to technologies such as APIs and RSS feeds that automatically collect the necessary information from data sources.

[0652] "Means for analyzing information" refers to natural language processing and statistical analysis techniques used to analyze collected data and extract important information about players and teams.

[0653] "Means for analyzing user questions" refers to natural language processing technology for understanding questions in natural language from users and analyzing their intent.

[0654] A "generative AI model" is an artificial intelligence model that generates appropriate answers to user questions, and specifically includes generative AI models such as GPT-3.

[0655] "Means for generating answers" refers to technology that uses a generative AI model to generate answers to users' questions.

[0656] The "means for providing an answer" refers to the communication means and user interface for conveying the generated answer to the user.

[0657] The present invention relates to a system for providing real-time commentary on live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[0658] Server Processing

[0659] The server provides the technical infrastructure for retrieving information from a variety of data sources, analyzing the retrieved information, and responding to user queries. The server includes the following functions:

[0660] 1. Data Collection:

[0661] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services. For example, it retrieves the latest match results from the official website and collects fan comments from social media.

[0662] 2. Data Analysis:

[0663] The collected data is parsed and key information about players and teams is extracted using natural language processing techniques. For example, it can extract only positive opinions about a specific player from social media comments, or derive statistical data such as a player's score and performance from game results.

[0664] 3. Update the database:

[0665] The extracted information is stored and updated in a database, which allows the latest information to be kept up to date. Examples of databases that can be used include MySQL and PostgreSQL.

[0666] 4. Question analysis:

[0667] It uses natural language processing models such as BERT and GPT-3 to analyze natural language questions from users and understand the intent of the question. For example, it analyzes the question "How is player X's recent performance?" and generates a query to retrieve the player's recent game data.

[0668] 5. Information generation:

[0669] Based on the analyzed data, answers to user questions are generated. Generative AI models such as GPT-3 are used for generation, and explanations are provided in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "player X has averaged 15.2 points in his last five games" is generated.

[0670] 6. Sending a response:

[0671] The generated answer is sent to the device in JSON format, and the response is displayed in the user interface (UI).

[0672] Terminal handling

[0673] The terminal provides an interface for users to enter questions and display answers from the server. The terminal includes the following features:

[0674] 1. User Interface (UI):

[0675] The device displays a live sports broadcast screen and a question input field. The UI is built using React.js and other tools, and is designed to make it easy for users to enter questions.

[0676] 2. Submit your request:

[0677] The question entered by the user is converted into JSON format and sent to the server. For example, if the user enters a question such as "How is player X's recent performance?", this question is sent to the server using the axios.post method.

[0678] 3. Receive and display the response:

[0679] Receives the response from the server and converts it into a displayable format. For example, if the server returns an answer such as "Player X's average score this season is 15.2 points," it displays it on the screen.

[0680] User processing

[0681] Users can watch live sports broadcasts through their devices and ask questions about information they are interested in. The following functions are included in the user-side processing:

[0682] 1. Enter your question:

[0683] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0684] 2. Verify the information:

[0685] The user sees the response from the server displayed on the device and can enter additional questions if needed.

[0686] 3. Feedback:

[0687] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[0688] Examples and prompts

[0689] For example, if a user types a question like, "How is player X performing recently?", the server collects and analyzes past game data and social media comments to generate an answer such as, "X player has averaged 15.2 points in his last five games."

[0690] Example prompt sentence:

[0691] "Please tell me about player XX's recent performance."

[0692] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of the user.

[0693] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0694] Step 1: Data collection

[0695] The server collects information from official websites, local information sites, blogs, and social networking services. As input, it provides API endpoints and RSS feed URLs, which it uses to retrieve data. As output, it obtains data in JSON or XML format. Specifically, it sends a "GET" request to the official website's API to retrieve match results and player information. It also uses the Twitter API to collect tweets containing specific hashtags.

[0696] Step 2: Data analysis

[0697] The server parses the collected data and extracts important information about players and teams. The JSON or XML data collected in Step 1 is used as input. The output is an analysis of players' performance and social media comments. Specifically, the server parses the data using the json.loads() function and uses natural language processing technology to classify comments as positive or negative. It also analyzes match results to extract players' points and assists.

[0698] Step 3: Update the database

[0699] The server stores and updates the parsed information in the database. The parsed results from step 2 are used as input. The output is to update the database with the latest information. Specifically, the SQLAlchemy library is used to execute a query to insert the latest player statistics into the database. For example, an SQL statement such as INSERT INTO player_stats (player_id, match_id, points) VALUES (...) is generated and executed.

[0700] Step 4: Question Analysis

[0701] The server analyzes natural language questions from users and understands their intent. The input is a text question from the user. The output is a query that reflects the intent of the question. Specifically, the BERT model is used to analyze the question "How is player X's recent performance?" and generate a query that identifies "player X" and asks for statistical information related to his or her "performance."

[0702] Step 5: Information Generation

[0703] The server generates an answer to the user's question based on the analyzed data. The query generated in step 4 and information obtained from the database are used as input. The output is an answer expressed in natural language. Specifically, the query result is input into a generative AI model such as GPT-3, and an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[0704] Step 6: Sending a response

[0705] The server sends the generated answer in JSON format to the terminal. The answer generated in step 5 is used as input. The output is a JSON response to be sent to the terminal. Specifically, a response is generated to return the generated answer in JSON format to the client side through Flask's API endpoint.

[0706] Step 7: User Interface (UI) Display

[0707] The device displays a sports broadcast screen and a question input field. HTML and JavaScript files are used as input. An interface where users can enter questions is displayed on the screen as output. Specifically, React.js is used to build the question input field and broadcast screen, providing a UI that users can operate intuitively.

[0708] Step 8: Submitting the request

[0709] The device converts the question entered by the user into JSON format and sends it to the server. The text question entered by the user is used as input. An HTTP request to the server is generated as output. Specifically, the question "How has player X performed recently?" is sent to the server using the axios.post method.

[0710] Step 9: Receive and display the response

[0711] The terminal receives the response from the server and converts it into a displayable format. The JSON response returned by the server is used as input. The output is a text answer that is displayed on the screen. Specifically, the response received by axios is analyzed and the text "Player X's average score this season is 15.2 points" is displayed on the screen.

[0712] Step 10: Feedback

[0713] Users provide feedback on the information provided. The text of the feedback provided by the user is used as input. The feedback content is sent to the server as output, and is used to improve the system. Specifically, a function is provided to enter feedback such as "The information was accurate" or "The explanation was easy to understand" in an input field and send it.

[0714] (Application example 1)

[0715] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0716] Traditional sports broadcasting systems often only provided general information, making it difficult for users to obtain real-time commentary tailored to their individual needs. Furthermore, users lacked the means to quickly obtain detailed information about specific players or teams while watching a game. This limited the user experience and reduced the enjoyment of viewing.

[0717] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0718] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about people and organizations, means for analyzing a question entered in natural language by a user and generating an answer based on the extracted information, means for providing the generated answer to the user, means for providing real-time commentary about live sports broadcasts using a mobile communication device, means for converting a question entered by a user into JSON format and sending the JSON format to the server, and means for receiving, analyzing, and converting a response from the server into a displayable format, thereby enabling users to quickly obtain detailed, real-time commentary tailored to their individual needs while watching a game.

[0719] "Data sources" are information sources for automatically obtaining information from official websites, local information sites, blogs, social networking services, etc.

[0720] "Acquired information" refers to information such as match results, player and team performance data, and comments on social media collected from data sources.

[0721] "Analysis" is the act of automatically processing and analyzing acquired information and extracting necessary information and patterns.

[0722] "Person" is a term that refers to a specific individual, such as a sports player or a related person.

[0723] "Organization" is a term that refers to an organization or group, such as a sports team or association.

[0724] "Users" refer to sports spectators and viewers who use this system.

[0725] "Natural language" refers to the language that users use in their daily lives, in the form of input before it is converted into a machine-interpretable format.

[0726] A "question" is something a user inputs in natural language about the information they want to know.

[0727] "JSON format" is a type of data exchange format and is an abbreviation for JavaScript Object Notation.

[0728] "Server" means a central processing unit that collects and analyzes data and generates and provides answers to user questions.

[0729] "Mobile communication device" refers to a mobile communication device such as a smartphone or tablet.

[0730] "Real-time commentary" refers to detailed information and commentary that is available instantly while watching live sports broadcasts.

[0731] A "response" is an answer that a server generates to a user's question.

[0732] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs. The system has the function of acquiring information from data sources, analyzing the acquired information, and generating answers to user questions.

[0733] Server Processing

[0734] The server retrieves information from data sources such as the official website, local information sites, blogs, and social networking services. Information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0735] The collected information is analyzed to extract important information about individuals and organizations. Natural language processing technology is used for the analysis, making it possible to extract only positive opinions about a specific individual from social media comments, for example. Statistical data such as player scores and performance can also be extracted from match results.

[0736] The extracted information is stored and updated in a database, and the latest information is always kept in real time. For example, player performance data is updated after every game.

[0737] Questions from users are sent to the server in natural language. The server analyzes the question and understands the user's intent. For example, if the server receives a question like "How has player X performed recently?", it generates a query to retrieve the player's recent game data.

[0738] Based on the analyzed data, answers to user questions are generated. A generative AI model is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[0739] The generated answer is sent to the user's device in JSON format.

[0740] Terminal handling

[0741] The device displays a live sports broadcast screen and a question input field. The user interface (UI) is designed to be intuitive, allowing users to easily input questions.

[0742] The question entered by the user is converted into JSON format and sent to the server. The response from the server is received, parsed, and converted into a displayable format. For example, the answer displayed is "Player X's average points this season is 15.2 points."

[0743] User Actions

[0744] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?". They can also provide feedback on the information provided, which helps improve the accuracy of the system.

[0745] Examples of concrete examples and prompts

[0746] As a concrete example, if a user uses a smartphone app while watching a game to enter a question such as "How many points did player X score in yesterday's game?", the following prompt sentence will be generated:

[0747] Prompt Sentence Examples

[0748] "How many points did player X score in yesterday's game?"

[0749] Using this system, users can quickly obtain detailed, real-time commentary tailored to their individual needs while watching a sporting event, making live sports broadcasts even more enjoyable.

[0750] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0751] Processing step flow

[0752] Step 1:

[0753] The server obtains information from data sources such as official websites, local information sites, blogs, and social networking services (SNS). Information is collected automatically through APIs and RSS feeds. The input is various information provided by the data sources, and the output is raw data stored in temporary data storage on the server.

[0754] Step 2:

[0755] The server analyzes the collected information and uses natural language processing technology to extract important information about individuals and organizations. For example, it can extract positive opinions about a specific individual from social media comments, or statistical data such as a player's score and performance from game results. The input is the acquired raw data, and the output is the analyzed important information.

[0756] Step 3:

[0757] The server stores and updates the extracted information in the database, which keeps the database up-to-date in real time. The input is the analyzed important information, and the output is the updated database information.

[0758] Step 4:

[0759] The device displays a live sports broadcast screen and a question input field through a user interface (UI). The user inputs a question in natural language. The input is the text question entered by the user, and the output is the question data converted to JSON format.

[0760] Step 5:

[0761] The terminal converts the input question into JSON format and sends it to the server. The converted question is sent via the API. The input is text-formatted question data, and the output is JSON-formatted question data sent to the server.

[0762] Step 6:

[0763] The server receives a question in JSON format sent by the user and uses natural language processing technology to analyze the question and understand the user's intent. The input is the question data in JSON format, and the output is the analyzed intent information.

[0764] Step 7:

[0765] The server extracts relevant data from the database based on the analyzed user intent and generates commentary in natural language using a generative AI model. For example, in response to the question "How is player X's recent performance?", it generates commentary such as "Player X has averaged 15.2 points in the last five games" based on the latest match data. The input is the analyzed intent information and related data from the database, and the output is commentary generated in natural language.

[0766] Step 8:

[0767] The server sends the generated explanation to the terminal in JSON format. The input is the generated natural language explanation, and the output is JSON format data.

[0768] Step 9:

[0769] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input is the response data in JSON format, and the output is a displayable text description.

[0770] Step 10:

[0771] The user checks the explanation displayed on the terminal and inputs additional questions as necessary. They can also provide feedback on the information provided. This feedback information is sent to the server and used to improve the accuracy of the system. The input is the displayed explanation information and feedback, and the output is increased user satisfaction and improved system performance.

[0772] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0773] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[0774] Server Processing

[0775] 1. Data Collection:

[0776] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0777] 2. Data Analysis:

[0778] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[0779] 3. Update the database:

[0780] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[0781] 4. Emotion analysis:

[0782] The server receives questions from users and facial expression data displayed on the device, and analyzes them using an emotion engine. Emotion analysis uses facial recognition and natural language processing technologies to identify emotional states from the text and facial expressions entered by the user.

[0783] Example: If a user types, "Player X was not good today either...", the emotion engine will detect "disappointment."

[0784] 5. Question analysis:

[0785] The server analyzes the user's question request received from the device, and uses natural language processing technology to understand the intent of the question and identify the required information.

[0786] Example: If a question is received such as "What is player X's average points this season?", a query will be generated to retrieve that player's game data this season.

[0787] 6. Data query generation and execution:

[0788] The server generates a query to a database based on the question to retrieve the required information.

[0789] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0790] 7. Information generation:

[0791] Based on the acquired data and the analysis results of the emotion engine, answers to user questions are generated using generative AI, which adjusts the answers according to the user's emotional state.

[0792] For example, if a user is detected as "disappointed," an encouraging commentary will be provided, such as, "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[0793] 8. Sending a response:

[0794] The generated response is formatted in JSON and sent to the terminal.

[0795] Terminal handling

[0796] 1. User Interface (UI):

[0797] The device displays a live sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[0798] 2. Submit your request:

[0799] The question entered by the user and the emotion-recognized facial expression data are converted into JSON format and sent to the server.

[0800] 3. Receive and display the response:

[0801] Receives responses from the server, analyzes them, and converts them into a displayable format. Displays answers based on the analyzed information and emotions on the screen.

[0802] User Action

[0803] 1. Enter your question:

[0804] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0805] 2. Facial expression recognition:

[0806] The user's facial expression data is automatically collected through the camera and analyzed by the emotion engine.

[0807] 3. Verify the information:

[0808] Check the response from the server displayed on the device. For example, it may say, "Player X's average points this season is 15.2 points."

[0809] 4. Feedback:

[0810] Users can provide feedback on the information they provide, which helps improve the system's accuracy and user experience.

[0811] In this way, the present invention can recognize user emotions and provide personalized real-time sports commentary, providing a significantly improved user experience compared to traditional broadcasts.

[0812] The processing flow will be explained below.

[0813] Server Processing

[0814] Step 1: Data collection

[0815] 1. Configure the data source:

[0816] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[0817] 2. Send API request:

[0818] The server periodically sends requests to the API of the configured data source.

[0819] 3. Data Acquisition:

[0820] The server receives the returned data in JSON or XML format.

[0821] Example: Get the latest match results from the official website.

[0822] Step 2: Data analysis

[0823] 1. Data Parsing:

[0824] The server parses the retrieved data and converts it into structured data.

[0825] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[0826] 2. Data Formatting:

[0827] The server formats the parsed data and filters out the necessary information.

[0828] Example: Extracting positive comments about a specific player from social media comments.

[0829] 3. Update the database:

[0830] The server stores and updates the formatted data in the database.

[0831] Example: Updating player performance data with the latest match results.

[0832] Step 3: Sentiment Analysis

[0833] 1. Receiving questions and facial expression data:

[0834] The server receives the user's questions and facial expression data sent from the terminal.

[0835] For example, if a user types in "How many points did player X score in yesterday's game?", the collected facial expression data will be received along with the text.

[0836] 2. Natural Language and Facial Expression Analysis:

[0837] The server analyzes the received question text and facial expression data to identify the user's emotions.

[0838] Example: Analyzing emotions such as "surprise" and "disappointment" from question text and facial expression data.

[0839] Step 4: Question Analysis

[0840] 1. Question analysis:

[0841] The server uses natural language processing technology to understand the intent of the user's question.

[0842] Example: Analyze the question "What is player X's average points this season?" and identify relevant information.

[0843] Step 5: Generate and execute a data query

[0844] 1. Query Generation:

[0845] The server generates a query to a database based on the question.

[0846] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0847] 2. Query execution:

[0848] The server executes the generated query against the database to obtain the required information.

[0849] Example: Retrieve player X's average points this season from the database.

[0850] Step 6: Information Generation

[0851] 1. Information generation:

[0852] The server generates answers to the user's questions based on the acquired data and the analysis results of the emotion engine.

[0853] For example, if a user is detected as "disappointed," an encouraging commentary will be generated, such as "Player X hasn't been in good form recently, but his average score this season is 15.2 points, which is still the highest on the team."

[0854] Step 7: Sending a response

[0855] 1. Response formatting:

[0856] The server formats the generated response in JSON format.

[0857] 2. Send response:

[0858] The server sends the formatted response to the terminal.

[0859] Terminal handling

[0860] Step 1: User Interface (UI)

[0861] 1. UI initialization:

[0862] The device displays a sports broadcast screen, a question input field, and a camera for facial recognition.

[0863] 2. User gesture detection:

[0864] The device uses a camera to detect the user's movements and facial expressions as they enter text into an input field.

[0865] Step 2: Submitting a request

[0866] 1. Enter your question:

[0867] The user types in a specific question.

[0868] For example, enter "What is player X's average points this season?"

[0869] 2. Acquiring facial expression data:

[0870] The device collects facial expression data from the user through the camera.

[0871] 3. Request Generation:

[0872] The device converts the user's question and facial expression data into JSON format to generate a request.

[0873] 4. Submit your request:

[0874] The terminal sends the generated request to the server.

[0875] Step 3: Receive and display the response

[0876] 1. Response received:

[0877] The terminal receives the response from the server.

[0878] 2. Response parsing:

[0879] The terminal parses the response and converts it into a displayable format.

[0880] 3. Information display:

[0881] The terminal displays the analyzed information on the screen.

[0882] Example: Display "Player X's average points this season is 15.2 points."

[0883] User Action

[0884] Step 1: Enter your question

[0885] 1. Enter your question:

[0886] While watching the game, users can enter questions about players or teams they are interested in.

[0887] For example, enter "How many points did player X score in yesterday's game?"

[0888] Step 2: Facial Expression Recognition

[0889] 1. Facial expression recognition:

[0890] The user's facial expression data is automatically collected through the camera.

[0891] Step 3: Verify the information

[0892] 1. Verify the information:

[0893] The user checks the response from the server displayed on the terminal.

[0894] 2. Enter additional questions:

[0895] If desired, the user can enter further questions.

[0896] Step 4: Feedback

[0897] 1. Providing Feedback:

[0898] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[0899] Example 2

[0900] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0901] Conventional sports broadcasting systems have difficulty providing real-time commentary that responds to the individual needs and emotional state of users. This leads to a poor user experience and difficulty in improving satisfaction. Another issue is that answers to questions entered by users are standardized, making it difficult to provide personalized responses.

[0902] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for identifying an emotional state by analyzing facial expression data of a user, means for analyzing a question from the user in natural language and generating an answer based on the extracted information and the identified emotional state, and means for providing the generated answer to the user. This makes it possible to provide real-time sports commentary that meets the individual needs and emotional state of each user.

[0903] "Data sources" are information sources for obtaining information from official websites, local information sites, blogs, social networking services, etc.

[0904] A "means of obtaining information" is a mechanism for automatically gathering information from data sources, such as through an API or RSS feed.

[0905] "Means of analyzing information" is the process of parsing the collected information and extracting key data about players and teams.

[0906] "Facial expression data" is image data used to capture a user's facial expression, and is basic data for analyzing the user's emotions.

[0907] "Emotional state" refers to the user's emotional state, such as disappointment or excitement, analyzed based on the user's facial expression data and natural language input.

[0908] "Means for question analysis" refers to technology for understanding natural language questions from users and identifying their intent.

[0909] The "answer generation means" is the process for creating an answer to the user's question based on the extracted information and the identified emotional state.

[0910] A "means for providing an answer" is a mechanism for communicating and displaying the generated answer to the user.

[0911] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[0912] Server Processing

[0913] Data collection:

[0914] The server collects information from data sources such as the official website, local information sites, blogs, and social networking services (SNS). This process uses APIs and RSS feeds. For example, the latest match results are obtained from the official website, and fan comments are collected from SNS.

[0915] Data Analysis:

[0916] The server analyzes the collected data and extracts important information about players and teams, using natural language processing technologies such as Apache OpenNLP and Google Cloud Natural Language API to extract positive opinions and statistical data from social media comments.

[0917] Database update:

[0918] The server stores the extracted information in a database and updates it in real time. For example, player performance data is updated after each game. The database uses MySQL or PostgreSQL.

[0919] Emotion analysis:

[0920] The server receives the questions and facial expression data sent by the user and analyzes them with the emotion engine. Using Azure Face API and Amazon Rekognition, the server uses facial recognition and natural language processing technology to identify the user's emotional state. For example, if a user types, "Player X, you're not doing so well today...", the emotion engine will detect "disappointment."

[0921] Question analysis:

[0922] The server uses natural language processing technology to analyze the user's question. For example, if it receives a question like "What is player X's average score this season?", it converts the question into a database query.

[0923] Data query generation and execution:

[0924] The server generates a query to the database based on the question and retrieves the information. Specifically, it generates and executes the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[0925] Information generation:

[0926] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The answers are adjusted according to the user's emotional state. For example, if the user is disappointed, an encouraging answer such as, "It's true that player X hasn't been playing well recently, but his average score of 15.2 points this season is still the highest on the team" will be generated.

[0927] Response sent:

[0928] The server formats the generated response in JSON format and sends it to the terminal.

[0929] Terminal handling

[0930] User Interface (UI):

[0931] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[0932] Submit a request:

[0933] The device converts the question entered by the user and the emotion-recognized facial expression data into JSON format and sends it to the server.

[0934] Response received and displayed:

[0935] The device receives the response from the server, analyzes it, and converts it into a displayable format. It then displays an answer based on the analyzed information and emotions on the screen.

[0936] User Action

[0937] Enter your question:

[0938] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[0939] Facial expression recognition:

[0940] The device's camera captures the user's face and sends the image data to the emotion engine in real time.

[0941] Verify the information:

[0942] The user checks the response from the server displayed on the device, for example, "Player X's average points this season is 15.2 points."

[0943] feedback:

[0944] Users can provide feedback on the information provided, such as whether it was helpful or not, which improves the accuracy of the system and the user experience.

[0945] Examples and prompts

[0946] For example, if a user asks, "When is player X's next game scheduled?", the server retrieves the date of the next game and information about the opposing team from the database and generates a positive response through the emotion engine.

[0947] Example prompt sentence:

[0948] "Provide information about a player's next game schedule and opposing team based on the user's question and facial expression. Example question: 'When is player X's next game scheduled?' Facial expression: Excitement"

[0949] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0950] Step 1: Data collection

[0951] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services (SNS). For example, the server accesses endpoints such as "http: / / officialsite.com / api / results" and "http: / / sns.com / api / comments" to retrieve match results and fan comments. The input data is in XML or JSON format, and the output data is saved in the server's storage.

[0952] Step 2: Data analysis

[0953] The server analyzes the collected data and extracts important information related to players and teams. It uses Apache OpenNLP and Google Cloud Natural Language API to parse the text data and extract positive opinions and statistical data. The input data is the raw data collected in the previous step, and the output data is the parsed structured data. For example, from a comment such as "Player X played well today," the server extracts the name of "Player X" and the sentiment "positive."

[0954] Step 3: Update the database

[0955] The server stores the analyzed data in a database such as MySQL or PostgreSQL. Based on the analysis results, it executes INSERT and UPDATE statements to update information such as player performance and fan comments. The input data is the analysis result of the previous step, and the output data is the updated database. For example, the data is stored by executing the SQL statement "INSERT INTO performance (player, date, points) VALUES ('〇〇', '2023-10-01', 25)".

[0956] Step 4: Sentiment Analysis

[0957] The server receives questions and facial expression data sent by users and analyzes them using an emotion engine. Azure Face API and Amazon Rekognition are used for facial recognition technology, and natural language processing technology is used to identify emotions from the user's text input. The input data is the user's question and facial expression image data, and the output data is the analyzed emotional state. For example, an input such as "Player X is doing poorly again today..." is identified as the emotion of "disappointment."

[0958] Step 5: Question Analysis

[0959] The server uses natural language processing technology to analyze natural language questions from users. The input data is the user's question text, and the output data is structured data that identifies the intent of the question. For example, the server analyzes the question "What is player X's average score this season?" and extracts the information "player='X' AND season='current'".

[0960] Step 6: Generate and execute a data query

[0961] The server generates a query to the database based on the question and retrieves the information. The input data is the result of question analysis, and the output data is the information retrieved from the database. For example, execute the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'" to retrieve the average score.

[0962] Step 7: Information Generation

[0963] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The input data is information from the database and the results of emotion analysis, and the output data is the generated answer text. For example, in response to the emotion detected as "disappointment," the server generates an answer such as "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[0964] Step 8: Sending a response

[0965] The server formats the generated answer in JSON format and sends it to the terminal. The input data is the generated answer text, and the output data is the sent JSON response.

[0966] Step 9: User Interface (UI)

[0967] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. It provides an interface that allows users to easily input questions. The input data are user actions, and the output data are the displayed UI elements.

[0968] Step 10: Submitting the request

[0969] The device converts the user's input question and emotion-recognized facial expression data into JSON format and sends it to the server. The input data is the user's question and facial expression data, and the output data is the request sent to the server.

[0970] Step 11: Receive and display the response

[0971] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input data is the JSON-formatted response from the server, and the output data is the answer text that is displayed on the screen.

[0972] Step 12: Enter your question

[0973] While watching a game, a user can input a question about a player or team that they are interested in. For example, they might input, "How many points did player X score in yesterday's game?" The input data is the user's question text, and the output data is the question data sent to the device.

[0974] Step 13: Facial Expression Recognition

[0975] The device camera captures the user's face and sends the image data to the emotion engine in real time. The input data is the user's facial expression image data, and the output data is the analyzed emotional state.

[0976] Step 14: Verify the information

[0977] The user confirms the response from the server displayed on the device. For example, "Player X's average points this season is 15.2 points." The input data is the text displayed on the screen, and the output data is the information confirmed by the user.

[0978] Step 15: Feedback

[0979] Users provide feedback on the information provided, such as "it was helpful" or "it wasn't." This allows the system to learn and improve the accuracy of answers from the next time onwards. The input data is the user's feedback, and the output data is the system's learning data.

[0980] (Application example 2)

[0981] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0982] The problem that this invention aims to solve is that conventional sports broadcast commentary systems have difficulty in providing information tailored to individual user needs, and in particular, they are unable to respond based on real-time emotion recognition. Furthermore, because they lack the functionality of an entertainment system in an autonomous vehicle, users lack a means to effectively obtain sports information while driving.

[0983] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question from a user in natural language and generating an answer based on the extracted information, means for recognizing the user's emotional state, means for adjusting and providing the generated answer according to the user's emotional state, means for capturing facial expression data of passengers in real time using an in-vehicle camera and analyzing their emotional state, and means for providing the generated answer to the user in the autonomous vehicle in voice and text. This makes it possible to provide sports information in real time that meets individual needs while taking user emotions into consideration.

[0984] "Data sources" refer to various sources of information, such as official websites, local information sites, blogs, and social networking services.

[0985] "Means of obtaining information" refers to the methods and techniques used to collect the required information from the data source.

[0986] "Means for analyzing information" refers to the mechanism for processing the acquired information and extracting important data about players and teams.

[0987] "Important information" refers to statistical data, evaluations, performance data, etc. about players and teams obtained through analysis.

[0988] A "natural language question" is a question posed by a user via free-form text or voice.

[0989] "User emotional state" refers to the user's emotions identified using facial recognition and natural language processing technologies.

[0990] "Means for recognizing emotional states" refers to technology that analyzes a user's facial expression data and text input to identify their emotions.

[0991] The "answer generation means" refers to a mechanism that generates an answer to a user's question based on the analyzed information and the perceived emotional state.

[0992] "Means for adjusting and providing" refers to methods and techniques for appropriately changing the generated answer according to the user's emotions and communicating it to the user.

[0993] An "in-car camera" is a camera device installed inside an autonomous vehicle to capture passengers' facial expressions in real time.

[0994] "Facial expression data" refers to information captured as image or video data of a user's facial expressions.

[0995] An "autonomous vehicle" is a vehicle that can drive autonomously by utilizing artificial intelligence and sensor technology.

[0996] "Voice and text delivery means" refers to methods and technologies for communicating generated answers to users using speech synthesis and display technologies.

[0997] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and also combines it with an emotion engine that recognizes user emotions. It is particularly applicable to entertainment systems that allow users to comfortably receive sports information while driving in autonomous vehicles.

[0998] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. Information collected from data sources is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[0999] The server then analyzes the collected data to extract key information about players and teams. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. It also extracts statistical data such as players' scores and performance from game results. This is done using technologies such as Hugging Face's Transformers and the Google Natural Language API.

[1000] The analyzed information is stored and updated in a database. Then, questions in natural language from users are analyzed. The server analyzes the user's question request received from the device, identifies the required information, and understands the intent of the question. Natural language processing technology is also used in this process.

[1001] In addition, the server recognizes the user's emotional state. It uses an in-car camera to capture facial expression data of passengers in the autonomous vehicle in real time, analyzes their emotions using facial recognition technology, and generates customized answers using a generative AI model (e.g., OpenAI's GPT-4) based on the analysis results.

[1002] The generated answers are adjusted appropriately according to the user's emotional state and are provided to the user in the autonomous vehicle via voice and text. By using speech synthesis and display technologies, real-time sports commentary is realized.

[1003] This system takes into consideration the user's emotions and makes it possible to provide real-time sports information tailored to individual needs. For example, if a user inputs a question such as "How many points did player X score in the current game?" and the system recognizes the user's emotional state as disappointment, it will provide commentary that includes encouragement, such as "Player X has currently scored 15 points in the game. He may not be in good form right now, but let's look forward to his future performances!"

[1004] Specific examples of prompt sentences are as follows:

[1005] example:

[1006] text

[1007] Question: How many points did player X score in the game?

[1008] Emotion: disappointment

[1009] Answer: Player XX is currently scoring 15 points per game. He may not be in good form right now, but let's look forward to his future performance!

[1010] In this way, the present invention can improve the user experience in an autonomous vehicle and provide comfortable live sports commentary.

[1011] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1012] Step 1:

[1013] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. The collected information is automatically retrieved through APIs and RSS feeds. For example, the latest match results can be obtained from the official website, and fan comments can be collected from social media. The input is data from the data sources, and the output is the collected raw data.

[1014] Step 2:

[1015] The server analyzes the collected raw data and extracts important information about players and teams. The analysis is performed using natural language processing techniques such as Hugging Face's Transformers and Google Natural Language API. The input is the collected raw data, and the output is the analyzed important information (e.g., player scores and performance).

[1016] Step 3:

[1017] The server stores and updates the analyzed information in a database, ensuring that the latest information is always available in real time. The input is the analyzed important information, and the output is an updated database.

[1018] Step 4:

[1019] The user inputs a question in natural language or asks it by voice. For example, a question like, "How many points did player X score in the current game?" The input is the user's question in natural language, and the output is the text data of the question.

[1020] Step 5:

[1021] The terminal captures the user's facial expression data in real time using an in-car camera. The captured image data is sent to a server. The input is the captured facial expression data, and the output is the facial expression image data.

[1022] Step 6:

[1023] The server analyzes the user's question request and facial expression image data received from the device. Facial expression image data is analyzed using facial recognition technology, and natural language processing technology is used to understand the intent of the question. For example, in response to the question "How many points did player X score in the current game?", a query is generated to obtain the score information of a specific player. The input is the question request and facial expression image data, and the output is the analysis result and the query.

[1024] Step 7:

[1025] The server retrieves the necessary information from the database based on the generated query, for example, the latest scores and performance data of a player. The input is the query, and the output is the retrieved information.

[1026] Step 8:

[1027] The server generates answers to the user's questions based on the emotion engine's analysis results and the acquired information. Generative AI such as OpenAI's GPT-4 is used for generation. If the user is disappointed, an answer containing encouragement is generated. The input is the emotion analysis results and the acquired information, and the output is the generated answer.

[1028] Step 9:

[1029] The server formats the generated answer in JSON format and sends it to the terminal. The terminal parses the received JSON format data, converts it into a displayable format, and provides it to the user in voice and text. The input is the generated answer, and the output is the voice and text format answer to the user.

[1030] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1031] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1032] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1033] [Third embodiment]

[1034] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1035] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1037] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1038] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1039] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1041] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1042] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1044] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1045] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1046] The present invention relates to a system for providing real-time commentary of live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[1047] Server Processing

[1048] 1. Data Collection:

[1049] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1050] 2. Data Analysis:

[1051] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[1052] 3. Update the database:

[1053] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[1054] 4. Question analysis:

[1055] It analyzes natural language questions from users and understands their intent. For example, if it receives a question like, "How is player X performing recently?", it generates a query to retrieve that player's recent match data.

[1056] 5. Information generation:

[1057] Based on the analyzed data, answers to user questions are generated. Generative AI is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, the generated explanation would be, "Player X has averaged 15.2 points in his last five games."

[1058] 6. Sending a response:

[1059] The generated answer is provided to the user, and the response is sent to the terminal in JSON format and displayed in the user interface (UI).

[1060] Terminal handling

[1061] 1. User Interface (UI):

[1062] The device displays a live sports broadcast screen and a question input field. The UI is designed to be intuitive, allowing users to easily input questions.

[1063] 2. Submit your request:

[1064] The question entered by the user is converted into JSON format and sent to the server.

[1065] 3. Receive and display the response:

[1066] It receives the response from the server, parses it, and converts it into a displayable format. For example, it displays the answer "Player X's average points this season is 15.2 points."

[1067] User processing

[1068] 1. Enter your question:

[1069] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1070] 2. Verify the information:

[1071] The server response will be displayed on your device, and you can enter additional questions if necessary.

[1072] 3. Feedback:

[1073] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[1074] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of users, providing a significantly improved user experience compared to traditional broadcasting.

[1075] The processing flow will be explained below.

[1076] Server Processing

[1077] Step 1: Data collection

[1078] 1. Configure the data source:

[1079] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[1080] 2. Send API request:

[1081] The server periodically sends requests to the API of the configured data source.

[1082] 3. Data Acquisition:

[1083] The server receives the returned data in JSON or XML format.

[1084] Example: Get the latest match results from the official website.

[1085] Step 2: Data analysis

[1086] 1. Data Parsing:

[1087] The server parses the retrieved data and converts it into structured data.

[1088] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[1089] 2. Data Formatting:

[1090] The server formats the parsed data and filters out the necessary information.

[1091] Example: Extracting positive comments about a specific player from social media comments.

[1092] 3. Update the database:

[1093] The server stores and updates the formatted data in the database.

[1094] Example: Updating player performance data with the latest match results.

[1095] Step 3: Analyze user questions

[1096] 1. User request received:

[1097] The server receives a user's question request sent from the terminal.

[1098] Example: You receive a question such as, "What is player X's average score this season?"

[1099] 2. Natural language analysis:

[1100] The server analyzes the received question using natural language processing technology and identifies the required information.

[1101] Example: Extract keywords to identify "X player's average points this season."

[1102] Step 4: Generate and execute a data query

[1103] 1. Query Generation:

[1104] The server generates a query to a database based on the question.

[1105] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1106] 2. Query execution:

[1107] The server executes the generated query against the database to obtain the required information.

[1108] Example: Retrieve player X's average points this season from the database.

[1109] Step 5: Information generation and transmission

[1110] 1. Information generation:

[1111] The server generates answers to the user's questions based on the acquired data.

[1112] Example: Generates the text "Player X's average points this season is 15.2 points."

[1113] 2. Send response:

[1114] The server sends the generated answer to the terminal as a response.

[1115] Terminal handling

[1116] Step 1: User Interface (UI)

[1117] 1. UI initialization:

[1118] The device displays a live sports broadcast screen and a question input field.

[1119] 2. User gesture detection:

[1120] The device detects when the user enters text into the input field.

[1121] Step 2: Submitting a request

[1122] 1. Enter your question:

[1123] The user types in a specific question.

[1124] For example, enter "What is player X's average points this season?"

[1125] 2. Request Generation:

[1126] The device converts the user's question into JSON format and generates a request.

[1127] 3. Submit your request:

[1128] The terminal sends the generated request to the server.

[1129] Step 3: Receive and display the response

[1130] 1. Response received:

[1131] The terminal receives the response from the server.

[1132] 2. Response parsing:

[1133] The terminal parses the response and converts it into a displayable format.

[1134] 3. Information display:

[1135] The terminal displays the analyzed information on the screen.

[1136] Example: Display "Player X's average points this season is 15.2 points."

[1137] User Action

[1138] Step 1: Enter your question

[1139] 1. Enter your question:

[1140] While watching the game, users can enter questions about players or teams they are interested in.

[1141] For example, enter "How many points did player X score in yesterday's game?"

[1142] Step 2: Verify the information

[1143] 1. Verify the information:

[1144] The user checks the response from the server displayed on the terminal.

[1145] 2. Enter additional questions:

[1146] If desired, the user can enter further questions.

[1147] Step 3: Feedback

[1148] 1. Providing Feedback:

[1149] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[1150] Example 1

[1151] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1152] In conventional sports broadcasts, it was difficult for users to obtain detailed information about the players or teams they were interested in in real time. Furthermore, there was a lack of a way to quickly and appropriately answer specific questions from users. Therefore, in order to increase user satisfaction, there is a need for more accurate information provision and faster responses.

[1153] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1154] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question in natural language from a user and understanding the intent of the question, means for generating an answer using a generative AI model based on the understood intent, and means for providing the generated answer to the user. This makes it possible to provide detailed information based on the individual needs of the user in real time and respond quickly and appropriately.

[1155] "Data sources" are sources of information such as official websites, local information sites, blogs, and social networking services for collecting information.

[1156] "Means of obtaining information" refers to technologies such as APIs and RSS feeds that automatically collect the necessary information from data sources.

[1157] "Means for analyzing information" refers to natural language processing and statistical analysis techniques used to analyze collected data and extract important information about players and teams.

[1158] "Means for analyzing user questions" refers to natural language processing technology for understanding questions in natural language from users and analyzing their intent.

[1159] A "generative AI model" is an artificial intelligence model that generates appropriate answers to user questions, and specifically includes generative AI models such as GPT-3.

[1160] "Means for generating answers" refers to technology that uses a generative AI model to generate answers to users' questions.

[1161] The "means for providing an answer" refers to the communication means and user interface for conveying the generated answer to the user.

[1162] The present invention relates to a system for providing real-time commentary on live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[1163] Server Processing

[1164] The server provides the technical infrastructure for retrieving information from a variety of data sources, analyzing the retrieved information, and responding to user queries. The server includes the following functions:

[1165] 1. Data Collection:

[1166] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services. For example, it retrieves the latest match results from the official website and collects fan comments from social media.

[1167] 2. Data Analysis:

[1168] The collected data is parsed and key information about players and teams is extracted using natural language processing techniques. For example, it can extract only positive opinions about a specific player from social media comments, or derive statistical data such as a player's score and performance from game results.

[1169] 3. Update the database:

[1170] The extracted information is stored and updated in a database, which allows the latest information to be kept up to date. Examples of databases that can be used include MySQL and PostgreSQL.

[1171] 4. Question analysis:

[1172] It uses natural language processing models such as BERT and GPT-3 to analyze natural language questions from users and understand the intent of the question. For example, it analyzes the question "How is player X's recent performance?" and generates a query to retrieve the player's recent game data.

[1173] 5. Information generation:

[1174] Based on the analyzed data, answers to user questions are generated. Generative AI models such as GPT-3 are used for generation, and explanations are provided in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "player X has averaged 15.2 points in his last five games" is generated.

[1175] 6. Sending a response:

[1176] The generated answer is sent to the device in JSON format, and the response is displayed in the user interface (UI).

[1177] Terminal handling

[1178] The terminal provides an interface for users to enter questions and display answers from the server. The terminal includes the following features:

[1179] 1. User Interface (UI):

[1180] The device displays a live sports broadcast screen and a question input field. The UI is built using React.js and other tools, and is designed to make it easy for users to enter questions.

[1181] 2. Submit your request:

[1182] The question entered by the user is converted into JSON format and sent to the server. For example, if the user enters a question such as "How is player X's recent performance?", this question is sent to the server using the axios.post method.

[1183] 3. Receive and display the response:

[1184] Receives the response from the server and converts it into a displayable format. For example, if the server returns an answer such as "Player X's average score this season is 15.2 points," it displays it on the screen.

[1185] User processing

[1186] Users can watch live sports broadcasts through their devices and ask questions about information they are interested in. The following functions are included in the user-side processing:

[1187] 1. Enter your question:

[1188] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1189] 2. Verify the information:

[1190] The user sees the response from the server displayed on the device and can enter additional questions if needed.

[1191] 3. Feedback:

[1192] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[1193] Examples and prompts

[1194] For example, if a user types a question like, "How is player X performing recently?", the server collects and analyzes past game data and social media comments to generate an answer such as, "X player has averaged 15.2 points in his last five games."

[1195] Example prompt sentence:

[1196] "Please tell me about player XX's recent performance."

[1197] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of the user.

[1198] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1199] Step 1: Data collection

[1200] The server collects information from official websites, local information sites, blogs, and social networking services. As input, it provides API endpoints and RSS feed URLs, which it uses to retrieve data. As output, it obtains data in JSON or XML format. Specifically, it sends a "GET" request to the official website's API to retrieve match results and player information. It also uses the Twitter API to collect tweets containing specific hashtags.

[1201] Step 2: Data analysis

[1202] The server parses the collected data and extracts important information about players and teams. The JSON or XML data collected in Step 1 is used as input. The output is an analysis of players' performance and social media comments. Specifically, the server parses the data using the json.loads() function and uses natural language processing technology to classify comments as positive or negative. It also analyzes match results to extract players' points and assists.

[1203] Step 3: Update the database

[1204] The server stores and updates the parsed information in the database. The parsed results from step 2 are used as input. The output is to update the database with the latest information. Specifically, the SQLAlchemy library is used to execute a query to insert the latest player statistics into the database. For example, an SQL statement such as INSERT INTO player_stats (player_id, match_id, points) VALUES (...) is generated and executed.

[1205] Step 4: Question Analysis

[1206] The server analyzes natural language questions from users and understands their intent. The input is a text question from the user. The output is a query that reflects the intent of the question. Specifically, the BERT model is used to analyze the question "How is player X's recent performance?" and generate a query that identifies "player X" and asks for statistical information related to his or her "performance."

[1207] Step 5: Information Generation

[1208] The server generates an answer to the user's question based on the analyzed data. The query generated in step 4 and information obtained from the database are used as input. The output is an answer expressed in natural language. Specifically, the query result is input into a generative AI model such as GPT-3, and an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[1209] Step 6: Sending a response

[1210] The server sends the generated answer in JSON format to the terminal. The answer generated in step 5 is used as input. The output is a JSON response to be sent to the terminal. Specifically, a response is generated to return the generated answer in JSON format to the client side through Flask's API endpoint.

[1211] Step 7: User Interface (UI) Display

[1212] The device displays a sports broadcast screen and a question input field. HTML and JavaScript files are used as input. An interface where users can enter questions is displayed on the screen as output. Specifically, React.js is used to build the question input field and broadcast screen, providing a UI that users can operate intuitively.

[1213] Step 8: Submitting the request

[1214] The device converts the question entered by the user into JSON format and sends it to the server. The text question entered by the user is used as input. An HTTP request to the server is generated as output. Specifically, the question "How has player X performed recently?" is sent to the server using the axios.post method.

[1215] Step 9: Receive and display the response

[1216] The terminal receives the response from the server and converts it into a displayable format. The JSON response returned by the server is used as input. The output is a text answer that is displayed on the screen. Specifically, the response received by axios is analyzed and the text "Player X's average score this season is 15.2 points" is displayed on the screen.

[1217] Step 10: Feedback

[1218] Users provide feedback on the information provided. The text of the feedback provided by the user is used as input. The feedback content is sent to the server as output, and is used to improve the system. Specifically, a function is provided to enter feedback such as "The information was accurate" or "The explanation was easy to understand" in an input field and send it.

[1219] (Application example 1)

[1220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1221] Traditional sports broadcasting systems often only provided general information, making it difficult for users to obtain real-time commentary tailored to their individual needs. Furthermore, users lacked the means to quickly obtain detailed information about specific players or teams while watching a game. This limited the user experience and reduced the enjoyment of viewing.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1223] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about people and organizations, means for analyzing a question entered in natural language by a user and generating an answer based on the extracted information, means for providing the generated answer to the user, means for providing real-time commentary about live sports broadcasts using a mobile communication device, means for converting a question entered by a user into JSON format and sending the JSON format to the server, and means for receiving, analyzing, and converting a response from the server into a displayable format, thereby enabling users to quickly obtain detailed, real-time commentary tailored to their individual needs while watching a game.

[1224] "Data sources" are information sources for automatically obtaining information from official websites, local information sites, blogs, social networking services, etc.

[1225] "Acquired information" refers to information such as match results, player and team performance data, and comments on social media collected from data sources.

[1226] "Analysis" is the act of automatically processing and analyzing acquired information and extracting necessary information and patterns.

[1227] "Person" is a term that refers to a specific individual, such as a sports player or a related person.

[1228] "Organization" is a term that refers to an organization or group, such as a sports team or association.

[1229] "Users" refer to sports spectators and viewers who use this system.

[1230] "Natural language" refers to the language that users use in their daily lives, in the form of input before it is converted into a machine-interpretable format.

[1231] A "question" is something a user inputs in natural language about the information they want to know.

[1232] "JSON format" is a type of data exchange format and is an abbreviation for JavaScript Object Notation.

[1233] "Server" means a central processing unit that collects and analyzes data and generates and provides answers to user questions.

[1234] "Mobile communication device" refers to a mobile communication device such as a smartphone or tablet.

[1235] "Real-time commentary" refers to detailed information and commentary that is available instantly while watching live sports broadcasts.

[1236] A "response" is an answer that a server generates to a user's question.

[1237] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs. The system has the function of acquiring information from data sources, analyzing the acquired information, and generating answers to user questions.

[1238] Server Processing

[1239] The server retrieves information from data sources such as the official website, local information sites, blogs, and social networking services. Information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1240] The collected information is analyzed to extract important information about individuals and organizations. Natural language processing technology is used for the analysis, making it possible to extract only positive opinions about a specific individual from social media comments, for example. Statistical data such as player scores and performance can also be extracted from match results.

[1241] The extracted information is stored and updated in a database, and the latest information is always kept in real time. For example, player performance data is updated after every game.

[1242] Questions from users are sent to the server in natural language. The server analyzes the question and understands the user's intent. For example, if the server receives a question like "How has player X performed recently?", it generates a query to retrieve the player's recent game data.

[1243] Based on the analyzed data, answers to user questions are generated. A generative AI model is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[1244] The generated answer is sent to the user's device in JSON format.

[1245] Terminal handling

[1246] The device displays a live sports broadcast screen and a question input field. The user interface (UI) is designed to be intuitive, allowing users to easily input questions.

[1247] The question entered by the user is converted into JSON format and sent to the server. The response from the server is received, parsed, and converted into a displayable format. For example, the answer displayed is "Player X's average points this season is 15.2 points."

[1248] User Actions

[1249] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?". They can also provide feedback on the information provided, which helps improve the accuracy of the system.

[1250] Examples of concrete examples and prompts

[1251] As a concrete example, if a user uses a smartphone app while watching a game to enter a question such as "How many points did player X score in yesterday's game?", the following prompt sentence will be generated:

[1252] Prompt Sentence Examples

[1253] "How many points did player X score in yesterday's game?"

[1254] Using this system, users can quickly obtain detailed, real-time commentary tailored to their individual needs while watching a sporting event, making live sports broadcasts even more enjoyable.

[1255] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1256] Processing step flow

[1257] Step 1:

[1258] The server obtains information from data sources such as official websites, local information sites, blogs, and social networking services (SNS). Information is collected automatically through APIs and RSS feeds. The input is various information provided by the data sources, and the output is raw data stored in temporary data storage on the server.

[1259] Step 2:

[1260] The server analyzes the collected information and uses natural language processing technology to extract important information about individuals and organizations. For example, it can extract positive opinions about a specific individual from social media comments, or statistical data such as a player's score and performance from game results. The input is the acquired raw data, and the output is the analyzed important information.

[1261] Step 3:

[1262] The server stores and updates the extracted information in the database, which keeps the database up-to-date in real time. The input is the analyzed important information, and the output is the updated database information.

[1263] Step 4:

[1264] The device displays a live sports broadcast screen and a question input field through a user interface (UI). The user inputs a question in natural language. The input is the text question entered by the user, and the output is the question data converted to JSON format.

[1265] Step 5:

[1266] The terminal converts the input question into JSON format and sends it to the server. The converted question is sent via the API. The input is text-formatted question data, and the output is JSON-formatted question data sent to the server.

[1267] Step 6:

[1268] The server receives a question in JSON format sent by the user and uses natural language processing technology to analyze the question and understand the user's intent. The input is the question data in JSON format, and the output is the analyzed intent information.

[1269] Step 7:

[1270] The server extracts relevant data from the database based on the analyzed user intent and generates commentary in natural language using a generative AI model. For example, in response to the question "How is player X's recent performance?", it generates commentary such as "Player X has averaged 15.2 points in the last five games" based on the latest match data. The input is the analyzed intent information and related data from the database, and the output is commentary generated in natural language.

[1271] Step 8:

[1272] The server sends the generated explanation to the terminal in JSON format. The input is the generated natural language explanation, and the output is JSON format data.

[1273] Step 9:

[1274] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input is the response data in JSON format, and the output is a displayable text description.

[1275] Step 10:

[1276] The user checks the explanation displayed on the terminal and inputs additional questions as necessary. They can also provide feedback on the information provided. This feedback information is sent to the server and used to improve the accuracy of the system. The input is the displayed explanation information and feedback, and the output is increased user satisfaction and improved system performance.

[1277] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1278] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[1279] Server Processing

[1280] 1. Data Collection:

[1281] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1282] 2. Data Analysis:

[1283] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[1284] 3. Update the database:

[1285] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[1286] 4. Emotion analysis:

[1287] The server receives questions from users and facial expression data displayed on the device, and analyzes them using an emotion engine. Emotion analysis uses facial recognition and natural language processing technologies to identify emotional states from the text and facial expressions entered by the user.

[1288] Example: If a user types, "Player X was not good today either...", the emotion engine will detect "disappointment."

[1289] 5. Question analysis:

[1290] The server analyzes the user's question request received from the device, and uses natural language processing technology to understand the intent of the question and identify the required information.

[1291] Example: If a question is received such as "What is player X's average points this season?", a query will be generated to retrieve that player's game data this season.

[1292] 6. Data query generation and execution:

[1293] The server generates a query to a database based on the question to retrieve the required information.

[1294] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1295] 7. Information generation:

[1296] Based on the acquired data and the analysis results of the emotion engine, answers to user questions are generated using generative AI, which adjusts the answers according to the user's emotional state.

[1297] For example, if a user is detected as "disappointed," an encouraging commentary will be provided, such as, "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[1298] 8. Sending a response:

[1299] The generated response is formatted in JSON and sent to the terminal.

[1300] Terminal handling

[1301] 1. User Interface (UI):

[1302] The device displays a live sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[1303] 2. Submit your request:

[1304] The question entered by the user and the emotion-recognized facial expression data are converted into JSON format and sent to the server.

[1305] 3. Receive and display the response:

[1306] Receives responses from the server, analyzes them, and converts them into a displayable format. Displays answers based on the analyzed information and emotions on the screen.

[1307] User Action

[1308] 1. Enter your question:

[1309] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1310] 2. Facial expression recognition:

[1311] The user's facial expression data is automatically collected through the camera and analyzed by the emotion engine.

[1312] 3. Verify the information:

[1313] Check the response from the server displayed on the device. For example, it may say, "Player X's average points this season is 15.2 points."

[1314] 4. Feedback:

[1315] Users can provide feedback on the information they provide, which helps improve the system's accuracy and user experience.

[1316] In this way, the present invention can recognize user emotions and provide personalized real-time sports commentary, providing a significantly improved user experience compared to traditional broadcasts.

[1317] The processing flow will be explained below.

[1318] Server Processing

[1319] Step 1: Data collection

[1320] 1. Configure the data source:

[1321] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[1322] 2. Send API request:

[1323] The server periodically sends requests to the API of the configured data source.

[1324] 3. Data Acquisition:

[1325] The server receives the returned data in JSON or XML format.

[1326] Example: Get the latest match results from the official website.

[1327] Step 2: Data analysis

[1328] 1. Data Parsing:

[1329] The server parses the retrieved data and converts it into structured data.

[1330] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[1331] 2. Data Formatting:

[1332] The server formats the parsed data and filters out the necessary information.

[1333] Example: Extracting positive comments about a specific player from social media comments.

[1334] 3. Update the database:

[1335] The server stores and updates the formatted data in the database.

[1336] Example: Updating player performance data with the latest match results.

[1337] Step 3: Sentiment Analysis

[1338] 1. Receiving questions and facial expression data:

[1339] The server receives the user's questions and facial expression data sent from the terminal.

[1340] For example, if a user types in "How many points did player X score in yesterday's game?", the collected facial expression data will be received along with the text.

[1341] 2. Natural Language and Facial Expression Analysis:

[1342] The server analyzes the received question text and facial expression data to identify the user's emotions.

[1343] Example: Analyzing emotions such as "surprise" and "disappointment" from question text and facial expression data.

[1344] Step 4: Question Analysis

[1345] 1. Question analysis:

[1346] The server uses natural language processing technology to understand the intent of the user's question.

[1347] Example: Analyze the question "What is player X's average points this season?" and identify relevant information.

[1348] Step 5: Generate and execute a data query

[1349] 1. Query Generation:

[1350] The server generates a query to a database based on the question.

[1351] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1352] 2. Query execution:

[1353] The server executes the generated query against the database to obtain the required information.

[1354] Example: Retrieve player X's average points this season from the database.

[1355] Step 6: Information Generation

[1356] 1. Information generation:

[1357] The server generates answers to the user's questions based on the acquired data and the analysis results of the emotion engine.

[1358] For example, if a user is detected as "disappointed," an encouraging commentary will be generated, such as "Player X hasn't been in good form recently, but his average score this season is 15.2 points, which is still the highest on the team."

[1359] Step 7: Sending a response

[1360] 1. Response formatting:

[1361] The server formats the generated response in JSON format.

[1362] 2. Send response:

[1363] The server sends the formatted response to the terminal.

[1364] Terminal handling

[1365] Step 1: User Interface (UI)

[1366] 1. UI initialization:

[1367] The device displays a sports broadcast screen, a question input field, and a camera for facial recognition.

[1368] 2. User gesture detection:

[1369] The device uses a camera to detect the user's movements and facial expressions as they enter text into an input field.

[1370] Step 2: Submitting a request

[1371] 1. Enter your question:

[1372] The user types in a specific question.

[1373] For example, enter "What is player X's average points this season?"

[1374] 2. Acquiring facial expression data:

[1375] The device collects facial expression data from the user through the camera.

[1376] 3. Request Generation:

[1377] The device converts the user's question and facial expression data into JSON format to generate a request.

[1378] 4. Submit your request:

[1379] The terminal sends the generated request to the server.

[1380] Step 3: Receive and display the response

[1381] 1. Response received:

[1382] The terminal receives the response from the server.

[1383] 2. Response parsing:

[1384] The terminal parses the response and converts it into a displayable format.

[1385] 3. Information display:

[1386] The terminal displays the analyzed information on the screen.

[1387] Example: Display "Player X's average points this season is 15.2 points."

[1388] User Action

[1389] Step 1: Enter your question

[1390] 1. Enter your question:

[1391] While watching the game, users can enter questions about players or teams they are interested in.

[1392] For example, enter "How many points did player X score in yesterday's game?"

[1393] Step 2: Facial Expression Recognition

[1394] 1. Facial expression recognition:

[1395] The user's facial expression data is automatically collected through the camera.

[1396] Step 3: Verify the information

[1397] 1. Verify the information:

[1398] The user checks the response from the server displayed on the terminal.

[1399] 2. Enter additional questions:

[1400] If desired, the user can enter further questions.

[1401] Step 4: Feedback

[1402] 1. Providing Feedback:

[1403] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[1404] Example 2

[1405] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1406] Conventional sports broadcasting systems have difficulty providing real-time commentary that responds to the individual needs and emotional state of users. This leads to a poor user experience and difficulty in improving satisfaction. Another issue is that answers to questions entered by users are standardized, making it difficult to provide personalized responses.

[1407] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for identifying an emotional state by analyzing facial expression data of a user, means for analyzing a question from the user in natural language and generating an answer based on the extracted information and the identified emotional state, and means for providing the generated answer to the user. This makes it possible to provide real-time sports commentary that meets the individual needs and emotional state of each user.

[1408] "Data sources" are information sources for obtaining information from official websites, local information sites, blogs, social networking services, etc.

[1409] A "means of obtaining information" is a mechanism for automatically gathering information from data sources, such as through an API or RSS feed.

[1410] "Means of analyzing information" is the process of parsing the collected information and extracting key data about players and teams.

[1411] "Facial expression data" is image data used to capture a user's facial expression, and is basic data for analyzing the user's emotions.

[1412] "Emotional state" refers to the user's emotional state, such as disappointment or excitement, analyzed based on the user's facial expression data and natural language input.

[1413] "Means for question analysis" refers to technology for understanding natural language questions from users and identifying their intent.

[1414] The "answer generation means" is the process for creating an answer to the user's question based on the extracted information and the identified emotional state.

[1415] A "means for providing an answer" is a mechanism for communicating and displaying the generated answer to the user.

[1416] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[1417] Server Processing

[1418] Data collection:

[1419] The server collects information from data sources such as the official website, local information sites, blogs, and social networking services (SNS). This process uses APIs and RSS feeds. For example, the latest match results are obtained from the official website, and fan comments are collected from SNS.

[1420] Data Analysis:

[1421] The server analyzes the collected data and extracts important information about players and teams, using natural language processing technologies such as Apache OpenNLP and Google Cloud Natural Language API to extract positive opinions and statistical data from social media comments.

[1422] Database update:

[1423] The server stores the extracted information in a database and updates it in real time. For example, player performance data is updated after each game. The database uses MySQL or PostgreSQL.

[1424] Emotion analysis:

[1425] The server receives the questions and facial expression data sent by the user and analyzes them with the emotion engine. Using Azure Face API and Amazon Rekognition, the server uses facial recognition and natural language processing technology to identify the user's emotional state. For example, if a user types, "Player X, you're not doing so well today...", the emotion engine will detect "disappointment."

[1426] Question analysis:

[1427] The server uses natural language processing technology to analyze the user's question. For example, if it receives a question like "What is player X's average score this season?", it converts the question into a database query.

[1428] Data query generation and execution:

[1429] The server generates a query to the database based on the question and retrieves the information. Specifically, it generates and executes the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1430] Information generation:

[1431] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The answers are adjusted according to the user's emotional state. For example, if the user is disappointed, an encouraging answer such as, "It's true that player X hasn't been playing well recently, but his average score of 15.2 points this season is still the highest on the team" will be generated.

[1432] Response sent:

[1433] The server formats the generated response in JSON format and sends it to the terminal.

[1434] Terminal handling

[1435] User Interface (UI):

[1436] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[1437] Submit a request:

[1438] The device converts the question entered by the user and the emotion-recognized facial expression data into JSON format and sends it to the server.

[1439] Response received and displayed:

[1440] The device receives the response from the server, analyzes it, and converts it into a displayable format. It then displays an answer based on the analyzed information and emotions on the screen.

[1441] User Action

[1442] Enter your question:

[1443] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1444] Facial expression recognition:

[1445] The device's camera captures the user's face and sends the image data to the emotion engine in real time.

[1446] Verify the information:

[1447] The user checks the response from the server displayed on the device, for example, "Player X's average points this season is 15.2 points."

[1448] feedback:

[1449] Users can provide feedback on the information provided, such as whether it was helpful or not, which improves the accuracy of the system and the user experience.

[1450] Examples and prompts

[1451] For example, if a user asks, "When is player X's next game scheduled?", the server retrieves the date of the next game and information about the opposing team from the database and generates a positive response through the emotion engine.

[1452] Example prompt sentence:

[1453] "Provide information about a player's next game schedule and opposing team based on the user's question and facial expression. Example question: 'When is player X's next game scheduled?' Facial expression: Excitement"

[1454] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1455] Step 1: Data collection

[1456] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services (SNS). For example, the server accesses endpoints such as "http: / / officialsite.com / api / results" and "http: / / sns.com / api / comments" to retrieve match results and fan comments. The input data is in XML or JSON format, and the output data is saved in the server's storage.

[1457] Step 2: Data analysis

[1458] The server analyzes the collected data and extracts important information related to players and teams. It uses Apache OpenNLP and Google Cloud Natural Language API to parse the text data and extract positive opinions and statistical data. The input data is the raw data collected in the previous step, and the output data is the parsed structured data. For example, from a comment such as "Player X played well today," the server extracts the name of "Player X" and the sentiment "positive."

[1459] Step 3: Update the database

[1460] The server stores the analyzed data in a database such as MySQL or PostgreSQL. Based on the analysis results, it executes INSERT and UPDATE statements to update information such as player performance and fan comments. The input data is the analysis result of the previous step, and the output data is the updated database. For example, the data is stored by executing the SQL statement "INSERT INTO performance (player, date, points) VALUES ('〇〇', '2023-10-01', 25)".

[1461] Step 4: Sentiment Analysis

[1462] The server receives questions and facial expression data sent by users and analyzes them using an emotion engine. Azure Face API and Amazon Rekognition are used for facial recognition technology, and natural language processing technology is used to identify emotions from the user's text input. The input data is the user's question and facial expression image data, and the output data is the analyzed emotional state. For example, an input such as "Player X is doing poorly again today..." is identified as the emotion of "disappointment."

[1463] Step 5: Question Analysis

[1464] The server uses natural language processing technology to analyze natural language questions from users. The input data is the user's question text, and the output data is structured data that identifies the intent of the question. For example, the server analyzes the question "What is player X's average score this season?" and extracts the information "player='X' AND season='current'".

[1465] Step 6: Generate and execute a data query

[1466] The server generates a query to the database based on the question and retrieves the information. The input data is the result of question analysis, and the output data is the information retrieved from the database. For example, execute the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'" to retrieve the average score.

[1467] Step 7: Information Generation

[1468] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The input data is information from the database and the results of emotion analysis, and the output data is the generated answer text. For example, in response to the emotion detected as "disappointment," the server generates an answer such as "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[1469] Step 8: Sending a response

[1470] The server formats the generated answer in JSON format and sends it to the terminal. The input data is the generated answer text, and the output data is the sent JSON response.

[1471] Step 9: User Interface (UI)

[1472] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. It provides an interface that allows users to easily input questions. The input data are user actions, and the output data are the displayed UI elements.

[1473] Step 10: Submitting the request

[1474] The device converts the user's input question and emotion-recognized facial expression data into JSON format and sends it to the server. The input data is the user's question and facial expression data, and the output data is the request sent to the server.

[1475] Step 11: Receive and display the response

[1476] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input data is the JSON-formatted response from the server, and the output data is the answer text that is displayed on the screen.

[1477] Step 12: Enter your question

[1478] While watching a game, a user can input a question about a player or team that they are interested in. For example, they might input, "How many points did player X score in yesterday's game?" The input data is the user's question text, and the output data is the question data sent to the device.

[1479] Step 13: Facial Expression Recognition

[1480] The device camera captures the user's face and sends the image data to the emotion engine in real time. The input data is the user's facial expression image data, and the output data is the analyzed emotional state.

[1481] Step 14: Verify the information

[1482] The user confirms the response from the server displayed on the device. For example, "Player X's average points this season is 15.2 points." The input data is the text displayed on the screen, and the output data is the information confirmed by the user.

[1483] Step 15: Feedback

[1484] Users provide feedback on the information provided, such as "it was helpful" or "it wasn't." This allows the system to learn and improve the accuracy of answers from the next time onwards. The input data is the user's feedback, and the output data is the system's learning data.

[1485] (Application example 2)

[1486] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1487] The problem that this invention aims to solve is that conventional sports broadcast commentary systems have difficulty in providing information tailored to individual user needs, and in particular, they are unable to respond based on real-time emotion recognition. Furthermore, because they lack the functionality of an entertainment system in an autonomous vehicle, users lack a means to effectively obtain sports information while driving.

[1488] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question from a user in natural language and generating an answer based on the extracted information, means for recognizing the user's emotional state, means for adjusting and providing the generated answer according to the user's emotional state, means for capturing facial expression data of passengers in real time using an in-vehicle camera and analyzing their emotional state, and means for providing the generated answer to the user in the autonomous vehicle in voice and text. This makes it possible to provide sports information in real time that meets individual needs while taking user emotions into consideration.

[1489] "Data sources" refer to various sources of information, such as official websites, local information sites, blogs, and social networking services.

[1490] "Means of obtaining information" refers to the methods and techniques used to collect the required information from the data source.

[1491] "Means for analyzing information" refers to the mechanism for processing the acquired information and extracting important data about players and teams.

[1492] "Important information" refers to statistical data, evaluations, performance data, etc. about players and teams obtained through analysis.

[1493] A "natural language question" is a question posed by a user via free-form text or voice.

[1494] "User emotional state" refers to the user's emotions identified using facial recognition and natural language processing technologies.

[1495] "Means for recognizing emotional states" refers to technology that analyzes a user's facial expression data and text input to identify their emotions.

[1496] The "answer generation means" refers to a mechanism that generates an answer to a user's question based on the analyzed information and the perceived emotional state.

[1497] "Means for adjusting and providing" refers to methods and techniques for appropriately changing the generated answer according to the user's emotions and communicating it to the user.

[1498] An "in-car camera" is a camera device installed inside an autonomous vehicle to capture passengers' facial expressions in real time.

[1499] "Facial expression data" refers to information captured as image or video data of a user's facial expressions.

[1500] An "autonomous vehicle" is a vehicle that can drive autonomously by utilizing artificial intelligence and sensor technology.

[1501] "Voice and text delivery means" refers to methods and technologies for communicating generated answers to users using speech synthesis and display technologies.

[1502] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and also combines it with an emotion engine that recognizes user emotions. It is particularly applicable to entertainment systems that allow users to comfortably receive sports information while driving in autonomous vehicles.

[1503] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. Information collected from data sources is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1504] The server then analyzes the collected data to extract key information about players and teams. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. It also extracts statistical data such as players' scores and performance from game results. This is done using technologies such as Hugging Face's Transformers and the Google Natural Language API.

[1505] The analyzed information is stored and updated in a database. Then, questions in natural language from users are analyzed. The server analyzes the user's question request received from the device, identifies the required information, and understands the intent of the question. Natural language processing technology is also used in this process.

[1506] In addition, the server recognizes the user's emotional state. It uses an in-car camera to capture facial expression data of passengers in the autonomous vehicle in real time, analyzes their emotions using facial recognition technology, and generates customized answers using a generative AI model (e.g., OpenAI's GPT-4) based on the analysis results.

[1507] The generated answers are adjusted appropriately according to the user's emotional state and are provided to the user in the autonomous vehicle via voice and text. By using speech synthesis and display technologies, real-time sports commentary is realized.

[1508] This system takes into consideration the user's emotions and makes it possible to provide real-time sports information tailored to individual needs. For example, if a user inputs a question such as "How many points did player X score in the current game?" and the system recognizes the user's emotional state as disappointment, it will provide commentary that includes encouragement, such as "Player X has currently scored 15 points in the game. He may not be in good form right now, but let's look forward to his future performances!"

[1509] Specific examples of prompt sentences are as follows:

[1510] example:

[1511] text

[1512] Question: How many points did player X score in the game?

[1513] Emotion: disappointment

[1514] Answer: Player XX is currently scoring 15 points per game. He may not be in good form right now, but let's look forward to his future performance!

[1515] In this way, the present invention can improve the user experience in an autonomous vehicle and provide comfortable live sports commentary.

[1516] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1517] Step 1:

[1518] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. The collected information is automatically retrieved through APIs and RSS feeds. For example, the latest match results can be obtained from the official website, and fan comments can be collected from social media. The input is data from the data sources, and the output is the collected raw data.

[1519] Step 2:

[1520] The server analyzes the collected raw data and extracts important information about players and teams. The analysis is performed using natural language processing techniques such as Hugging Face's Transformers and Google Natural Language API. The input is the collected raw data, and the output is the analyzed important information (e.g., player scores and performance).

[1521] Step 3:

[1522] The server stores and updates the analyzed information in a database, ensuring that the latest information is always available in real time. The input is the analyzed important information, and the output is an updated database.

[1523] Step 4:

[1524] The user inputs a question in natural language or asks it by voice. For example, a question like, "How many points did player X score in the current game?" The input is the user's question in natural language, and the output is the text data of the question.

[1525] Step 5:

[1526] The terminal captures the user's facial expression data in real time using an in-car camera. The captured image data is sent to a server. The input is the captured facial expression data, and the output is the facial expression image data.

[1527] Step 6:

[1528] The server analyzes the user's question request and facial expression image data received from the device. Facial expression image data is analyzed using facial recognition technology, and natural language processing technology is used to understand the intent of the question. For example, in response to the question "How many points did player X score in the current game?", a query is generated to obtain the score information of a specific player. The input is the question request and facial expression image data, and the output is the analysis result and the query.

[1529] Step 7:

[1530] The server retrieves the necessary information from the database based on the generated query, for example, the latest scores and performance data of a player. The input is the query, and the output is the retrieved information.

[1531] Step 8:

[1532] The server generates answers to the user's questions based on the emotion engine's analysis results and the acquired information. Generative AI such as OpenAI's GPT-4 is used for generation. If the user is disappointed, an answer containing encouragement is generated. The input is the emotion analysis results and the acquired information, and the output is the generated answer.

[1533] Step 9:

[1534] The server formats the generated answer in JSON format and sends it to the terminal. The terminal parses the received JSON format data, converts it into a displayable format, and provides it to the user in voice and text. The input is the generated answer, and the output is the voice and text format answer to the user.

[1535] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1536] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1537] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1538] [Fourth embodiment]

[1539] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1540] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1541] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1542] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1543] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1544] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1545] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1546] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1547] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1548] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1549] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1550] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1551] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1552] The present invention relates to a system for providing real-time commentary of live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[1553] Server Processing

[1554] 1. Data Collection:

[1555] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1556] 2. Data Analysis:

[1557] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[1558] 3. Update the database:

[1559] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[1560] 4. Question analysis:

[1561] It analyzes natural language questions from users and understands their intent. For example, if it receives a question like, "How is player X performing recently?", it generates a query to retrieve that player's recent match data.

[1562] 5. Information generation:

[1563] Based on the analyzed data, answers to user questions are generated. Generative AI is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, the generated explanation would be, "Player X has averaged 15.2 points in his last five games."

[1564] 6. Sending a response:

[1565] The generated answer is provided to the user, and the response is sent to the terminal in JSON format and displayed in the user interface (UI).

[1566] Terminal handling

[1567] 1. User Interface (UI):

[1568] The device displays a live sports broadcast screen and a question input field. The UI is designed to be intuitive, allowing users to easily input questions.

[1569] 2. Submit your request:

[1570] The question entered by the user is converted into JSON format and sent to the server.

[1571] 3. Receive and display the response:

[1572] It receives the response from the server, parses it, and converts it into a displayable format. For example, it displays the answer "Player X's average points this season is 15.2 points."

[1573] User processing

[1574] 1. Enter your question:

[1575] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1576] 2. Verify the information:

[1577] The server response will be displayed on your device, and you can enter additional questions if necessary.

[1578] 3. Feedback:

[1579] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[1580] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of users, providing a significantly improved user experience compared to traditional broadcasting.

[1581] The processing flow will be explained below.

[1582] Server Processing

[1583] Step 1: Data collection

[1584] 1. Configure the data source:

[1585] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[1586] 2. Send API request:

[1587] The server periodically sends requests to the API of the configured data source.

[1588] 3. Data Acquisition:

[1589] The server receives the returned data in JSON or XML format.

[1590] Example: Get the latest match results from the official website.

[1591] Step 2: Data analysis

[1592] 1. Data Parsing:

[1593] The server parses the retrieved data and converts it into structured data.

[1594] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[1595] 2. Data Formatting:

[1596] The server formats the parsed data and filters out the necessary information.

[1597] Example: Extracting positive comments about a specific player from social media comments.

[1598] 3. Update the database:

[1599] The server stores and updates the formatted data in the database.

[1600] Example: Updating player performance data with the latest match results.

[1601] Step 3: Analyze user questions

[1602] 1. User request received:

[1603] The server receives a user's question request sent from the terminal.

[1604] Example: You receive a question such as, "What is player X's average score this season?"

[1605] 2. Natural language analysis:

[1606] The server analyzes the received question using natural language processing technology and identifies the required information.

[1607] Example: Extract keywords to identify "X player's average points this season."

[1608] Step 4: Generate and execute a data query

[1609] 1. Query Generation:

[1610] The server generates a query to a database based on the question.

[1611] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1612] 2. Query execution:

[1613] The server executes the generated query against the database to obtain the required information.

[1614] Example: Retrieve player X's average points this season from the database.

[1615] Step 5: Information generation and transmission

[1616] 1. Information generation:

[1617] The server generates answers to the user's questions based on the acquired data.

[1618] Example: Generates the text "Player X's average points this season is 15.2 points."

[1619] 2. Send response:

[1620] The server sends the generated answer to the terminal as a response.

[1621] Terminal handling

[1622] Step 1: User Interface (UI)

[1623] 1. UI initialization:

[1624] The device displays a live sports broadcast screen and a question input field.

[1625] 2. User gesture detection:

[1626] The device detects when the user enters text into the input field.

[1627] Step 2: Submitting a request

[1628] 1. Enter your question:

[1629] The user types in a specific question.

[1630] For example, enter "What is player X's average points this season?"

[1631] 2. Request Generation:

[1632] The device converts the user's question into JSON format and generates a request.

[1633] 3. Submit your request:

[1634] The terminal sends the generated request to the server.

[1635] Step 3: Receive and display the response

[1636] 1. Response received:

[1637] The terminal receives the response from the server.

[1638] 2. Response parsing:

[1639] The terminal parses the response and converts it into a displayable format.

[1640] 3. Information display:

[1641] The terminal displays the analyzed information on the screen.

[1642] Example: Display "Player X's average points this season is 15.2 points."

[1643] User Action

[1644] Step 1: Enter your question

[1645] 1. Enter your question:

[1646] While watching the game, users can enter questions about players or teams they are interested in.

[1647] For example, enter "How many points did player X score in yesterday's game?"

[1648] Step 2: Verify the information

[1649] 1. Verify the information:

[1650] The user checks the response from the server displayed on the terminal.

[1651] 2. Enter additional questions:

[1652] If desired, the user can enter further questions.

[1653] Step 3: Feedback

[1654] 1. Providing Feedback:

[1655] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[1656] Example 1

[1657] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1658] In conventional sports broadcasts, it was difficult for users to obtain detailed information about the players or teams they were interested in in real time. Furthermore, there was a lack of a way to quickly and appropriately answer specific questions from users. Therefore, in order to increase user satisfaction, there is a need for more accurate information provision and faster responses.

[1659] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1660] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question in natural language from a user and understanding the intent of the question, means for generating an answer using a generative AI model based on the understood intent, and means for providing the generated answer to the user. This makes it possible to provide detailed information based on the individual needs of the user in real time and respond quickly and appropriately.

[1661] "Data sources" are sources of information such as official websites, local information sites, blogs, and social networking services for collecting information.

[1662] "Means of obtaining information" refers to technologies such as APIs and RSS feeds that automatically collect the necessary information from data sources.

[1663] "Means for analyzing information" refers to natural language processing and statistical analysis techniques used to analyze collected data and extract important information about players and teams.

[1664] "Means for analyzing user questions" refers to natural language processing technology for understanding questions in natural language from users and analyzing their intent.

[1665] A "generative AI model" is an artificial intelligence model that generates appropriate answers to user questions, and specifically includes generative AI models such as GPT-3.

[1666] "Means for generating answers" refers to technology that uses a generative AI model to generate answers to users' questions.

[1667] The "means for providing an answer" refers to the communication means and user interface for conveying the generated answer to the user.

[1668] The present invention relates to a system for providing real-time commentary on live sports broadcasts that meets the needs of individual users. Specific embodiments for carrying out the present invention are described below.

[1669] Server Processing

[1670] The server provides the technical infrastructure for retrieving information from a variety of data sources, analyzing the retrieved information, and responding to user queries. The server includes the following functions:

[1671] 1. Data Collection:

[1672] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services. For example, it retrieves the latest match results from the official website and collects fan comments from social media.

[1673] 2. Data Analysis:

[1674] The collected data is parsed and key information about players and teams is extracted using natural language processing techniques. For example, it can extract only positive opinions about a specific player from social media comments, or derive statistical data such as a player's score and performance from game results.

[1675] 3. Update the database:

[1676] The extracted information is stored and updated in a database, which allows the latest information to be kept up to date. Examples of databases that can be used include MySQL and PostgreSQL.

[1677] 4. Question analysis:

[1678] It uses natural language processing models such as BERT and GPT-3 to analyze natural language questions from users and understand the intent of the question. For example, it analyzes the question "How is player X's recent performance?" and generates a query to retrieve the player's recent game data.

[1679] 5. Information generation:

[1680] Based on the analyzed data, answers to user questions are generated. Generative AI models such as GPT-3 are used for generation, and explanations are provided in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "player X has averaged 15.2 points in his last five games" is generated.

[1681] 6. Sending a response:

[1682] The generated answer is sent to the device in JSON format, and the response is displayed in the user interface (UI).

[1683] Terminal handling

[1684] The terminal provides an interface for users to enter questions and display answers from the server. The terminal includes the following features:

[1685] 1. User Interface (UI):

[1686] The device displays a live sports broadcast screen and a question input field. The UI is built using React.js and other tools, and is designed to make it easy for users to enter questions.

[1687] 2. Submit your request:

[1688] The question entered by the user is converted into JSON format and sent to the server. For example, if the user enters a question such as "How is player X's recent performance?", this question is sent to the server using the axios.post method.

[1689] 3. Receive and display the response:

[1690] Receives the response from the server and converts it into a displayable format. For example, if the server returns an answer such as "Player X's average score this season is 15.2 points," it displays it on the screen.

[1691] User processing

[1692] Users can watch live sports broadcasts through their devices and ask questions about information they are interested in. The following functions are included in the user-side processing:

[1693] 1. Enter your question:

[1694] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1695] 2. Verify the information:

[1696] The user sees the response from the server displayed on the device and can enter additional questions if needed.

[1697] 3. Feedback:

[1698] Users can provide feedback on the information they provide, which helps improve the accuracy of the system.

[1699] Examples and prompts

[1700] For example, if a user types a question like, "How is player X performing recently?", the server collects and analyzes past game data and social media comments to generate an answer such as, "X player has averaged 15.2 points in his last five games."

[1701] Example prompt sentence:

[1702] "Please tell me about player XX's recent performance."

[1703] In this way, the present invention can provide real-time sports commentary tailored to the individual needs of the user.

[1704] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1705] Step 1: Data collection

[1706] The server collects information from official websites, local information sites, blogs, and social networking services. As input, it provides API endpoints and RSS feed URLs, which it uses to retrieve data. As output, it obtains data in JSON or XML format. Specifically, it sends a "GET" request to the official website's API to retrieve match results and player information. It also uses the Twitter API to collect tweets containing specific hashtags.

[1707] Step 2: Data analysis

[1708] The server parses the collected data and extracts important information about players and teams. The JSON or XML data collected in Step 1 is used as input. The output is an analysis of players' performance and social media comments. Specifically, the server parses the data using the json.loads() function and uses natural language processing technology to classify comments as positive or negative. It also analyzes match results to extract players' points and assists.

[1709] Step 3: Update the database

[1710] The server stores and updates the parsed information in the database. The parsed results from step 2 are used as input. The output is to update the database with the latest information. Specifically, the SQLAlchemy library is used to execute a query to insert the latest player statistics into the database. For example, an SQL statement such as INSERT INTO player_stats (player_id, match_id, points) VALUES (...) is generated and executed.

[1711] Step 4: Question Analysis

[1712] The server analyzes natural language questions from users and understands their intent. The input is a text question from the user. The output is a query that reflects the intent of the question. Specifically, the BERT model is used to analyze the question "How is player X's recent performance?" and generate a query that identifies "player X" and asks for statistical information related to his or her "performance."

[1713] Step 5: Information Generation

[1714] The server generates an answer to the user's question based on the analyzed data. The query generated in step 4 and information obtained from the database are used as input. The output is an answer expressed in natural language. Specifically, the query result is input into a generative AI model such as GPT-3, and an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[1715] Step 6: Sending a response

[1716] The server sends the generated answer in JSON format to the terminal. The answer generated in step 5 is used as input. The output is a JSON response to be sent to the terminal. Specifically, a response is generated to return the generated answer in JSON format to the client side through Flask's API endpoint.

[1717] Step 7: User Interface (UI) Display

[1718] The device displays a sports broadcast screen and a question input field. HTML and JavaScript files are used as input. An interface where users can enter questions is displayed on the screen as output. Specifically, React.js is used to build the question input field and broadcast screen, providing a UI that users can operate intuitively.

[1719] Step 8: Submitting the request

[1720] The device converts the question entered by the user into JSON format and sends it to the server. The text question entered by the user is used as input. An HTTP request to the server is generated as output. Specifically, the question "How has player X performed recently?" is sent to the server using the axios.post method.

[1721] Step 9: Receive and display the response

[1722] The terminal receives the response from the server and converts it into a displayable format. The JSON response returned by the server is used as input. The output is a text answer that is displayed on the screen. Specifically, the response received by axios is analyzed and the text "Player X's average score this season is 15.2 points" is displayed on the screen.

[1723] Step 10: Feedback

[1724] Users provide feedback on the information provided. The text of the feedback provided by the user is used as input. The feedback content is sent to the server as output, and is used to improve the system. Specifically, a function is provided to enter feedback such as "The information was accurate" or "The explanation was easy to understand" in an input field and send it.

[1725] (Application example 1)

[1726] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1727] Traditional sports broadcasting systems often only provided general information, making it difficult for users to obtain real-time commentary tailored to their individual needs. Furthermore, users lacked the means to quickly obtain detailed information about specific players or teams while watching a game. This limited the user experience and reduced the enjoyment of viewing.

[1728] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1729] In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about people and organizations, means for analyzing a question entered in natural language by a user and generating an answer based on the extracted information, means for providing the generated answer to the user, means for providing real-time commentary about live sports broadcasts using a mobile communication device, means for converting a question entered by a user into JSON format and sending the JSON format to the server, and means for receiving, analyzing, and converting a response from the server into a displayable format, thereby enabling users to quickly obtain detailed, real-time commentary tailored to their individual needs while watching a game.

[1730] "Data sources" are information sources for automatically obtaining information from official websites, local information sites, blogs, social networking services, etc.

[1731] "Acquired information" refers to information such as match results, player and team performance data, and comments on social media collected from data sources.

[1732] "Analysis" is the act of automatically processing and analyzing acquired information and extracting necessary information and patterns.

[1733] "Person" is a term that refers to a specific individual, such as a sports player or a related person.

[1734] "Organization" is a term that refers to an organization or group, such as a sports team or association.

[1735] "Users" refer to sports spectators and viewers who use this system.

[1736] "Natural language" refers to the language that users use in their daily lives, in the form of input before it is converted into a machine-interpretable format.

[1737] A "question" is something a user inputs in natural language about the information they want to know.

[1738] "JSON format" is a type of data exchange format and is an abbreviation for JavaScript Object Notation.

[1739] "Server" means a central processing unit that collects and analyzes data and generates and provides answers to user questions.

[1740] "Mobile communication device" refers to a mobile communication device such as a smartphone or tablet.

[1741] "Real-time commentary" refers to detailed information and commentary that is available instantly while watching live sports broadcasts.

[1742] A "response" is an answer that a server generates to a user's question.

[1743] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs. The system has the function of acquiring information from data sources, analyzing the acquired information, and generating answers to user questions.

[1744] Server Processing

[1745] The server retrieves information from data sources such as the official website, local information sites, blogs, and social networking services. Information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1746] The collected information is analyzed to extract important information about individuals and organizations. Natural language processing technology is used for the analysis, making it possible to extract only positive opinions about a specific individual from social media comments, for example. Statistical data such as player scores and performance can also be extracted from match results.

[1747] The extracted information is stored and updated in a database, and the latest information is always kept in real time. For example, player performance data is updated after every game.

[1748] Questions from users are sent to the server in natural language. The server analyzes the question and understands the user's intent. For example, if the server receives a question like "How has player X performed recently?", it generates a query to retrieve the player's recent game data.

[1749] Based on the analyzed data, answers to user questions are generated. A generative AI model is used for generation, providing explanations in natural language. For example, if player X's average score in his last five games is 15.2 points, an explanation such as "Player X has averaged 15.2 points in his last five games" is generated.

[1750] The generated answer is sent to the user's device in JSON format.

[1751] Terminal handling

[1752] The device displays a live sports broadcast screen and a question input field. The user interface (UI) is designed to be intuitive, allowing users to easily input questions.

[1753] The question entered by the user is converted into JSON format and sent to the server. The response from the server is received, parsed, and converted into a displayable format. For example, the answer displayed is "Player X's average points this season is 15.2 points."

[1754] User Actions

[1755] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?". They can also provide feedback on the information provided, which helps improve the accuracy of the system.

[1756] Examples of concrete examples and prompts

[1757] As a concrete example, if a user uses a smartphone app while watching a game to enter a question such as "How many points did player X score in yesterday's game?", the following prompt sentence will be generated:

[1758] Prompt Sentence Examples

[1759] "How many points did player X score in yesterday's game?"

[1760] Using this system, users can quickly obtain detailed, real-time commentary tailored to their individual needs while watching a sporting event, making live sports broadcasts even more enjoyable.

[1761] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1762] Processing step flow

[1763] Step 1:

[1764] The server obtains information from data sources such as official websites, local information sites, blogs, and social networking services (SNS). Information is collected automatically through APIs and RSS feeds. The input is various information provided by the data sources, and the output is raw data stored in temporary data storage on the server.

[1765] Step 2:

[1766] The server analyzes the collected information and uses natural language processing technology to extract important information about individuals and organizations. For example, it can extract positive opinions about a specific individual from social media comments, or statistical data such as a player's score and performance from game results. The input is the acquired raw data, and the output is the analyzed important information.

[1767] Step 3:

[1768] The server stores and updates the extracted information in the database, which keeps the database up-to-date in real time. The input is the analyzed important information, and the output is the updated database information.

[1769] Step 4:

[1770] The device displays a live sports broadcast screen and a question input field through a user interface (UI). The user inputs a question in natural language. The input is the text question entered by the user, and the output is the question data converted to JSON format.

[1771] Step 5:

[1772] The terminal converts the input question into JSON format and sends it to the server. The converted question is sent via the API. The input is text-formatted question data, and the output is JSON-formatted question data sent to the server.

[1773] Step 6:

[1774] The server receives a question in JSON format sent by the user and uses natural language processing technology to analyze the question and understand the user's intent. The input is the question data in JSON format, and the output is the analyzed intent information.

[1775] Step 7:

[1776] The server extracts relevant data from the database based on the analyzed user intent and generates commentary in natural language using a generative AI model. For example, in response to the question "How is player X's recent performance?", it generates commentary such as "Player X has averaged 15.2 points in the last five games" based on the latest match data. The input is the analyzed intent information and related data from the database, and the output is commentary generated in natural language.

[1777] Step 8:

[1778] The server sends the generated explanation to the terminal in JSON format. The input is the generated natural language explanation, and the output is JSON format data.

[1779] Step 9:

[1780] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input is the response data in JSON format, and the output is a displayable text description.

[1781] Step 10:

[1782] The user checks the explanation displayed on the terminal and inputs additional questions as necessary. They can also provide feedback on the information provided. This feedback information is sent to the server and used to improve the accuracy of the system. The input is the displayed explanation information and feedback, and the output is increased user satisfaction and improved system performance.

[1783] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1784] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[1785] Server Processing

[1786] 1. Data Collection:

[1787] The server collects information from various data sources, such as the official website, local information sites, blogs, and social networking services. This information is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[1788] 2. Data Analysis:

[1789] The collected data is parsed and key information about players and teams is extracted. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. Statistical data such as player scores and performance is also extracted from match results.

[1790] 3. Update the database:

[1791] The extracted information is stored and updated in a database, ensuring that the latest information is always available in real time. For example, player performance data is updated after each game.

[1792] 4. Emotion analysis:

[1793] The server receives questions from users and facial expression data displayed on the device, and analyzes them using an emotion engine. Emotion analysis uses facial recognition and natural language processing technologies to identify emotional states from the text and facial expressions entered by the user.

[1794] Example: If a user types, "Player X was not good today either...", the emotion engine will detect "disappointment."

[1795] 5. Question analysis:

[1796] The server analyzes the user's question request received from the device, and uses natural language processing technology to understand the intent of the question and identify the required information.

[1797] Example: If a question is received such as "What is player X's average points this season?", a query will be generated to retrieve that player's game data this season.

[1798] 6. Data query generation and execution:

[1799] The server generates a query to a database based on the question to retrieve the required information.

[1800] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1801] 7. Information generation:

[1802] Based on the acquired data and the analysis results of the emotion engine, answers to user questions are generated using generative AI, which adjusts the answers according to the user's emotional state.

[1803] For example, if a user is detected as "disappointed," an encouraging commentary will be provided, such as, "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[1804] 8. Sending a response:

[1805] The generated response is formatted in JSON and sent to the terminal.

[1806] Terminal handling

[1807] 1. User Interface (UI):

[1808] The device displays a live sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[1809] 2. Submit your request:

[1810] The question entered by the user and the emotion-recognized facial expression data are converted into JSON format and sent to the server.

[1811] 3. Receive and display the response:

[1812] Receives responses from the server, analyzes them, and converts them into a displayable format. Displays answers based on the analyzed information and emotions on the screen.

[1813] User Action

[1814] 1. Enter your question:

[1815] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1816] 2. Facial expression recognition:

[1817] The user's facial expression data is automatically collected through the camera and analyzed by the emotion engine.

[1818] 3. Verify the information:

[1819] Check the response from the server displayed on the device. For example, it may say, "Player X's average points this season is 15.2 points."

[1820] 4. Feedback:

[1821] Users can provide feedback on the information they provide, which helps improve the system's accuracy and user experience.

[1822] In this way, the present invention can recognize user emotions and provide personalized real-time sports commentary, providing a significantly improved user experience compared to traditional broadcasts.

[1823] The processing flow will be explained below.

[1824] Server Processing

[1825] Step 1: Data collection

[1826] 1. Configure the data source:

[1827] The server sets up data sources such as official homepages, local information sites, blogs, and social networking services.

[1828] 2. Send API request:

[1829] The server periodically sends requests to the API of the configured data source.

[1830] 3. Data Acquisition:

[1831] The server receives the returned data in JSON or XML format.

[1832] Example: Get the latest match results from the official website.

[1833] Step 2: Data analysis

[1834] 1. Data Parsing:

[1835] The server parses the retrieved data and converts it into structured data.

[1836] Example: Parse JSON data to extract player names, scores, dates and times, etc.

[1837] 2. Data Formatting:

[1838] The server formats the parsed data and filters out the necessary information.

[1839] Example: Extracting positive comments about a specific player from social media comments.

[1840] 3. Update the database:

[1841] The server stores and updates the formatted data in the database.

[1842] Example: Updating player performance data with the latest match results.

[1843] Step 3: Sentiment Analysis

[1844] 1. Receiving questions and facial expression data:

[1845] The server receives the user's questions and facial expression data sent from the terminal.

[1846] For example, if a user types in "How many points did player X score in yesterday's game?", the collected facial expression data will be received along with the text.

[1847] 2. Natural Language and Facial Expression Analysis:

[1848] The server analyzes the received question text and facial expression data to identify the user's emotions.

[1849] Example: Analyzing emotions such as "surprise" and "disappointment" from question text and facial expression data.

[1850] Step 4: Question Analysis

[1851] 1. Question analysis:

[1852] The server uses natural language processing technology to understand the intent of the user's question.

[1853] Example: Analyze the question "What is player X's average points this season?" and identify relevant information.

[1854] Step 5: Generate and execute a data query

[1855] 1. Query Generation:

[1856] The server generates a query to a database based on the question.

[1857] Example: Generates the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1858] 2. Query execution:

[1859] The server executes the generated query against the database to obtain the required information.

[1860] Example: Retrieve player X's average points this season from the database.

[1861] Step 6: Information Generation

[1862] 1. Information generation:

[1863] The server generates answers to the user's questions based on the acquired data and the analysis results of the emotion engine.

[1864] For example, if a user is detected as "disappointed," an encouraging commentary will be generated, such as "Player X hasn't been in good form recently, but his average score this season is 15.2 points, which is still the highest on the team."

[1865] Step 7: Sending a response

[1866] 1. Response formatting:

[1867] The server formats the generated response in JSON format.

[1868] 2. Send response:

[1869] The server sends the formatted response to the terminal.

[1870] Terminal handling

[1871] Step 1: User Interface (UI)

[1872] 1. UI initialization:

[1873] The device displays a sports broadcast screen, a question input field, and a camera for facial recognition.

[1874] 2. User gesture detection:

[1875] The device uses a camera to detect the user's movements and facial expressions as they enter text into an input field.

[1876] Step 2: Submitting a request

[1877] 1. Enter your question:

[1878] The user types in a specific question.

[1879] For example, enter "What is player X's average points this season?"

[1880] 2. Acquiring facial expression data:

[1881] The device collects facial expression data from the user through the camera.

[1882] 3. Request Generation:

[1883] The device converts the user's question and facial expression data into JSON format to generate a request.

[1884] 4. Submit your request:

[1885] The terminal sends the generated request to the server.

[1886] Step 3: Receive and display the response

[1887] 1. Response received:

[1888] The terminal receives the response from the server.

[1889] 2. Response parsing:

[1890] The terminal parses the response and converts it into a displayable format.

[1891] 3. Information display:

[1892] The terminal displays the analyzed information on the screen.

[1893] Example: Display "Player X's average points this season is 15.2 points."

[1894] User Action

[1895] Step 1: Enter your question

[1896] 1. Enter your question:

[1897] While watching the game, users can enter questions about players or teams they are interested in.

[1898] For example, enter "How many points did player X score in yesterday's game?"

[1899] Step 2: Facial Expression Recognition

[1900] 1. Facial expression recognition:

[1901] The user's facial expression data is automatically collected through the camera.

[1902] Step 3: Verify the information

[1903] 1. Verify the information:

[1904] The user checks the response from the server displayed on the terminal.

[1905] 2. Enter additional questions:

[1906] If desired, the user can enter further questions.

[1907] Step 4: Feedback

[1908] 1. Providing Feedback:

[1909] Users can provide feedback on the information they provide, which is used to improve the accuracy of the system.

[1910] Example 2

[1911] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1912] Conventional sports broadcasting systems have difficulty providing real-time commentary that responds to the individual needs and emotional state of users. This leads to a poor user experience and difficulty in improving satisfaction. Another issue is that answers to questions entered by users are standardized, making it difficult to provide personalized responses.

[1913] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for identifying an emotional state by analyzing facial expression data of a user, means for analyzing a question from the user in natural language and generating an answer based on the extracted information and the identified emotional state, and means for providing the generated answer to the user. This makes it possible to provide real-time sports commentary that meets the individual needs and emotional state of each user.

[1914] "Data sources" are information sources for obtaining information from official websites, local information sites, blogs, social networking services, etc.

[1915] A "means of obtaining information" is a mechanism for automatically gathering information from data sources, such as through an API or RSS feed.

[1916] "Means of analyzing information" is the process of parsing the collected information and extracting key data about players and teams.

[1917] "Facial expression data" is image data used to capture a user's facial expression, and is basic data for analyzing the user's emotions.

[1918] "Emotional state" refers to the user's emotional state, such as disappointment or excitement, analyzed based on the user's facial expression data and natural language input.

[1919] "Means for question analysis" refers to technology for understanding natural language questions from users and identifying their intent.

[1920] The "answer generation means" is the process for creating an answer to the user's question based on the extracted information and the identified emotional state.

[1921] A "means for providing an answer" is a mechanism for communicating and displaying the generated answer to the user.

[1922] The present invention relates to a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and further combines it with an emotion engine that recognizes user emotions. Specific embodiments for carrying out the present invention are described below.

[1923] Server Processing

[1924] Data collection:

[1925] The server collects information from data sources such as the official website, local information sites, blogs, and social networking services (SNS). This process uses APIs and RSS feeds. For example, the latest match results are obtained from the official website, and fan comments are collected from SNS.

[1926] Data Analysis:

[1927] The server analyzes the collected data and extracts important information about players and teams, using natural language processing technologies such as Apache OpenNLP and Google Cloud Natural Language API to extract positive opinions and statistical data from social media comments.

[1928] Database update:

[1929] The server stores the extracted information in a database and updates it in real time. For example, player performance data is updated after each game. The database uses MySQL or PostgreSQL.

[1930] Emotion analysis:

[1931] The server receives the questions and facial expression data sent by the user and analyzes them with the emotion engine. Using Azure Face API and Amazon Rekognition, the server uses facial recognition and natural language processing technology to identify the user's emotional state. For example, if a user types, "Player X, you're not doing so well today...", the emotion engine will detect "disappointment."

[1932] Question analysis:

[1933] The server uses natural language processing technology to analyze the user's question. For example, if it receives a question like "What is player X's average score this season?", it converts the question into a database query.

[1934] Data query generation and execution:

[1935] The server generates a query to the database based on the question and retrieves the information. Specifically, it generates and executes the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'".

[1936] Information generation:

[1937] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The answers are adjusted according to the user's emotional state. For example, if the user is disappointed, an encouraging answer such as, "It's true that player X hasn't been playing well recently, but his average score of 15.2 points this season is still the highest on the team" will be generated.

[1938] Response sent:

[1939] The server formats the generated response in JSON format and sends it to the terminal.

[1940] Terminal handling

[1941] User Interface (UI):

[1942] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. The UI is designed to be intuitive, allowing users to easily input questions and automatically recognize facial expressions.

[1943] Submit a request:

[1944] The device converts the question entered by the user and the emotion-recognized facial expression data into JSON format and sends it to the server.

[1945] Response received and displayed:

[1946] The device receives the response from the server, analyzes it, and converts it into a displayable format. It then displays an answer based on the analyzed information and emotions on the screen.

[1947] User Action

[1948] Enter your question:

[1949] While watching a game, users can input questions about players or teams they are interested in. For example, they can input, "How many points did player X score in yesterday's game?"

[1950] Facial expression recognition:

[1951] The device's camera captures the user's face and sends the image data to the emotion engine in real time.

[1952] Verify the information:

[1953] The user checks the response from the server displayed on the device, for example, "Player X's average points this season is 15.2 points."

[1954] feedback:

[1955] Users can provide feedback on the information provided, such as whether it was helpful or not, which improves the accuracy of the system and the user experience.

[1956] Examples and prompts

[1957] For example, if a user asks, "When is player X's next game scheduled?", the server retrieves the date of the next game and information about the opposing team from the database and generates a positive response through the emotion engine.

[1958] Example prompt sentence:

[1959] "Provide information about a player's next game schedule and opposing team based on the user's question and facial expression. Example question: 'When is player X's next game scheduled?' Facial expression: Excitement"

[1960] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1961] Step 1: Data collection

[1962] The server uses APIs and RSS feeds to collect information from official websites, local information sites, blogs, and social networking services (SNS). For example, the server accesses endpoints such as "http: / / officialsite.com / api / results" and "http: / / sns.com / api / comments" to retrieve match results and fan comments. The input data is in XML or JSON format, and the output data is saved in the server's storage.

[1963] Step 2: Data analysis

[1964] The server analyzes the collected data and extracts important information related to players and teams. It uses Apache OpenNLP and Google Cloud Natural Language API to parse the text data and extract positive opinions and statistical data. The input data is the raw data collected in the previous step, and the output data is the parsed structured data. For example, from a comment such as "Player X played well today," the server extracts the name of "Player X" and the sentiment "positive."

[1965] Step 3: Update the database

[1966] The server stores the analyzed data in a database such as MySQL or PostgreSQL. Based on the analysis results, it executes INSERT and UPDATE statements to update information such as player performance and fan comments. The input data is the analysis result of the previous step, and the output data is the updated database. For example, the data is stored by executing the SQL statement "INSERT INTO performance (player, date, points) VALUES ('〇〇', '2023-10-01', 25)".

[1967] Step 4: Sentiment Analysis

[1968] The server receives questions and facial expression data sent by users and analyzes them using an emotion engine. Azure Face API and Amazon Rekognition are used for facial recognition technology, and natural language processing technology is used to identify emotions from the user's text input. The input data is the user's question and facial expression image data, and the output data is the analyzed emotional state. For example, an input such as "Player X is doing poorly again today..." is identified as the emotion of "disappointment."

[1969] Step 5: Question Analysis

[1970] The server uses natural language processing technology to analyze natural language questions from users. The input data is the user's question text, and the output data is structured data that identifies the intent of the question. For example, the server analyzes the question "What is player X's average score this season?" and extracts the information "player='X' AND season='current'".

[1971] Step 6: Generate and execute a data query

[1972] The server generates a query to the database based on the question and retrieves the information. The input data is the result of question analysis, and the output data is the information retrieved from the database. For example, execute the query "SELECT AVG(points) FROM performance WHERE player='〇〇' AND season='current'" to retrieve the average score.

[1973] Step 7: Information Generation

[1974] The server uses generative AI (such as GPT-3) to generate answers to user questions based on the acquired data and the analysis results of the emotion engine. The input data is information from the database and the results of emotion analysis, and the output data is the generated answer text. For example, in response to the emotion detected as "disappointment," the server generates an answer such as "It's true that player X hasn't been in good form recently, but his average score this season of 15.2 points is still the highest on the team."

[1975] Step 8: Sending a response

[1976] The server formats the generated answer in JSON format and sends it to the terminal. The input data is the generated answer text, and the output data is the sent JSON response.

[1977] Step 9: User Interface (UI)

[1978] The device uses front-end frameworks such as React.js and Vue.js to display a sports broadcast screen, a question input field, and a camera for facial recognition. It provides an interface that allows users to easily input questions. The input data are user actions, and the output data are the displayed UI elements.

[1979] Step 10: Submitting the request

[1980] The device converts the user's input question and emotion-recognized facial expression data into JSON format and sends it to the server. The input data is the user's question and facial expression data, and the output data is the request sent to the server.

[1981] Step 11: Receive and display the response

[1982] The terminal receives the response from the server, parses it, and converts it into a displayable format. The input data is the JSON-formatted response from the server, and the output data is the answer text that is displayed on the screen.

[1983] Step 12: Enter your question

[1984] While watching a game, a user can input a question about a player or team that they are interested in. For example, they might input, "How many points did player X score in yesterday's game?" The input data is the user's question text, and the output data is the question data sent to the device.

[1985] Step 13: Facial Expression Recognition

[1986] The device camera captures the user's face and sends the image data to the emotion engine in real time. The input data is the user's facial expression image data, and the output data is the analyzed emotional state.

[1987] Step 14: Verify the information

[1988] The user confirms the response from the server displayed on the device. For example, "Player X's average points this season is 15.2 points." The input data is the text displayed on the screen, and the output data is the information confirmed by the user.

[1989] Step 15: Feedback

[1990] Users provide feedback on the information provided, such as "it was helpful" or "it wasn't." This allows the system to learn and improve the accuracy of answers from the next time onwards. The input data is the user's feedback, and the output data is the system's learning data.

[1991] (Application example 2)

[1992] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1993] The problem that this invention aims to solve is that conventional sports broadcast commentary systems have difficulty in providing information tailored to individual user needs, and in particular, they are unable to respond based on real-time emotion recognition. Furthermore, because they lack the functionality of an entertainment system in an autonomous vehicle, users lack a means to effectively obtain sports information while driving.

[1994] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from a data source, means for analyzing the acquired information and extracting important information about players and teams, means for analyzing a question from a user in natural language and generating an answer based on the extracted information, means for recognizing the user's emotional state, means for adjusting and providing the generated answer according to the user's emotional state, means for capturing facial expression data of passengers in real time using an in-vehicle camera and analyzing their emotional state, and means for providing the generated answer to the user in the autonomous vehicle in voice and text. This makes it possible to provide sports information in real time that meets individual needs while taking user emotions into consideration.

[1995] "Data sources" refer to various sources of information, such as official websites, local information sites, blogs, and social networking services.

[1996] "Means of obtaining information" refers to the methods and techniques used to collect the required information from the data source.

[1997] "Means for analyzing information" refers to the mechanism for processing the acquired information and extracting important data about players and teams.

[1998] "Important information" refers to statistical data, evaluations, performance data, etc. about players and teams obtained through analysis.

[1999] A "natural language question" is a question posed by a user via free-form text or voice.

[2000] "User emotional state" refers to the user's emotions identified using facial recognition and natural language processing technologies.

[2001] "Means for recognizing emotional states" refers to technology that analyzes a user's facial expression data and text input to identify their emotions.

[2002] The "answer generation means" refers to a mechanism that generates an answer to a user's question based on the analyzed information and the perceived emotional state.

[2003] "Means for adjusting and providing" refers to methods and techniques for appropriately changing the generated answer according to the user's emotions and communicating it to the user.

[2004] An "in-car camera" is a camera device installed inside an autonomous vehicle to capture passengers' facial expressions in real time.

[2005] "Facial expression data" refers to information captured as image or video data of a user's facial expressions.

[2006] An "autonomous vehicle" is a vehicle that can drive autonomously by utilizing artificial intelligence and sensor technology.

[2007] "Voice and text delivery means" refers to methods and technologies for communicating generated answers to users using speech synthesis and display technologies.

[2008] This invention is a system that provides real-time commentary on live sports broadcasts tailored to individual user needs, and also combines it with an emotion engine that recognizes user emotions. It is particularly applicable to entertainment systems that allow users to comfortably receive sports information while driving in autonomous vehicles.

[2009] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. Information collected from data sources is automatically retrieved through APIs and RSS feeds. For example, the latest match results are retrieved from the official website, and fan comments are collected from social media.

[2010] The server then analyzes the collected data to extract key information about players and teams. Natural language processing technology is used for the analysis, allowing it to extract only positive opinions about specific players from social media comments. It also extracts statistical data such as players' scores and performance from game results. This is done using technologies such as Hugging Face's Transformers and the Google Natural Language API.

[2011] The analyzed information is stored and updated in a database. Then, questions in natural language from users are analyzed. The server analyzes the user's question request received from the device, identifies the required information, and understands the intent of the question. Natural language processing technology is also used in this process.

[2012] In addition, the server recognizes the user's emotional state. It uses an in-car camera to capture facial expression data of passengers in the autonomous vehicle in real time, analyzes their emotions using facial recognition technology, and generates customized answers using a generative AI model (e.g., OpenAI's GPT-4) based on the analysis results.

[2013] The generated answers are adjusted appropriately according to the user's emotional state and are provided to the user in the autonomous vehicle via voice and text. By using speech synthesis and display technologies, real-time sports commentary is realized.

[2014] This system takes into consideration the user's emotions and makes it possible to provide real-time sports information tailored to individual needs. For example, if a user inputs a question such as "How many points did player X score in the current game?" and the system recognizes the user's emotional state as disappointment, it will provide commentary that includes encouragement, such as "Player X has currently scored 15 points in the game. He may not be in good form right now, but let's look forward to his future performances!"

[2015] Specific examples of prompt sentences are as follows:

[2016] example:

[2017] text

[2018] Question: How many points did player X score in the game?

[2019] Emotion: disappointment

[2020] Answer: Player XX is currently scoring 15 points per game. He may not be in good form right now, but let's look forward to his future performance!

[2021] In this way, the present invention can improve the user experience in an autonomous vehicle and provide comfortable live sports commentary.

[2022] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2023] Step 1:

[2024] The server collects sports-related information from data sources such as official websites, local information sites, blogs, and social networking services. The collected information is automatically retrieved through APIs and RSS feeds. For example, the latest match results can be obtained from the official website, and fan comments can be collected from social media. The input is data from the data sources, and the output is the collected raw data.

[2025] Step 2:

[2026] The server analyzes the collected raw data and extracts important information about players and teams. The analysis is performed using natural language processing techniques such as Hugging Face's Transformers and Google Natural Language API. The input is the collected raw data, and the output is the analyzed important information (e.g., player scores and performance).

[2027] Step 3:

[2028] The server stores and updates the analyzed information in a database, ensuring that the latest information is always available in real time. The input is the analyzed important information, and the output is an updated database.

[2029] Step 4:

[2030] The user inputs a question in natural language or asks it by voice. For example, a question like, "How many points did player X score in the current game?" The input is the user's question in natural language, and the output is the text data of the question.

[2031] Step 5:

[2032] The terminal captures the user's facial expression data in real time using an in-car camera. The captured image data is sent to a server. The input is the captured facial expression data, and the output is the facial expression image data.

[2033] Step 6:

[2034] The server analyzes the user's question request and facial expression image data received from the device. Facial expression image data is analyzed using facial recognition technology, and natural language processing technology is used to understand the intent of the question. For example, in response to the question "How many points did player X score in the current game?", a query is generated to obtain the score information of a specific player. The input is the question request and facial expression image data, and the output is the analysis result and the query.

[2035] Step 7:

[2036] The server retrieves the necessary information from the database based on the generated query, for example, the latest scores and performance data of a player. The input is the query, and the output is the retrieved information.

[2037] Step 8:

[2038] The server generates answers to the user's questions based on the emotion engine's analysis results and the acquired information. Generative AI such as OpenAI's GPT-4 is used for generation. If the user is disappointed, an answer containing encouragement is generated. The input is the emotion analysis results and the acquired information, and the output is the generated answer.

[2039] Step 9:

[2040] The server formats the generated answer in JSON format and sends it to the terminal. The terminal parses the received JSON format data, converts it into a displayable format, and provides it to the user in voice and text. The input is the generated answer, and the output is the voice and text format answer to the user.

[2041] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2042] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neur...

Claims

1. a means for obtaining information from a data source; and A means of analyzing the acquired information and extracting important information about players and teams; A means of analyzing natural language questions from users and generating answers based on the extracted information; The system includes a means for providing the generated answer to the user.

2. The system of claim 1, wherein the data sources include official websites, local information sites, blogs, and social networking services.

3. 2. The system according to claim 1, wherein the system analyzes information and generates answers according to individual needs according to the user's hobbies and preferences.

Images