system

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a user's input information and creating a user account; means for storing the user's authentication information and authenticating the user at the next login; means for generating and delivering a streaming URL corresponding to an event selected by the user; means for recommending a specific live event or performer based on the user's past listening history; and means for relaying a real-time chat between users and delivering to other users.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Participating in live events requires physical travel, time, and expense, making it difficult for users with certain conditions to participate. Furthermore, hosting live events is also difficult during times of restricted movement, such as the coronavirus pandemic. It is necessary to overcome these constraints and provide a way for a wide range of users to easily experience live events remotely. Furthermore, a means is also needed to enable users to enjoy a more personalized experience. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for receiving user input information and creating a user account, a means for saving the user's authentication information and authenticating the user the next time the user logs in, a means for generating and distributing a streaming URL corresponding to an event selected by the user, a means for recommending specific live events and performers based on the user's past listening history, and a means for relaying real-time chats between users and broadcasting them to other users. The system aims to enable users to experience an immersive live event from the comfort of their own home using virtual reality and augmented reality technology. Furthermore, by using generative AI technology, it is possible to provide a personalized experience by making individually customized recommendations based on the user's preferences. This allows users to enjoy a high-quality live experience without being subject to physical, time, or financial constraints.

[0006] "User" means an individual or end user of the System.

[0007] A "user account" is a digital profile that contains authentication information and profile information for a user to access the system.

[0008] "Authentication Information" means information used to verify a User's identity, including a username, password, or other means of authentication.

[0009] A "streaming URL" is a resource address on the Internet that streams video and audio from a live event to users in real time.

[0010] "Listening History" is a record of music and events that a User has previously viewed or listened to through the System.

[0011] "Generative AI technology" is an artificial intelligence technology that analyzes user data and makes individually customized recommendations.

[0012] "Virtual reality (VR)" is a technology that uses computer technology to provide users with a virtual environment that is different from the real world.

[0013] Augmented reality (AR) is a technology that overlays digital information and virtual objects onto the real-world environment.

[0014] A "live event" is an event that broadcasts real-time performances by artists or performers.

[0015] A "personalized experience" is an experience that is customized based on a user's individual preferences and history, making it more relevant and interesting to the user.

[0016] "Real-time chat" is a means of real-time communication between users participating in the same event. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies, and also provides personalized recommendations for live events and artists based on the user's listening history and preferences, as well as the ability to interact with other users in real time.

[0039] User registration and login function

[0040] User Registration

[0041] The user enters their information (username, password, email address) on the account creation screen and sends that information to the server via their device. The server encrypts the received password (using a hashing function) and stores it in a database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user, creating the account.

[0042] User Login

[0043] When the user accesses the system again, they enter their username and password on the login screen and send that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the device, allowing the user to access the system.

[0044] Virtual live viewing function

[0045] Select a Live Event

[0046] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[0047] Start Streaming

[0048] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0049] Content customization function

[0050] Acquiring and analyzing user history

[0051] When a user logs in, the server retrieves their listening history from the database. Based on this history data, the AI ​​analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that matches their tastes.

[0052] User interaction function

[0053] Sending and receiving chat messages

[0054] While watching a live event, users can use the chat function to interact with other users. When a user types a chat message and presses the send button, the device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event.

[0055] Specific examples

[0056] 1. A user logs in to the system and accesses the TOP page.

[0057] 2. The user selects the live event they are interested in from the available options and starts watching.

[0058] 3. The server provides a streaming URL and sends it to the device.

[0059] 4. The device will begin providing live footage to the user through the VR device.

[0060] 5. Users exchange chat messages with other participants in real time.

[0061] 6. The server uses AI to analyze the user's listening history and recommend live events and artists to watch next.

[0062] This system allows users to enjoy live events beyond physical constraints, interact with other fans in real time, and more easily find content that suits their tastes.

[0063] The processing flow will be explained below.

[0064] User registration and login function

[0065] User Registration

[0066] Step 1:

[0067] The user enters their username, password, and email address on the account creation screen.

[0068] Step 2:

[0069] The terminal transmits the input information to the server.

[0070] Step 3:

[0071] The server encrypts the received password and stores it in a database along with the username and email address.

[0072] Step 4:

[0073] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[0074] User Login

[0075] Step 1:

[0076] The user enters their username and password on the login screen.

[0077] Step 2:

[0078] The terminal transmits the entered authentication information to the server.

[0079] Step 3:

[0080] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[0081] Step 4:

[0082] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[0083] Step 5:

[0084] The user accesses the system based on a session token.

[0085] Virtual live viewing function

[0086] Select a Live Event

[0087] Step 1:

[0088] Users select the event they want to attend from the list of live events within the app.

[0089] Step 2:

[0090] The terminal sends the selected event ID to the server.

[0091] Start Streaming

[0092] Step 1:

[0093] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[0094] Step 2:

[0095] The device uses the received URL to set up a video stream on the VR or AR device and start providing live video.

[0096] Content customization function

[0097] Acquiring and analyzing user history

[0098] Step 1:

[0099] When a user logs in, the server retrieves past listening history from the database.

[0100] Step 2:

[0101] The server uses generative AI to analyze the user's preferences based on listening history data.

[0102] Step 3:

[0103] Based on the analysis results, the server generates appropriate live events and artists and sends them to the device.

[0104] Step 4:

[0105] Users review recommended content.

[0106] User interaction function

[0107] Sending and receiving chat messages

[0108] Step 1:

[0109] A user types a chat message while watching a live event.

[0110] Step 2:

[0111] The terminal transmits the entered chat message to the server.

[0112] Step 3:

[0113] The server stores the received messages in a database.

[0114] Step 4:

[0115] The server relays the incoming message to other users attending the same live event.

[0116] Step 5:

[0117] Other users receive and respond to messages in real time.

[0118] Example 1

[0119] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0120] Today's users want to experience immersive live events from the comfort of their own homes. They also want to find content tailored to their preferences and interact with other users in real time. Conventional systems have struggled to meet these needs, limiting the user experience. This invention aims to provide a system that allows users to transcend physical constraints, providing a more personalized virtual reality experience and enabling real-time interaction with other users.

[0121] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0122] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time, means for generating and distributing a streaming address corresponding to content selected by the user, means for analyzing the user's past usage history using a generative AI model and making individually customized recommendations based on the user's preferences, means for providing users with an experience using virtual reality or augmented reality, and means for relaying real-time message exchanges between users and distributing them to other users. This allows users to experience an immersive live event from the comfort of their own home and enjoy individually customized content. At the same time, they can interact with other users in real time.

[0123] "User" means any individual or entity that uses the System.

[0124] "Input Information" means data or information that a user provides to a system, including, for example, a username, password, or email address.

[0125] "User Account" means individual identification information registered by a User in order to use the System.

[0126] "Authentication Information" means data used to identify and authenticate a user to access a system, including, by way of example, a hashed password.

[0127] "Streaming Address" means the URL or link used to stream a live event or content.

[0128] "Usage history" is a record of past operations and selections made by a user within the system.

[0129] A "generative AI model" is an algorithm or program that uses artificial intelligence techniques to analyze user data and generate specific results or recommendations.

[0130] "Virtual reality" is a three-dimensional visual and sensory experience generated using computer technology.

[0131] "Augmented reality" is a technology that overlays computer-generated information onto real-world visual information.

[0132] "Real-time message exchange" is a function that allows users to send and receive messages instantly.

[0133] "Relaying" is the process of receiving data or information and then forwarding it to other users or devices.

[0134] This invention is a system that allows users to experience immersive live events from the comfort of their own home using VR and AR technologies. Furthermore, the system uses a generative AI model based on the user's past usage history to provide customized content recommendations and allows users to interact with other users in real time. A specific implementation of this system is described below.

[0135] User registration and login function

[0136] User Registration

[0137] When a user creates an account, they enter a username, password, and email address. This information is sent from the device to the server. The server then encrypts the received password using a hashing function (e.g., SHA-256) and stores it in a database. Once registration is complete, the server generates a user ID and returns a registration completion message to the user.

[0138] User Login

[0139] When a user logs in, they enter their username and password. This information is sent from the terminal to the server. The server retrieves the authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal, allowing the user to access the system.

[0140] Virtual live viewing function

[0141] Select a Live Event

[0142] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[0143] Start Streaming

[0144] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0145] Content customization function

[0146] Acquiring and analyzing user history

[0147] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[0148] Specific prompt examples:

[0149] "Based on my recent listening history, what live event would you recommend next?"

[0150] "Show me a list of live events worth attending"

[0151] User interaction function

[0152] Sending and receiving chat messages

[0153] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0154] The system allows users to enjoy live events beyond physical constraints, interact with other attendees in real time, and discover content that suits their tastes.

[0155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0156] Step 1: User Registration

[0157] The user enters a username, password, and email address on the account creation screen. This input data is sent from the device to the server. The server receives the received data and encrypts the password using a hashing function (e.g., SHA-256). The encrypted password and other user information are stored in a database. The server then generates a new user ID and returns it to the device along with a message confirming the user's registration. This completes the creation of the user account.

[0158] Specific behavior:

[0159] 1. The user enters information into the form on the browser and presses the "Register" button.

[0160] 2. The terminal sends the input data in JSON format to the server.

[0161] 3. The server receives the data, encrypts the password, and stores it in the database.

[0162] 4. The server generates a new user ID and returns a registration completion message to the terminal.

[0163] Step 2: User Login

[0164] The user enters their username and password on the login screen. This input data is sent from the device to the server. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, allowing the user to access the system.

[0165] Specific behavior:

[0166] 1. The user enters their username and password on the login screen and presses the "Login" button.

[0167] 2. The terminal sends the input data in JSON format to the server.

[0168] 3. The server retrieves the authentication information from the database and verifies the password.

[0169] 4. If the server is successful in authentication, it generates a session token and returns it to the device.

[0170] Step 3: Select a live event

[0171] Users select the event they want to attend from a list of live events provided within the app. The selected event ID is sent from the device to the server. The server receives the event ID and generates the corresponding streaming URL.

[0172] Specific behavior:

[0173] 1. A user browses the list of live events on the app and selects the event they want to attend.

[0174] 2. The device sends the selected event ID to the server.

[0175] 3. The server generates a streaming URL based on the received event ID.

[0176] Step 4: Start Streaming

[0177] The server sends the generated streaming URL to the device, which then uses the URL to set up a video stream on the VR or AR device and provide the user with live video.

[0178] Specific behavior:

[0179] 1. The server sends the streaming URL to the device.

[0180] 2. Using the URL received by the terminal, set up a stream on the VR device using WebRTC or HLS technology.

[0181] 3. Users watch the live footage through a VR device.

[0182] Step 5: Capture and analyze user history

[0183] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends customized live events and performers.

[0184] Specific behavior:

[0185] 1. The server retrieves the history from the database using the user ID.

[0186] 2. The server inputs historical data into the generated AI model and analyzes preferences.

[0187] 3. The server recommends customized content to the user based on the analysis results.

[0188] Step 6: Sending and Receiving Chat Messages

[0189] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0190] Specific behavior:

[0191] 1. The user types a message in the chat window and presses the "Send" button.

[0192] 2. The device sends a message to the server using WebSocket or an HTTP POST request.

[0193] 3. The server stores the message in a database and relays it to other participants in real time.

[0194] (Application example 1)

[0195] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] Conventional live event viewing systems require users to physically travel to the event venue, consuming a lot of time and money and lacking in convenience. Furthermore, there are limited ways for users to interact with other users in real time, which lacks the immersive and intimate feeling of a live event. Furthermore, the lack of customized event recommendations based on past listening history makes it difficult for users to find events that suit their preferences.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0198] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history, means for relaying real-time chats between users and distributing them to other users, and viewing means for the user to watch the live event on a device using virtual reality or augmented reality.This allows users to experience an immersive live event from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[0199] "User input information" refers to information provided by a user when creating an account or logging in, including a username, password, email address, etc.

[0200] "User Account" means an account created by a User to use the System, which includes the User's personal information and authentication information.

[0201] "Authentication Information" means information used to verify a user's access privileges, typically including a username and password.

[0202] A "streaming URL" is a link that allows you to watch live events or video content in real time over the Internet.

[0203] "Listening History" means a record of live events or audio content that a User has previously viewed or listened to.

[0204] A "live event" is a performance, show, or other event that takes place in real time.

[0205] "Performer" means an artist or performer who performs at a live event.

[0206] "Recommendation methods" are methods for presenting appropriate live events and performers based on a user's past listening history and preferences.

[0207] "Real-time chat" means a feature that allows users to instantly exchange messages with other users during a live event.

[0208] A "relaying means" is a method for delivering a chat message received from one user to another user.

[0209] "Virtual reality (VR)" is a technology that uses computer graphics technology to allow users to experience a virtual environment that is different from reality.

[0210] "Augmented reality (AR)" is a technology that overlays digital information onto a real-world environment, allowing users to visually experience additional information.

[0211] A "viewing instrument" is a method by which a user views a live event using a virtual reality or augmented reality device.

[0212] A "generative AI model" is an artificial intelligence model that learns user preferences and behaviors based on large amounts of data and makes appropriate recommendations and personalization.

[0213] A "prompt" is text data that is input into a generative AI model and serves as a trigger for generating a specific result or response.

[0214] "Device" refers to hardware equipment such as a computer, smartphone, or VR headset.

[0215] The system for realizing the present invention has the following main functions.

[0216] 1. User Account Creation and Authentication

[0217] The server receives the user's input and creates a user account. The user's credentials are saved and used to authenticate the user the next time they log in.

[0218] For example, when a user enters their information (username, password, email address) on the account creation screen, the server receives that information, encrypts the password (using a hashing function) and stores it in the database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user.

[0219] 2. Watching Live Events

[0220] The server generates a streaming URL corresponding to the user's selected event and delivers it to the user, who can then watch the live event using their virtual reality (VR) or augmented reality (AR) device.

[0221] For example, when a user selects an event they want to attend from a list of live events provided within the app, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device then uses the URL to set up a video stream on the VR or AR device and provide live video.

[0222] 3. Customized content recommendations to users

[0223] The server analyzes the user's past listening history and uses a generative AI model to make personalized recommendations based on the user's preferences.

[0224] For example, when a user logs in, the server retrieves their past listening history from the database, and the generative AI model analyzes the user's preferences based on this historical data, suggesting live events and artists to watch next.

[0225] 4. Real-time chat between users

[0226] The server provides users with the ability to exchange chat messages with other users in real time while watching a live event, which promotes a sense of community among users.

[0227] For example, when a user types a chat message and hits the send button, the device sends the message to a server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0228] Hardware and software used

[0229] Hardware: VR device (e.g., Oculus Quest 2), AR device (e.g., Microsoft HoloLens), smartphone, PC

[0230] Software: Server applications (e.g., Flask framework), databases (e.g., MySQL), generative AI models

[0231] Prompt Sentence Examples

[0232] "Recommend the best live event based on the user's listening history. The data to recommend the next event to watch is as follows: - User ID: 1234 - Listening history: ['Artist A', 'Artist B', 'Event C']"

[0233] As a result, the system allows users to experience immersive live events from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[0234] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0235] Step 1:

[0236] The user enters their information (username, password, email address) on the account creation screen and sends the information to the server via the device. The server creates a user account based on the received information. Specifically, the server encrypts the entered password (using a hashing function) and stores the encrypted information in a database. Finally, the server generates a user ID and returns a registration completion message to the device.

[0237] Input: Username, Password, Email Address

[0238] Data processing: password hashing

[0239] Output: User ID, registration completion message

[0240] Step 2:

[0241] The user enters their username and password on the login screen and sends that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, authenticating the user.

[0242] Input: Username, Password

[0243] Data processing: password hashing, authentication information matching

[0244] Output: Session token

[0245] Step 3:

[0246] The user selects the event they want to attend from a list of live events provided within the app. The device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0247] Input: Event ID

[0248] Data processing: Streaming URL generation

[0249] Output: Streaming URL

[0250] Step 4:

[0251] When a user logs in, the server retrieves the user's past listening history from a database. The server then analyzes the retrieved listening history using a generative AI model to recommend live events and performers that are individually tailored to the user's preferences. The recommendations are then sent to the device.

[0252] Input: User ID

[0253] Data processing: Acquisition and analysis of listening history, and recommendations based on generative AI models

[0254] Output: Recommended live events and performers

[0255] Step 5:

[0256] While watching a live event, a user types a chat message and presses the send button. The device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event. The chat message is then distributed to other devices in real time.

[0257] Input: Chat message

[0258] Data processing: message storage and relay

[0259] Output: Chat messages delivered to other users' devices

[0260] These steps allow users to enjoy immersive live events from the comfort of their own home, interact with other users in real time, and receive personalized live event recommendations based on their past listening history.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] This invention combines a system that allows users to experience immersive live events from the comfort of their own homes using virtual reality (VR) and augmented reality (AR) technologies with an emotion engine that recognizes users' emotions. Furthermore, it has the ability to recommend live events and artists that are individually customized based on the user's past listening history and emotions, and to promote real-time interaction between users.

[0263] User registration and login function

[0264] User Registration

[0265] The user enters a username, password, and email address on the account creation screen and sends that information to the server via the device. The server then encrypts the received password (using a hashing function) and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[0266] User Login

[0267] When accessing the system again, the user enters their username and password on the login screen and sends that information to the server via their terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[0268] Virtual live viewing function

[0269] Select a Live Event

[0270] Users select the event they want to attend from the list of live events in the app, and the device sends the selected event ID to the server.

[0271] Start Streaming

[0272] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and starts providing live video.

[0273] Content customization function

[0274] Acquiring and analyzing user history

[0275] When a user logs in, the server retrieves their listening history from the database. Based on this history data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[0276] Emotion Engine Functions

[0277] Emotion Recognition and Analysis

[0278] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions.

[0279] User interaction function

[0280] Sending and receiving chat messages

[0281] Users can enter chat messages while watching a live event. The device sends the messages to a server, which then stores them in a database and relays them in real time to other users participating in the same live event. The emotion engine can also analyze users' emotions and automatically adjust or suggest chat messages.

[0282] Specific examples

[0283] User registration and login

[0284] 1. The user accesses the system and enters information on the account creation screen.

[0285] 2. The device sends the input information to the server, which stores the information in a database.

[0286] 3. The user enters their credentials on the login screen and the server authenticates them.

[0287] Watching live events

[0288] 1. The user selects the live event they want to attend, and the device sends the selection information to the server.

[0289] 2. The server sends the streaming URL to the device, and the device sets up the video stream.

[0290] Utilizing the Emotion Engine

[0291] 1. While the user is watching live video, the device collects emotional data through the camera and microphone.

[0292] 2. The server analyzes the data using an emotion engine and provides optimal live events and interactions based on the user's emotions.

[0293] User interaction

[0294] 1. The user types a chat message while watching live, and the device sends the message to the server.

[0295] 2. The server receives the message and relays it to other users attending the same event.

[0296] The system allows users to transcend physical constraints and enjoy high-quality live events, while leveraging an emotion engine for a more personalized experience and the ability to interact with other users in real time.

[0297] The processing flow will be explained below.

[0298] User registration and login function

[0299] User Registration

[0300] Step 1:

[0301] The user enters their username, password, and email address on the account creation screen.

[0302] Step 2:

[0303] The terminal transmits the input information to the server.

[0304] Step 3:

[0305] The server encrypts the received password and stores it in a database along with the username and email address.

[0306] Step 4:

[0307] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[0308] User Login

[0309] Step 1:

[0310] The user enters their username and password on the login screen.

[0311] Step 2:

[0312] The terminal transmits the entered authentication information to the server.

[0313] Step 3:

[0314] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[0315] Step 4:

[0316] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[0317] Step 5:

[0318] The user accesses the system using a session token.

[0319] Virtual live viewing function

[0320] Select a Live Event

[0321] Step 1:

[0322] Users select the event they want to attend from the list of live events within the app.

[0323] Step 2:

[0324] The terminal sends the selected event ID to the server.

[0325] Start Streaming

[0326] Step 1:

[0327] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[0328] Step 2:

[0329] The terminal uses the received URL to set up a video stream on the VR or AR device.

[0330] Step 3:

[0331] The device will start providing live video.

[0332] Content customization function

[0333] Acquiring and analyzing user history

[0334] Step 1:

[0335] When a user logs in, the server retrieves past listening history from the database.

[0336] Step 2:

[0337] The server provides listening history data to the generation AI, which analyzes the user's preferences.

[0338] Step 3:

[0339] Based on the analysis results, the server generates individually customized live events and performers and sends them to the device.

[0340] Step 4:

[0341] The user reviews and watches the recommended content.

[0342] Emotion Engine Functions

[0343] Emotion Recognition and Analysis

[0344] Step 1:

[0345] The device uses a camera and microphone to capture the facial expressions and voice of the user watching the live video.

[0346] Step 2:

[0347] The data collected by the device is sent to the server in real time.

[0348] Step 3:

[0349] The server analyzes the data using an emotion engine.

[0350] Step 4:

[0351] The server sends information to the device to adjust live events and interactions based on the analyzed user emotions.

[0352] Step 5:

[0353] The device presents appropriate interactions and content to the user.

[0354] User interaction function

[0355] Sending and receiving chat messages

[0356] Step 1:

[0357] A user types a chat message while watching a live event.

[0358] Step 2:

[0359] The terminal transmits the entered chat message to the server.

[0360] Step 3:

[0361] The server stores the received messages in a database.

[0362] Step 4:

[0363] The server relays messages in real time to other users participating in the same live event.

[0364] Step 5:

[0365] The server uses an emotion engine to analyze the content of the message and make adjustments or suggestions as needed.

[0366] Step 6:

[0367] Other users receive and respond to messages in real time.

[0368] Example 2

[0369] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0370] In today's digital entertainment market, users are looking for ways to enjoy immersive live events beyond physical limitations. Furthermore, there is a need to customize content based on each user's preferences and emotions, providing greater satisfaction. Furthermore, there is a demand for the ability to interact with other users in real time during live events. However, current systems struggle to integrate and effectively realize these elements.

[0371] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history and emotional data, means for recognizing and analyzing the user's emotions in real time and optimizing live events and interactions based on the results, and means for relaying real-time chats between users and distributing them to other users. This allows users to enjoy immersive live events beyond physical constraints, and further enables individually customized experiences and real-time user interaction.

[0372] "User" refers to an individual who uses this system.

[0373] "Input Information" means information provided by a user to the system, including, for example, a username, password, or email address.

[0374] "User Account" means an individual record in the System created based on a User's personal information and authentication information.

[0375] "Authentication Information" means information used to verify a User's identity, including a username and password.

[0376] "Login" refers to the process by which a user accesses a system using authentication information.

[0377] A "streaming URL" is a link that is generated by the server and sent to the user's device, indicating the destination of the live video stream.

[0378] "Listening history" is a record of music and live events that a user has listened to in the past.

[0379] "Emotional data" refers to information about emotions captured from a user's facial expressions, voice, etc.

[0380] "Recommendation" is the act of suggesting suitable content or services based on a user's listening history and emotional data.

[0381] "Real-time" means that events and interactions occur immediately, without delay.

[0382] "Chat" is a function that allows users to exchange messages and information in real time.

[0383] The "emotion engine" is a function within the system that analyzes emotional data in real time and provides optimal content and interactions based on the results.

[0384] "Virtual reality (VR)" is a technology that allows users to experience a three-dimensional space similar to reality, generated using computer technology.

[0385] "Augmented reality (AR)" is a technology that overlays digital information onto the real physical environment.

[0386] An "interaction" is an action or process in which a user interacts with a system or with another user.

[0387] This invention relates to a system that allows users to enjoy immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies. This system further enhances the user experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the system and their processing are described below.

[0388] First, the user accesses the account creation screen using a device and enters information such as a username, password, and email address. The device then sends this information to the server. The server then encrypts the received password (using a hashing function) and stores it in a database along with other information. The server then generates a new user ID and returns a registration completion message to the device.

[0389] Next, when the user accesses the system again, they enter their username and password on the login screen. The terminal sends the input information to the server, which retrieves the corresponding user's authentication information from the database. The server compares the hash value of the input password with the value stored in the database, and if authentication is successful, it generates a session token and returns it to the terminal. The terminal stores this session token, allowing the user to access the system.

[0390] When a user selects an event they want to attend from a list of live events, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begins providing live video. In some cases, a CDN (Content Delivery Network) service is used for streaming.

[0391] When a user logs in, the server retrieves their listening history from the database and analyzes their preferences using an emotion engine. Based on the results, the generative AI recommends individually customized live events and performers, allowing users to easily find content that matches their preferences.

[0392] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone and sends the data to a server. The server then uses an emotion engine to analyze the user's emotional data and provides optimal live events and interactions based on the results.

[0393] Furthermore, when a user types a chat message while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event. The emotion engine analyzes the message and can make automatic adjustments or suggestions.

[0394] For example, if a user types "I want to watch a live event of my favorite artist," the system will recommend the most suitable live event based on the user's past listening history and current emotional state. Also, if a user types "I want to share my impressions with friends in real time," the system will utilize the chat function to encourage interaction with other users.

[0395] In this way, the system provides users with an optimized live event experience, utilizing virtual and augmented reality to enhance the sense of presence, while also enabling real-time interaction with other users.

[0396] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0397] User registration process steps

[0398] Step 1:

[0399] The user accesses the account creation screen using their device and enters their username, password, and email address.

[0400] Input: Username, Password, Email Address

[0401] Output: User information ready to send

[0402] Step 2:

[0403] The terminal transmits the entered user information to the server.

[0404] Input: User information (username, password, email address)

[0405] Output: Request to server

[0406] Step 3:

[0407] The server applies a hashing function to the received password to generate an encrypted password.

[0408] Enter: Password

[0409] Output: Encrypted password

[0410] Step 4:

[0411] The server stores the username, email address, and hashed password in a database.

[0412] Input: Username, Email Address, and Hashed Password

[0413] Output: Saved to database

[0414] Step 5:

[0415] The server generates a new user ID and returns a registration completion message to the terminal, which displays this message to the user.

[0416] Input: Registered user information

[0417] Output: New user ID, registration completion message

[0418] User login process steps

[0419] Step 1:

[0420] The user accesses the login screen using the device and enters their username and password.

[0421] Input: Username, Password

[0422] Output: Login information ready to be sent

[0423] Step 2:

[0424] The terminal transmits the entered authentication information to the server.

[0425] Input: Username, Password

[0426] Output: Send authentication information to the server

[0427] Step 3:

[0428] The server retrieves the corresponding user's authentication information (hashed password) from the database.

[0429] Input: Username

[0430] Output: Retrieving authentication information from the database

[0431] Step 4:

[0432] The server hashes the entered password and compares it with the hash value retrieved from the database. If authentication is successful, it generates a session token, otherwise it generates an error message.

[0433] Input: Password, hashed password for database

[0434] Output: Session token or error message

[0435] Step 5:

[0436] If authentication is successful, the server generates a session token and sends it to the device, which retains it and allows the user to access the system.

[0437] Input: Session token

[0438] Output: User approved to access the system

[0439] Processing steps for watching virtual live shows

[0440] Step 1:

[0441] Users select the event they want to attend from the list of live events within the app via their device.

[0442] Input: Live Event Selection

[0443] Output: Get the event ID

[0444] Step 2:

[0445] The device sends the ID of the selected event to the server.

[0446] Input: Event ID

[0447] Output: Send event ID to server

[0448] Step 3:

[0449] The server generates a streaming URL based on the event ID. This streaming may use a CDN (Content Delivery Network) service.

[0450] Input: Event ID

[0451] Output: Streaming URL

[0452] Step 4:

[0453] The server generates a streaming URL and sends it to the device.

[0454] Input: Streaming URL

[0455] Output: Send streaming URL to device

[0456] Step 5:

[0457] The device uses the received URL to set up a video stream on the VR or AR device, and the user can then start watching the live video.

[0458] Input: Streaming URL

[0459] Output: Start of video stream

[0460] Emotion Engine Processing Steps

[0461] Step 1:

[0462] While the user is watching the live video, the device's camera and microphone capture the user's facial expressions and voice.

[0463] Input: User's facial expressions, voice

[0464] Output: Obtaining emotion data

[0465] Step 2:

[0466] The device transmits the captured emotion data to the server.

[0467] Input: Emotion data

[0468] Output: Send emotion data to the server

[0469] Step 3:

[0470] The server analyzes the emotional data using an emotion engine, which uses technologies such as facial expression recognition and voice tone analysis.

[0471] Input: Emotion data

[0472] Output: Emotion analysis results

[0473] Step 4:

[0474] The server uses the analysis results to provide optimal live events and interactions tailored to the user's emotions. For example, if a specific emotion is strong, a request can be sent to an artist in real time.

[0475] Input: Sentiment analysis results

[0476] Output: Optimized live events and interactions

[0477] Steps in processing user interactions

[0478] Step 1:

[0479] A user types a chat message while watching a live event.

[0480] Input: Chat message

[0481] Output: Message prepared for sending

[0482] Step 2:

[0483] The terminal transmits the entered chat message to the server.

[0484] Input: Chat message

[0485] Output: Send message to server

[0486] Step 3:

[0487] The server stores the received messages in a database.

[0488] Input: Chat message

[0489] Output: Message saved

[0490] Step 4:

[0491] The server relays messages in real time to other users attending the same live event, and an emotion engine analyzes users' emotions and can automatically adjust or suggest chat messages as needed.

[0492] Input: Chat message

[0493] Output: Message relay to other users, auto-adjustment suggestions

[0494] The above are the specific processing steps of the program of this system.

[0495] (Application example 2)

[0496] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0497] In conventional online live event viewing systems, users were limited to simply watching live footage, and were unable to enjoy real-time interaction or a personalized experience. Furthermore, they lacked the means to recognize users' emotions and enhance the sense of presence or provide recommended content based on those emotions. As a result, the entertainment value and user experience were limited.

[0498] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0499] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for analyzing the user's past viewing history and emotional data and recommending individually customized live events and performers, means for relaying real-time messages between users and distributing them to other users, and means for recognizing user emotions and dynamically adjusting the experience content based on the recognition results. This allows users to experience a more personalized live event and interact with other users in real time, as well as enjoy an emotionally-driven, interactive, and immersive experience.

[0500] "Input Information" means data provided by a user to the system for purposes such as account creation and event selection.

[0501] "Authentication information" refers to data such as username and password used when a user logs in, and is used to accurately identify the user and verify access rights.

[0502] "Streaming URL" means a link on the Internet that streams video of a selected live event to users in real time.

[0503] "Viewing history" is data about content a user has viewed in the past, and personalized recommendations are based on this data.

[0504] "Recommendation" means selecting and presenting appropriate content and performers based on the user's tastes, preferences, and emotions.

[0505] "Real-time Messages" are text messages that allow Users to instantly communicate with other Users while watching a live event.

[0506] "Emotional data" refers to emotional information recognized from a user's facial expressions and voice, and is used to make the user's experience more personalized and relevant.

[0507] "Virtual reality" is a technology that uses specialized devices to allow users to experience a computer-generated environment that differs from reality.

[0508] "Augmented reality" is a technology that overlays digitally generated elements onto the real world, providing users with a blended experience of reality and virtuality.

[0509] "Dynamic adjustment" is the process of optimizing the experience in real time based on the user's emotional data and other variables.

[0510] This invention combines a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technology with an emotion engine that recognizes the user's emotions. The system also has the function of recommending individually customized live events and performers based on the user's past viewing history and emotions, and promoting real-time interaction between users. The following describes in detail the modes for implementing this invention.

[0511] User registration and login function

[0512] User Registration

[0513] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen and sends this information to the server via their device. The server encrypts the received password and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[0514] User Login

[0515] When the user accesses the system again, the server receives the user name and password on the login screen and sends the information to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[0516] Virtual live viewing function

[0517] Select a Live Event

[0518] When a user selects an event they want to attend from a list of live events within the app, the server receives the selection information. The device sends the selected event ID to the server, and the server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[0519] Content customization function

[0520] Acquiring and analyzing user history

[0521] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their preferences.

[0522] Emotion Engine Functions

[0523] Emotion Recognition and Analysis

[0524] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone. This data is sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions. Emotional data recognized from the user's facial expressions and voice is used to make the user's experience more personalized and relevant.

[0525] User interaction function

[0526] Sending and receiving chat messages

[0527] When a user types a chat message while watching a live event, the server can receive the message information and relay it in real time to other users participating in the same live event.The emotion engine can also analyze the user's emotions and automatically adjust or suggest chat messages.

[0528] Hardware and software used

[0529] The following hardware and software are used to implement this system:

[0530] Hardware: PC, smartphone, tablet, VR headset, AR device, camera, microphone

[0531] Software: Flask web framework, Python, database (SQLite or PostgreSQL), OpenCV, a provisional emotion recognition library (EmotionRecognizer), and a recommendation system (Recommender).

[0532] Specific examples

[0533] 1. When a user enters and submits information on the account creation screen, the server stores the encrypted information in a database.

[0534] 2. The user selects the desired live event, and the server generates and returns a streaming URL.

[0535] 3. The emotion engine recognizes emotions based on the user's facial expressions and voice data, and provides recommendations to the user based on those emotions.

[0536] Prompt Sentence Examples

[0537] "Use an emotion engine to analyze users' facial expressions and recommend appropriate live events based on their past viewing history."

[0538] "Share chat messages typed by users while watching a live event with other users in real time."

[0539] In this way, the present invention is a system that utilizes the user's emotions and past viewing history to provide a more personalized, interactive, and immersive live experience.

[0540] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0541] Step 1:

[0542] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen. The device sends this input information to the server. The server uses a hashing function to encrypt the password and stores it in a database along with the username and email address. As output, it generates a new user ID and returns a registration completion message to the user.

[0543] Step 2:

[0544] The server saves the user's authentication information and authenticates the user the next time they log in. When accessing the system again, the user enters their username and password on the login screen and sends them to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal. The user can then use this session token to access the system.

[0545] Step 3:

[0546] When a user selects an event they want to attend from the list of live events in the app, the server receives the selection information. The device sends the selected event ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[0547] Step 4:

[0548] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers. As an output, users can easily find content that suits their preferences.

[0549] Step 5:

[0550] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the server provides live events and interactions optimized for the user's emotions. Specifically, content recommendations and interaction adjustments are made based on emotion recognition results.

[0551] Step 6:

[0552] When a user enters a chat message while watching a live event, the server receives the message information. The device then sends the entered chat message to the server, which then relays the message to other users participating in the same live event in real time. As an output, other users receive the chat message in real time, allowing them to interact with each other.

[0553] In this way, each step works together to allow users to enjoy an immersive live event, with a more personalized experience and real-time interaction with other users.

[0554] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0555] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0556] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0557] [Second embodiment]

[0558] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0559] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0560] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0561] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0562] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0563] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0564] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0565] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0566] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0567] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0568] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0569] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0570] This invention is a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies, and also provides personalized recommendations for live events and artists based on the user's listening history and preferences, as well as the ability to interact with other users in real time.

[0571] User registration and login function

[0572] User Registration

[0573] The user enters their information (username, password, email address) on the account creation screen and sends that information to the server via their device. The server encrypts the received password (using a hashing function) and stores it in a database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user, creating the account.

[0574] User Login

[0575] When the user accesses the system again, they enter their username and password on the login screen and send that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the device, allowing the user to access the system.

[0576] Virtual live viewing function

[0577] Select a Live Event

[0578] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[0579] Start Streaming

[0580] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0581] Content customization function

[0582] Acquiring and analyzing user history

[0583] When a user logs in, the server retrieves their listening history from the database. Based on this history data, the AI ​​analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that matches their tastes.

[0584] User interaction function

[0585] Sending and receiving chat messages

[0586] While watching a live event, users can use the chat function to interact with other users. When a user types a chat message and presses the send button, the device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event.

[0587] Specific examples

[0588] 1. A user logs in to the system and accesses the TOP page.

[0589] 2. The user selects the live event they are interested in from the available options and starts watching.

[0590] 3. The server provides a streaming URL and sends it to the device.

[0591] 4. The device will begin providing live footage to the user through the VR device.

[0592] 5. Users exchange chat messages with other participants in real time.

[0593] 6. The server uses AI to analyze the user's listening history and recommend live events and artists to watch next.

[0594] This system allows users to enjoy live events beyond physical constraints, interact with other fans in real time, and more easily find content that suits their tastes.

[0595] The processing flow will be explained below.

[0596] User registration and login function

[0597] User Registration

[0598] Step 1:

[0599] The user enters their username, password, and email address on the account creation screen.

[0600] Step 2:

[0601] The terminal transmits the input information to the server.

[0602] Step 3:

[0603] The server encrypts the received password and stores it in a database along with the username and email address.

[0604] Step 4:

[0605] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[0606] User Login

[0607] Step 1:

[0608] The user enters their username and password on the login screen.

[0609] Step 2:

[0610] The terminal transmits the entered authentication information to the server.

[0611] Step 3:

[0612] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[0613] Step 4:

[0614] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[0615] Step 5:

[0616] The user accesses the system based on a session token.

[0617] Virtual live viewing function

[0618] Select a Live Event

[0619] Step 1:

[0620] Users select the event they want to attend from the list of live events within the app.

[0621] Step 2:

[0622] The terminal sends the selected event ID to the server.

[0623] Start Streaming

[0624] Step 1:

[0625] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[0626] Step 2:

[0627] The device uses the received URL to set up a video stream on the VR or AR device and start providing live video.

[0628] Content customization function

[0629] Acquiring and analyzing user history

[0630] Step 1:

[0631] When a user logs in, the server retrieves past listening history from the database.

[0632] Step 2:

[0633] The server uses generative AI to analyze the user's preferences based on listening history data.

[0634] Step 3:

[0635] Based on the analysis results, the server generates appropriate live events and artists and sends them to the device.

[0636] Step 4:

[0637] Users review recommended content.

[0638] User interaction function

[0639] Sending and receiving chat messages

[0640] Step 1:

[0641] A user types a chat message while watching a live event.

[0642] Step 2:

[0643] The terminal transmits the entered chat message to the server.

[0644] Step 3:

[0645] The server stores the received messages in a database.

[0646] Step 4:

[0647] The server relays the incoming message to other users attending the same live event.

[0648] Step 5:

[0649] Other users receive and respond to messages in real time.

[0650] Example 1

[0651] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] Today's users want to experience immersive live events from the comfort of their own homes. They also want to find content tailored to their preferences and interact with other users in real time. Conventional systems have struggled to meet these needs, limiting the user experience. This invention aims to provide a system that allows users to transcend physical constraints, providing a more personalized virtual reality experience and enabling real-time interaction with other users.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0654] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time, means for generating and distributing a streaming address corresponding to content selected by the user, means for analyzing the user's past usage history using a generative AI model and making individually customized recommendations based on the user's preferences, means for providing users with an experience using virtual reality or augmented reality, and means for relaying real-time message exchanges between users and distributing them to other users. This allows users to experience an immersive live event from the comfort of their own home and enjoy individually customized content. At the same time, they can interact with other users in real time.

[0655] "User" means any individual or entity that uses the System.

[0656] "Input Information" means data or information that a user provides to a system, including, for example, a username, password, or email address.

[0657] "User Account" means individual identification information registered by a User in order to use the System.

[0658] "Authentication Information" means data used to identify and authenticate a user to access a system, including, by way of example, a hashed password.

[0659] "Streaming Address" means the URL or link used to stream a live event or content.

[0660] "Usage history" is a record of past operations and selections made by a user within the system.

[0661] A "generative AI model" is an algorithm or program that uses artificial intelligence techniques to analyze user data and generate specific results or recommendations.

[0662] "Virtual reality" is a three-dimensional visual and sensory experience generated using computer technology.

[0663] "Augmented reality" is a technology that overlays computer-generated information onto real-world visual information.

[0664] "Real-time message exchange" is a function that allows users to send and receive messages instantly.

[0665] "Relaying" is the process of receiving data or information and then forwarding it to other users or devices.

[0666] This invention is a system that allows users to experience immersive live events from the comfort of their own home using VR and AR technologies. Furthermore, the system uses a generative AI model based on the user's past usage history to provide customized content recommendations and allows users to interact with other users in real time. A specific implementation of this system is described below.

[0667] User registration and login function

[0668] User Registration

[0669] When a user creates an account, they enter a username, password, and email address. This information is sent from the device to the server. The server then encrypts the received password using a hashing function (e.g., SHA-256) and stores it in a database. Once registration is complete, the server generates a user ID and returns a registration completion message to the user.

[0670] User Login

[0671] When a user logs in, they enter their username and password. This information is sent from the terminal to the server. The server retrieves the authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal, allowing the user to access the system.

[0672] Virtual live viewing function

[0673] Select a Live Event

[0674] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[0675] Start Streaming

[0676] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0677] Content customization function

[0678] Acquiring and analyzing user history

[0679] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[0680] Specific prompt examples:

[0681] "Based on my recent listening history, what live event would you recommend next?"

[0682] "Show me a list of live events worth attending"

[0683] User interaction function

[0684] Sending and receiving chat messages

[0685] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0686] The system allows users to enjoy live events beyond physical constraints, interact with other attendees in real time, and discover content that suits their tastes.

[0687] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0688] Step 1: User Registration

[0689] The user enters a username, password, and email address on the account creation screen. This input data is sent from the device to the server. The server receives the received data and encrypts the password using a hashing function (e.g., SHA-256). The encrypted password and other user information are stored in a database. The server then generates a new user ID and returns it to the device along with a message confirming the user's registration. This completes the creation of the user account.

[0690] Specific behavior:

[0691] 1. The user enters information into the form on the browser and presses the "Register" button.

[0692] 2. The terminal sends the input data in JSON format to the server.

[0693] 3. The server receives the data, encrypts the password, and stores it in the database.

[0694] 4. The server generates a new user ID and returns a registration completion message to the terminal.

[0695] Step 2: User Login

[0696] The user enters their username and password on the login screen. This input data is sent from the device to the server. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, allowing the user to access the system.

[0697] Specific behavior:

[0698] 1. The user enters their username and password on the login screen and presses the "Login" button.

[0699] 2. The terminal sends the input data in JSON format to the server.

[0700] 3. The server retrieves the authentication information from the database and verifies the password.

[0701] 4. If the server is successful in authentication, it generates a session token and returns it to the device.

[0702] Step 3: Select a live event

[0703] Users select the event they want to attend from a list of live events provided within the app. The selected event ID is sent from the device to the server. The server receives the event ID and generates the corresponding streaming URL.

[0704] Specific behavior:

[0705] 1. A user browses the list of live events on the app and selects the event they want to attend.

[0706] 2. The device sends the selected event ID to the server.

[0707] 3. The server generates a streaming URL based on the received event ID.

[0708] Step 4: Start Streaming

[0709] The server sends the generated streaming URL to the device, which then uses the URL to set up a video stream on the VR or AR device and provide the user with live video.

[0710] Specific behavior:

[0711] 1. The server sends the streaming URL to the device.

[0712] 2. Using the URL received by the terminal, set up a stream on the VR device using WebRTC or HLS technology.

[0713] 3. Users watch the live footage through a VR device.

[0714] Step 5: Capture and analyze user history

[0715] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends customized live events and performers.

[0716] Specific behavior:

[0717] 1. The server retrieves the history from the database using the user ID.

[0718] 2. The server inputs historical data into the generated AI model and analyzes preferences.

[0719] 3. The server recommends customized content to the user based on the analysis results.

[0720] Step 6: Sending and Receiving Chat Messages

[0721] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0722] Specific behavior:

[0723] 1. The user types a message in the chat window and presses the "Send" button.

[0724] 2. The device sends a message to the server using WebSocket or an HTTP POST request.

[0725] 3. The server stores the message in a database and relays it to other participants in real time.

[0726] (Application example 1)

[0727] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Conventional live event viewing systems require users to physically travel to the event venue, consuming a lot of time and money and lacking in convenience. Furthermore, there are limited ways for users to interact with other users in real time, which lacks the immersive and intimate feeling of a live event. Furthermore, the lack of customized event recommendations based on past listening history makes it difficult for users to find events that suit their preferences.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0730] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history, means for relaying real-time chats between users and distributing them to other users, and viewing means for the user to watch the live event on a device using virtual reality or augmented reality.This allows users to experience an immersive live event from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[0731] "User input information" refers to information provided by a user when creating an account or logging in, including a username, password, email address, etc.

[0732] "User Account" means an account created by a User to use the System, which includes the User's personal information and authentication information.

[0733] "Authentication Information" means information used to verify a user's access privileges, typically including a username and password.

[0734] A "streaming URL" is a link that allows you to watch live events or video content in real time over the Internet.

[0735] "Listening History" means a record of live events or audio content that a User has previously viewed or listened to.

[0736] A "live event" is a performance, show, or other event that takes place in real time.

[0737] "Performer" means an artist or performer who performs at a live event.

[0738] "Recommendation methods" are methods for presenting appropriate live events and performers based on a user's past listening history and preferences.

[0739] "Real-time chat" means a feature that allows users to instantly exchange messages with other users during a live event.

[0740] A "relaying means" is a method for delivering a chat message received from one user to another user.

[0741] "Virtual reality (VR)" is a technology that uses computer graphics technology to allow users to experience a virtual environment that is different from reality.

[0742] "Augmented reality (AR)" is a technology that overlays digital information onto a real-world environment, allowing users to visually experience additional information.

[0743] A "viewing instrument" is a method by which a user views a live event using a virtual reality or augmented reality device.

[0744] A "generative AI model" is an artificial intelligence model that learns user preferences and behaviors based on large amounts of data and makes appropriate recommendations and personalization.

[0745] A "prompt" is text data that is input into a generative AI model and serves as a trigger for generating a specific result or response.

[0746] "Device" refers to hardware equipment such as a computer, smartphone, or VR headset.

[0747] The system for realizing the present invention has the following main functions.

[0748] 1. User Account Creation and Authentication

[0749] The server receives the user's input and creates a user account. The user's credentials are saved and used to authenticate the user the next time they log in.

[0750] For example, when a user enters their information (username, password, email address) on the account creation screen, the server receives that information, encrypts the password (using a hashing function) and stores it in the database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user.

[0751] 2. Watching Live Events

[0752] The server generates a streaming URL corresponding to the user's selected event and delivers it to the user, who can then watch the live event using their virtual reality (VR) or augmented reality (AR) device.

[0753] For example, when a user selects an event they want to attend from a list of live events provided within the app, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device then uses the URL to set up a video stream on the VR or AR device and provide live video.

[0754] 3. Customized content recommendations to users

[0755] The server analyzes the user's past listening history and uses a generative AI model to make personalized recommendations based on the user's preferences.

[0756] For example, when a user logs in, the server retrieves their past listening history from the database, and the generative AI model analyzes the user's preferences based on this historical data, suggesting live events and artists to watch next.

[0757] 4. Real-time chat between users

[0758] The server provides users with the ability to exchange chat messages with other users in real time while watching a live event, which promotes a sense of community among users.

[0759] For example, when a user types a chat message and hits the send button, the device sends the message to a server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[0760] Hardware and software used

[0761] Hardware: VR device (e.g., Oculus Quest 2), AR device (e.g., Microsoft HoloLens), smartphone, PC

[0762] Software: Server applications (e.g., Flask framework), databases (e.g., MySQL), generative AI models

[0763] Prompt Sentence Examples

[0764] "Recommend the best live event based on the user's listening history. The data to recommend the next event to watch is as follows: - User ID: 1234 - Listening history: ['Artist A', 'Artist B', 'Event C']"

[0765] As a result, the system allows users to experience immersive live events from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[0766] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0767] Step 1:

[0768] The user enters their information (username, password, email address) on the account creation screen and sends the information to the server via the device. The server creates a user account based on the received information. Specifically, the server encrypts the entered password (using a hashing function) and stores the encrypted information in a database. Finally, the server generates a user ID and returns a registration completion message to the device.

[0769] Input: Username, Password, Email Address

[0770] Data processing: password hashing

[0771] Output: User ID, registration completion message

[0772] Step 2:

[0773] The user enters their username and password on the login screen and sends that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, authenticating the user.

[0774] Input: Username, Password

[0775] Data processing: password hashing, authentication information matching

[0776] Output: Session token

[0777] Step 3:

[0778] The user selects the event they want to attend from a list of live events provided within the app. The device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[0779] Input: Event ID

[0780] Data processing: Streaming URL generation

[0781] Output: Streaming URL

[0782] Step 4:

[0783] When a user logs in, the server retrieves the user's past listening history from a database. The server then analyzes the retrieved listening history using a generative AI model to recommend live events and performers that are individually tailored to the user's preferences. The recommendations are then sent to the device.

[0784] Input: User ID

[0785] Data processing: Acquisition and analysis of listening history, and recommendations based on generative AI models

[0786] Output: Recommended live events and performers

[0787] Step 5:

[0788] While watching a live event, a user types a chat message and presses the send button. The device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event. The chat message is then distributed to other devices in real time.

[0789] Input: Chat message

[0790] Data processing: message storage and relay

[0791] Output: Chat messages delivered to other users' devices

[0792] These steps allow users to enjoy immersive live events from the comfort of their own home, interact with other users in real time, and receive personalized live event recommendations based on their past listening history.

[0793] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0794] This invention combines a system that allows users to experience immersive live events from the comfort of their own homes using virtual reality (VR) and augmented reality (AR) technologies with an emotion engine that recognizes users' emotions. Furthermore, it has the ability to recommend live events and artists that are individually customized based on the user's past listening history and emotions, and to promote real-time interaction between users.

[0795] User registration and login function

[0796] User Registration

[0797] The user enters a username, password, and email address on the account creation screen and sends that information to the server via the device. The server then encrypts the received password (using a hashing function) and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[0798] User Login

[0799] When accessing the system again, the user enters their username and password on the login screen and sends that information to the server via their terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[0800] Virtual live viewing function

[0801] Select a Live Event

[0802] Users select the event they want to attend from the list of live events in the app, and the device sends the selected event ID to the server.

[0803] Start Streaming

[0804] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and starts providing live video.

[0805] Content customization function

[0806] Acquiring and analyzing user history

[0807] When a user logs in, the server retrieves their listening history from the database. Based on this history data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[0808] Emotion Engine Functions

[0809] Emotion Recognition and Analysis

[0810] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions.

[0811] User interaction function

[0812] Sending and receiving chat messages

[0813] Users can enter chat messages while watching a live event. The device sends the messages to a server, which then stores them in a database and relays them in real time to other users participating in the same live event. The emotion engine can also analyze users' emotions and automatically adjust or suggest chat messages.

[0814] Specific examples

[0815] User registration and login

[0816] 1. The user accesses the system and enters information on the account creation screen.

[0817] 2. The device sends the input information to the server, which stores the information in a database.

[0818] 3. The user enters their credentials on the login screen and the server authenticates them.

[0819] Watching live events

[0820] 1. The user selects the live event they want to attend, and the device sends the selection information to the server.

[0821] 2. The server sends the streaming URL to the device, and the device sets up the video stream.

[0822] Utilizing the Emotion Engine

[0823] 1. While the user is watching live video, the device collects emotional data through the camera and microphone.

[0824] 2. The server analyzes the data using an emotion engine and provides optimal live events and interactions based on the user's emotions.

[0825] User interaction

[0826] 1. The user types a chat message while watching live, and the device sends the message to the server.

[0827] 2. The server receives the message and relays it to other users attending the same event.

[0828] The system allows users to transcend physical constraints and enjoy high-quality live events, while leveraging an emotion engine for a more personalized experience and the ability to interact with other users in real time.

[0829] The processing flow will be explained below.

[0830] User registration and login function

[0831] User Registration

[0832] Step 1:

[0833] The user enters their username, password, and email address on the account creation screen.

[0834] Step 2:

[0835] The terminal transmits the input information to the server.

[0836] Step 3:

[0837] The server encrypts the received password and stores it in a database along with the username and email address.

[0838] Step 4:

[0839] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[0840] User Login

[0841] Step 1:

[0842] The user enters their username and password on the login screen.

[0843] Step 2:

[0844] The terminal transmits the entered authentication information to the server.

[0845] Step 3:

[0846] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[0847] Step 4:

[0848] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[0849] Step 5:

[0850] The user accesses the system using a session token.

[0851] Virtual live viewing function

[0852] Select a Live Event

[0853] Step 1:

[0854] Users select the event they want to attend from the list of live events within the app.

[0855] Step 2:

[0856] The terminal sends the selected event ID to the server.

[0857] Start Streaming

[0858] Step 1:

[0859] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[0860] Step 2:

[0861] The terminal uses the received URL to set up a video stream on the VR or AR device.

[0862] Step 3:

[0863] The device will start providing live video.

[0864] Content customization function

[0865] Acquiring and analyzing user history

[0866] Step 1:

[0867] When a user logs in, the server retrieves past listening history from the database.

[0868] Step 2:

[0869] The server provides listening history data to the generation AI, which analyzes the user's preferences.

[0870] Step 3:

[0871] Based on the analysis results, the server generates individually customized live events and performers and sends them to the device.

[0872] Step 4:

[0873] The user reviews and watches the recommended content.

[0874] Emotion Engine Functions

[0875] Emotion Recognition and Analysis

[0876] Step 1:

[0877] The device uses a camera and microphone to capture the facial expressions and voice of the user watching the live video.

[0878] Step 2:

[0879] The data collected by the device is sent to the server in real time.

[0880] Step 3:

[0881] The server analyzes the data using an emotion engine.

[0882] Step 4:

[0883] The server sends information to the device to adjust live events and interactions based on the analyzed user emotions.

[0884] Step 5:

[0885] The device presents appropriate interactions and content to the user.

[0886] User interaction function

[0887] Sending and receiving chat messages

[0888] Step 1:

[0889] A user types a chat message while watching a live event.

[0890] Step 2:

[0891] The terminal transmits the entered chat message to the server.

[0892] Step 3:

[0893] The server stores the received messages in a database.

[0894] Step 4:

[0895] The server relays messages in real time to other users participating in the same live event.

[0896] Step 5:

[0897] The server uses an emotion engine to analyze the content of the message and make adjustments or suggestions as needed.

[0898] Step 6:

[0899] Other users receive and respond to messages in real time.

[0900] Example 2

[0901] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] In today's digital entertainment market, users are looking for ways to enjoy immersive live events beyond physical limitations. Furthermore, there is a need to customize content based on each user's preferences and emotions, providing greater satisfaction. Furthermore, there is a demand for the ability to interact with other users in real time during live events. However, current systems struggle to integrate and effectively realize these elements.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history and emotional data, means for recognizing and analyzing the user's emotions in real time and optimizing live events and interactions based on the results, and means for relaying real-time chats between users and distributing them to other users. This allows users to enjoy immersive live events beyond physical constraints, and further enables individually customized experiences and real-time user interaction.

[0904] "User" refers to an individual who uses this system.

[0905] "Input Information" means information provided by a user to the system, including, for example, a username, password, or email address.

[0906] "User Account" means an individual record in the System created based on a User's personal information and authentication information.

[0907] "Authentication Information" means information used to verify a User's identity, including a username and password.

[0908] "Login" refers to the process by which a user accesses a system using authentication information.

[0909] A "streaming URL" is a link that is generated by the server and sent to the user's device, indicating the destination of the live video stream.

[0910] "Listening history" is a record of music and live events that a user has listened to in the past.

[0911] "Emotional data" refers to information about emotions captured from a user's facial expressions, voice, etc.

[0912] "Recommendation" is the act of suggesting suitable content or services based on a user's listening history and emotional data.

[0913] "Real-time" means that events and interactions occur immediately, without delay.

[0914] "Chat" is a function that allows users to exchange messages and information in real time.

[0915] The "emotion engine" is a function within the system that analyzes emotional data in real time and provides optimal content and interactions based on the results.

[0916] "Virtual reality (VR)" is a technology that allows users to experience a three-dimensional space similar to reality, generated using computer technology.

[0917] "Augmented reality (AR)" is a technology that overlays digital information onto the real physical environment.

[0918] An "interaction" is an action or process in which a user interacts with a system or with another user.

[0919] This invention relates to a system that allows users to enjoy immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies. This system further enhances the user experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the system and their processing are described below.

[0920] First, the user accesses the account creation screen using a device and enters information such as a username, password, and email address. The device then sends this information to the server. The server then encrypts the received password (using a hashing function) and stores it in a database along with other information. The server then generates a new user ID and returns a registration completion message to the device.

[0921] Next, when the user accesses the system again, they enter their username and password on the login screen. The terminal sends the input information to the server, which retrieves the corresponding user's authentication information from the database. The server compares the hash value of the input password with the value stored in the database, and if authentication is successful, it generates a session token and returns it to the terminal. The terminal stores this session token, allowing the user to access the system.

[0922] When a user selects an event they want to attend from a list of live events, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begins providing live video. In some cases, a CDN (Content Delivery Network) service is used for streaming.

[0923] When a user logs in, the server retrieves their listening history from the database and analyzes their preferences using an emotion engine. Based on the results, the generative AI recommends individually customized live events and performers, allowing users to easily find content that matches their preferences.

[0924] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone and sends the data to a server. The server then uses an emotion engine to analyze the user's emotional data and provides optimal live events and interactions based on the results.

[0925] Furthermore, when a user types a chat message while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event. The emotion engine analyzes the message and can make automatic adjustments or suggestions.

[0926] For example, if a user types "I want to watch a live event of my favorite artist," the system will recommend the most suitable live event based on the user's past listening history and current emotional state. Also, if a user types "I want to share my impressions with friends in real time," the system will utilize the chat function to encourage interaction with other users.

[0927] In this way, the system provides users with an optimized live event experience, utilizing virtual and augmented reality to enhance the sense of presence, while also enabling real-time interaction with other users.

[0928] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0929] User registration process steps

[0930] Step 1:

[0931] The user accesses the account creation screen using their device and enters their username, password, and email address.

[0932] Input: Username, Password, Email Address

[0933] Output: User information ready to send

[0934] Step 2:

[0935] The terminal transmits the entered user information to the server.

[0936] Input: User information (username, password, email address)

[0937] Output: Request to server

[0938] Step 3:

[0939] The server applies a hashing function to the received password to generate an encrypted password.

[0940] Enter: Password

[0941] Output: Encrypted password

[0942] Step 4:

[0943] The server stores the username, email address, and hashed password in a database.

[0944] Input: Username, Email Address, and Hashed Password

[0945] Output: Saved to database

[0946] Step 5:

[0947] The server generates a new user ID and returns a registration completion message to the terminal, which displays this message to the user.

[0948] Input: Registered user information

[0949] Output: New user ID, registration completion message

[0950] User login process steps

[0951] Step 1:

[0952] The user accesses the login screen using the device and enters their username and password.

[0953] Input: Username, Password

[0954] Output: Login information ready to be sent

[0955] Step 2:

[0956] The terminal transmits the entered authentication information to the server.

[0957] Input: Username, Password

[0958] Output: Send authentication information to the server

[0959] Step 3:

[0960] The server retrieves the corresponding user's authentication information (hashed password) from the database.

[0961] Input: Username

[0962] Output: Retrieving authentication information from the database

[0963] Step 4:

[0964] The server hashes the entered password and compares it with the hash value retrieved from the database. If authentication is successful, it generates a session token, otherwise it generates an error message.

[0965] Input: Password, hashed password for database

[0966] Output: Session token or error message

[0967] Step 5:

[0968] If authentication is successful, the server generates a session token and sends it to the device, which retains it and allows the user to access the system.

[0969] Input: Session token

[0970] Output: User approved to access the system

[0971] Processing steps for watching virtual live shows

[0972] Step 1:

[0973] Users select the event they want to attend from the list of live events within the app via their device.

[0974] Input: Live Event Selection

[0975] Output: Get the event ID

[0976] Step 2:

[0977] The device sends the ID of the selected event to the server.

[0978] Input: Event ID

[0979] Output: Send event ID to server

[0980] Step 3:

[0981] The server generates a streaming URL based on the event ID. This streaming may use a CDN (Content Delivery Network) service.

[0982] Input: Event ID

[0983] Output: Streaming URL

[0984] Step 4:

[0985] The server generates a streaming URL and sends it to the device.

[0986] Input: Streaming URL

[0987] Output: Send streaming URL to device

[0988] Step 5:

[0989] The device uses the received URL to set up a video stream on the VR or AR device, and the user can then start watching the live video.

[0990] Input: Streaming URL

[0991] Output: Start of video stream

[0992] Emotion Engine Processing Steps

[0993] Step 1:

[0994] While the user is watching the live video, the device's camera and microphone capture the user's facial expressions and voice.

[0995] Input: User's facial expressions, voice

[0996] Output: Obtaining emotion data

[0997] Step 2:

[0998] The device transmits the captured emotion data to the server.

[0999] Input: Emotion data

[1000] Output: Send emotion data to the server

[1001] Step 3:

[1002] The server analyzes the emotional data using an emotion engine, which uses technologies such as facial expression recognition and voice tone analysis.

[1003] Input: Emotion data

[1004] Output: Emotion analysis results

[1005] Step 4:

[1006] The server uses the analysis results to provide optimal live events and interactions tailored to the user's emotions. For example, if a specific emotion is strong, a request can be sent to an artist in real time.

[1007] Input: Sentiment analysis results

[1008] Output: Optimized live events and interactions

[1009] Steps in processing user interactions

[1010] Step 1:

[1011] A user types a chat message while watching a live event.

[1012] Input: Chat message

[1013] Output: Message prepared for sending

[1014] Step 2:

[1015] The terminal transmits the entered chat message to the server.

[1016] Input: Chat message

[1017] Output: Send message to server

[1018] Step 3:

[1019] The server stores the received messages in a database.

[1020] Input: Chat message

[1021] Output: Message saved

[1022] Step 4:

[1023] The server relays messages in real time to other users attending the same live event, and an emotion engine analyzes users' emotions and can automatically adjust or suggest chat messages as needed.

[1024] Input: Chat message

[1025] Output: Message relay to other users, auto-adjustment suggestions

[1026] The above are the specific processing steps of the program of this system.

[1027] (Application example 2)

[1028] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1029] In conventional online live event viewing systems, users were limited to simply watching live footage, and were unable to enjoy real-time interaction or a personalized experience. Furthermore, they lacked the means to recognize users' emotions and enhance the sense of presence or provide recommended content based on those emotions. As a result, the entertainment value and user experience were limited.

[1030] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1031] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for analyzing the user's past viewing history and emotional data and recommending individually customized live events and performers, means for relaying real-time messages between users and distributing them to other users, and means for recognizing user emotions and dynamically adjusting the experience content based on the recognition results. This allows users to experience a more personalized live event and interact with other users in real time, as well as enjoy an emotionally-driven, interactive, and immersive experience.

[1032] "Input Information" means data provided by a user to the system for purposes such as account creation and event selection.

[1033] "Authentication information" refers to data such as username and password used when a user logs in, and is used to accurately identify the user and verify access rights.

[1034] "Streaming URL" means a link on the Internet that streams video of a selected live event to users in real time.

[1035] "Viewing history" is data about content a user has viewed in the past, and personalized recommendations are based on this data.

[1036] "Recommendation" means selecting and presenting appropriate content and performers based on the user's tastes, preferences, and emotions.

[1037] "Real-time Messages" are text messages that allow Users to instantly communicate with other Users while watching a live event.

[1038] "Emotional data" refers to emotional information recognized from a user's facial expressions and voice, and is used to make the user's experience more personalized and relevant.

[1039] "Virtual reality" is a technology that uses specialized devices to allow users to experience a computer-generated environment that differs from reality.

[1040] "Augmented reality" is a technology that overlays digitally generated elements onto the real world, providing users with a blended experience of reality and virtuality.

[1041] "Dynamic adjustment" is the process of optimizing the experience in real time based on the user's emotional data and other variables.

[1042] This invention combines a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technology with an emotion engine that recognizes the user's emotions. The system also has the function of recommending individually customized live events and performers based on the user's past viewing history and emotions, and promoting real-time interaction between users. The following describes in detail the modes for implementing this invention.

[1043] User registration and login function

[1044] User Registration

[1045] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen and sends this information to the server via their device. The server encrypts the received password and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[1046] User Login

[1047] When the user accesses the system again, the server receives the user name and password on the login screen and sends the information to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[1048] Virtual live viewing function

[1049] Select a Live Event

[1050] When a user selects an event they want to attend from a list of live events within the app, the server receives the selection information. The device sends the selected event ID to the server, and the server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[1051] Content customization function

[1052] Acquiring and analyzing user history

[1053] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their preferences.

[1054] Emotion Engine Functions

[1055] Emotion Recognition and Analysis

[1056] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone. This data is sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions. Emotional data recognized from the user's facial expressions and voice is used to make the user's experience more personalized and relevant.

[1057] User interaction function

[1058] Sending and receiving chat messages

[1059] When a user types a chat message while watching a live event, the server can receive the message information and relay it in real time to other users participating in the same live event.The emotion engine can also analyze the user's emotions and automatically adjust or suggest chat messages.

[1060] Hardware and software used

[1061] The following hardware and software are used to implement this system:

[1062] Hardware: PC, smartphone, tablet, VR headset, AR device, camera, microphone

[1063] Software: Flask web framework, Python, database (SQLite or PostgreSQL), OpenCV, a provisional emotion recognition library (EmotionRecognizer), and a recommendation system (Recommender).

[1064] Specific examples

[1065] 1. When a user enters and submits information on the account creation screen, the server stores the encrypted information in a database.

[1066] 2. The user selects the desired live event, and the server generates and returns a streaming URL.

[1067] 3. The emotion engine recognizes emotions based on the user's facial expressions and voice data, and provides recommendations to the user based on those emotions.

[1068] Prompt Sentence Examples

[1069] "Use an emotion engine to analyze users' facial expressions and recommend appropriate live events based on their past viewing history."

[1070] "Share chat messages typed by users while watching a live event with other users in real time."

[1071] In this way, the present invention is a system that utilizes the user's emotions and past viewing history to provide a more personalized, interactive, and immersive live experience.

[1072] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1073] Step 1:

[1074] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen. The device sends this input information to the server. The server uses a hashing function to encrypt the password and stores it in a database along with the username and email address. As output, it generates a new user ID and returns a registration completion message to the user.

[1075] Step 2:

[1076] The server saves the user's authentication information and authenticates the user the next time they log in. When accessing the system again, the user enters their username and password on the login screen and sends them to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal. The user can then use this session token to access the system.

[1077] Step 3:

[1078] When a user selects an event they want to attend from the list of live events in the app, the server receives the selection information. The device sends the selected event ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[1079] Step 4:

[1080] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers. As an output, users can easily find content that suits their preferences.

[1081] Step 5:

[1082] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the server provides live events and interactions optimized for the user's emotions. Specifically, content recommendations and interaction adjustments are made based on emotion recognition results.

[1083] Step 6:

[1084] When a user enters a chat message while watching a live event, the server receives the message information. The device then sends the entered chat message to the server, which then relays the message to other users participating in the same live event in real time. As an output, other users receive the chat message in real time, allowing them to interact with each other.

[1085] In this way, each step works together to allow users to enjoy an immersive live event, with a more personalized experience and real-time interaction with other users.

[1086] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1087] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1088] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1089] [Third embodiment]

[1090] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1091] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1092] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1093] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1094] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1095] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1096] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1097] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1098] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1099] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1100] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1101] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1102] This invention is a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies, and also provides personalized recommendations for live events and artists based on the user's listening history and preferences, as well as the ability to interact with other users in real time.

[1103] User registration and login function

[1104] User Registration

[1105] The user enters their information (username, password, email address) on the account creation screen and sends that information to the server via their device. The server encrypts the received password (using a hashing function) and stores it in a database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user, creating the account.

[1106] User Login

[1107] When the user accesses the system again, they enter their username and password on the login screen and send that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the device, allowing the user to access the system.

[1108] Virtual live viewing function

[1109] Select a Live Event

[1110] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[1111] Start Streaming

[1112] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1113] Content customization function

[1114] Acquiring and analyzing user history

[1115] When a user logs in, the server retrieves their listening history from the database. Based on this history data, the AI ​​analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that matches their tastes.

[1116] User interaction function

[1117] Sending and receiving chat messages

[1118] While watching a live event, users can use the chat function to interact with other users. When a user types a chat message and presses the send button, the device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event.

[1119] Specific examples

[1120] 1. A user logs in to the system and accesses the TOP page.

[1121] 2. The user selects the live event they are interested in from the available options and starts watching.

[1122] 3. The server provides a streaming URL and sends it to the device.

[1123] 4. The device will begin providing live footage to the user through the VR device.

[1124] 5. Users exchange chat messages with other participants in real time.

[1125] 6. The server uses AI to analyze the user's listening history and recommend live events and artists to watch next.

[1126] This system allows users to enjoy live events beyond physical constraints, interact with other fans in real time, and more easily find content that suits their tastes.

[1127] The processing flow will be explained below.

[1128] User registration and login function

[1129] User Registration

[1130] Step 1:

[1131] The user enters their username, password, and email address on the account creation screen.

[1132] Step 2:

[1133] The terminal transmits the input information to the server.

[1134] Step 3:

[1135] The server encrypts the received password and stores it in a database along with the username and email address.

[1136] Step 4:

[1137] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[1138] User Login

[1139] Step 1:

[1140] The user enters their username and password on the login screen.

[1141] Step 2:

[1142] The terminal transmits the entered authentication information to the server.

[1143] Step 3:

[1144] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[1145] Step 4:

[1146] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[1147] Step 5:

[1148] The user accesses the system based on a session token.

[1149] Virtual live viewing function

[1150] Select a Live Event

[1151] Step 1:

[1152] Users select the event they want to attend from the list of live events within the app.

[1153] Step 2:

[1154] The terminal sends the selected event ID to the server.

[1155] Start Streaming

[1156] Step 1:

[1157] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[1158] Step 2:

[1159] The device uses the received URL to set up a video stream on the VR or AR device and start providing live video.

[1160] Content customization function

[1161] Acquiring and analyzing user history

[1162] Step 1:

[1163] When a user logs in, the server retrieves past listening history from the database.

[1164] Step 2:

[1165] The server uses generative AI to analyze the user's preferences based on listening history data.

[1166] Step 3:

[1167] Based on the analysis results, the server generates appropriate live events and artists and sends them to the device.

[1168] Step 4:

[1169] Users review recommended content.

[1170] User interaction function

[1171] Sending and receiving chat messages

[1172] Step 1:

[1173] A user types a chat message while watching a live event.

[1174] Step 2:

[1175] The terminal transmits the entered chat message to the server.

[1176] Step 3:

[1177] The server stores the received messages in a database.

[1178] Step 4:

[1179] The server relays the incoming message to other users attending the same live event.

[1180] Step 5:

[1181] Other users receive and respond to messages in real time.

[1182] Example 1

[1183] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1184] Today's users want to experience immersive live events from the comfort of their own homes. They also want to find content tailored to their preferences and interact with other users in real time. Conventional systems have struggled to meet these needs, limiting the user experience. This invention aims to provide a system that allows users to transcend physical constraints, providing a more personalized virtual reality experience and enabling real-time interaction with other users.

[1185] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1186] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time, means for generating and distributing a streaming address corresponding to content selected by the user, means for analyzing the user's past usage history using a generative AI model and making individually customized recommendations based on the user's preferences, means for providing users with an experience using virtual reality or augmented reality, and means for relaying real-time message exchanges between users and distributing them to other users. This allows users to experience an immersive live event from the comfort of their own home and enjoy individually customized content. At the same time, they can interact with other users in real time.

[1187] "User" means any individual or entity that uses the System.

[1188] "Input Information" means data or information that a user provides to a system, including, for example, a username, password, or email address.

[1189] "User Account" means individual identification information registered by a User in order to use the System.

[1190] "Authentication Information" means data used to identify and authenticate a user to access a system, including, by way of example, a hashed password.

[1191] "Streaming Address" means the URL or link used to stream a live event or content.

[1192] "Usage history" is a record of past operations and selections made by a user within the system.

[1193] A "generative AI model" is an algorithm or program that uses artificial intelligence techniques to analyze user data and generate specific results or recommendations.

[1194] "Virtual reality" is a three-dimensional visual and sensory experience generated using computer technology.

[1195] "Augmented reality" is a technology that overlays computer-generated information onto real-world visual information.

[1196] "Real-time message exchange" is a function that allows users to send and receive messages instantly.

[1197] "Relaying" is the process of receiving data or information and then forwarding it to other users or devices.

[1198] This invention is a system that allows users to experience immersive live events from the comfort of their own home using VR and AR technologies. Furthermore, the system uses a generative AI model based on the user's past usage history to provide customized content recommendations and allows users to interact with other users in real time. A specific implementation of this system is described below.

[1199] User registration and login function

[1200] User Registration

[1201] When a user creates an account, they enter a username, password, and email address. This information is sent from the device to the server. The server then encrypts the received password using a hashing function (e.g., SHA-256) and stores it in a database. Once registration is complete, the server generates a user ID and returns a registration completion message to the user.

[1202] User Login

[1203] When a user logs in, they enter their username and password. This information is sent from the terminal to the server. The server retrieves the authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal, allowing the user to access the system.

[1204] Virtual live viewing function

[1205] Select a Live Event

[1206] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[1207] Start Streaming

[1208] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1209] Content customization function

[1210] Acquiring and analyzing user history

[1211] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[1212] Specific prompt examples:

[1213] "Based on my recent listening history, what live event would you recommend next?"

[1214] "Show me a list of live events worth attending"

[1215] User interaction function

[1216] Sending and receiving chat messages

[1217] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1218] The system allows users to enjoy live events beyond physical constraints, interact with other attendees in real time, and discover content that suits their tastes.

[1219] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1220] Step 1: User Registration

[1221] The user enters a username, password, and email address on the account creation screen. This input data is sent from the device to the server. The server receives the received data and encrypts the password using a hashing function (e.g., SHA-256). The encrypted password and other user information are stored in a database. The server then generates a new user ID and returns it to the device along with a message confirming the user's registration. This completes the creation of the user account.

[1222] Specific behavior:

[1223] 1. The user enters information into the form on the browser and presses the "Register" button.

[1224] 2. The terminal sends the input data in JSON format to the server.

[1225] 3. The server receives the data, encrypts the password, and stores it in the database.

[1226] 4. The server generates a new user ID and returns a registration completion message to the terminal.

[1227] Step 2: User Login

[1228] The user enters their username and password on the login screen. This input data is sent from the device to the server. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, allowing the user to access the system.

[1229] Specific behavior:

[1230] 1. The user enters their username and password on the login screen and presses the "Login" button.

[1231] 2. The terminal sends the input data in JSON format to the server.

[1232] 3. The server retrieves the authentication information from the database and verifies the password.

[1233] 4. If the server is successful in authentication, it generates a session token and returns it to the device.

[1234] Step 3: Select a live event

[1235] Users select the event they want to attend from a list of live events provided within the app. The selected event ID is sent from the device to the server. The server receives the event ID and generates the corresponding streaming URL.

[1236] Specific behavior:

[1237] 1. A user browses the list of live events on the app and selects the event they want to attend.

[1238] 2. The device sends the selected event ID to the server.

[1239] 3. The server generates a streaming URL based on the received event ID.

[1240] Step 4: Start Streaming

[1241] The server sends the generated streaming URL to the device, which then uses the URL to set up a video stream on the VR or AR device and provide the user with live video.

[1242] Specific behavior:

[1243] 1. The server sends the streaming URL to the device.

[1244] 2. Using the URL received by the terminal, set up a stream on the VR device using WebRTC or HLS technology.

[1245] 3. Users watch the live footage through a VR device.

[1246] Step 5: Capture and analyze user history

[1247] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends customized live events and performers.

[1248] Specific behavior:

[1249] 1. The server retrieves the history from the database using the user ID.

[1250] 2. The server inputs historical data into the generated AI model and analyzes preferences.

[1251] 3. The server recommends customized content to the user based on the analysis results.

[1252] Step 6: Sending and Receiving Chat Messages

[1253] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1254] Specific behavior:

[1255] 1. The user types a message in the chat window and presses the "Send" button.

[1256] 2. The device sends a message to the server using WebSocket or an HTTP POST request.

[1257] 3. The server stores the message in a database and relays it to other participants in real time.

[1258] (Application example 1)

[1259] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1260] Conventional live event viewing systems require users to physically travel to the event venue, consuming a lot of time and money and lacking in convenience. Furthermore, there are limited ways for users to interact with other users in real time, which lacks the immersive and intimate feeling of a live event. Furthermore, the lack of customized event recommendations based on past listening history makes it difficult for users to find events that suit their preferences.

[1261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1262] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history, means for relaying real-time chats between users and distributing them to other users, and viewing means for the user to watch the live event on a device using virtual reality or augmented reality.This allows users to experience an immersive live event from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[1263] "User input information" refers to information provided by a user when creating an account or logging in, including a username, password, email address, etc.

[1264] "User Account" means an account created by a User to use the System, which includes the User's personal information and authentication information.

[1265] "Authentication Information" means information used to verify a user's access privileges, typically including a username and password.

[1266] A "streaming URL" is a link that allows you to watch live events or video content in real time over the Internet.

[1267] "Listening History" means a record of live events or audio content that a User has previously viewed or listened to.

[1268] A "live event" is a performance, show, or other event that takes place in real time.

[1269] "Performer" means an artist or performer who performs at a live event.

[1270] "Recommendation methods" are methods for presenting appropriate live events and performers based on a user's past listening history and preferences.

[1271] "Real-time chat" means a feature that allows users to instantly exchange messages with other users during a live event.

[1272] A "relaying means" is a method for delivering a chat message received from one user to another user.

[1273] "Virtual reality (VR)" is a technology that uses computer graphics technology to allow users to experience a virtual environment that is different from reality.

[1274] "Augmented reality (AR)" is a technology that overlays digital information onto a real-world environment, allowing users to visually experience additional information.

[1275] A "viewing instrument" is a method by which a user views a live event using a virtual reality or augmented reality device.

[1276] A "generative AI model" is an artificial intelligence model that learns user preferences and behaviors based on large amounts of data and makes appropriate recommendations and personalization.

[1277] A "prompt" is text data that is input into a generative AI model and serves as a trigger for generating a specific result or response.

[1278] "Device" refers to hardware equipment such as a computer, smartphone, or VR headset.

[1279] The system for realizing the present invention has the following main functions.

[1280] 1. User Account Creation and Authentication

[1281] The server receives the user's input and creates a user account. The user's credentials are saved and used to authenticate the user the next time they log in.

[1282] For example, when a user enters their information (username, password, email address) on the account creation screen, the server receives that information, encrypts the password (using a hashing function) and stores it in the database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user.

[1283] 2. Watching Live Events

[1284] The server generates a streaming URL corresponding to the user's selected event and delivers it to the user, who can then watch the live event using their virtual reality (VR) or augmented reality (AR) device.

[1285] For example, when a user selects an event they want to attend from a list of live events provided within the app, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device then uses the URL to set up a video stream on the VR or AR device and provide live video.

[1286] 3. Customized content recommendations to users

[1287] The server analyzes the user's past listening history and uses a generative AI model to make personalized recommendations based on the user's preferences.

[1288] For example, when a user logs in, the server retrieves their past listening history from the database, and the generative AI model analyzes the user's preferences based on this historical data, suggesting live events and artists to watch next.

[1289] 4. Real-time chat between users

[1290] The server provides users with the ability to exchange chat messages with other users in real time while watching a live event, which promotes a sense of community among users.

[1291] For example, when a user types a chat message and hits the send button, the device sends the message to a server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1292] Hardware and software used

[1293] Hardware: VR device (e.g., Oculus Quest 2), AR device (e.g., Microsoft HoloLens), smartphone, PC

[1294] Software: Server applications (e.g., Flask framework), databases (e.g., MySQL), generative AI models

[1295] Prompt Sentence Examples

[1296] "Recommend the best live event based on the user's listening history. The data to recommend the next event to watch is as follows: - User ID: 1234 - Listening history: ['Artist A', 'Artist B', 'Event C']"

[1297] As a result, the system allows users to experience immersive live events from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[1298] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1299] Step 1:

[1300] The user enters their information (username, password, email address) on the account creation screen and sends the information to the server via the device. The server creates a user account based on the received information. Specifically, the server encrypts the entered password (using a hashing function) and stores the encrypted information in a database. Finally, the server generates a user ID and returns a registration completion message to the device.

[1301] Input: Username, Password, Email Address

[1302] Data processing: password hashing

[1303] Output: User ID, registration completion message

[1304] Step 2:

[1305] The user enters their username and password on the login screen and sends that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, authenticating the user.

[1306] Input: Username, Password

[1307] Data processing: password hashing, authentication information matching

[1308] Output: Session token

[1309] Step 3:

[1310] The user selects the event they want to attend from a list of live events provided within the app. The device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1311] Input: Event ID

[1312] Data processing: Streaming URL generation

[1313] Output: Streaming URL

[1314] Step 4:

[1315] When a user logs in, the server retrieves the user's past listening history from a database. The server then analyzes the retrieved listening history using a generative AI model to recommend live events and performers that are individually tailored to the user's preferences. The recommendations are then sent to the device.

[1316] Input: User ID

[1317] Data processing: Acquisition and analysis of listening history, and recommendations based on generative AI models

[1318] Output: Recommended live events and performers

[1319] Step 5:

[1320] While watching a live event, a user types a chat message and presses the send button. The device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event. The chat message is then distributed to other devices in real time.

[1321] Input: Chat message

[1322] Data processing: message storage and relay

[1323] Output: Chat messages delivered to other users' devices

[1324] These steps allow users to enjoy immersive live events from the comfort of their own home, interact with other users in real time, and receive personalized live event recommendations based on their past listening history.

[1325] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1326] This invention combines a system that allows users to experience immersive live events from the comfort of their own homes using virtual reality (VR) and augmented reality (AR) technologies with an emotion engine that recognizes users' emotions. Furthermore, it has the ability to recommend live events and artists that are individually customized based on the user's past listening history and emotions, and to promote real-time interaction between users.

[1327] User registration and login function

[1328] User Registration

[1329] The user enters a username, password, and email address on the account creation screen and sends that information to the server via the device. The server then encrypts the received password (using a hashing function) and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[1330] User Login

[1331] When accessing the system again, the user enters their username and password on the login screen and sends that information to the server via their terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[1332] Virtual live viewing function

[1333] Select a Live Event

[1334] Users select the event they want to attend from the list of live events in the app, and the device sends the selected event ID to the server.

[1335] Start Streaming

[1336] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and starts providing live video.

[1337] Content customization function

[1338] Acquiring and analyzing user history

[1339] When a user logs in, the server retrieves their listening history from the database. Based on this history data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[1340] Emotion Engine Functions

[1341] Emotion Recognition and Analysis

[1342] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions.

[1343] User interaction function

[1344] Sending and receiving chat messages

[1345] Users can enter chat messages while watching a live event. The device sends the messages to a server, which then stores them in a database and relays them in real time to other users participating in the same live event. The emotion engine can also analyze users' emotions and automatically adjust or suggest chat messages.

[1346] Specific examples

[1347] User registration and login

[1348] 1. The user accesses the system and enters information on the account creation screen.

[1349] 2. The device sends the input information to the server, which stores the information in a database.

[1350] 3. The user enters their credentials on the login screen and the server authenticates them.

[1351] Watching live events

[1352] 1. The user selects the live event they want to attend, and the device sends the selection information to the server.

[1353] 2. The server sends the streaming URL to the device, and the device sets up the video stream.

[1354] Utilizing the Emotion Engine

[1355] 1. While the user is watching live video, the device collects emotional data through the camera and microphone.

[1356] 2. The server analyzes the data using an emotion engine and provides optimal live events and interactions based on the user's emotions.

[1357] User interaction

[1358] 1. The user types a chat message while watching live, and the device sends the message to the server.

[1359] 2. The server receives the message and relays it to other users attending the same event.

[1360] The system allows users to transcend physical constraints and enjoy high-quality live events, while leveraging an emotion engine for a more personalized experience and the ability to interact with other users in real time.

[1361] The processing flow will be explained below.

[1362] User registration and login function

[1363] User Registration

[1364] Step 1:

[1365] The user enters their username, password, and email address on the account creation screen.

[1366] Step 2:

[1367] The terminal transmits the input information to the server.

[1368] Step 3:

[1369] The server encrypts the received password and stores it in a database along with the username and email address.

[1370] Step 4:

[1371] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[1372] User Login

[1373] Step 1:

[1374] The user enters their username and password on the login screen.

[1375] Step 2:

[1376] The terminal transmits the entered authentication information to the server.

[1377] Step 3:

[1378] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[1379] Step 4:

[1380] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[1381] Step 5:

[1382] The user accesses the system using a session token.

[1383] Virtual live viewing function

[1384] Select a Live Event

[1385] Step 1:

[1386] Users select the event they want to attend from the list of live events within the app.

[1387] Step 2:

[1388] The terminal sends the selected event ID to the server.

[1389] Start Streaming

[1390] Step 1:

[1391] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[1392] Step 2:

[1393] The terminal uses the received URL to set up a video stream on the VR or AR device.

[1394] Step 3:

[1395] The device will start providing live video.

[1396] Content customization function

[1397] Acquiring and analyzing user history

[1398] Step 1:

[1399] When a user logs in, the server retrieves past listening history from the database.

[1400] Step 2:

[1401] The server provides listening history data to the generation AI, which analyzes the user's preferences.

[1402] Step 3:

[1403] Based on the analysis results, the server generates individually customized live events and performers and sends them to the device.

[1404] Step 4:

[1405] The user reviews and watches the recommended content.

[1406] Emotion Engine Functions

[1407] Emotion Recognition and Analysis

[1408] Step 1:

[1409] The device uses a camera and microphone to capture the facial expressions and voice of the user watching the live video.

[1410] Step 2:

[1411] The data collected by the device is sent to the server in real time.

[1412] Step 3:

[1413] The server analyzes the data using an emotion engine.

[1414] Step 4:

[1415] The server sends information to the device to adjust live events and interactions based on the analyzed user emotions.

[1416] Step 5:

[1417] The device presents appropriate interactions and content to the user.

[1418] User interaction function

[1419] Sending and receiving chat messages

[1420] Step 1:

[1421] A user types a chat message while watching a live event.

[1422] Step 2:

[1423] The terminal transmits the entered chat message to the server.

[1424] Step 3:

[1425] The server stores the received messages in a database.

[1426] Step 4:

[1427] The server relays messages in real time to other users participating in the same live event.

[1428] Step 5:

[1429] The server uses an emotion engine to analyze the content of the message and make adjustments or suggestions as needed.

[1430] Step 6:

[1431] Other users receive and respond to messages in real time.

[1432] Example 2

[1433] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1434] In today's digital entertainment market, users are looking for ways to enjoy immersive live events beyond physical limitations. Furthermore, there is a need to customize content based on each user's preferences and emotions, providing greater satisfaction. Furthermore, there is a demand for the ability to interact with other users in real time during live events. However, current systems struggle to integrate and effectively realize these elements.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history and emotional data, means for recognizing and analyzing the user's emotions in real time and optimizing live events and interactions based on the results, and means for relaying real-time chats between users and distributing them to other users. This allows users to enjoy immersive live events beyond physical constraints, and further enables individually customized experiences and real-time user interaction.

[1436] "User" refers to an individual who uses this system.

[1437] "Input Information" means information provided by a user to the system, including, for example, a username, password, or email address.

[1438] "User Account" means an individual record in the System created based on a User's personal information and authentication information.

[1439] "Authentication Information" means information used to verify a User's identity, including a username and password.

[1440] "Login" refers to the process by which a user accesses a system using authentication information.

[1441] A "streaming URL" is a link that is generated by the server and sent to the user's device, indicating the destination of the live video stream.

[1442] "Listening history" is a record of music and live events that a user has listened to in the past.

[1443] "Emotional data" refers to information about emotions captured from a user's facial expressions, voice, etc.

[1444] "Recommendation" is the act of suggesting suitable content or services based on a user's listening history and emotional data.

[1445] "Real-time" means that events and interactions occur immediately, without delay.

[1446] "Chat" is a function that allows users to exchange messages and information in real time.

[1447] The "emotion engine" is a function within the system that analyzes emotional data in real time and provides optimal content and interactions based on the results.

[1448] "Virtual reality (VR)" is a technology that allows users to experience a three-dimensional space similar to reality, generated using computer technology.

[1449] "Augmented reality (AR)" is a technology that overlays digital information onto the real physical environment.

[1450] An "interaction" is an action or process in which a user interacts with a system or with another user.

[1451] This invention relates to a system that allows users to enjoy immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies. This system further enhances the user experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the system and their processing are described below.

[1452] First, the user accesses the account creation screen using a device and enters information such as a username, password, and email address. The device then sends this information to the server. The server then encrypts the received password (using a hashing function) and stores it in a database along with other information. The server then generates a new user ID and returns a registration completion message to the device.

[1453] Next, when the user accesses the system again, they enter their username and password on the login screen. The terminal sends the input information to the server, which retrieves the corresponding user's authentication information from the database. The server compares the hash value of the input password with the value stored in the database, and if authentication is successful, it generates a session token and returns it to the terminal. The terminal stores this session token, allowing the user to access the system.

[1454] When a user selects an event they want to attend from a list of live events, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begins providing live video. In some cases, a CDN (Content Delivery Network) service is used for streaming.

[1455] When a user logs in, the server retrieves their listening history from the database and analyzes their preferences using an emotion engine. Based on the results, the generative AI recommends individually customized live events and performers, allowing users to easily find content that matches their preferences.

[1456] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone and sends the data to a server. The server then uses an emotion engine to analyze the user's emotional data and provides optimal live events and interactions based on the results.

[1457] Furthermore, when a user types a chat message while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event. The emotion engine analyzes the message and can make automatic adjustments or suggestions.

[1458] For example, if a user types "I want to watch a live event of my favorite artist," the system will recommend the most suitable live event based on the user's past listening history and current emotional state. Also, if a user types "I want to share my impressions with friends in real time," the system will utilize the chat function to encourage interaction with other users.

[1459] In this way, the system provides users with an optimized live event experience, utilizing virtual and augmented reality to enhance the sense of presence, while also enabling real-time interaction with other users.

[1460] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1461] User registration process steps

[1462] Step 1:

[1463] The user accesses the account creation screen using their device and enters their username, password, and email address.

[1464] Input: Username, Password, Email Address

[1465] Output: User information ready to send

[1466] Step 2:

[1467] The terminal transmits the entered user information to the server.

[1468] Input: User information (username, password, email address)

[1469] Output: Request to server

[1470] Step 3:

[1471] The server applies a hashing function to the received password to generate an encrypted password.

[1472] Enter: Password

[1473] Output: Encrypted password

[1474] Step 4:

[1475] The server stores the username, email address, and hashed password in a database.

[1476] Input: Username, Email Address, and Hashed Password

[1477] Output: Saved to database

[1478] Step 5:

[1479] The server generates a new user ID and returns a registration completion message to the terminal, which displays this message to the user.

[1480] Input: Registered user information

[1481] Output: New user ID, registration completion message

[1482] User login process steps

[1483] Step 1:

[1484] The user accesses the login screen using the device and enters their username and password.

[1485] Input: Username, Password

[1486] Output: Login information ready to be sent

[1487] Step 2:

[1488] The terminal transmits the entered authentication information to the server.

[1489] Input: Username, Password

[1490] Output: Send authentication information to the server

[1491] Step 3:

[1492] The server retrieves the corresponding user's authentication information (hashed password) from the database.

[1493] Input: Username

[1494] Output: Retrieving authentication information from the database

[1495] Step 4:

[1496] The server hashes the entered password and compares it with the hash value retrieved from the database. If authentication is successful, it generates a session token, otherwise it generates an error message.

[1497] Input: Password, hashed password for database

[1498] Output: Session token or error message

[1499] Step 5:

[1500] If authentication is successful, the server generates a session token and sends it to the device, which retains it and allows the user to access the system.

[1501] Input: Session token

[1502] Output: User approved to access the system

[1503] Processing steps for watching virtual live shows

[1504] Step 1:

[1505] Users select the event they want to attend from the list of live events within the app via their device.

[1506] Input: Live Event Selection

[1507] Output: Get the event ID

[1508] Step 2:

[1509] The device sends the ID of the selected event to the server.

[1510] Input: Event ID

[1511] Output: Send event ID to server

[1512] Step 3:

[1513] The server generates a streaming URL based on the event ID. This streaming may use a CDN (Content Delivery Network) service.

[1514] Input: Event ID

[1515] Output: Streaming URL

[1516] Step 4:

[1517] The server generates a streaming URL and sends it to the device.

[1518] Input: Streaming URL

[1519] Output: Send streaming URL to device

[1520] Step 5:

[1521] The device uses the received URL to set up a video stream on the VR or AR device, and the user can then start watching the live video.

[1522] Input: Streaming URL

[1523] Output: Start of video stream

[1524] Emotion Engine Processing Steps

[1525] Step 1:

[1526] While the user is watching the live video, the device's camera and microphone capture the user's facial expressions and voice.

[1527] Input: User's facial expressions, voice

[1528] Output: Obtaining emotion data

[1529] Step 2:

[1530] The device transmits the captured emotion data to the server.

[1531] Input: Emotion data

[1532] Output: Send emotion data to the server

[1533] Step 3:

[1534] The server analyzes the emotional data using an emotion engine, which uses technologies such as facial expression recognition and voice tone analysis.

[1535] Input: Emotion data

[1536] Output: Emotion analysis results

[1537] Step 4:

[1538] The server uses the analysis results to provide optimal live events and interactions tailored to the user's emotions. For example, if a specific emotion is strong, a request can be sent to an artist in real time.

[1539] Input: Sentiment analysis results

[1540] Output: Optimized live events and interactions

[1541] Steps in processing user interactions

[1542] Step 1:

[1543] A user types a chat message while watching a live event.

[1544] Input: Chat message

[1545] Output: Message prepared for sending

[1546] Step 2:

[1547] The terminal transmits the entered chat message to the server.

[1548] Input: Chat message

[1549] Output: Send message to server

[1550] Step 3:

[1551] The server stores the received messages in a database.

[1552] Input: Chat message

[1553] Output: Message saved

[1554] Step 4:

[1555] The server relays messages in real time to other users attending the same live event, and an emotion engine analyzes users' emotions and can automatically adjust or suggest chat messages as needed.

[1556] Input: Chat message

[1557] Output: Message relay to other users, auto-adjustment suggestions

[1558] The above are the specific processing steps of the program of this system.

[1559] (Application example 2)

[1560] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1561] In conventional online live event viewing systems, users were limited to simply watching live footage, and were unable to enjoy real-time interaction or a personalized experience. Furthermore, they lacked the means to recognize users' emotions and enhance the sense of presence or provide recommended content based on those emotions. As a result, the entertainment value and user experience were limited.

[1562] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1563] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for analyzing the user's past viewing history and emotional data and recommending individually customized live events and performers, means for relaying real-time messages between users and distributing them to other users, and means for recognizing user emotions and dynamically adjusting the experience content based on the recognition results. This allows users to experience a more personalized live event and interact with other users in real time, as well as enjoy an emotionally-driven, interactive, and immersive experience.

[1564] "Input Information" means data provided by a user to the system for purposes such as account creation and event selection.

[1565] "Authentication information" refers to data such as username and password used when a user logs in, and is used to accurately identify the user and verify access rights.

[1566] "Streaming URL" means a link on the Internet that streams video of a selected live event to users in real time.

[1567] "Viewing history" is data about content a user has viewed in the past, and personalized recommendations are based on this data.

[1568] "Recommendation" means selecting and presenting appropriate content and performers based on the user's tastes, preferences, and emotions.

[1569] "Real-time Messages" are text messages that allow Users to instantly communicate with other Users while watching a live event.

[1570] "Emotional data" refers to emotional information recognized from a user's facial expressions and voice, and is used to make the user's experience more personalized and relevant.

[1571] "Virtual reality" is a technology that uses specialized devices to allow users to experience a computer-generated environment that differs from reality.

[1572] "Augmented reality" is a technology that overlays digitally generated elements onto the real world, providing users with a blended experience of reality and virtuality.

[1573] "Dynamic adjustment" is the process of optimizing the experience in real time based on the user's emotional data and other variables.

[1574] This invention combines a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technology with an emotion engine that recognizes the user's emotions. The system also has the function of recommending individually customized live events and performers based on the user's past viewing history and emotions, and promoting real-time interaction between users. The following describes in detail the modes for implementing this invention.

[1575] User registration and login function

[1576] User Registration

[1577] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen and sends this information to the server via their device. The server encrypts the received password and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[1578] User Login

[1579] When the user accesses the system again, the server receives the user name and password on the login screen and sends the information to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[1580] Virtual live viewing function

[1581] Select a Live Event

[1582] When a user selects an event they want to attend from a list of live events within the app, the server receives the selection information. The device sends the selected event ID to the server, and the server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[1583] Content customization function

[1584] Acquiring and analyzing user history

[1585] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their preferences.

[1586] Emotion Engine Functions

[1587] Emotion Recognition and Analysis

[1588] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone. This data is sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions. Emotional data recognized from the user's facial expressions and voice is used to make the user's experience more personalized and relevant.

[1589] User interaction function

[1590] Sending and receiving chat messages

[1591] When a user types a chat message while watching a live event, the server can receive the message information and relay it in real time to other users participating in the same live event.The emotion engine can also analyze the user's emotions and automatically adjust or suggest chat messages.

[1592] Hardware and software used

[1593] The following hardware and software are used to implement this system:

[1594] Hardware: PC, smartphone, tablet, VR headset, AR device, camera, microphone

[1595] Software: Flask web framework, Python, database (SQLite or PostgreSQL), OpenCV, a provisional emotion recognition library (EmotionRecognizer), and a recommendation system (Recommender).

[1596] Specific examples

[1597] 1. When a user enters and submits information on the account creation screen, the server stores the encrypted information in a database.

[1598] 2. The user selects the desired live event, and the server generates and returns a streaming URL.

[1599] 3. The emotion engine recognizes emotions based on the user's facial expressions and voice data, and provides recommendations to the user based on those emotions.

[1600] Prompt Sentence Examples

[1601] "Use an emotion engine to analyze users' facial expressions and recommend appropriate live events based on their past viewing history."

[1602] "Share chat messages typed by users while watching a live event with other users in real time."

[1603] In this way, the present invention is a system that utilizes the user's emotions and past viewing history to provide a more personalized, interactive, and immersive live experience.

[1604] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1605] Step 1:

[1606] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen. The device sends this input information to the server. The server uses a hashing function to encrypt the password and stores it in a database along with the username and email address. As output, it generates a new user ID and returns a registration completion message to the user.

[1607] Step 2:

[1608] The server saves the user's authentication information and authenticates the user the next time they log in. When accessing the system again, the user enters their username and password on the login screen and sends them to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal. The user can then use this session token to access the system.

[1609] Step 3:

[1610] When a user selects an event they want to attend from the list of live events in the app, the server receives the selection information. The device sends the selected event ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[1611] Step 4:

[1612] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers. As an output, users can easily find content that suits their preferences.

[1613] Step 5:

[1614] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the server provides live events and interactions optimized for the user's emotions. Specifically, content recommendations and interaction adjustments are made based on emotion recognition results.

[1615] Step 6:

[1616] When a user enters a chat message while watching a live event, the server receives the message information. The device then sends the entered chat message to the server, which then relays the message to other users participating in the same live event in real time. As an output, other users receive the chat message in real time, allowing them to interact with each other.

[1617] In this way, each step works together to allow users to enjoy an immersive live event, with a more personalized experience and real-time interaction with other users.

[1618] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1619] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1620] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1621] [Fourth embodiment]

[1622] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1623] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1624] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1626] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1628] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1629] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1630] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1631] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1632] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1633] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1634] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1635] This invention is a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies, and also provides personalized recommendations for live events and artists based on the user's listening history and preferences, as well as the ability to interact with other users in real time.

[1636] User registration and login function

[1637] User Registration

[1638] The user enters their information (username, password, email address) on the account creation screen and sends that information to the server via their device. The server encrypts the received password (using a hashing function) and stores it in a database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user, creating the account.

[1639] User Login

[1640] When the user accesses the system again, they enter their username and password on the login screen and send that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the device, allowing the user to access the system.

[1641] Virtual live viewing function

[1642] Select a Live Event

[1643] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[1644] Start Streaming

[1645] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1646] Content customization function

[1647] Acquiring and analyzing user history

[1648] When a user logs in, the server retrieves their listening history from the database. Based on this history data, the AI ​​analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that matches their tastes.

[1649] User interaction function

[1650] Sending and receiving chat messages

[1651] While watching a live event, users can use the chat function to interact with other users. When a user types a chat message and presses the send button, the device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event.

[1652] Specific examples

[1653] 1. A user logs in to the system and accesses the TOP page.

[1654] 2. The user selects the live event they are interested in from the available options and starts watching.

[1655] 3. The server provides a streaming URL and sends it to the device.

[1656] 4. The device will begin providing live footage to the user through the VR device.

[1657] 5. Users exchange chat messages with other participants in real time.

[1658] 6. The server uses AI to analyze the user's listening history and recommend live events and artists to watch next.

[1659] This system allows users to enjoy live events beyond physical constraints, interact with other fans in real time, and more easily find content that suits their tastes.

[1660] The processing flow will be explained below.

[1661] User registration and login function

[1662] User Registration

[1663] Step 1:

[1664] The user enters their username, password, and email address on the account creation screen.

[1665] Step 2:

[1666] The terminal transmits the input information to the server.

[1667] Step 3:

[1668] The server encrypts the received password and stores it in a database along with the username and email address.

[1669] Step 4:

[1670] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[1671] User Login

[1672] Step 1:

[1673] The user enters their username and password on the login screen.

[1674] Step 2:

[1675] The terminal transmits the entered authentication information to the server.

[1676] Step 3:

[1677] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[1678] Step 4:

[1679] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[1680] Step 5:

[1681] The user accesses the system based on a session token.

[1682] Virtual live viewing function

[1683] Select a Live Event

[1684] Step 1:

[1685] Users select the event they want to attend from the list of live events within the app.

[1686] Step 2:

[1687] The terminal sends the selected event ID to the server.

[1688] Start Streaming

[1689] Step 1:

[1690] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[1691] Step 2:

[1692] The device uses the received URL to set up a video stream on the VR or AR device and start providing live video.

[1693] Content customization function

[1694] Acquiring and analyzing user history

[1695] Step 1:

[1696] When a user logs in, the server retrieves past listening history from the database.

[1697] Step 2:

[1698] The server uses generative AI to analyze the user's preferences based on listening history data.

[1699] Step 3:

[1700] Based on the analysis results, the server generates appropriate live events and artists and sends them to the device.

[1701] Step 4:

[1702] Users review recommended content.

[1703] User interaction function

[1704] Sending and receiving chat messages

[1705] Step 1:

[1706] A user types a chat message while watching a live event.

[1707] Step 2:

[1708] The terminal transmits the entered chat message to the server.

[1709] Step 3:

[1710] The server stores the received messages in a database.

[1711] Step 4:

[1712] The server relays the incoming message to other users attending the same live event.

[1713] Step 5:

[1714] Other users receive and respond to messages in real time.

[1715] Example 1

[1716] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1717] Today's users want to experience immersive live events from the comfort of their own homes. They also want to find content tailored to their preferences and interact with other users in real time. Conventional systems have struggled to meet these needs, limiting the user experience. This invention aims to provide a system that allows users to transcend physical constraints, providing a more personalized virtual reality experience and enabling real-time interaction with other users.

[1718] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1719] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time, means for generating and distributing a streaming address corresponding to content selected by the user, means for analyzing the user's past usage history using a generative AI model and making individually customized recommendations based on the user's preferences, means for providing users with an experience using virtual reality or augmented reality, and means for relaying real-time message exchanges between users and distributing them to other users. This allows users to experience an immersive live event from the comfort of their own home and enjoy individually customized content. At the same time, they can interact with other users in real time.

[1720] "User" means any individual or entity that uses the System.

[1721] "Input Information" means data or information that a user provides to a system, including, for example, a username, password, or email address.

[1722] "User Account" means individual identification information registered by a User in order to use the System.

[1723] "Authentication Information" means data used to identify and authenticate a user to access a system, including, by way of example, a hashed password.

[1724] "Streaming Address" means the URL or link used to stream a live event or content.

[1725] "Usage history" is a record of past operations and selections made by a user within the system.

[1726] A "generative AI model" is an algorithm or program that uses artificial intelligence techniques to analyze user data and generate specific results or recommendations.

[1727] "Virtual reality" is a three-dimensional visual and sensory experience generated using computer technology.

[1728] "Augmented reality" is a technology that overlays computer-generated information onto real-world visual information.

[1729] "Real-time message exchange" is a function that allows users to send and receive messages instantly.

[1730] "Relaying" is the process of receiving data or information and then forwarding it to other users or devices.

[1731] This invention is a system that allows users to experience immersive live events from the comfort of their own home using VR and AR technologies. Furthermore, the system uses a generative AI model based on the user's past usage history to provide customized content recommendations and allows users to interact with other users in real time. A specific implementation of this system is described below.

[1732] User registration and login function

[1733] User Registration

[1734] When a user creates an account, they enter a username, password, and email address. This information is sent from the device to the server. The server then encrypts the received password using a hashing function (e.g., SHA-256) and stores it in a database. Once registration is complete, the server generates a user ID and returns a registration completion message to the user.

[1735] User Login

[1736] When a user logs in, they enter their username and password. This information is sent from the terminal to the server. The server retrieves the authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal, allowing the user to access the system.

[1737] Virtual live viewing function

[1738] Select a Live Event

[1739] Users select the event they want to attend from a list of live events provided within the app, and the device sends the selected event's ID to the server.

[1740] Start Streaming

[1741] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1742] Content customization function

[1743] Acquiring and analyzing user history

[1744] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[1745] Specific prompt examples:

[1746] "Based on my recent listening history, what live event would you recommend next?"

[1747] "Show me a list of live events worth attending"

[1748] User interaction function

[1749] Sending and receiving chat messages

[1750] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1751] The system allows users to enjoy live events beyond physical constraints, interact with other attendees in real time, and discover content that suits their tastes.

[1752] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1753] Step 1: User Registration

[1754] The user enters a username, password, and email address on the account creation screen. This input data is sent from the device to the server. The server receives the received data and encrypts the password using a hashing function (e.g., SHA-256). The encrypted password and other user information are stored in a database. The server then generates a new user ID and returns it to the device along with a message confirming the user's registration. This completes the creation of the user account.

[1755] Specific behavior:

[1756] 1. The user enters information into the form on the browser and presses the "Register" button.

[1757] 2. The terminal sends the input data in JSON format to the server.

[1758] 3. The server receives the data, encrypts the password, and stores it in the database.

[1759] 4. The server generates a new user ID and returns a registration completion message to the terminal.

[1760] Step 2: User Login

[1761] The user enters their username and password on the login screen. This input data is sent from the device to the server. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, allowing the user to access the system.

[1762] Specific behavior:

[1763] 1. The user enters their username and password on the login screen and presses the "Login" button.

[1764] 2. The terminal sends the input data in JSON format to the server.

[1765] 3. The server retrieves the authentication information from the database and verifies the password.

[1766] 4. If the server is successful in authentication, it generates a session token and returns it to the device.

[1767] Step 3: Select a live event

[1768] Users select the event they want to attend from a list of live events provided within the app. The selected event ID is sent from the device to the server. The server receives the event ID and generates the corresponding streaming URL.

[1769] Specific behavior:

[1770] 1. A user browses the list of live events on the app and selects the event they want to attend.

[1771] 2. The device sends the selected event ID to the server.

[1772] 3. The server generates a streaming URL based on the received event ID.

[1773] Step 4: Start Streaming

[1774] The server sends the generated streaming URL to the device, which then uses the URL to set up a video stream on the VR or AR device and provide the user with live video.

[1775] Specific behavior:

[1776] 1. The server sends the streaming URL to the device.

[1777] 2. Using the URL received by the terminal, set up a stream on the VR device using WebRTC or HLS technology.

[1778] 3. Users watch the live footage through a VR device.

[1779] Step 5: Capture and analyze user history

[1780] When a user logs in, the server retrieves their past usage history from the database. Based on this historical data, a generative AI model analyzes the user's preferences and recommends customized live events and performers.

[1781] Specific behavior:

[1782] 1. The server retrieves the history from the database using the user ID.

[1783] 2. The server inputs historical data into the generated AI model and analyzes preferences.

[1784] 3. The server recommends customized content to the user based on the analysis results.

[1785] Step 6: Sending and Receiving Chat Messages

[1786] When a user enters a chat message and presses the send button while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1787] Specific behavior:

[1788] 1. The user types a message in the chat window and presses the "Send" button.

[1789] 2. The device sends a message to the server using WebSocket or an HTTP POST request.

[1790] 3. The server stores the message in a database and relays it to other participants in real time.

[1791] (Application example 1)

[1792] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1793] Conventional live event viewing systems require users to physically travel to the event venue, consuming a lot of time and money and lacking in convenience. Furthermore, there are limited ways for users to interact with other users in real time, which lacks the immersive and intimate feeling of a live event. Furthermore, the lack of customized event recommendations based on past listening history makes it difficult for users to find events that suit their preferences.

[1794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1795] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history, means for relaying real-time chats between users and distributing them to other users, and viewing means for the user to watch the live event on a device using virtual reality or augmented reality.This allows users to experience an immersive live event from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[1796] "User input information" refers to information provided by a user when creating an account or logging in, including a username, password, email address, etc.

[1797] "User Account" means an account created by a User to use the System, which includes the User's personal information and authentication information.

[1798] "Authentication Information" means information used to verify a user's access privileges, typically including a username and password.

[1799] A "streaming URL" is a link that allows you to watch live events or video content in real time over the Internet.

[1800] "Listening History" means a record of live events or audio content that a User has previously viewed or listened to.

[1801] A "live event" is a performance, show, or other event that takes place in real time.

[1802] "Performer" means an artist or performer who performs at a live event.

[1803] "Recommendation methods" are methods for presenting appropriate live events and performers based on a user's past listening history and preferences.

[1804] "Real-time chat" means a feature that allows users to instantly exchange messages with other users during a live event.

[1805] A "relaying means" is a method for delivering a chat message received from one user to another user.

[1806] "Virtual reality (VR)" is a technology that uses computer graphics technology to allow users to experience a virtual environment that is different from reality.

[1807] "Augmented reality (AR)" is a technology that overlays digital information onto a real-world environment, allowing users to visually experience additional information.

[1808] A "viewing instrument" is a method by which a user views a live event using a virtual reality or augmented reality device.

[1809] A "generative AI model" is an artificial intelligence model that learns user preferences and behaviors based on large amounts of data and makes appropriate recommendations and personalization.

[1810] A "prompt" is text data that is input into a generative AI model and serves as a trigger for generating a specific result or response.

[1811] "Device" refers to hardware equipment such as a computer, smartphone, or VR headset.

[1812] The system for realizing the present invention has the following main functions.

[1813] 1. User Account Creation and Authentication

[1814] The server receives the user's input and creates a user account. The user's credentials are saved and used to authenticate the user the next time they log in.

[1815] For example, when a user enters their information (username, password, email address) on the account creation screen, the server receives that information, encrypts the password (using a hashing function) and stores it in the database. Once registration is complete, a user ID is generated and a registration completion message is returned to the user.

[1816] 2. Watching Live Events

[1817] The server generates a streaming URL corresponding to the user's selected event and delivers it to the user, who can then watch the live event using their virtual reality (VR) or augmented reality (AR) device.

[1818] For example, when a user selects an event they want to attend from a list of live events provided within the app, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device then uses the URL to set up a video stream on the VR or AR device and provide live video.

[1819] 3. Customized content recommendations to users

[1820] The server analyzes the user's past listening history and uses a generative AI model to make personalized recommendations based on the user's preferences.

[1821] For example, when a user logs in, the server retrieves their past listening history from the database, and the generative AI model analyzes the user's preferences based on this historical data, suggesting live events and artists to watch next.

[1822] 4. Real-time chat between users

[1823] The server provides users with the ability to exchange chat messages with other users in real time while watching a live event, which promotes a sense of community among users.

[1824] For example, when a user types a chat message and hits the send button, the device sends the message to a server, which stores the message in a database and relays it in real time to other users participating in the same live event.

[1825] Hardware and software used

[1826] Hardware: VR device (e.g., Oculus Quest 2), AR device (e.g., Microsoft HoloLens), smartphone, PC

[1827] Software: Server applications (e.g., Flask framework), databases (e.g., MySQL), generative AI models

[1828] Prompt Sentence Examples

[1829] "Recommend the best live event based on the user's listening history. The data to recommend the next event to watch is as follows: - User ID: 1234 - Listening history: ['Artist A', 'Artist B', 'Event C']"

[1830] As a result, the system allows users to experience immersive live events from the comfort of their own home, interact with other users in real time, and easily find content that suits their preferences.

[1831] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1832] Step 1:

[1833] The user enters their information (username, password, email address) on the account creation screen and sends the information to the server via the device. The server creates a user account based on the received information. Specifically, the server encrypts the entered password (using a hashing function) and stores the encrypted information in a database. Finally, the server generates a user ID and returns a registration completion message to the device.

[1834] Input: Username, Password, Email Address

[1835] Data processing: password hashing

[1836] Output: User ID, registration completion message

[1837] Step 2:

[1838] The user enters their username and password on the login screen and sends that information to the server via their device. The server retrieves the stored authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the device, authenticating the user.

[1839] Input: Username, Password

[1840] Data processing: password hashing, authentication information matching

[1841] Output: Session token

[1842] Step 3:

[1843] The user selects the event they want to attend from a list of live events provided within the app. The device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and provide live video.

[1844] Input: Event ID

[1845] Data processing: Streaming URL generation

[1846] Output: Streaming URL

[1847] Step 4:

[1848] When a user logs in, the server retrieves the user's past listening history from a database. The server then analyzes the retrieved listening history using a generative AI model to recommend live events and performers that are individually tailored to the user's preferences. The recommendations are then sent to the device.

[1849] Input: User ID

[1850] Data processing: Acquisition and analysis of listening history, and recommendations based on generative AI models

[1851] Output: Recommended live events and performers

[1852] Step 5:

[1853] While watching a live event, a user types a chat message and presses the send button. The device sends the message to the server. The server stores the received message in a database and relays it in real time to other users participating in the same live event. The chat message is then distributed to other devices in real time.

[1854] Input: Chat message

[1855] Data processing: message storage and relay

[1856] Output: Chat messages delivered to other users' devices

[1857] These steps allow users to enjoy immersive live events from the comfort of their own home, interact with other users in real time, and receive personalized live event recommendations based on their past listening history.

[1858] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1859] This invention combines a system that allows users to experience immersive live events from the comfort of their own homes using virtual reality (VR) and augmented reality (AR) technologies with an emotion engine that recognizes users' emotions. Furthermore, it has the ability to recommend live events and artists that are individually customized based on the user's past listening history and emotions, and to promote real-time interaction between users.

[1860] User registration and login function

[1861] User Registration

[1862] The user enters a username, password, and email address on the account creation screen and sends that information to the server via the device. The server then encrypts the received password (using a hashing function) and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[1863] User Login

[1864] When accessing the system again, the user enters their username and password on the login screen and sends that information to the server via their terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[1865] Virtual live viewing function

[1866] Select a Live Event

[1867] Users select the event they want to attend from the list of live events in the app, and the device sends the selected event ID to the server.

[1868] Start Streaming

[1869] The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and starts providing live video.

[1870] Content customization function

[1871] Acquiring and analyzing user history

[1872] When a user logs in, the server retrieves their listening history from the database. Based on this history data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their tastes.

[1873] Emotion Engine Functions

[1874] Emotion Recognition and Analysis

[1875] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions.

[1876] User interaction function

[1877] Sending and receiving chat messages

[1878] Users can enter chat messages while watching a live event. The device sends the messages to a server, which then stores them in a database and relays them in real time to other users participating in the same live event. The emotion engine can also analyze users' emotions and automatically adjust or suggest chat messages.

[1879] Specific examples

[1880] User registration and login

[1881] 1. The user accesses the system and enters information on the account creation screen.

[1882] 2. The device sends the input information to the server, which stores the information in a database.

[1883] 3. The user enters their credentials on the login screen and the server authenticates them.

[1884] Watching live events

[1885] 1. The user selects the live event they want to attend, and the device sends the selection information to the server.

[1886] 2. The server sends the streaming URL to the device, and the device sets up the video stream.

[1887] Utilizing the Emotion Engine

[1888] 1. While the user is watching live video, the device collects emotional data through the camera and microphone.

[1889] 2. The server analyzes the data using an emotion engine and provides optimal live events and interactions based on the user's emotions.

[1890] User interaction

[1891] 1. The user types a chat message while watching live, and the device sends the message to the server.

[1892] 2. The server receives the message and relays it to other users attending the same event.

[1893] The system allows users to transcend physical constraints and enjoy high-quality live events, while leveraging an emotion engine for a more personalized experience and the ability to interact with other users in real time.

[1894] The processing flow will be explained below.

[1895] User registration and login function

[1896] User Registration

[1897] Step 1:

[1898] The user enters their username, password, and email address on the account creation screen.

[1899] Step 2:

[1900] The terminal transmits the input information to the server.

[1901] Step 3:

[1902] The server encrypts the received password and stores it in a database along with the username and email address.

[1903] Step 4:

[1904] The server generates a new user ID and returns it to the terminal along with a registration completion message.

[1905] User Login

[1906] Step 1:

[1907] The user enters their username and password on the login screen.

[1908] Step 2:

[1909] The terminal transmits the entered authentication information to the server.

[1910] Step 3:

[1911] The server retrieves the corresponding user's credentials from the database and compares them with the hashed value of the entered password.

[1912] Step 4:

[1913] If the server is successful in the authentication, it generates a session token and returns it to the terminal.

[1914] Step 5:

[1915] The user accesses the system using a session token.

[1916] Virtual live viewing function

[1917] Select a Live Event

[1918] Step 1:

[1919] Users select the event they want to attend from the list of live events within the app.

[1920] Step 2:

[1921] The terminal sends the selected event ID to the server.

[1922] Start Streaming

[1923] Step 1:

[1924] The server generates a streaming URL corresponding to the event ID and sends it to the device.

[1925] Step 2:

[1926] The terminal uses the received URL to set up a video stream on the VR or AR device.

[1927] Step 3:

[1928] The device will start providing live video.

[1929] Content customization function

[1930] Acquiring and analyzing user history

[1931] Step 1:

[1932] When a user logs in, the server retrieves past listening history from the database.

[1933] Step 2:

[1934] The server provides listening history data to the generation AI, which analyzes the user's preferences.

[1935] Step 3:

[1936] Based on the analysis results, the server generates individually customized live events and performers and sends them to the device.

[1937] Step 4:

[1938] The user reviews and watches the recommended content.

[1939] Emotion Engine Functions

[1940] Emotion Recognition and Analysis

[1941] Step 1:

[1942] The device uses a camera and microphone to capture the facial expressions and voice of the user watching the live video.

[1943] Step 2:

[1944] The data collected by the device is sent to the server in real time.

[1945] Step 3:

[1946] The server analyzes the data using an emotion engine.

[1947] Step 4:

[1948] The server sends information to the device to adjust live events and interactions based on the analyzed user emotions.

[1949] Step 5:

[1950] The device presents appropriate interactions and content to the user.

[1951] User interaction function

[1952] Sending and receiving chat messages

[1953] Step 1:

[1954] A user types a chat message while watching a live event.

[1955] Step 2:

[1956] The terminal transmits the entered chat message to the server.

[1957] Step 3:

[1958] The server stores the received messages in a database.

[1959] Step 4:

[1960] The server relays messages in real time to other users participating in the same live event.

[1961] Step 5:

[1962] The server uses an emotion engine to analyze the content of the message and make adjustments or suggestions as needed.

[1963] Step 6:

[1964] Other users receive and respond to messages in real time.

[1965] Example 2

[1966] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1967] In today's digital entertainment market, users are looking for ways to enjoy immersive live events beyond physical limitations. Furthermore, there is a need to customize content based on each user's preferences and emotions, providing greater satisfaction. Furthermore, there is a demand for the ability to interact with other users in real time during live events. However, current systems struggle to integrate and effectively realize these elements.

[1968] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for recommending specific live events and performers based on the user's past listening history and emotional data, means for recognizing and analyzing the user's emotions in real time and optimizing live events and interactions based on the results, and means for relaying real-time chats between users and distributing them to other users. This allows users to enjoy immersive live events beyond physical constraints, and further enables individually customized experiences and real-time user interaction.

[1969] "User" refers to an individual who uses this system.

[1970] "Input Information" means information provided by a user to the system, including, for example, a username, password, or email address.

[1971] "User Account" means an individual record in the System created based on a User's personal information and authentication information.

[1972] "Authentication Information" means information used to verify a User's identity, including a username and password.

[1973] "Login" refers to the process by which a user accesses a system using authentication information.

[1974] A "streaming URL" is a link that is generated by the server and sent to the user's device, indicating the destination of the live video stream.

[1975] "Listening history" is a record of music and live events that a user has listened to in the past.

[1976] "Emotional data" refers to information about emotions captured from a user's facial expressions, voice, etc.

[1977] "Recommendation" is the act of suggesting suitable content or services based on a user's listening history and emotional data.

[1978] "Real-time" means that events and interactions occur immediately, without delay.

[1979] "Chat" is a function that allows users to exchange messages and information in real time.

[1980] The "emotion engine" is a function within the system that analyzes emotional data in real time and provides optimal content and interactions based on the results.

[1981] "Virtual reality (VR)" is a technology that allows users to experience a three-dimensional space similar to reality, generated using computer technology.

[1982] "Augmented reality (AR)" is a technology that overlays digital information onto the real physical environment.

[1983] An "interaction" is an action or process in which a user interacts with a system or with another user.

[1984] This invention relates to a system that allows users to enjoy immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technologies. This system further enhances the user experience by incorporating an emotion engine that recognizes the user's emotions. Specific embodiments for implementing the system and their processing are described below.

[1985] First, the user accesses the account creation screen using a device and enters information such as a username, password, and email address. The device then sends this information to the server. The server then encrypts the received password (using a hashing function) and stores it in a database along with other information. The server then generates a new user ID and returns a registration completion message to the device.

[1986] Next, when the user accesses the system again, they enter their username and password on the login screen. The terminal sends the input information to the server, which retrieves the corresponding user's authentication information from the database. The server compares the hash value of the input password with the value stored in the database, and if authentication is successful, it generates a session token and returns it to the terminal. The terminal stores this session token, allowing the user to access the system.

[1987] When a user selects an event they want to attend from a list of live events, the device sends the selected event's ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begins providing live video. In some cases, a CDN (Content Delivery Network) service is used for streaming.

[1988] When a user logs in, the server retrieves their listening history from the database and analyzes their preferences using an emotion engine. Based on the results, the generative AI recommends individually customized live events and performers, allowing users to easily find content that matches their preferences.

[1989] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone and sends the data to a server. The server then uses an emotion engine to analyze the user's emotional data and provides optimal live events and interactions based on the results.

[1990] Furthermore, when a user types a chat message while watching a live event, the device sends the message to the server, which stores the message in a database and relays it in real time to other users participating in the same live event. The emotion engine analyzes the message and can make automatic adjustments or suggestions.

[1991] For example, if a user types "I want to watch a live event of my favorite artist," the system will recommend the most suitable live event based on the user's past listening history and current emotional state. Also, if a user types "I want to share my impressions with friends in real time," the system will utilize the chat function to encourage interaction with other users.

[1992] In this way, the system provides users with an optimized live event experience, utilizing virtual and augmented reality to enhance the sense of presence, while also enabling real-time interaction with other users.

[1993] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1994] User registration process steps

[1995] Step 1:

[1996] The user accesses the account creation screen using their device and enters their username, password, and email address.

[1997] Input: Username, Password, Email Address

[1998] Output: User information ready to send

[1999] Step 2:

[2000] The terminal transmits the entered user information to the server.

[2001] Input: User information (username, password, email address)

[2002] Output: Request to server

[2003] Step 3:

[2004] The server applies a hashing function to the received password to generate an encrypted password.

[2005] Enter: Password

[2006] Output: Encrypted password

[2007] Step 4:

[2008] The server stores the username, email address, and hashed password in a database.

[2009] Input: Username, Email Address, and Hashed Password

[2010] Output: Saved to database

[2011] Step 5:

[2012] The server generates a new user ID and returns a registration completion message to the terminal, which displays this message to the user.

[2013] Input: Registered user information

[2014] Output: New user ID, registration completion message

[2015] User login process steps

[2016] Step 1:

[2017] The user accesses the login screen using the device and enters their username and password.

[2018] Input: Username, Password

[2019] Output: Login information ready to be sent

[2020] Step 2:

[2021] The terminal transmits the entered authentication information to the server.

[2022] Input: Username, Password

[2023] Output: Send authentication information to the server

[2024] Step 3:

[2025] The server retrieves the corresponding user's authentication information (hashed password) from the database.

[2026] Input: Username

[2027] Output: Retrieving authentication information from the database

[2028] Step 4:

[2029] The server hashes the entered password and compares it with the hash value retrieved from the database. If authentication is successful, it generates a session token, otherwise it generates an error message.

[2030] Input: Password, hashed password for database

[2031] Output: Session token or error message

[2032] Step 5:

[2033] If authentication is successful, the server generates a session token and sends it to the device, which retains it and allows the user to access the system.

[2034] Input: Session token

[2035] Output: User approved to access the system

[2036] Processing steps for watching virtual live shows

[2037] Step 1:

[2038] Users select the event they want to attend from the list of live events within the app via their device.

[2039] Input: Live Event Selection

[2040] Output: Get the event ID

[2041] Step 2:

[2042] The device sends the ID of the selected event to the server.

[2043] Input: Event ID

[2044] Output: Send event ID to server

[2045] Step 3:

[2046] The server generates a streaming URL based on the event ID. This streaming may use a CDN (Content Delivery Network) service.

[2047] Input: Event ID

[2048] Output: Streaming URL

[2049] Step 4:

[2050] The server generates a streaming URL and sends it to the device.

[2051] Input: Streaming URL

[2052] Output: Send streaming URL to device

[2053] Step 5:

[2054] The device uses the received URL to set up a video stream on the VR or AR device, and the user can then start watching the live video.

[2055] Input: Streaming URL

[2056] Output: Start of video stream

[2057] Emotion Engine Processing Steps

[2058] Step 1:

[2059] While the user is watching the live video, the device's camera and microphone capture the user's facial expressions and voice.

[2060] Input: User's facial expressions, voice

[2061] Output: Obtaining emotion data

[2062] Step 2:

[2063] The device transmits the captured emotion data to the server.

[2064] Input: Emotion data

[2065] Output: Send emotion data to the server

[2066] Step 3:

[2067] The server analyzes the emotional data using an emotion engine, which uses technologies such as facial expression recognition and voice tone analysis.

[2068] Input: Emotion data

[2069] Output: Emotion analysis results

[2070] Step 4:

[2071] The server uses the analysis results to provide optimal live events and interactions tailored to the user's emotions. For example, if a specific emotion is strong, a request can be sent to an artist in real time.

[2072] Input: Sentiment analysis results

[2073] Output: Optimized live events and interactions

[2074] Steps in processing user interactions

[2075] Step 1:

[2076] A user types a chat message while watching a live event.

[2077] Input: Chat message

[2078] Output: Message prepared for sending

[2079] Step 2:

[2080] The terminal transmits the entered chat message to the server.

[2081] Input: Chat message

[2082] Output: Send message to server

[2083] Step 3:

[2084] The server stores the received messages in a database.

[2085] Input: Chat message

[2086] Output: Message saved

[2087] Step 4:

[2088] The server relays messages in real time to other users attending the same live event, and an emotion engine analyzes users' emotions and can automatically adjust or suggest chat messages as needed.

[2089] Input: Chat message

[2090] Output: Message relay to other users, auto-adjustment suggestions

[2091] The above are the specific processing steps of the program of this system.

[2092] (Application example 2)

[2093] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2094] In conventional online live event viewing systems, users were limited to simply watching live footage, and were unable to enjoy real-time interaction or a personalized experience. Furthermore, they lacked the means to recognize users' emotions and enhance the sense of presence or provide recommended content based on those emotions. As a result, the entertainment value and user experience were limited.

[2095] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2096] In this invention, the server includes means for receiving user input information and creating a user account, means for saving the user's authentication information and authenticating the user the next time the user logs in, means for generating and distributing a streaming URL corresponding to an event selected by the user, means for analyzing the user's past viewing history and emotional data and recommending individually customized live events and performers, means for relaying real-time messages between users and distributing them to other users, and means for recognizing user emotions and dynamically adjusting the experience content based on the recognition results. This allows users to experience a more personalized live event and interact with other users in real time, as well as enjoy an emotionally-driven, interactive, and immersive experience.

[2097] "Input Information" means data provided by a user to the system for purposes such as account creation and event selection.

[2098] "Authentication information" refers to data such as username and password used when a user logs in, and is used to accurately identify the user and verify access rights.

[2099] "Streaming URL" means a link on the Internet that streams video of a selected live event to users in real time.

[2100] "Viewing history" is data about content a user has viewed in the past, and personalized recommendations are based on this data.

[2101] "Recommendation" means selecting and presenting appropriate content and performers based on the user's tastes, preferences, and emotions.

[2102] "Real-time Messages" are text messages that allow Users to instantly communicate with other Users while watching a live event.

[2103] "Emotional data" refers to emotional information recognized from a user's facial expressions and voice, and is used to make the user's experience more personalized and relevant.

[2104] "Virtual reality" is a technology that uses specialized devices to allow users to experience a computer-generated environment that differs from reality.

[2105] "Augmented reality" is a technology that overlays digitally generated elements onto the real world, providing users with a blended experience of reality and virtuality.

[2106] "Dynamic adjustment" is the process of optimizing the experience in real time based on the user's emotional data and other variables.

[2107] This invention combines a system that allows users to experience immersive live events from the comfort of their own home using virtual reality (VR) and augmented reality (AR) technology with an emotion engine that recognizes the user's emotions. The system also has the function of recommending individually customized live events and performers based on the user's past viewing history and emotions, and promoting real-time interaction between users. The following describes in detail the modes for implementing this invention.

[2108] User registration and login function

[2109] User Registration

[2110] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen and sends this information to the server via their device. The server encrypts the received password and stores it in a database along with the username and email address. Once registration is complete, a new user ID is generated and a registration completion message is returned to the user.

[2111] User Login

[2112] When the user accesses the system again, the server receives the user name and password on the login screen and sends the information to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, it generates a session token and returns it to the terminal, allowing the user to access the system.

[2113] Virtual live viewing function

[2114] Select a Live Event

[2115] When a user selects an event they want to attend from a list of live events within the app, the server receives the selection information. The device sends the selected event ID to the server, and the server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[2116] Content customization function

[2117] Acquiring and analyzing user history

[2118] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers, allowing users to easily find content that suits their preferences.

[2119] Emotion Engine Functions

[2120] Emotion Recognition and Analysis

[2121] When a user watches live video, the device captures the user's facial expressions and voice through a camera and microphone. This data is sent to a server where an emotion engine analyzes it. Based on the analysis results, the device provides live events and interactions optimized for the user's emotions. Emotional data recognized from the user's facial expressions and voice is used to make the user's experience more personalized and relevant.

[2122] User interaction function

[2123] Sending and receiving chat messages

[2124] When a user types a chat message while watching a live event, the server can receive the message information and relay it in real time to other users participating in the same live event.The emotion engine can also analyze the user's emotions and automatically adjust or suggest chat messages.

[2125] Hardware and software used

[2126] The following hardware and software are used to implement this system:

[2127] Hardware: PC, smartphone, tablet, VR headset, AR device, camera, microphone

[2128] Software: Flask web framework, Python, database (SQLite or PostgreSQL), OpenCV, a provisional emotion recognition library (EmotionRecognizer), and a recommendation system (Recommender).

[2129] Specific examples

[2130] 1. When a user enters and submits information on the account creation screen, the server stores the encrypted information in a database.

[2131] 2. The user selects the desired live event, and the server generates and returns a streaming URL.

[2132] 3. The emotion engine recognizes emotions based on the user's facial expressions and voice data, and provides recommendations to the user based on those emotions.

[2133] Prompt Sentence Examples

[2134] "Use an emotion engine to analyze users' facial expressions and recommend appropriate live events based on their past viewing history."

[2135] "Share chat messages typed by users while watching a live event with other users in real time."

[2136] In this way, the present invention is a system that utilizes the user's emotions and past viewing history to provide a more personalized, interactive, and immersive live experience.

[2137] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2138] Step 1:

[2139] The server receives the user's input information and creates a user account. Specifically, the user enters a username, password, and email address on the account creation screen. The device sends this input information to the server. The server uses a hashing function to encrypt the password and stores it in a database along with the username and email address. As output, it generates a new user ID and returns a registration completion message to the user.

[2140] Step 2:

[2141] The server saves the user's authentication information and authenticates the user the next time they log in. When accessing the system again, the user enters their username and password on the login screen and sends them to the server via the terminal. The server retrieves the corresponding user's authentication information from the database and compares it with the hash value of the entered password. If authentication is successful, the server generates a session token and returns it to the terminal. The user can then use this session token to access the system.

[2142] Step 3:

[2143] When a user selects an event they want to attend from the list of live events in the app, the server receives the selection information. The device sends the selected event ID to the server. The server generates a streaming URL corresponding to the event ID and sends it to the device. The device uses the received URL to set up a video stream on the VR or AR device and begin providing live video.

[2144] Step 4:

[2145] When a user logs in, the server retrieves their past viewing history from the database. Based on this historical data and the analysis results of the emotion engine, the generative AI analyzes the user's preferences and recommends individually customized live events and performers. As an output, users can easily find content that suits their preferences.

[2146] Step 5:

[2147] When a user watches live video, the device captures the user's facial expressions and voice using a camera and microphone. This data is then sent to a server where an emotion engine analyzes it. Based on the analysis results, the server provides live events and interactions optimized for the user's emotions. Specifically, content recommendations and interaction adjustments are made based on emotion recognition results.

[2148] Step 6:

[2149] When a user enters a chat message while watching a live event, the server receives the message information. The device then sends the entered chat message to the server, which then relays the message to other users participating in the same live event in real time. As an output, other users receive the chat message in real time, allowing them to interact with each other.

[2150] In this way, each step works together to allow users to enjoy an immersive live event, with a more personalized experience and real-time interaction with other users.

[2151] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2152] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2153] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2154] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2155] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2156] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2157] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2158] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2159] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2160] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2161] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2162] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2163] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2164] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2165] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2166] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2167] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2168] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2169] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2170] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2171] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2172] The following is further disclosed regarding the above embodiment.

[2173] (Claim 1)

[2174] means for receiving user input and creating a user account;

[2175] A means to store the user's credentials and authenticate them the next time they log in, and

[2176] A means for generating and delivering a streaming URL corresponding to the user-selected event;

[2177] A means to recommend specific live events and performers based on a user's past listening history;

[2178] A means of relaying real-time chats between users and broadcasting them to other users;

[2179] A system including:

[2180] (Claim 2)

[2181] 10. The system of claim 1, further comprising means for analyzing a user's past listening history and providing personalized recommendations based on the user's preferences.

[2182] (Claim 3)

[2183] 10. The system of claim 1, further comprising means for allowing a user to experience a live event using virtual reality or augmented reality.

[2184] "Example 1"

[2185] (Claim 1)

[2186] means for receiving user input and creating a user account;

[2187] A means to store the user's authentication information and authenticate the user the next time they authenticate, and

[2188] means for generating and delivering a streaming address corresponding to the content selected by the user;

[2189] A means to recommend specific virtual reality experiences and providers based on a user's past usage history; and

[2190] a means for relaying real-time message exchanges between users and distributing them to other users;

[2191] A system including:

[2192] (Claim 2)

[2193] The system of claim 1, further comprising means for analyzing a user's past usage history using a generative AI model to provide individually customized recommendations based on the user's preferences.

[2194] (Claim 3)

[2195] 10. The system of claim 1, further comprising means for providing a user with an experience of reality using virtual reality or augmented reality.

[2196] "Application Example 1"

[2197] (Claim 1)

[2198] means for receiving user input and creating a user account;

[2199] A means to store the user's credentials and authenticate them the next time they log in, and

[2200] A means for generating and delivering a streaming URL corresponding to the user-selected event;

[2201] A means to recommend specific live events and performers based on a user's past listening history;

[2202] A means of relaying real-time chats between users and broadcasting them to other users;

[2203] a viewing means for a user to view the live event on a virtual reality or augmented reality device;

[2204] A system including:

[2205] (Claim 2)

[2206] 10. The system of claim 1, further comprising means for analyzing a user's past listening history and using a generative AI model to provide personalized recommendations based on the user's preferences.

[2207] (Claim 3)

[2208] 10. The system of claim 1, further comprising means for enabling a user to exchange chat messages with other users in real time while viewing the live event.

[2209] "Example 2: Combining Emotion Engines"

[2210] (Claim 1)

[2211] means for receiving user input and creating a user account;

[2212] A means to store the user's credentials and authenticate them the next time they log in, and

[2213] A means for generating and delivering a streaming URL corresponding to the user-selected event;

[2214] A means to recommend specific live events and performers based on a user's past listening history and emotional data;

[2215] A means to recognize and analyze user sentiment in real time and optimize live events and interactions based on the results;

[2216] A means of relaying real-time chats between users and broadcasting them to other users;

[2217] A system including:

[2218] (Claim 2)

[2219] 10. The system of claim 1, further comprising means for analyzing a user's past listening history and emotional data to provide personalized recommendations based on the user's preferences.

[2220] (Claim 3)

[2221] 10. The system of claim 1, further comprising means for allowing a user to experience a live event using virtual reality or augmented reality, and further comprising an emotion engine for providing interaction based on the user's emotions.

[2222] "Application example 2 when combining emotion engines"

[2223] (Claim 1)

[2224] means for receiving user input and creating a user account;

[2225] A means to store the user's credentials and authenticate them the next time they log in, and

[2226] A means for generating and delivering a streaming URL corresponding to the user-selected event;

[2227] A means to recommend specific live events and organizers based on a user's past viewing history, and

[2228] a means for relaying real-time messages between users and distributing them to other users;

[2229] A means of analyzing users' emotional data and recommending individually customized live events and performers based on that data;

[2230] A system including:

[2231] (Claim 2)

[2232] 10. The system of claim 1, further comprising means for analyzing a user's past viewing history and providing individually customized recommendations based on the user's preferences.

[2233] (Claim 3)

[2234] The system of claim 1 further comprises means for allowing a user to experience a live event using virtual reality or augmented reality, as well as means for recognizing the user's emotions and dynamically adjusting the experience content based on the results of the recognition. [Explanation of symbols]

[2235] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving user input and creating a user account; A means to store the user's credentials and authenticate them the next time they log in, and A means for generating and delivering a streaming URL corresponding to the user-selected event; A means to recommend specific live events and performers based on a user's past listening history; A means of relaying real-time chats between users and broadcasting them to other users; A system including:

2. 10. The system of claim 1, further comprising means for analyzing a user's past listening history and providing personalized recommendations based on the user's preferences.

3. 10. The system of claim 1, further comprising means for allowing a user to experience a live event using virtual reality or augmented reality.

Images