system

Abstract

Provide a system. The method includes: collecting posting data from a user's social networking service account; A means for analyzing the collected posting data to identify user characteristics and interests; means for generating characters and stories based on the identified characteristics and interests; means for transmitting the generated characters and stories to a user's terminal; A means of synchronizing each user's data so that multiple users can play simultaneously in real time; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] While today's young generation expresses themselves through social networking services, traditional games have faced the challenge of making it difficult to reflect individual personalities and the content of user posts in the game story. Furthermore, the gaming experience is uniform, and there are few ways to provide a personalized experience optimized for each user. Furthermore, there are insufficient environments for multiple users to cooperate in real time to progress through the game. Given these circumstances, there is a demand for a system that can automatically generate characters and stories based on users' social networking service posts, providing a unique RPG experience in which the user is the protagonist. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. First, a user links their social networking service account to the system. This linking allows data posted by the user to be collected. The collected data is analyzed using a generative AI model to identify the user's characteristics and interests. Based on the analysis results, a character and story dedicated to the user are generated. The generated character and story are sent to the user's device and displayed on the game interface, allowing the user to enjoy the game with their own avatar. In addition, by providing a function that allows multiple users to play simultaneously in real time and introducing a system that synchronizes each user's data, users can enjoy the game with other players. As a result, a personalized RPG experience that reflects the user's personality and interests is provided, and new communities can be formed.

[0006] "User" means an individual who uses a social networking service and plays RPG games through this system.

[0007] A "social networking service" is an online platform that enables users to share their information and content and interact with other users.

[0008] "Posted data" refers to various content such as text, images, and videos posted by users to social networking services.

[0009] "Means of collection" refers to the technical means by which the system obtains posting data from a user's social networking service account.

[0010] "Means for analysis" refers to the technical means for analyzing collected posting data and identifying the characteristics and interests of users.

[0011] "Characteristics" refer to the unique attributes and personality of a user identified from the analyzed posting data.

[0012] "Interests" refer to a user's preferences and areas of interest that are revealed from the analyzed posting data.

[0013] "Characters" are in-game avatars and characters that are generated based on the user's characteristics and interests.

[0014] "Story" is the story or scenario of an RPG game that reflects the user's characteristics and interests.

[0015] "Generating means" refers to the technical means for automatically creating characters and stories based on the characteristics and interests of the user.

[0016] "Transmission means" refers to the technical means for transmitting the generated characters and story to the user's terminal.

[0017] "Means for synchronization" refers to the technical means for collating game data of multiple users in real time and matching data between players.

[0018] A "terminal" is an electronic device such as a computer, smartphone, or tablet that a user uses to play a game.

[0019] A "generative AI model" is an artificial intelligence algorithm that analyzes user-submitted data and generates characters and stories.

[0020] A "game interface" refers to the screen and operating means through which a user operates a game and enjoys an experience via a terminal. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The system collects data posted by users on social networking service accounts, analyzes the data to identify the user's characteristics and interests, generates characters and stories based on the data, and finally transmits and displays them on the user's device. The system also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data.

[0043] Program processing

[0044] Data collection

[0045] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0046] Data analysis

[0047] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[0048] Character and story generation

[0049] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[0050] Sending data and starting the game

[0051] The generated character and story data is sent from the server to the user's device, which receives this data and displays the game interface, where the user can view their avatar and begin playing the game.

[0052] Real-time play function

[0053] As users progress through the game, all actions (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0054] Specific examples

[0055] Let's say a user links their Twitter account. If their daily tweets often contain topics like "cats" and "travel," the following characters and stories will be generated.

[0056] 1. The user connects their Twitter account and launches the application.

[0057] 2. The server collects past tweets via the Twitter API and analyzes the data using a generative AI model.

[0058] 3. As a result of the analysis, it is determined that this user is a "cat lover" and a "travel lover."

[0059] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role.

[0060] 5. The generated data is sent to the device and the game interface is displayed.

[0061] 6. Users experience a story in which their avatar goes on an adventure to save a cat.

[0062] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0063] This allows users to have a unique RPG experience that reflects their own characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The user downloads and installs the application.

[0067] Step 2:

[0068] The user launches the application, enters their information on the account registration screen, and creates an account.

[0069] Step 3:

[0070] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[0071] Step 4:

[0072] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[0073] Step 5:

[0074] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[0075] Step 6:

[0076] The server then supplies the saved post data to a generative AI model, which uses natural language processing (NLP) technology to analyze the sentiment of posts, extract frequently occurring keywords, and identify users' areas of interest.

[0077] Step 7:

[0078] A generative AI model identifies user characteristics and interests, such as "cat lover" or "travel lover" from posts.

[0079] Step 8:

[0080] The server generates characters and stories based on the analysis results. Using a generative AI model, it creates a character specific to the user (e.g., a cat-loving adventurer) and generates a story based on the user's interests (e.g., an adventure to save cats).

[0081] Step 9:

[0082] The server transmits the generated character and story data to the user's terminal.

[0083] Step 10:

[0084] The device displays the game interface based on the received character and story data, allowing the user to confirm their avatar and begin the game.

[0085] Step 11:

[0086] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[0087] Step 12:

[0088] The server synchronizes the data of multiple users in real time, so that the movements and progress of other players are also reflected on the user's device.

[0089] Step 13:

[0090] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[0091] Step 14:

[0092] When a user finishes a game session, the terminal transmits current game progress data to the server.

[0093] Step 15:

[0094] The server saves your game progress, allowing you to resume from the same point the next time you play.

[0095] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Today's users often demand personalized content based on their individual interests and characteristics. However, content generation using social networking service (SNS) data to date has faced challenges due to its inability to fully reflect user characteristics. Furthermore, there is no established method for providing an environment in which multiple users can play simultaneously in real time while synchronizing each user's operation data. Furthermore, there are still issues with SNS account authentication, secure data storage, and understanding daily patterns.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for authenticating the linking of social networking accounts, means for analyzing the acquired posted data using natural language processing technology, means for creating characters and stories using a generative AI model, and means for transmitting operation data from the user's device to the server and receiving operation data from other users at the device in real time. This enables the generation of personalized characters and stories that reflect users' characteristics, providing a synchronized gaming experience in real time. Furthermore, the authentication means enables secure social networking and ensures data security.

[0101] A "social networking service account" is identification information that allows a user to access a social networking service.

[0102] "Posted data" refers to information such as text, images, and videos that users make public on social networking services.

[0103] "Analysis" is the process of processing collected data using natural language processing techniques to identify user characteristics and interests.

[0104] "Features" refer to individual attributes such as user preferences and tendencies obtained from analyzed data.

[0105] "Interests" indicate topics or activities that a user is particularly interested in.

[0106] A "character" is a user's avatar or a representative of a player within a game.

[0107] A "story" is the narrative or scenario that the player progresses through within the game.

[0108] A "terminal" is a user's device (e.g., smartphone, tablet) for running an application.

[0109] "Data synchronization" is the process of making all users' operation data consistent in real time when multiple users are playing a game simultaneously.

[0110] "Authentication means" refers to a method for linking a user's social networking service account with an application.

[0111] A "generative AI model" is an artificial intelligence model that automatically generates characters and stories based on the user's characteristics and interests.

[0112] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0113] "Operation data" is information relating to operations and actions performed by the user within the game.

[0114] "Data Storage Means" means a method or system for securely storing collected data.

[0115] "Daily patterns" are trends in daily activities and behaviors that are analyzed from data posted by users.

[0116] "Personalized" means customized based on the characteristics and interests of each individual user.

[0117] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that, and ultimately sends and displays them on the user's terminal.

[0118] This system provides an environment in which multiple users can play games simultaneously in real time, and is characterized by synchronizing the data of each user. The operation of the system is carried out as follows.

[0119] First, the user downloads and launches the application. This displays an account registration screen, where the user enters basic information such as their name and email address and links their social networking service account (e.g., Twitter, Facebook). Next, once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0120] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[0121] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[0122] The generated character and story data is sent from the server to the user's device, which receives the data and displays the game interface, where the user can view their avatar and begin playing the game.

[0123] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. In this way, users can cooperate with each other to complete missions.

[0124] As a concrete example, let's assume that a user has linked their Twitter account. Suppose that the user's daily tweets often contain topics like "cats" and "travel." In this case, the following characters and stories will be generated.

[0125] 1. The user connects their Twitter account and launches the application.

[0126] 2. The server collects past tweet data through the Twitter API.

[0127] 3. The collected tweet data is analyzed using a generative AI model.

[0128] 4. The analysis results reveal that this user is a "cat lover" and a "travel lover."

[0129] 5. Based on the analysis, the server generates a story in which the cat plays an important role.

[0130] 6. The generated data is sent to the device and the game interface is displayed.

[0131] 7. Users experience a story in which their avatar goes on an adventure to save a cat.

[0132] 8. The server synchronizes all player data in real time, and everyone's actions are reflected in the game.

[0133] An example of a prompt you could give to a generative AI model would be:

[0134] "Identify the user's interests and characteristics from their posted data, and generate a unique character and story based on that data. For example, if the user loves cats and travels, generate a character with an adventure story about saving cats."

[0135] This allows users to have a unique RPG experience that reflects their characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[0136] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0137] Program processing flow

[0138] Step 1:

[0139] A user downloads an application and launches it on their device. As input, the user installs and launches the application. As output, an account registration screen is displayed.

[0140] Specifically, when the application is launched for the first time, the terminal generates a user interface and displays an account registration screen.

[0141] Step 2:

[0142] The user enters their account registration information and links their social media account. As input, the user enters their name, email address, and social media link information. As output, an authentication token is obtained once linking is complete.

[0143] Specifically, the terminal receives user input, displays an authentication screen for the social networking service, and transmits an authentication token to the server once authentication is complete.

[0144] Step 3:

[0145] The server uses the SNS API to collect user posted data. As input, it receives an authentication token. As output, the user posted data is collected and saved in a database.

[0146] Specifically, the server accesses the SNS API using the authentication token, obtains the user's past posting data, and stores the collected data in a database.

[0147] Step 4:

[0148] The server supplies the collected posting data to the generative AI model for analysis. Posting data obtained from the database is supplied as input to the generative AI model. User characteristics and interests are identified as output.

[0149] Specifically, the generative AI model analyzes the data using natural language processing (NLP) techniques to identify the user's characteristics (e.g., cat lover, travel lover) and interests. The server then stores the analysis results in a database.

[0150] Step 5:

[0151] The server generates characters and stories based on the analysis results. As input, it generates prompt sentences based on the analysis results and feeds them to the generative AI model. As output, personalized characters and stories are generated.

[0152] Specifically, the server generates a prompt, for example, "If the user loves cats and travels, please generate a character with an adventure story about saving cats," and inputs this into the generative AI model. The generative AI model generates the character and story and returns the generated data to the server, which stores it in a database.

[0153] Step 6:

[0154] The server sends the generated character and story data to the user's device. As input, it receives the generated character and story data. As output, it sends the data to the user's device and displays the game interface.

[0155] Specifically, the server sends the generated data to the user's device, which receives the data and displays the game interface, allowing the user to view their avatar and story.

[0156] Step 7:

[0157] When a user starts a game, action data is sent from the device to the server as the user progresses. As input, the user's action data (e.g., movement, combat, item collection) is acquired. As output, the action data is sent to the server.

[0158] Specifically, the terminal captures the user's operations and transmits the data to the server.

[0159] Step 8:

[0160] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other users on their devices. As input, it receives the operation data of all users. As output, it sends the synchronized data in real time to each user's device.

[0161] Specifically, the server processes all users' operation data in real time and sends the synchronized data to each user's device. The device then reflects the movements of other users, enabling real-time cooperative play.

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] In recent autonomous vehicles, there is a growing demand for entertainment systems that allow passengers to make effective use of their time while on board. However, existing systems lack the means to provide a personalized entertainment experience. Furthermore, when there are multiple passengers, there is a problem in that they lack the functionality to allow passengers to interact with each other in real time. Therefore, the present invention aims to provide a system that uses users' social networking service data to provide a personalized and interactive entertainment experience.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' terminals, means for synchronizing data for each user so that multiple users can play simultaneously in real time, and means for operating within the mobile communication terminal to provide users with an interactive entertainment experience. This allows users on board to enjoy a personalized entertainment experience based on their interests and characteristics, and if there are multiple users, they can interact with each other in real time.

[0167] Definitions of important words

[0168] "User's Social Networking Service Account" means the credentials used by a User to post and interact on a Social Media Platform.

[0169] "Posted data" refers to content such as text, images, and videos posted by users to social networking services, as well as metadata.

[0170] "Analysis" is a data processing method that uses natural language processing technology and machine learning algorithms to identify user characteristics and interests from collected posted data.

[0171] A "character" is a virtual entity that is generated exclusively for a user, reflects the characteristics of the user, and plays an active role within the story.

[0172] A "story" is entertainment content that includes a sequence of events or scenarios generated based on the user's interests.

[0173] "Terminal" refers to an electronic device that can be directly operated by a user, including a smartphone, tablet, computer, in-vehicle display, etc.

[0174] "Real-time" means that data and operations are processed instantly, without any delay.

[0175] "Synchronizing data" means matching data with each other in order to keep the operation data and progress status of multiple users in the same state.

[0176] "Mobile communication terminal" refers to an electronic device with communication functions intended for use in a vehicle, including displays and infotainment systems in autonomous vehicles.

[0177] An "interactive entertainment experience" is an experience in which users can interact with the content and influence the story or character behavior through actions and choices.

[0178] "Server" means a computer system running computational resources and software for collecting, analyzing, storing, and synchronizing data.

[0179] "Natural language processing technology" refers to the technology used by computers to understand, interpret, and generate human language, and uses machine learning algorithms and deep learning models.

[0180] A "generative AI model" is an artificial intelligence model that can generate new data from input data using machine learning algorithms.

[0181] "Multiple users on board" refers to multiple users inside an autonomous vehicle, each of whom can operate the system using their own terminal.

[0182] The OAuth 2.0 authentication system is a widely used protocol for authentication and authorization on the web, allowing third parties to access a user's resources.

[0183] A "real-time synchronization system" is a system that instantly processes changing data and shares the latest information with all related devices.

[0184] "Infotainment system" refers to a system that integrates information and entertainment within a vehicle, including displays, audio, and internet connectivity.

[0185] MODE FOR CARRYING OUT THE INVENTION

[0186] 1. System Program

[0187] The system that realizes this application example consists of the following program modules:

[0188] 1. User authentication module

[0189] 2. Data Collection Module

[0190] 3. Data Analysis Module

[0191] 4. Character Story Generation Module

[0192] 5. Data transmission and game interface module

[0193] 6. Real-time synchronization module

[0194] 2. Natural language explanation

[0195] User Authentication Module

[0196] When a user logs into the vehicle's entertainment system, an authentication module is started. The authentication module authenticates the user's social networking service account (e.g., Twitter, Facebook) using the OAuth 2.0 authentication system. This authentication allows the server to obtain an authentication token, which can then be used to collect the user's posted data.

[0197] Data Collection Module

[0198] The server uses the authentication token to retrieve user posted data through the SNS API. This includes content such as text, images, and videos. The collected data is securely stored in a database system. This module can use database systems such as Amazon RDS.

[0199] Data Analysis Module

[0200] The collected data is analyzed on the server using natural language processing (NLP) technology, particularly NLP models such as BERT and GPT-4 (registered trademark). These models extract user characteristics and interests from text and analyze user trends and patterns.

[0201] Character Story Generation Module

[0202] Based on the analysis results, a generative AI model (e.g., GPT-4) generates a character and story specifically for the user. During this process, character attributes and storylines that match the user's characteristics and interests are generated. The server generates this data and prepares it for later transmission to the device.

[0203] Data transmission and game interface module

[0204] The generated characters and story are sent from the server to the vehicle's entertainment system. The user operates the interactive game interface using the in-vehicle display or touch panel. This module is implemented as an ANDROID (registered trademark) / iOS application.

[0205] Real-time Synchronization Module

[0206] The server synchronizes the data of multiple users riding in the vehicle in real time. Using Firebase and WebSocket technology, all users' operation data is updated simultaneously, and the progress of other users is also reflected in real time. In this way, users can interact with each other.

[0207] 3. Examples and prompts

[0208] Specific examples

[0209] For example, if a user links their Facebook account and it is analyzed that they frequently post about "cats" and "music" while on board, a story such as "Adventuring in a musical land where cats live" will be generated for that user. A unique feature of this service is that it allows passengers to enjoy the adventure together.

[0210] Prompt Sentence Examples

[0211] Example of prompt input:

[0212] Analyze the user's posted data and generate a story based on their interest in "cats" and "music." The user is already logged into the vehicle's entertainment system.

[0213] This allows users to spend their travel time enjoying personalized entertainment in an autonomous vehicle.

[0214] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0215] Program processing steps

[0216] Step 1:

[0217] When a user logs into the vehicle's entertainment system, the terminal displays an authentication screen. The user logs in with a social networking service account (e.g., Twitter, Facebook). As input, the user's authentication information is required, and as output, an authentication token is obtained. The terminal sends this token to the server. The authentication system uses OAuth 2.0.

[0218] Step 2:

[0219] The server uses the obtained authentication token to collect user posted data through the SNS API. The authentication token is required as input, and the user posted data is obtained as output. The server securely stores this data in a database. Amazon RDS is used as the database system.

[0220] Step 3:

[0221] The server analyzes the collected posting data using natural language processing technology. Posting data is required as input, and user characteristics and interests are obtained as output. Specifically, NLP models such as BERT and GPT-4 are used to identify user interest patterns (e.g., "cat lover" or "music lover"). This allows the user's tendencies and patterns to be identified.

[0222] Step 4:

[0223] The server uses a generative AI model (e.g., GPT-4) based on the analysis results to generate a character and story specifically for the user. The analysis results are required as input, and the characters and story are generated as output. For example, a story based on the user's interests is created, such as "Adventure in a musical land where cats live."

[0224] Step 5:

[0225] The server sends the generated characters and stories to the terminal. The generated characters and stories are required as input, and the data is displayed on the terminal as output. The terminal provides the user with an interactive game interface, which is achieved using an Android / iOS application.

[0226] Step 6:

[0227] The terminal collects user operation data and sends it to the server. The user operation data is required as input and sent to the server as output. The server also synchronizes other users' operation data in real time.

[0228] Step 7:

[0229] The server synchronizes all user operation data in real time using Firebase and WebSocket technology. Each user's operation data is required as input, and the game state updated in real time is obtained as output. This allows multiple users to share an interactive entertainment experience simultaneously.

[0230] Through these steps, users can enjoy a personalized and interactive entertainment experience inside their autonomous vehicle.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[0233] Program processing

[0234] Data collection

[0235] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0236] Data analysis

[0237] The collected post data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) techniques to identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes user emotions (e.g., positive, negative, excited, calm, etc.) from the post.

[0238] Character and story generation

[0239] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[0240] Sending data and starting the game

[0241] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin playing the game. The music, background, and character dialogue are also dynamically adjusted according to the user's emotions.

[0242] Real-time play function

[0243] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0244] Specific examples

[0245] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[0246] 1. The user connects their Twitter account and launches the application.

[0247] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[0248] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[0249] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[0250] 5. The generated data is sent to the device and the game interface is displayed.

[0251] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[0252] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0253] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[0254] The processing flow will be explained below.

[0255] Step 1:

[0256] The user downloads and installs the application.

[0257] Step 2:

[0258] The user launches the application, enters their information on the account registration screen, and creates an account.

[0259] Step 3:

[0260] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[0261] Step 4:

[0262] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[0263] Step 5:

[0264] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[0265] Step 6:

[0266] The server provides the saved post data to the generative AI model and emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) technology to identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotion (e.g., positive, negative, excited, calm, etc.) from the post.

[0267] Step 7:

[0268] Based on the analysis results of the generative AI model and emotion engine, the server generates a character and story specifically for the user. For example, if a user posts many cat-related messages and the emotion engine recognizes positive emotions from those messages, the character will be given the attribute of "cat lover" and the story will include "a happy encounter with a cat."

[0269] Step 8:

[0270] The server transmits the generated character and story data to the user's terminal.

[0271] Step 9:

[0272] The device displays the game interface based on the character and story data received. The user can then confirm their avatar and begin the game. The in-game music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[0273] Step 10:

[0274] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[0275] Step 11:

[0276] The server synchronizes the data of multiple users in real time, so the movements and progress of other players are also reflected on the device.

[0277] Step 12:

[0278] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[0279] Step 13:

[0280] When a user finishes a game session, the terminal transmits current game progress data to the server.

[0281] Step 14:

[0282] The server saves your game progress, allowing you to resume from the same point the next time you play.

[0283] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data and emotions.

[0284] Example 2

[0285] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0286] Current game systems and entertainment services struggle to provide personalized experiences that fully reflect each user's individual characteristics, interests, and emotions. Furthermore, when multiple users play simultaneously in real time, smoothly integrating all users' data and enabling cooperative play is also problematic. Furthermore, it is currently difficult to effectively utilize the data posted by users on social networking services and dynamically reflect it in the characters and story of the game. Systems that can solve these challenges and provide greater user satisfaction are needed.

[0287] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0288] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for linking social networking service accounts using authentication means, means for analyzing the posted data acquired via an API using natural language processing technology, means for recognizing emotions from the analyzed posted data using an emotion engine, means for reflecting dynamic changes in the characters and stories based on the recognized emotions, means for dynamically adjusting music, backgrounds, character dialogue, etc. on the user's device based on the generated characters and stories, and means for transmitting actions performed by users as they progress through the game to the server and synchronizing operation data of other users in real time. This realizes a personalized gaming experience that reflects each user's individual characteristics and emotions, and enables smooth cooperative play in real time.

[0289] "User" means an individual or organization that has a social networking service account and uses this system.

[0290] A "server" is a computer system that collects, stores, and analyzes user data and transmits generated characters and stories to the user's device.

[0291] A "terminal" is a device (e.g., smartphone, tablet, computer) that a user uses to play a game.

[0292] A "Social Networking Service Account" is an individual account that a User has on an SNS platform.

[0293] "Posted data" refers to text, images, videos, and other content posted by users on social networking services.

[0294] "Collection methods" refer to the processes and tools used to obtain posted data through the API of a social networking service.

[0295] "Analysis means" refers to the process and tools for analyzing collected posted data using natural language processing technology and emotion recognition technology.

[0296] "Characteristics" refers to information such as individual personality, interests, and behavioral patterns identified from user posted data.

[0297] "Interests" are interests and preferences expressed by users through their posted data.

[0298] A "character" is an in-game avatar or virtual persona that is generated based on the user's characteristics and interests.

[0299] A "story" is a narrative scenario that is generated based on the user's characteristics and interests.

[0300] "Transmission means" refers to the process and tools that transfer the generated character and story data from the server to the user's terminal.

[0301] "Synchronization means" refers to the process and tools that transmit and receive operation data of multiple users between the server and the terminals in real time, ensuring consistency of in-game actions.

[0302] "Authentication Method" means the authentication technology and process for linking a social networking service account with the system.

[0303] "Natural language processing technology" refers to algorithms and methods for analyzing the text of posted data and understanding its meaning and sentiment.

[0304] The "emotion engine" is an algorithm and tool that recognizes user emotions from posted data and reflects them in the in-game experience.

[0305] "Dynamic adjustment" is the process of changing in-game music, backgrounds, character dialogue, and more in real time in response to the user's emotions and actions.

[0306] "Real-time cooperative play" is a play style in which multiple users play the game simultaneously, share data, and work together to complete missions.

[0307] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[0308] To implement this system, the following hardware and software are required:

[0309] Hardware

[0310] Server: A high-performance computer system that manages all processes of data collection, analysis, storage, and transmission.

[0311] Device: The device on which a user plays a game, such as a smartphone, tablet, or personal computer.

[0312] software

[0313] Generative AI model: An algorithm that uses natural language processing techniques to analyze user-submitted data and identify user characteristics and interests.

[0314] Emotion engine: Software that recognizes user emotions from posted data and reflects them in the game experience.

[0315] Data storage system: A database management system for securely storing collected data.

[0316] Real-time synchronization system: A network communication system for synchronizing data of multiple users in real time.

[0317] Specific operation of the system

[0318] Data collection

[0319] When a user downloads and launches the application, an account registration screen appears. Here, the user links their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token, which is used to collect the user's posted data via the SNS API. At this time, high security is ensured by using authentication methods.

[0320] Data analysis

[0321] The collected posting data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model uses natural language processing technology to analyze the posting data and identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotions (e.g., positive, negative, excited, calm, etc.) from the posts.

[0322] Character and story generation

[0323] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[0324] Sending data and starting the game

[0325] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin the game. In addition, the music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[0326] Real-time play function

[0327] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0328] Specific examples

[0329] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[0330] 1. The user connects their Twitter account and launches the application.

[0331] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[0332] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[0333] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[0334] 5. The generated data is sent to the device and the game interface is displayed.

[0335] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[0336] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0337] Prompt Sentence Examples

[0338] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[0339] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[0340] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0341] Program processing flow

[0342] Step 1: Download and launch the application

[0343] A user downloads an application from the GOOGLE PLAY® Store or the App Store.

[0344] The device installs and launches the downloaded application.

[0345] Input: Application package

[0346] Output: Launched application interface

[0347] Step 2: Register an account and connect to social media

[0348] The device displays an account registration screen, and the user enters their SNS account information.

[0349] The server obtains an authentication token from the social media platform.

[0350] Input: User's social media account information

[0351] Output: Authentication token

[0352] Step 3: Data collection

[0353] The server uses the authentication token to collect user posting data from the SNS via API.

[0354] The server stores the collected data in a database.

[0355] Input: Authentication token

[0356] Output: User submitted data

[0357] Step 4: Data analysis

[0358] The server supplies the collected post data to the generative AI model and emotion engine.

[0359] The generative AI model uses natural language processing technology to analyze posted data and identify user characteristics and interests.

[0360] The emotion engine recognizes user emotions from posts.

[0361] Input: User submitted data

[0362] Output: User characteristics, interests, emotions

[0363] Step 5: Character and story generation

[0364] Based on the analysis results, the server generates a character and story specifically for the user.

[0365] It uses generative AI models to create characters and customized stories that reflect the user's characteristics.

[0366] Character behavior and story development are dynamically adjusted based on the recognition results of the emotion engine.

[0367] Input: User characteristics, interests, and emotions

[0368] Output: Character and story data

[0369] Step 6: Send data

[0370] The server transmits the generated character and story data to the user's terminal.

[0371] Input: Character and story data

[0372] Output: Data sent to the terminal

[0373] Step 7: Game Start and Dynamic Adjustments

[0374] The device uses the received data to display the game interface.

[0375] The user starts the game and experiences the characters and story.

[0376] The device dynamically adjusts the music, background, character dialogue, etc. according to the user's emotions.

[0377] Input: Received character and story data

[0378] Output: Dynamically adjusted game interface

[0379] Step 8: User actions and real-time synchronization

[0380] The actions that the user performs in the game (e.g., movement, combat, item collection) are sent from the device to the server.

[0381] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other players on the device.

[0382] Input: User action data

[0383] Output: Real-time synchronized game data

[0384] Specific operation example

[0385] Let's assume that a user has linked their Twitter account. Their daily tweets are mostly about "cats" and "travel," and most of the content is positive.

[0386] 1. The user links their account and launches the application (Steps 1 and 2).

[0387] 2. The server collects past tweets via the Twitter API and stores them in a database (Step 3).

[0388] 3. Generative AI models and emotion engines analyze this data to identify user characteristics, interests, and emotions (Step 4).

[0389] 4. Based on the analysis results, the server generates an adventure story for users who are "cat lovers" or "travel lovers," adding elements of happiness (Step 5).

[0390] 5. The generated data is sent to the device and the game interface is displayed (Step 6).

[0391] 6. The user experiences an adventure story in which their avatar rescues a cat, with music and backgrounds dynamically adjusted to match their emotions. (Step 7)

[0392] 7. If other players are also playing the game at the same time, all player data is synchronized in real time (Step 8).

[0393] Prompt Sentence Examples

[0394] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[0395] (Application example 2)

[0396] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0397] Many existing game systems that utilize social networking services collect users' interests and characteristics and generate characters and stories based on them. However, these systems are inadequate in terms of real-time play and emotional reflection, making it difficult to improve the quality of individual user experiences. Furthermore, there has been no system that can provide a shopping experience that reflects the user's emotions and interests. This has led to the problem of not being able to meet users' demands for a personalized shopping experience.

[0398] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0399] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to the user's terminal, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for generating a personalized virtual environment by reflecting emotions recognized from each user's posted data, and means for providing an interactive shopping experience based on exploration and actions within the personalized virtual environment, thereby enabling a personalized shopping experience that reflects the user's interests and emotions.

[0400] A "social networking service account" is personal authentication information and profile information that enables a user to access and use a social networking service.

[0401] "Posted data" refers to information such as messages, images, and videos posted by users on social networking services.

[0402] "Analysis" is the process of classifying, understanding, and evaluating collected data using natural language processing techniques and machine learning models.

[0403] "Features" refer to attributes and information that indicate a user's interests, behavior, preferences, etc.

[0404] A "character" is a depiction of a virtual person, animal, etc. that is generated based on the user's characteristics and interests.

[0405] A "story" is a virtual narrative progression or scenario generated based on the user's interests and emotions.

[0406] A "terminal" is an electronic device such as a smartphone, tablet, or PC that a user accesses.

[0407] "Synchronization" refers to the process of matching the data and actions of multiple users in real time.

[0408] "Reflecting emotions" means influencing the behavior of the virtual environment and characters based on emotional information analyzed from user-submitted data.

[0409] "Virtual environment" refers to a computer-generated virtual space or virtual store.

[0410] An "interactive shopping experience" refers to an experience in which users explore, select, and purchase products in a virtual environment while interacting with other users in real time.

[0411] The system for realizing this invention comprises the following means: First, a user downloads and launches an application using their own terminal (smartphone, tablet, PC, etc.). The roles of the server, terminal, and user are explained in detail below.

[0412] Server Roles

[0413] 1. Data Collection:

[0414] When a user links their account with a social networking service (e.g., Twitter or Instagram), the server collects the user's posting data via the SNS API, using an authentication token.

[0415] 2. Data Analysis:

[0416] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine, utilizing a generative AI model (e.g., GPT-4) to identify user characteristics (e.g., specific hobbies and interests) and emotional states (e.g., positive, negative).

[0417] 3. Character and Story Generation:

[0418] The server generates characters and stories based on the identified user's interests, characteristics, and emotional state. A generative AI model automatically generates characters and stories personalized to the user.

[0419] 4. Real-time data synchronization:

[0420] The server has the ability to synchronize the data of multiple users in real time, providing an environment where users can play simultaneously.

[0421] 5. Creating personalized virtual environments:

[0422] Generate a virtual environment (e.g., a virtual store) that reflects the user's emotions and characteristics. The environment is customized based on the products and themes that the user is interested in.

[0423] Device Role

[0424] 1. Data reception and interface provision:

[0425] The user's device receives the generated characters and story sent from the server and displays them through a user interface, which also includes adjusting the music and background according to the emotions.

[0426] 2. Sending operational data:

[0427] When a user operates the device, the operation data (e.g., movement, selection, etc.) is sent to the server and synchronized in real time.

[0428] 3. Exploration and interaction within virtual environments:

[0429] Users can explore the virtual environment on their device and are provided with interactive experiences such as searching for products and completing missions.

[0430] User Roles

[0431] 1. Account linking and data provision:

[0432] A user links a social networking service account to the application and provides posting data.

[0433] 2. Gameplay and Shopping Experience:

[0434] Users not only enjoy individually customized characters and stories, but also have a shopping experience while connecting with other users in real time.

[0435] Specific examples

[0436] For example, if a user links their Instagram account and their posts are mostly about fashion and travel and contain positive content, they will be offered a personalized shopping adventure through the following steps:

[0437] 1. The user installs the application and connects their Instagram account.

[0438] 2. The server collects post data through the Instagram API and analyzes it using a generative AI model and emotion engine.

[0439] 3. Based on the analysis results, we recognize that the user is interested in "fashion" and "travel" and has positive emotions.

[0440] 4. The generative AI model generates a virtual fashion and travel-themed store and characters specifically for the user.

[0441] 5. The generated data is sent to the user's device, and their personalized shopping adventure begins.

[0442] Prompt Sentence Examples

[0443] "User 1 has provided us with their social media account and posting data. This user's posts are mostly about fashion and travel, and contain a lot of positive emotions. Please create a personalized virtual store that reflects the user's interests and emotions, and create a character and story just for the user."

[0444] In this way, users can have a unique shopping experience that reflects their own characteristics and interests, and can enjoy interacting with other users in real time.

[0445] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0446] Step 1:

[0447] The user installs the application and connects their social media account.

[0448] A user downloads and launches an application onto a device such as a smartphone or tablet. Next, they link their social networking service (SNS) account on the account registration screen. For example, they link their account by entering their Twitter or Instagram credentials. The input for this step is their SNS account credentials, and the output is an authentication token.

[0449] Step 2:

[0450] The server collects post data through the SNS API

[0451] The server uses the obtained authentication token to collect user post data through the SNS API. This data includes text, images, videos, etc., and is all stored on the server. The input of this step is the authentication token, and the output is the post data.

[0452] Step 3:

[0453] The server analyzes the collected submission data.

[0454] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine. NLP is used to analyze text data and identify user characteristics and interests. Image recognition technology is also used to extract information from image data. The emotion engine recognizes the user's emotions (positive, negative, etc.) from the posting data. The input of this step is the posting data, and the output is the user's characteristics, interests, and emotional information.

[0455] Step 4:

[0456] The server generates a character and story for the user.

[0457] Based on the analysis results, the server uses a generative AI model (e.g., GPT-4) to generate a character and story specifically for the user. The generated character reflects the user's characteristics, and the story incorporates the user's interests. The input for this step is the user's characteristics, interests, and emotional information, and the output is the generated character and story.

[0458] Step 5:

[0459] The server sends the generated character and story data to the user's device.

[0460] The server sends the generated character and story data to the user's device. The user's device displays a game interface based on the received data. The input for this step is the generated character data and story data, and the output is the game interface displayed on the user's device.

[0461] Step 6:

[0462] Users explore the virtual environment and engage in an interactive shopping experience

[0463] The user explores the virtual environment displayed on the device and controls the character. All exploration and actions within the virtual environment (e.g., selecting items, checking products) are sent to the server in real time. The server uses this operation data to provide an interactive shopping experience synchronized with other users. The input of this step is the user's operation data, and the output is a real-time synchronized shopping experience.

[0464] Step 7:

[0465] The server provides dynamic environment settings according to the user's emotions and actions.

[0466] The server dynamically adjusts the virtual environment settings (e.g., background music, visual effects) according to the emotions and real-time operation data analyzed from the user's posted data. The input of this step is the user's emotional information and operation data, and the output is the adjusted virtual environment settings.

[0467] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0468] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0469] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0470] [Second embodiment]

[0471] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0472] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0473] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0474] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0475] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0476] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0477] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0478] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0479] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0480] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0481] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0482] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0483] The system collects data posted by users on social networking service accounts, analyzes the data to identify the user's characteristics and interests, generates characters and stories based on the data, and finally transmits and displays them on the user's device. The system also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data.

[0484] Program processing

[0485] Data collection

[0486] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0487] Data analysis

[0488] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[0489] Character and story generation

[0490] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[0491] Sending data and starting the game

[0492] The generated character and story data is sent from the server to the user's device, which receives this data and displays the game interface, where the user can view their avatar and begin playing the game.

[0493] Real-time play function

[0494] As users progress through the game, all actions (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0495] Specific examples

[0496] Let's say a user links their Twitter account. If their daily tweets often contain topics like "cats" and "travel," the following characters and stories will be generated.

[0497] 1. The user connects their Twitter account and launches the application.

[0498] 2. The server collects past tweets via the Twitter API and analyzes the data using a generative AI model.

[0499] 3. As a result of the analysis, it is determined that this user is a "cat lover" and a "travel lover."

[0500] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role.

[0501] 5. The generated data is sent to the device and the game interface is displayed.

[0502] 6. Users experience a story in which their avatar goes on an adventure to save a cat.

[0503] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0504] This allows users to have a unique RPG experience that reflects their own characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[0505] The processing flow will be explained below.

[0506] Step 1:

[0507] The user downloads and installs the application.

[0508] Step 2:

[0509] The user launches the application, enters their information on the account registration screen, and creates an account.

[0510] Step 3:

[0511] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[0512] Step 4:

[0513] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[0514] Step 5:

[0515] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[0516] Step 6:

[0517] The server then supplies the saved post data to a generative AI model, which uses natural language processing (NLP) technology to analyze the sentiment of posts, extract frequently occurring keywords, and identify users' areas of interest.

[0518] Step 7:

[0519] A generative AI model identifies user characteristics and interests, such as "cat lover" or "travel lover" from posts.

[0520] Step 8:

[0521] The server generates characters and stories based on the analysis results. Using a generative AI model, it creates a character specific to the user (e.g., a cat-loving adventurer) and generates a story based on the user's interests (e.g., an adventure to save cats).

[0522] Step 9:

[0523] The server transmits the generated character and story data to the user's terminal.

[0524] Step 10:

[0525] The device displays the game interface based on the received character and story data, allowing the user to confirm their avatar and begin the game.

[0526] Step 11:

[0527] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[0528] Step 12:

[0529] The server synchronizes the data of multiple users in real time, so that the movements and progress of other players are also reflected on the user's device.

[0530] Step 13:

[0531] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[0532] Step 14:

[0533] When a user finishes a game session, the terminal transmits current game progress data to the server.

[0534] Step 15:

[0535] The server saves your game progress, allowing you to resume from the same point the next time you play.

[0536] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data.

[0537] Example 1

[0538] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0539] Today's users often demand personalized content based on their individual interests and characteristics. However, content generation using social networking service (SNS) data to date has faced challenges due to its inability to fully reflect user characteristics. Furthermore, there is no established method for providing an environment in which multiple users can play simultaneously in real time while synchronizing each user's operation data. Furthermore, there are still issues with SNS account authentication, secure data storage, and understanding daily patterns.

[0540] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0541] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for authenticating the linking of social networking accounts, means for analyzing the acquired posted data using natural language processing technology, means for creating characters and stories using a generative AI model, and means for transmitting operation data from the user's device to the server and receiving operation data from other users at the device in real time. This enables the generation of personalized characters and stories that reflect users' characteristics, providing a synchronized gaming experience in real time. Furthermore, the authentication means enables secure social networking and ensures data security.

[0542] A "social networking service account" is identification information that allows a user to access a social networking service.

[0543] "Posted data" refers to information such as text, images, and videos that users make public on social networking services.

[0544] "Analysis" is the process of processing collected data using natural language processing techniques to identify user characteristics and interests.

[0545] "Features" refer to individual attributes such as user preferences and tendencies obtained from analyzed data.

[0546] "Interests" indicate topics or activities that a user is particularly interested in.

[0547] A "character" is a user's avatar or a representative of a player within a game.

[0548] A "story" is the narrative or scenario that the player progresses through within the game.

[0549] A "terminal" is a user's device (e.g., smartphone, tablet) for running an application.

[0550] "Data synchronization" is the process of making all users' operation data consistent in real time when multiple users are playing a game simultaneously.

[0551] "Authentication means" refers to a method for linking a user's social networking service account with an application.

[0552] A "generative AI model" is an artificial intelligence model that automatically generates characters and stories based on the user's characteristics and interests.

[0553] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0554] "Operation data" is information relating to operations and actions performed by the user within the game.

[0555] "Data Storage Means" means a method or system for securely storing collected data.

[0556] "Daily patterns" are trends in daily activities and behaviors that are analyzed from data posted by users.

[0557] "Personalized" means customized based on the characteristics and interests of each individual user.

[0558] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that, and ultimately sends and displays them on the user's terminal.

[0559] This system provides an environment in which multiple users can play games simultaneously in real time, and is characterized by synchronizing the data of each user. The operation of the system is carried out as follows.

[0560] First, the user downloads and launches the application. This displays an account registration screen, where the user enters basic information such as their name and email address and links their social networking service account (e.g., Twitter, Facebook). Next, once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0561] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[0562] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[0563] The generated character and story data is sent from the server to the user's device, which receives the data and displays the game interface, where the user can view their avatar and begin playing the game.

[0564] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. In this way, users can cooperate with each other to complete missions.

[0565] As a concrete example, let's assume that a user has linked their Twitter account. Suppose that the user's daily tweets often contain topics like "cats" and "travel." In this case, the following characters and stories will be generated.

[0566] 1. The user connects their Twitter account and launches the application.

[0567] 2. The server collects past tweet data through the Twitter API.

[0568] 3. The collected tweet data is analyzed using a generative AI model.

[0569] 4. The analysis results reveal that this user is a "cat lover" and a "travel lover."

[0570] 5. Based on the analysis, the server generates a story in which the cat plays an important role.

[0571] 6. The generated data is sent to the device and the game interface is displayed.

[0572] 7. Users experience a story in which their avatar goes on an adventure to save a cat.

[0573] 8. The server synchronizes all player data in real time, and everyone's actions are reflected in the game.

[0574] An example of a prompt you could give to a generative AI model would be:

[0575] "Identify the user's interests and characteristics from their posted data, and generate a unique character and story based on that data. For example, if the user loves cats and travels, generate a character with an adventure story about saving cats."

[0576] This allows users to have a unique RPG experience that reflects their characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Program processing flow

[0579] Step 1:

[0580] A user downloads an application and launches it on their device. As input, the user installs and launches the application. As output, an account registration screen is displayed.

[0581] Specifically, when the application is launched for the first time, the terminal generates a user interface and displays an account registration screen.

[0582] Step 2:

[0583] The user enters their account registration information and links their social media account. As input, the user enters their name, email address, and social media link information. As output, an authentication token is obtained once linking is complete.

[0584] Specifically, the terminal receives user input, displays an authentication screen for the social networking service, and transmits an authentication token to the server once authentication is complete.

[0585] Step 3:

[0586] The server uses the SNS API to collect user posted data. As input, it receives an authentication token. As output, the user posted data is collected and saved in a database.

[0587] Specifically, the server accesses the SNS API using the authentication token, obtains the user's past posting data, and stores the collected data in a database.

[0588] Step 4:

[0589] The server supplies the collected posting data to the generative AI model for analysis. Posting data obtained from the database is supplied as input to the generative AI model. User characteristics and interests are identified as output.

[0590] Specifically, the generative AI model analyzes the data using natural language processing (NLP) techniques to identify the user's characteristics (e.g., cat lover, travel lover) and interests. The server then stores the analysis results in a database.

[0591] Step 5:

[0592] The server generates characters and stories based on the analysis results. As input, it generates prompt sentences based on the analysis results and feeds them to the generative AI model. As output, personalized characters and stories are generated.

[0593] Specifically, the server generates a prompt, for example, "If the user loves cats and travels, please generate a character with an adventure story about saving cats," and inputs this into the generative AI model. The generative AI model generates the character and story and returns the generated data to the server, which stores it in a database.

[0594] Step 6:

[0595] The server sends the generated character and story data to the user's device. As input, it receives the generated character and story data. As output, it sends the data to the user's device and displays the game interface.

[0596] Specifically, the server sends the generated data to the user's device, which receives the data and displays the game interface, allowing the user to view their avatar and story.

[0597] Step 7:

[0598] When a user starts a game, action data is sent from the device to the server as the user progresses. As input, the user's action data (e.g., movement, combat, item collection) is acquired. As output, the action data is sent to the server.

[0599] Specifically, the terminal captures the user's operations and transmits the data to the server.

[0600] Step 8:

[0601] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other users on their devices. As input, it receives the operation data of all users. As output, it sends the synchronized data in real time to each user's device.

[0602] Specifically, the server processes all users' operation data in real time and sends the synchronized data to each user's device. The device then reflects the movements of other users, enabling real-time cooperative play.

[0603] (Application example 1)

[0604] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0605] In recent autonomous vehicles, there is a growing demand for entertainment systems that allow passengers to make effective use of their time while on board. However, existing systems lack the means to provide a personalized entertainment experience. Furthermore, when there are multiple passengers, there is a problem in that they lack the functionality to allow passengers to interact with each other in real time. Therefore, the present invention aims to provide a system that uses users' social networking service data to provide a personalized and interactive entertainment experience.

[0606] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0607] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' terminals, means for synchronizing data for each user so that multiple users can play simultaneously in real time, and means for operating within the mobile communication terminal to provide users with an interactive entertainment experience. This allows users on board to enjoy a personalized entertainment experience based on their interests and characteristics, and if there are multiple users, they can interact with each other in real time.

[0608] Definitions of important words

[0609] "User's Social Networking Service Account" means the credentials used by a User to post and interact on a Social Media Platform.

[0610] "Posted data" refers to content such as text, images, and videos posted by users to social networking services, as well as metadata.

[0611] "Analysis" is a data processing method that uses natural language processing technology and machine learning algorithms to identify user characteristics and interests from collected posted data.

[0612] A "character" is a virtual entity that is generated exclusively for a user, reflects the characteristics of the user, and plays an active role within the story.

[0613] A "story" is entertainment content that includes a sequence of events or scenarios generated based on the user's interests.

[0614] "Terminal" refers to an electronic device that can be directly operated by a user, including a smartphone, tablet, computer, in-vehicle display, etc.

[0615] "Real-time" means that data and operations are processed instantly, without any delay.

[0616] "Synchronizing data" means matching data with each other in order to keep the operation data and progress status of multiple users in the same state.

[0617] "Mobile communication terminal" refers to an electronic device with communication functions intended for use in a vehicle, including displays and infotainment systems in autonomous vehicles.

[0618] An "interactive entertainment experience" is an experience in which users can interact with the content and influence the story or character behavior through actions and choices.

[0619] "Server" means a computer system running computational resources and software for collecting, analyzing, storing, and synchronizing data.

[0620] "Natural language processing technology" refers to the technology used by computers to understand, interpret, and generate human language, and uses machine learning algorithms and deep learning models.

[0621] A "generative AI model" is an artificial intelligence model that can generate new data from input data using machine learning algorithms.

[0622] "Multiple users on board" refers to multiple users inside an autonomous vehicle, each of whom can operate the system using their own terminal.

[0623] The OAuth 2.0 authentication system is a widely used protocol for authentication and authorization on the web, allowing third parties to access a user's resources.

[0624] A "real-time synchronization system" is a system that instantly processes changing data and shares the latest information with all related devices.

[0625] "Infotainment system" refers to a system that integrates information and entertainment within a vehicle, including displays, audio, and internet connectivity.

[0626] MODE FOR CARRYING OUT THE INVENTION

[0627] 1. System Program

[0628] The system that realizes this application example consists of the following program modules:

[0629] 1. User authentication module

[0630] 2. Data Collection Module

[0631] 3. Data Analysis Module

[0632] 4. Character Story Generation Module

[0633] 5. Data transmission and game interface module

[0634] 6. Real-time synchronization module

[0635] 2. Natural language explanation

[0636] User Authentication Module

[0637] When a user logs into the vehicle's entertainment system, an authentication module is started. The authentication module authenticates the user's social networking service account (e.g., Twitter, Facebook) using the OAuth 2.0 authentication system. This authentication allows the server to obtain an authentication token, which can then be used to collect the user's posted data.

[0638] Data Collection Module

[0639] The server uses the authentication token to retrieve user posted data through the SNS API. This includes content such as text, images, and videos. The collected data is securely stored in a database system. This module can use database systems such as Amazon RDS.

[0640] Data Analysis Module

[0641] The collected data is analyzed on the server using natural language processing (NLP) technology, particularly NLP models such as BERT and GPT-4. These models extract user characteristics and interests from text and analyze user trends and patterns.

[0642] Character Story Generation Module

[0643] Based on the analysis results, a generative AI model (e.g., GPT-4) generates a character and story specifically for the user. During this process, character attributes and storylines that match the user's characteristics and interests are generated. The server generates this data and prepares it for later transmission to the device.

[0644] Data transmission and game interface module

[0645] The generated characters and story are sent from the server to the vehicle's entertainment system. The user operates the interactive game interface using the in-vehicle display or touch panel. This module is implemented as an Android / iOS application.

[0646] Real-time Synchronization Module

[0647] The server synchronizes the data of multiple users riding in the vehicle in real time. Using Firebase and WebSocket technology, all users' operation data is updated simultaneously, and the progress of other users is also reflected in real time. In this way, users can interact with each other.

[0648] 3. Examples and prompts

[0649] Specific examples

[0650] For example, if a user links their Facebook account and it is analyzed that they frequently post about "cats" and "music" while on board, a story such as "Adventuring in a musical land where cats live" will be generated for that user. A unique feature of this service is that it allows passengers to enjoy the adventure together.

[0651] Prompt Sentence Examples

[0652] Example of prompt input:

[0653] Analyze the user's posted data and generate a story based on their interest in "cats" and "music." The user is already logged into the vehicle's entertainment system.

[0654] This allows users to spend their travel time enjoying personalized entertainment in an autonomous vehicle.

[0655] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0656] Program processing steps

[0657] Step 1:

[0658] When a user logs into the vehicle's entertainment system, the terminal displays an authentication screen. The user logs in with a social networking service account (e.g., Twitter, Facebook). As input, the user's authentication information is required, and as output, an authentication token is obtained. The terminal sends this token to the server. The authentication system uses OAuth 2.0.

[0659] Step 2:

[0660] The server uses the obtained authentication token to collect user posted data through the SNS API. The authentication token is required as input, and the user posted data is obtained as output. The server securely stores this data in a database. Amazon RDS is used as the database system.

[0661] Step 3:

[0662] The server analyzes the collected posting data using natural language processing technology. Posting data is required as input, and user characteristics and interests are obtained as output. Specifically, NLP models such as BERT and GPT-4 are used to identify user interest patterns (e.g., "cat lover" or "music lover"). This allows the user's tendencies and patterns to be identified.

[0663] Step 4:

[0664] The server uses a generative AI model (e.g., GPT-4) based on the analysis results to generate a character and story specifically for the user. The analysis results are required as input, and the characters and story are generated as output. For example, a story based on the user's interests is created, such as "Adventure in a musical land where cats live."

[0665] Step 5:

[0666] The server sends the generated characters and stories to the terminal. The generated characters and stories are required as input, and the data is displayed on the terminal as output. The terminal provides the user with an interactive game interface, which is achieved using an Android / iOS application.

[0667] Step 6:

[0668] The terminal collects user operation data and sends it to the server. The user operation data is required as input and sent to the server as output. The server also synchronizes other users' operation data in real time.

[0669] Step 7:

[0670] The server synchronizes all user operation data in real time using Firebase and WebSocket technology. Each user's operation data is required as input, and the game state updated in real time is obtained as output. This allows multiple users to share an interactive entertainment experience simultaneously.

[0671] Through these steps, users can enjoy a personalized and interactive entertainment experience inside their autonomous vehicle.

[0672] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0673] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[0674] Program processing

[0675] Data collection

[0676] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0677] Data analysis

[0678] The collected post data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) techniques to identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes user emotions (e.g., positive, negative, excited, calm, etc.) from the post.

[0679] Character and story generation

[0680] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[0681] Sending data and starting the game

[0682] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin playing the game. The music, background, and character dialogue are also dynamically adjusted according to the user's emotions.

[0683] Real-time play function

[0684] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0685] Specific examples

[0686] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[0687] 1. The user connects their Twitter account and launches the application.

[0688] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[0689] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[0690] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[0691] 5. The generated data is sent to the device and the game interface is displayed.

[0692] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[0693] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0694] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[0695] The processing flow will be explained below.

[0696] Step 1:

[0697] The user downloads and installs the application.

[0698] Step 2:

[0699] The user launches the application, enters their information on the account registration screen, and creates an account.

[0700] Step 3:

[0701] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[0702] Step 4:

[0703] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[0704] Step 5:

[0705] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[0706] Step 6:

[0707] The server provides the saved post data to the generative AI model and emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) technology to identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotion (e.g., positive, negative, excited, calm, etc.) from the post.

[0708] Step 7:

[0709] Based on the analysis results of the generative AI model and emotion engine, the server generates a character and story specifically for the user. For example, if a user posts many cat-related messages and the emotion engine recognizes positive emotions from those messages, the character will be given the attribute of "cat lover" and the story will include "a happy encounter with a cat."

[0710] Step 8:

[0711] The server transmits the generated character and story data to the user's terminal.

[0712] Step 9:

[0713] The device displays the game interface based on the character and story data received. The user can then confirm their avatar and begin the game. The in-game music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[0714] Step 10:

[0715] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[0716] Step 11:

[0717] The server synchronizes the data of multiple users in real time, so the movements and progress of other players are also reflected on the device.

[0718] Step 12:

[0719] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[0720] Step 13:

[0721] When a user finishes a game session, the terminal transmits current game progress data to the server.

[0722] Step 14:

[0723] The server saves your game progress, allowing you to resume from the same point the next time you play.

[0724] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data and emotions.

[0725] Example 2

[0726] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0727] Current game systems and entertainment services struggle to provide personalized experiences that fully reflect each user's individual characteristics, interests, and emotions. Furthermore, when multiple users play simultaneously in real time, smoothly integrating all users' data and enabling cooperative play is also problematic. Furthermore, it is currently difficult to effectively utilize the data posted by users on social networking services and dynamically reflect it in the characters and story of the game. Systems that can solve these challenges and provide greater user satisfaction are needed.

[0728] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0729] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for linking social networking service accounts using authentication means, means for analyzing the posted data acquired via an API using natural language processing technology, means for recognizing emotions from the analyzed posted data using an emotion engine, means for reflecting dynamic changes in the characters and stories based on the recognized emotions, means for dynamically adjusting music, backgrounds, character dialogue, etc. on the user's device based on the generated characters and stories, and means for transmitting actions performed by users as they progress through the game to the server and synchronizing operation data of other users in real time. This realizes a personalized gaming experience that reflects each user's individual characteristics and emotions, and enables smooth cooperative play in real time.

[0730] "User" means an individual or organization that has a social networking service account and uses this system.

[0731] A "server" is a computer system that collects, stores, and analyzes user data and transmits generated characters and stories to the user's device.

[0732] A "terminal" is a device (e.g., smartphone, tablet, computer) that a user uses to play a game.

[0733] A "Social Networking Service Account" is an individual account that a User has on an SNS platform.

[0734] "Posted data" refers to text, images, videos, and other content posted by users on social networking services.

[0735] "Collection methods" refer to the processes and tools used to obtain posted data through the API of a social networking service.

[0736] "Analysis means" refers to the process and tools for analyzing collected posted data using natural language processing technology and emotion recognition technology.

[0737] "Characteristics" refers to information such as individual personality, interests, and behavioral patterns identified from user posted data.

[0738] "Interests" are interests and preferences expressed by users through their posted data.

[0739] A "character" is an in-game avatar or virtual persona that is generated based on the user's characteristics and interests.

[0740] A "story" is a narrative scenario that is generated based on the user's characteristics and interests.

[0741] "Transmission means" refers to the process and tools that transfer the generated character and story data from the server to the user's terminal.

[0742] "Synchronization means" refers to the process and tools that transmit and receive operation data of multiple users between the server and the terminals in real time, ensuring consistency of in-game actions.

[0743] "Authentication Method" means the authentication technology and process for linking a social networking service account with the system.

[0744] "Natural language processing technology" refers to algorithms and methods for analyzing the text of posted data and understanding its meaning and sentiment.

[0745] The "emotion engine" is an algorithm and tool that recognizes user emotions from posted data and reflects them in the in-game experience.

[0746] "Dynamic adjustment" is the process of changing in-game music, backgrounds, character dialogue, and more in real time in response to the user's emotions and actions.

[0747] "Real-time cooperative play" is a play style in which multiple users play the game simultaneously, share data, and work together to complete missions.

[0748] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[0749] To implement this system, the following hardware and software are required:

[0750] Hardware

[0751] Server: A high-performance computer system that manages all processes of data collection, analysis, storage, and transmission.

[0752] Device: The device on which a user plays a game, such as a smartphone, tablet, or personal computer.

[0753] software

[0754] Generative AI model: An algorithm that uses natural language processing techniques to analyze user-submitted data and identify user characteristics and interests.

[0755] Emotion engine: Software that recognizes user emotions from posted data and reflects them in the game experience.

[0756] Data storage system: A database management system for securely storing collected data.

[0757] Real-time synchronization system: A network communication system for synchronizing data of multiple users in real time.

[0758] Specific operation of the system

[0759] Data collection

[0760] When a user downloads and launches the application, an account registration screen appears. Here, the user links their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token, which is used to collect the user's posted data via the SNS API. At this time, high security is ensured by using authentication methods.

[0761] Data analysis

[0762] The collected posting data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model uses natural language processing technology to analyze the posting data and identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotions (e.g., positive, negative, excited, calm, etc.) from the posts.

[0763] Character and story generation

[0764] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[0765] Sending data and starting the game

[0766] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin the game. In addition, the music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[0767] Real-time play function

[0768] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0769] Specific examples

[0770] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[0771] 1. The user connects their Twitter account and launches the application.

[0772] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[0773] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[0774] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[0775] 5. The generated data is sent to the device and the game interface is displayed.

[0776] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[0777] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0778] Prompt Sentence Examples

[0779] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[0780] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[0781] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0782] Program processing flow

[0783] Step 1: Download and launch the application

[0784] A user downloads an application from the Google (registered trademark) Play Store or the App Store.

[0785] The device installs and launches the downloaded application.

[0786] Input: Application package

[0787] Output: Launched application interface

[0788] Step 2: Register an account and connect to social media

[0789] The device displays an account registration screen, and the user enters their SNS account information.

[0790] The server obtains an authentication token from the social media platform.

[0791] Input: User's social media account information

[0792] Output: Authentication token

[0793] Step 3: Data collection

[0794] The server uses the authentication token to collect user posting data from the SNS via API.

[0795] The server stores the collected data in a database.

[0796] Input: Authentication token

[0797] Output: User submitted data

[0798] Step 4: Data analysis

[0799] The server supplies the collected post data to the generative AI model and emotion engine.

[0800] The generative AI model uses natural language processing technology to analyze posted data and identify user characteristics and interests.

[0801] The emotion engine recognizes user emotions from posts.

[0802] Input: User submitted data

[0803] Output: User characteristics, interests, emotions

[0804] Step 5: Character and story generation

[0805] Based on the analysis results, the server generates a character and story specifically for the user.

[0806] It uses generative AI models to create characters and customized stories that reflect the user's characteristics.

[0807] Character behavior and story development are dynamically adjusted based on the recognition results of the emotion engine.

[0808] Input: User characteristics, interests, and emotions

[0809] Output: Character and story data

[0810] Step 6: Send data

[0811] The server transmits the generated character and story data to the user's terminal.

[0812] Input: Character and story data

[0813] Output: Data sent to the terminal

[0814] Step 7: Game Start and Dynamic Adjustments

[0815] The device uses the received data to display the game interface.

[0816] The user starts the game and experiences the characters and story.

[0817] The device dynamically adjusts the music, background, character dialogue, etc. according to the user's emotions.

[0818] Input: Received character and story data

[0819] Output: Dynamically adjusted game interface

[0820] Step 8: User actions and real-time synchronization

[0821] The actions that the user performs in the game (e.g., movement, combat, item collection) are sent from the device to the server.

[0822] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other players on the device.

[0823] Input: User action data

[0824] Output: Real-time synchronized game data

[0825] Specific operation example

[0826] Let's assume that a user has linked their Twitter account. Their daily tweets are mostly about "cats" and "travel," and most of the content is positive.

[0827] 1. The user links their account and launches the application (Steps 1 and 2).

[0828] 2. The server collects past tweets via the Twitter API and stores them in a database (Step 3).

[0829] 3. Generative AI models and emotion engines analyze this data to identify user characteristics, interests, and emotions (Step 4).

[0830] 4. Based on the analysis results, the server generates an adventure story for users who are "cat lovers" or "travel lovers," adding elements of happiness (Step 5).

[0831] 5. The generated data is sent to the device and the game interface is displayed (Step 6).

[0832] 6. The user experiences an adventure story in which their avatar rescues a cat, with music and backgrounds dynamically adjusted to match their emotions. (Step 7)

[0833] 7. If other players are also playing the game at the same time, all player data is synchronized in real time (Step 8).

[0834] Prompt Sentence Examples

[0835] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[0836] (Application example 2)

[0837] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0838] Many existing game systems that utilize social networking services collect users' interests and characteristics and generate characters and stories based on them. However, these systems are inadequate in terms of real-time play and emotional reflection, making it difficult to improve the quality of individual user experiences. Furthermore, there has been no system that can provide a shopping experience that reflects the user's emotions and interests. This has led to the problem of not being able to meet users' demands for a personalized shopping experience.

[0839] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0840] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to the user's terminal, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for generating a personalized virtual environment by reflecting emotions recognized from each user's posted data, and means for providing an interactive shopping experience based on exploration and actions within the personalized virtual environment, thereby enabling a personalized shopping experience that reflects the user's interests and emotions.

[0841] A "social networking service account" is personal authentication information and profile information that enables a user to access and use a social networking service.

[0842] "Posted data" refers to information such as messages, images, and videos posted by users on social networking services.

[0843] "Analysis" is the process of classifying, understanding, and evaluating collected data using natural language processing techniques and machine learning models.

[0844] "Features" refer to attributes and information that indicate a user's interests, behavior, preferences, etc.

[0845] A "character" is a depiction of a virtual person, animal, etc. that is generated based on the user's characteristics and interests.

[0846] A "story" is a virtual narrative progression or scenario generated based on the user's interests and emotions.

[0847] A "terminal" is an electronic device such as a smartphone, tablet, or PC that a user accesses.

[0848] "Synchronization" refers to the process of matching the data and actions of multiple users in real time.

[0849] "Reflecting emotions" means influencing the behavior of the virtual environment and characters based on emotional information analyzed from user-submitted data.

[0850] "Virtual environment" refers to a computer-generated virtual space or virtual store.

[0851] An "interactive shopping experience" refers to an experience in which users explore, select, and purchase products in a virtual environment while interacting with other users in real time.

[0852] The system for realizing this invention comprises the following means: First, a user downloads and launches an application using their own terminal (smartphone, tablet, PC, etc.). The roles of the server, terminal, and user are explained in detail below.

[0853] Server Roles

[0854] 1. Data Collection:

[0855] When a user links their account with a social networking service (e.g., Twitter or Instagram), the server collects the user's posting data via the SNS API, using an authentication token.

[0856] 2. Data Analysis:

[0857] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine, utilizing a generative AI model (e.g., GPT-4) to identify user characteristics (e.g., specific hobbies and interests) and emotional states (e.g., positive, negative).

[0858] 3. Character and Story Generation:

[0859] The server generates characters and stories based on the identified user's interests, characteristics, and emotional state. A generative AI model automatically generates characters and stories personalized to the user.

[0860] 4. Real-time data synchronization:

[0861] The server has the ability to synchronize the data of multiple users in real time, providing an environment where users can play simultaneously.

[0862] 5. Creating personalized virtual environments:

[0863] Generate a virtual environment (e.g., a virtual store) that reflects the user's emotions and characteristics. The environment is customized based on the products and themes that the user is interested in.

[0864] Device Role

[0865] 1. Data reception and interface provision:

[0866] The user's device receives the generated characters and story sent from the server and displays them through a user interface, which also includes adjusting the music and background according to the emotions.

[0867] 2. Sending operational data:

[0868] When a user operates the device, the operation data (e.g., movement, selection, etc.) is sent to the server and synchronized in real time.

[0869] 3. Exploration and interaction within virtual environments:

[0870] Users can explore the virtual environment on their device and are provided with interactive experiences such as searching for products and completing missions.

[0871] User Roles

[0872] 1. Account linking and data provision:

[0873] A user links a social networking service account to the application and provides posting data.

[0874] 2. Gameplay and Shopping Experience:

[0875] Users not only enjoy individually customized characters and stories, but also have a shopping experience while connecting with other users in real time.

[0876] Specific examples

[0877] For example, if a user links their Instagram account and their posts are mostly about fashion and travel and contain positive content, they will be offered a personalized shopping adventure through the following steps:

[0878] 1. The user installs the application and connects their Instagram account.

[0879] 2. The server collects post data through the Instagram API and analyzes it using a generative AI model and emotion engine.

[0880] 3. Based on the analysis results, we recognize that the user is interested in "fashion" and "travel" and has positive emotions.

[0881] 4. The generative AI model generates a virtual fashion and travel-themed store and characters specifically for the user.

[0882] 5. The generated data is sent to the user's device, and their personalized shopping adventure begins.

[0883] Prompt Sentence Examples

[0884] "User 1 has provided us with their social media account and posting data. This user's posts are mostly about fashion and travel, and contain a lot of positive emotions. Please create a personalized virtual store that reflects the user's interests and emotions, and create a character and story just for the user."

[0885] In this way, users can have a unique shopping experience that reflects their own characteristics and interests, and can enjoy interacting with other users in real time.

[0886] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0887] Step 1:

[0888] The user installs the application and connects their social media account.

[0889] A user downloads and launches an application onto a device such as a smartphone or tablet. Next, they link their social networking service (SNS) account on the account registration screen. For example, they link their account by entering their Twitter or Instagram credentials. The input for this step is their SNS account credentials, and the output is an authentication token.

[0890] Step 2:

[0891] The server collects post data through the SNS API

[0892] The server uses the obtained authentication token to collect user post data through the SNS API. This data includes text, images, videos, etc., and is all stored on the server. The input of this step is the authentication token, and the output is the post data.

[0893] Step 3:

[0894] The server analyzes the collected submission data.

[0895] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine. NLP is used to analyze text data and identify user characteristics and interests. Image recognition technology is also used to extract information from image data. The emotion engine recognizes the user's emotions (positive, negative, etc.) from the posting data. The input of this step is the posting data, and the output is the user's characteristics, interests, and emotional information.

[0896] Step 4:

[0897] The server generates a character and story for the user.

[0898] Based on the analysis results, the server uses a generative AI model (e.g., GPT-4) to generate a character and story specifically for the user. The generated character reflects the user's characteristics, and the story incorporates the user's interests. The input for this step is the user's characteristics, interests, and emotional information, and the output is the generated character and story.

[0899] Step 5:

[0900] The server sends the generated character and story data to the user's device.

[0901] The server sends the generated character and story data to the user's device. The user's device displays a game interface based on the received data. The input for this step is the generated character data and story data, and the output is the game interface displayed on the user's device.

[0902] Step 6:

[0903] Users explore the virtual environment and engage in an interactive shopping experience

[0904] The user explores the virtual environment displayed on the device and controls the character. All exploration and actions within the virtual environment (e.g., selecting items, checking products) are sent to the server in real time. The server uses this operation data to provide an interactive shopping experience synchronized with other users. The input of this step is the user's operation data, and the output is a real-time synchronized shopping experience.

[0905] Step 7:

[0906] The server provides dynamic environment settings according to the user's emotions and actions.

[0907] The server dynamically adjusts the virtual environment settings (e.g., background music, visual effects) according to the emotions and real-time operation data analyzed from the user's posted data. The input of this step is the user's emotional information and operation data, and the output is the adjusted virtual environment settings.

[0908] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0909] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0910] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0911] [Third embodiment]

[0912] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0913] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0914] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0915] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0916] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0917] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0918] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0919] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0920] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0921] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0922] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0923] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0924] The system collects data posted by users on social networking service accounts, analyzes the data to identify the user's characteristics and interests, generates characters and stories based on the data, and finally transmits and displays them on the user's device. The system also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data.

[0925] Program processing

[0926] Data collection

[0927] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[0928] Data analysis

[0929] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[0930] Character and story generation

[0931] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[0932] Sending data and starting the game

[0933] The generated character and story data is sent from the server to the user's device, which receives this data and displays the game interface, where the user can view their avatar and begin playing the game.

[0934] Real-time play function

[0935] As users progress through the game, all actions (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. This allows users to cooperate with each other to complete missions.

[0936] Specific examples

[0937] Let's say a user links their Twitter account. If their daily tweets often contain topics like "cats" and "travel," the following characters and stories will be generated.

[0938] 1. The user connects their Twitter account and launches the application.

[0939] 2. The server collects past tweets via the Twitter API and analyzes the data using a generative AI model.

[0940] 3. As a result of the analysis, it is determined that this user is a "cat lover" and a "travel lover."

[0941] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role.

[0942] 5. The generated data is sent to the device and the game interface is displayed.

[0943] 6. Users experience a story in which their avatar goes on an adventure to save a cat.

[0944] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[0945] This allows users to have a unique RPG experience that reflects their own characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[0946] The processing flow will be explained below.

[0947] Step 1:

[0948] The user downloads and installs the application.

[0949] Step 2:

[0950] The user launches the application, enters their information on the account registration screen, and creates an account.

[0951] Step 3:

[0952] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[0953] Step 4:

[0954] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[0955] Step 5:

[0956] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[0957] Step 6:

[0958] The server then supplies the saved post data to a generative AI model, which uses natural language processing (NLP) technology to analyze the sentiment of posts, extract frequently occurring keywords, and identify users' areas of interest.

[0959] Step 7:

[0960] A generative AI model identifies user characteristics and interests, such as "cat lover" or "travel lover" from posts.

[0961] Step 8:

[0962] The server generates characters and stories based on the analysis results. Using a generative AI model, it creates a character specific to the user (e.g., a cat-loving adventurer) and generates a story based on the user's interests (e.g., an adventure to save cats).

[0963] Step 9:

[0964] The server transmits the generated character and story data to the user's terminal.

[0965] Step 10:

[0966] The device displays the game interface based on the received character and story data, allowing the user to confirm their avatar and begin the game.

[0967] Step 11:

[0968] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[0969] Step 12:

[0970] The server synchronizes the data of multiple users in real time, so that the movements and progress of other players are also reflected on the user's device.

[0971] Step 13:

[0972] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[0973] Step 14:

[0974] When a user finishes a game session, the terminal transmits current game progress data to the server.

[0975] Step 15:

[0976] The server saves your game progress, allowing you to resume from the same point the next time you play.

[0977] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data.

[0978] Example 1

[0979] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0980] Today's users often demand personalized content based on their individual interests and characteristics. However, content generation using social networking service (SNS) data to date has faced challenges due to its inability to fully reflect user characteristics. Furthermore, there is no established method for providing an environment in which multiple users can play simultaneously in real time while synchronizing each user's operation data. Furthermore, there are still issues with SNS account authentication, secure data storage, and understanding daily patterns.

[0981] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0982] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for authenticating the linking of social networking accounts, means for analyzing the acquired posted data using natural language processing technology, means for creating characters and stories using a generative AI model, and means for transmitting operation data from the user's device to the server and receiving operation data from other users at the device in real time. This enables the generation of personalized characters and stories that reflect users' characteristics, providing a synchronized gaming experience in real time. Furthermore, the authentication means enables secure social networking and ensures data security.

[0983] A "social networking service account" is identification information that allows a user to access a social networking service.

[0984] "Posted data" refers to information such as text, images, and videos that users make public on social networking services.

[0985] "Analysis" is the process of processing collected data using natural language processing techniques to identify user characteristics and interests.

[0986] "Features" refer to individual attributes such as user preferences and tendencies obtained from analyzed data.

[0987] "Interests" indicate topics or activities that a user is particularly interested in.

[0988] A "character" is a user's avatar or a representative of a player within a game.

[0989] A "story" is the narrative or scenario that the player progresses through within the game.

[0990] A "terminal" is a user's device (e.g., smartphone, tablet) for running an application.

[0991] "Data synchronization" is the process of making all users' operation data consistent in real time when multiple users are playing a game simultaneously.

[0992] "Authentication means" refers to a method for linking a user's social networking service account with an application.

[0993] A "generative AI model" is an artificial intelligence model that automatically generates characters and stories based on the user's characteristics and interests.

[0994] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[0995] "Operation data" is information relating to operations and actions performed by the user within the game.

[0996] "Data Storage Means" means a method or system for securely storing collected data.

[0997] "Daily patterns" are trends in daily activities and behaviors that are analyzed from data posted by users.

[0998] "Personalized" means customized based on the characteristics and interests of each individual user.

[0999] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that, and ultimately sends and displays them on the user's terminal.

[1000] This system provides an environment in which multiple users can play games simultaneously in real time, and is characterized by synchronizing the data of each user. The operation of the system is carried out as follows.

[1001] First, the user downloads and launches the application. This displays an account registration screen, where the user enters basic information such as their name and email address and links their social networking service account (e.g., Twitter, Facebook). Next, once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[1002] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[1003] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[1004] The generated character and story data is sent from the server to the user's device, which receives the data and displays the game interface, where the user can view their avatar and begin playing the game.

[1005] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. In this way, users can cooperate with each other to complete missions.

[1006] As a concrete example, let's assume that a user has linked their Twitter account. Suppose that the user's daily tweets often contain topics like "cats" and "travel." In this case, the following characters and stories will be generated.

[1007] 1. The user connects their Twitter account and launches the application.

[1008] 2. The server collects past tweet data through the Twitter API.

[1009] 3. The collected tweet data is analyzed using a generative AI model.

[1010] 4. The analysis results reveal that this user is a "cat lover" and a "travel lover."

[1011] 5. Based on the analysis, the server generates a story in which the cat plays an important role.

[1012] 6. The generated data is sent to the device and the game interface is displayed.

[1013] 7. Users experience a story in which their avatar goes on an adventure to save a cat.

[1014] 8. The server synchronizes all player data in real time, and everyone's actions are reflected in the game.

[1015] An example of a prompt you could give to a generative AI model would be:

[1016] "Identify the user's interests and characteristics from their posted data, and generate a unique character and story based on that data. For example, if the user loves cats and travels, generate a character with an adventure story about saving cats."

[1017] This allows users to have a unique RPG experience that reflects their characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[1018] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1019] Program processing flow

[1020] Step 1:

[1021] A user downloads an application and launches it on their device. As input, the user installs and launches the application. As output, an account registration screen is displayed.

[1022] Specifically, when the application is launched for the first time, the terminal generates a user interface and displays an account registration screen.

[1023] Step 2:

[1024] The user enters their account registration information and links their social media account. As input, the user enters their name, email address, and social media link information. As output, an authentication token is obtained once linking is complete.

[1025] Specifically, the terminal receives user input, displays an authentication screen for the social networking service, and transmits an authentication token to the server once authentication is complete.

[1026] Step 3:

[1027] The server uses the SNS API to collect user posted data. As input, it receives an authentication token. As output, the user posted data is collected and saved in a database.

[1028] Specifically, the server accesses the SNS API using the authentication token, obtains the user's past posting data, and stores the collected data in a database.

[1029] Step 4:

[1030] The server supplies the collected posting data to the generative AI model for analysis. Posting data obtained from the database is supplied as input to the generative AI model. User characteristics and interests are identified as output.

[1031] Specifically, the generative AI model analyzes the data using natural language processing (NLP) techniques to identify the user's characteristics (e.g., cat lover, travel lover) and interests. The server then stores the analysis results in a database.

[1032] Step 5:

[1033] The server generates characters and stories based on the analysis results. As input, it generates prompt sentences based on the analysis results and feeds them to the generative AI model. As output, personalized characters and stories are generated.

[1034] Specifically, the server generates a prompt, for example, "If the user loves cats and travels, please generate a character with an adventure story about saving cats," and inputs this into the generative AI model. The generative AI model generates the character and story and returns the generated data to the server, which stores it in a database.

[1035] Step 6:

[1036] The server sends the generated character and story data to the user's device. As input, it receives the generated character and story data. As output, it sends the data to the user's device and displays the game interface.

[1037] Specifically, the server sends the generated data to the user's device, which receives the data and displays the game interface, allowing the user to view their avatar and story.

[1038] Step 7:

[1039] When a user starts a game, action data is sent from the device to the server as the user progresses. As input, the user's action data (e.g., movement, combat, item collection) is acquired. As output, the action data is sent to the server.

[1040] Specifically, the terminal captures the user's operations and transmits the data to the server.

[1041] Step 8:

[1042] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other users on their devices. As input, it receives the operation data of all users. As output, it sends the synchronized data in real time to each user's device.

[1043] Specifically, the server processes all users' operation data in real time and sends the synchronized data to each user's device. The device then reflects the movements of other users, enabling real-time cooperative play.

[1044] (Application example 1)

[1045] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1046] In recent autonomous vehicles, there is a growing demand for entertainment systems that allow passengers to make effective use of their time while on board. However, existing systems lack the means to provide a personalized entertainment experience. Furthermore, when there are multiple passengers, there is a problem in that they lack the functionality to allow passengers to interact with each other in real time. Therefore, the present invention aims to provide a system that uses users' social networking service data to provide a personalized and interactive entertainment experience.

[1047] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1048] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' terminals, means for synchronizing data for each user so that multiple users can play simultaneously in real time, and means for operating within the mobile communication terminal to provide users with an interactive entertainment experience. This allows users on board to enjoy a personalized entertainment experience based on their interests and characteristics, and if there are multiple users, they can interact with each other in real time.

[1049] Definitions of important words

[1050] "User's Social Networking Service Account" means the credentials used by a User to post and interact on a Social Media Platform.

[1051] "Posted data" refers to content such as text, images, and videos posted by users to social networking services, as well as metadata.

[1052] "Analysis" is a data processing method that uses natural language processing technology and machine learning algorithms to identify user characteristics and interests from collected posted data.

[1053] A "character" is a virtual entity that is generated exclusively for a user, reflects the characteristics of the user, and plays an active role within the story.

[1054] A "story" is entertainment content that includes a sequence of events or scenarios generated based on the user's interests.

[1055] "Terminal" refers to an electronic device that can be directly operated by a user, including a smartphone, tablet, computer, in-vehicle display, etc.

[1056] "Real-time" means that data and operations are processed instantly, without any delay.

[1057] "Synchronizing data" means matching data with each other in order to keep the operation data and progress status of multiple users in the same state.

[1058] "Mobile communication terminal" refers to an electronic device with communication functions intended for use in a vehicle, including displays and infotainment systems in autonomous vehicles.

[1059] An "interactive entertainment experience" is an experience in which users can interact with the content and influence the story or character behavior through actions and choices.

[1060] "Server" means a computer system running computational resources and software for collecting, analyzing, storing, and synchronizing data.

[1061] "Natural language processing technology" refers to the technology used by computers to understand, interpret, and generate human language, and uses machine learning algorithms and deep learning models.

[1062] A "generative AI model" is an artificial intelligence model that can generate new data from input data using machine learning algorithms.

[1063] "Multiple users on board" refers to multiple users inside an autonomous vehicle, each of whom can operate the system using their own terminal.

[1064] The OAuth 2.0 authentication system is a widely used protocol for authentication and authorization on the web, allowing third parties to access a user's resources.

[1065] A "real-time synchronization system" is a system that instantly processes changing data and shares the latest information with all related devices.

[1066] "Infotainment system" refers to a system that integrates information and entertainment within a vehicle, including displays, audio, and internet connectivity.

[1067] MODE FOR CARRYING OUT THE INVENTION

[1068] 1. System Program

[1069] The system that realizes this application example consists of the following program modules:

[1070] 1. User authentication module

[1071] 2. Data Collection Module

[1072] 3. Data Analysis Module

[1073] 4. Character Story Generation Module

[1074] 5. Data transmission and game interface module

[1075] 6. Real-time synchronization module

[1076] 2. Natural language explanation

[1077] User Authentication Module

[1078] When a user logs into the vehicle's entertainment system, an authentication module is started. The authentication module authenticates the user's social networking service account (e.g., Twitter, Facebook) using the OAuth 2.0 authentication system. This authentication allows the server to obtain an authentication token, which can then be used to collect the user's posted data.

[1079] Data Collection Module

[1080] The server uses the authentication token to retrieve user posted data through the SNS API. This includes content such as text, images, and videos. The collected data is securely stored in a database system. This module can use database systems such as Amazon RDS.

[1081] Data Analysis Module

[1082] The collected data is analyzed on the server using natural language processing (NLP) technology, particularly NLP models such as BERT and GPT-4. These models extract user characteristics and interests from text and analyze user trends and patterns.

[1083] Character Story Generation Module

[1084] Based on the analysis results, a generative AI model (e.g., GPT-4) generates a character and story specifically for the user. During this process, character attributes and storylines that match the user's characteristics and interests are generated. The server generates this data and prepares it for later transmission to the device.

[1085] Data transmission and game interface module

[1086] The generated characters and story are sent from the server to the vehicle's entertainment system. The user operates the interactive game interface using the in-vehicle display or touch panel. This module is implemented as an Android / iOS application.

[1087] Real-time Synchronization Module

[1088] The server synchronizes the data of multiple users riding in the vehicle in real time. Using Firebase and WebSocket technology, all users' operation data is updated simultaneously, and the progress of other users is also reflected in real time. In this way, users can interact with each other.

[1089] 3. Examples and prompts

[1090] Specific examples

[1091] For example, if a user links their Facebook account and it is analyzed that they frequently post about "cats" and "music" while on board, a story such as "Adventuring in a musical land where cats live" will be generated for that user. A unique feature of this service is that it allows passengers to enjoy the adventure together.

[1092] Prompt Sentence Examples

[1093] Example of prompt input:

[1094] Analyze the user's posted data and generate a story based on their interest in "cats" and "music." The user is already logged into the vehicle's entertainment system.

[1095] This allows users to spend their travel time enjoying personalized entertainment in an autonomous vehicle.

[1096] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1097] Program processing steps

[1098] Step 1:

[1099] When a user logs into the vehicle's entertainment system, the terminal displays an authentication screen. The user logs in with a social networking service account (e.g., Twitter, Facebook). As input, the user's authentication information is required, and as output, an authentication token is obtained. The terminal sends this token to the server. The authentication system uses OAuth 2.0.

[1100] Step 2:

[1101] The server uses the obtained authentication token to collect user posted data through the SNS API. The authentication token is required as input, and the user posted data is obtained as output. The server securely stores this data in a database. Amazon RDS is used as the database system.

[1102] Step 3:

[1103] The server analyzes the collected posting data using natural language processing technology. Posting data is required as input, and user characteristics and interests are obtained as output. Specifically, NLP models such as BERT and GPT-4 are used to identify user interest patterns (e.g., "cat lover" or "music lover"). This allows the user's tendencies and patterns to be identified.

[1104] Step 4:

[1105] The server uses a generative AI model (e.g., GPT-4) based on the analysis results to generate a character and story specifically for the user. The analysis results are required as input, and the characters and story are generated as output. For example, a story based on the user's interests is created, such as "Adventure in a musical land where cats live."

[1106] Step 5:

[1107] The server sends the generated characters and stories to the terminal. The generated characters and stories are required as input, and the data is displayed on the terminal as output. The terminal provides the user with an interactive game interface, which is achieved using an Android / iOS application.

[1108] Step 6:

[1109] The terminal collects user operation data and sends it to the server. The user operation data is required as input and sent to the server as output. The server also synchronizes other users' operation data in real time.

[1110] Step 7:

[1111] The server synchronizes all user operation data in real time using Firebase and WebSocket technology. Each user's operation data is required as input, and the game state updated in real time is obtained as output. This allows multiple users to share an interactive entertainment experience simultaneously.

[1112] Through these steps, users can enjoy a personalized and interactive entertainment experience inside their autonomous vehicle.

[1113] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1114] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[1115] Program processing

[1116] Data collection

[1117] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[1118] Data analysis

[1119] The collected post data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) techniques to identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes user emotions (e.g., positive, negative, excited, calm, etc.) from the post.

[1120] Character and story generation

[1121] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[1122] Sending data and starting the game

[1123] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin playing the game. The music, background, and character dialogue are also dynamically adjusted according to the user's emotions.

[1124] Real-time play function

[1125] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[1126] Specific examples

[1127] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[1128] 1. The user connects their Twitter account and launches the application.

[1129] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[1130] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[1131] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[1132] 5. The generated data is sent to the device and the game interface is displayed.

[1133] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[1134] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[1135] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[1136] The processing flow will be explained below.

[1137] Step 1:

[1138] The user downloads and installs the application.

[1139] Step 2:

[1140] The user launches the application, enters their information on the account registration screen, and creates an account.

[1141] Step 3:

[1142] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[1143] Step 4:

[1144] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[1145] Step 5:

[1146] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[1147] Step 6:

[1148] The server provides the saved post data to the generative AI model and emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) technology to identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotion (e.g., positive, negative, excited, calm, etc.) from the post.

[1149] Step 7:

[1150] Based on the analysis results of the generative AI model and emotion engine, the server generates a character and story specifically for the user. For example, if a user posts many cat-related messages and the emotion engine recognizes positive emotions from those messages, the character will be given the attribute of "cat lover" and the story will include "a happy encounter with a cat."

[1151] Step 8:

[1152] The server transmits the generated character and story data to the user's terminal.

[1153] Step 9:

[1154] The device displays the game interface based on the character and story data received. The user can then confirm their avatar and begin the game. The in-game music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[1155] Step 10:

[1156] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[1157] Step 11:

[1158] The server synchronizes the data of multiple users in real time, so the movements and progress of other players are also reflected on the device.

[1159] Step 12:

[1160] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[1161] Step 13:

[1162] When a user finishes a game session, the terminal transmits current game progress data to the server.

[1163] Step 14:

[1164] The server saves your game progress, allowing you to resume from the same point the next time you play.

[1165] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data and emotions.

[1166] Example 2

[1167] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1168] Current game systems and entertainment services struggle to provide personalized experiences that fully reflect each user's individual characteristics, interests, and emotions. Furthermore, when multiple users play simultaneously in real time, smoothly integrating all users' data and enabling cooperative play is also problematic. Furthermore, it is currently difficult to effectively utilize the data posted by users on social networking services and dynamically reflect it in the characters and story of the game. Systems that can solve these challenges and provide greater user satisfaction are needed.

[1169] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1170] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for linking social networking service accounts using authentication means, means for analyzing the posted data acquired via an API using natural language processing technology, means for recognizing emotions from the analyzed posted data using an emotion engine, means for reflecting dynamic changes in the characters and stories based on the recognized emotions, means for dynamically adjusting music, backgrounds, character dialogue, etc. on the user's device based on the generated characters and stories, and means for transmitting actions performed by users as they progress through the game to the server and synchronizing operation data of other users in real time. This realizes a personalized gaming experience that reflects each user's individual characteristics and emotions, and enables smooth cooperative play in real time.

[1171] "User" means an individual or organization that has a social networking service account and uses this system.

[1172] A "server" is a computer system that collects, stores, and analyzes user data and transmits generated characters and stories to the user's device.

[1173] A "terminal" is a device (e.g., smartphone, tablet, computer) that a user uses to play a game.

[1174] A "Social Networking Service Account" is an individual account that a User has on an SNS platform.

[1175] "Posted data" refers to text, images, videos, and other content posted by users on social networking services.

[1176] "Collection methods" refer to the processes and tools used to obtain posted data through the API of a social networking service.

[1177] "Analysis means" refers to the process and tools for analyzing collected posted data using natural language processing technology and emotion recognition technology.

[1178] "Characteristics" refers to information such as individual personality, interests, and behavioral patterns identified from user posted data.

[1179] "Interests" are interests and preferences expressed by users through their posted data.

[1180] A "character" is an in-game avatar or virtual persona that is generated based on the user's characteristics and interests.

[1181] A "story" is a narrative scenario that is generated based on the user's characteristics and interests.

[1182] "Transmission means" refers to the process and tools that transfer the generated character and story data from the server to the user's terminal.

[1183] "Synchronization means" refers to the process and tools that transmit and receive operation data of multiple users between the server and the terminals in real time, ensuring consistency of in-game actions.

[1184] "Authentication Method" means the authentication technology and process for linking a social networking service account with the system.

[1185] "Natural language processing technology" refers to algorithms and methods for analyzing the text of posted data and understanding its meaning and sentiment.

[1186] The "emotion engine" is an algorithm and tool that recognizes user emotions from posted data and reflects them in the in-game experience.

[1187] "Dynamic adjustment" is the process of changing in-game music, backgrounds, character dialogue, and more in real time in response to the user's emotions and actions.

[1188] "Real-time cooperative play" is a play style in which multiple users play the game simultaneously, share data, and work together to complete missions.

[1189] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[1190] To implement this system, the following hardware and software are required:

[1191] Hardware

[1192] Server: A high-performance computer system that manages all processes of data collection, analysis, storage, and transmission.

[1193] Device: The device on which a user plays a game, such as a smartphone, tablet, or personal computer.

[1194] software

[1195] Generative AI model: An algorithm that uses natural language processing techniques to analyze user-submitted data and identify user characteristics and interests.

[1196] Emotion engine: Software that recognizes user emotions from posted data and reflects them in the game experience.

[1197] Data storage system: A database management system for securely storing collected data.

[1198] Real-time synchronization system: A network communication system for synchronizing data of multiple users in real time.

[1199] Specific operation of the system

[1200] Data collection

[1201] When a user downloads and launches the application, an account registration screen appears. Here, the user links their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token, which is used to collect the user's posted data via the SNS API. At this time, high security is ensured by using authentication methods.

[1202] Data analysis

[1203] The collected posting data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model uses natural language processing technology to analyze the posting data and identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotions (e.g., positive, negative, excited, calm, etc.) from the posts.

[1204] Character and story generation

[1205] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[1206] Sending data and starting the game

[1207] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin the game. In addition, the music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[1208] Real-time play function

[1209] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[1210] Specific examples

[1211] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[1212] 1. The user connects their Twitter account and launches the application.

[1213] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[1214] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[1215] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[1216] 5. The generated data is sent to the device and the game interface is displayed.

[1217] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[1218] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[1219] Prompt Sentence Examples

[1220] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[1221] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[1222] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1223] Program processing flow

[1224] Step 1: Download and launch the application

[1225] A user downloads an application from the Google Play Store or the App Store.

[1226] The device installs and launches the downloaded application.

[1227] Input: Application package

[1228] Output: Launched application interface

[1229] Step 2: Register an account and connect to social media

[1230] The device displays an account registration screen, and the user enters their SNS account information.

[1231] The server obtains an authentication token from the social media platform.

[1232] Input: User's social media account information

[1233] Output: Authentication token

[1234] Step 3: Data collection

[1235] The server uses the authentication token to collect user posting data from the SNS via API.

[1236] The server stores the collected data in a database.

[1237] Input: Authentication token

[1238] Output: User submitted data

[1239] Step 4: Data analysis

[1240] The server supplies the collected post data to the generative AI model and emotion engine.

[1241] The generative AI model uses natural language processing technology to analyze posted data and identify user characteristics and interests.

[1242] The emotion engine recognizes user emotions from posts.

[1243] Input: User submitted data

[1244] Output: User characteristics, interests, emotions

[1245] Step 5: Character and story generation

[1246] Based on the analysis results, the server generates a character and story specifically for the user.

[1247] It uses generative AI models to create characters and customized stories that reflect the user's characteristics.

[1248] Character behavior and story development are dynamically adjusted based on the recognition results of the emotion engine.

[1249] Input: User characteristics, interests, and emotions

[1250] Output: Character and story data

[1251] Step 6: Send data

[1252] The server transmits the generated character and story data to the user's terminal.

[1253] Input: Character and story data

[1254] Output: Data sent to the terminal

[1255] Step 7: Game Start and Dynamic Adjustments

[1256] The device uses the received data to display the game interface.

[1257] The user starts the game and experiences the characters and story.

[1258] The device dynamically adjusts the music, background, character dialogue, etc. according to the user's emotions.

[1259] Input: Received character and story data

[1260] Output: Dynamically adjusted game interface

[1261] Step 8: User actions and real-time synchronization

[1262] The actions that the user performs in the game (e.g., movement, combat, item collection) are sent from the device to the server.

[1263] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other players on the device.

[1264] Input: User action data

[1265] Output: Real-time synchronized game data

[1266] Specific operation example

[1267] Let's assume that a user has linked their Twitter account. Their daily tweets are mostly about "cats" and "travel," and most of the content is positive.

[1268] 1. The user links their account and launches the application (Steps 1 and 2).

[1269] 2. The server collects past tweets via the Twitter API and stores them in a database (Step 3).

[1270] 3. Generative AI models and emotion engines analyze this data to identify user characteristics, interests, and emotions (Step 4).

[1271] 4. Based on the analysis results, the server generates an adventure story for users who are "cat lovers" or "travel lovers," adding elements of happiness (Step 5).

[1272] 5. The generated data is sent to the device and the game interface is displayed (Step 6).

[1273] 6. The user experiences an adventure story in which their avatar rescues a cat, with music and backgrounds dynamically adjusted to match their emotions. (Step 7)

[1274] 7. If other players are also playing the game at the same time, all player data is synchronized in real time (Step 8).

[1275] Prompt Sentence Examples

[1276] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[1277] (Application example 2)

[1278] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1279] Many existing game systems that utilize social networking services collect users' interests and characteristics and generate characters and stories based on them. However, these systems are inadequate in terms of real-time play and emotional reflection, making it difficult to improve the quality of individual user experiences. Furthermore, there has been no system that can provide a shopping experience that reflects the user's emotions and interests. This has led to the problem of not being able to meet users' demands for a personalized shopping experience.

[1280] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1281] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to the user's terminal, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for generating a personalized virtual environment by reflecting emotions recognized from each user's posted data, and means for providing an interactive shopping experience based on exploration and actions within the personalized virtual environment, thereby enabling a personalized shopping experience that reflects the user's interests and emotions.

[1282] A "social networking service account" is personal authentication information and profile information that enables a user to access and use a social networking service.

[1283] "Posted data" refers to information such as messages, images, and videos posted by users on social networking services.

[1284] "Analysis" is the process of classifying, understanding, and evaluating collected data using natural language processing techniques and machine learning models.

[1285] "Features" refer to attributes and information that indicate a user's interests, behavior, preferences, etc.

[1286] A "character" is a depiction of a virtual person, animal, etc. that is generated based on the user's characteristics and interests.

[1287] A "story" is a virtual narrative progression or scenario generated based on the user's interests and emotions.

[1288] A "terminal" is an electronic device such as a smartphone, tablet, or PC that a user accesses.

[1289] "Synchronization" refers to the process of matching the data and actions of multiple users in real time.

[1290] "Reflecting emotions" means influencing the behavior of the virtual environment and characters based on emotional information analyzed from user-submitted data.

[1291] "Virtual environment" refers to a computer-generated virtual space or virtual store.

[1292] An "interactive shopping experience" refers to an experience in which users explore, select, and purchase products in a virtual environment while interacting with other users in real time.

[1293] The system for realizing this invention comprises the following means: First, a user downloads and launches an application using their own terminal (smartphone, tablet, PC, etc.). The roles of the server, terminal, and user are explained in detail below.

[1294] Server Roles

[1295] 1. Data Collection:

[1296] When a user links their account with a social networking service (e.g., Twitter or Instagram), the server collects the user's posting data via the SNS API, using an authentication token.

[1297] 2. Data Analysis:

[1298] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine, utilizing a generative AI model (e.g., GPT-4) to identify user characteristics (e.g., specific hobbies and interests) and emotional states (e.g., positive, negative).

[1299] 3. Character and Story Generation:

[1300] The server generates characters and stories based on the identified user's interests, characteristics, and emotional state. A generative AI model automatically generates characters and stories personalized to the user.

[1301] 4. Real-time data synchronization:

[1302] The server has the ability to synchronize the data of multiple users in real time, providing an environment where users can play simultaneously.

[1303] 5. Creating personalized virtual environments:

[1304] Generate a virtual environment (e.g., a virtual store) that reflects the user's emotions and characteristics. The environment is customized based on the products and themes that the user is interested in.

[1305] Device Role

[1306] 1. Data reception and interface provision:

[1307] The user's device receives the generated characters and story sent from the server and displays them through a user interface, which also includes adjusting the music and background according to the emotions.

[1308] 2. Sending operational data:

[1309] When a user operates the device, the operation data (e.g., movement, selection, etc.) is sent to the server and synchronized in real time.

[1310] 3. Exploration and interaction within virtual environments:

[1311] Users can explore the virtual environment on their device and are provided with interactive experiences such as searching for products and completing missions.

[1312] User Roles

[1313] 1. Account linking and data provision:

[1314] A user links a social networking service account to the application and provides posting data.

[1315] 2. Gameplay and Shopping Experience:

[1316] Users not only enjoy individually customized characters and stories, but also have a shopping experience while connecting with other users in real time.

[1317] Specific examples

[1318] For example, if a user links their Instagram account and their posts are mostly about fashion and travel and contain positive content, they will be offered a personalized shopping adventure through the following steps:

[1319] 1. The user installs the application and connects their Instagram account.

[1320] 2. The server collects post data through the Instagram API and analyzes it using a generative AI model and emotion engine.

[1321] 3. Based on the analysis results, we recognize that the user is interested in "fashion" and "travel" and has positive emotions.

[1322] 4. The generative AI model generates a virtual fashion and travel-themed store and characters specifically for the user.

[1323] 5. The generated data is sent to the user's device, and their personalized shopping adventure begins.

[1324] Prompt Sentence Examples

[1325] "User 1 has provided us with their social media account and posting data. This user's posts are mostly about fashion and travel, and contain a lot of positive emotions. Please create a personalized virtual store that reflects the user's interests and emotions, and create a character and story just for the user."

[1326] In this way, users can have a unique shopping experience that reflects their own characteristics and interests, and can enjoy interacting with other users in real time.

[1327] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1328] Step 1:

[1329] The user installs the application and connects their social media account.

[1330] A user downloads and launches an application onto a device such as a smartphone or tablet. Next, they link their social networking service (SNS) account on the account registration screen. For example, they link their account by entering their Twitter or Instagram credentials. The input for this step is their SNS account credentials, and the output is an authentication token.

[1331] Step 2:

[1332] The server collects post data through the SNS API

[1333] The server uses the obtained authentication token to collect user post data through the SNS API. This data includes text, images, videos, etc., and is all stored on the server. The input of this step is the authentication token, and the output is the post data.

[1334] Step 3:

[1335] The server analyzes the collected submission data.

[1336] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine. NLP is used to analyze text data and identify user characteristics and interests. Image recognition technology is also used to extract information from image data. The emotion engine recognizes the user's emotions (positive, negative, etc.) from the posting data. The input of this step is the posting data, and the output is the user's characteristics, interests, and emotional information.

[1337] Step 4:

[1338] The server generates a character and story for the user.

[1339] Based on the analysis results, the server uses a generative AI model (e.g., GPT-4) to generate a character and story specifically for the user. The generated character reflects the user's characteristics, and the story incorporates the user's interests. The input for this step is the user's characteristics, interests, and emotional information, and the output is the generated character and story.

[1340] Step 5:

[1341] The server sends the generated character and story data to the user's device.

[1342] The server sends the generated character and story data to the user's device. The user's device displays a game interface based on the received data. The input for this step is the generated character data and story data, and the output is the game interface displayed on the user's device.

[1343] Step 6:

[1344] Users explore the virtual environment and engage in an interactive shopping experience

[1345] The user explores the virtual environment displayed on the device and controls the character. All exploration and actions within the virtual environment (e.g., selecting items, checking products) are sent to the server in real time. The server uses this operation data to provide an interactive shopping experience synchronized with other users. The input of this step is the user's operation data, and the output is a real-time synchronized shopping experience.

[1346] Step 7:

[1347] The server provides dynamic environment settings according to the user's emotions and actions.

[1348] The server dynamically adjusts the virtual environment settings (e.g., background music, visual effects) according to the emotions and real-time operation data analyzed from the user's posted data. The input of this step is the user's emotional information and operation data, and the output is the adjusted virtual environment settings.

[1349] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1350] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1351] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1352] [Fourth embodiment]

[1353] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1354] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1355] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1356] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1357] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1358] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1359] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1360] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1361] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1362] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1363] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1364] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1365] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1366] The system collects data posted by users on social networking service accounts, analyzes the data to identify the user's characteristics and interests, generates characters and stories based on the data, and finally transmits and displays them on the user's device. The system also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data.

[1367] Program processing

[1368] Data collection

[1369] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[1370] Data analysis

[1371] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[1372] Character and story generation

[1373] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[1374] Sending data and starting the game

[1375] The generated character and story data is sent from the server to the user's device, which receives this data and displays the game interface, where the user can view their avatar and begin playing the game.

[1376] Real-time play function

[1377] As users progress through the game, all actions (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. This allows users to cooperate with each other to complete missions.

[1378] Specific examples

[1379] Let's say a user links their Twitter account. If their daily tweets often contain topics like "cats" and "travel," the following characters and stories will be generated.

[1380] 1. The user connects their Twitter account and launches the application.

[1381] 2. The server collects past tweets via the Twitter API and analyzes the data using a generative AI model.

[1382] 3. As a result of the analysis, it is determined that this user is a "cat lover" and a "travel lover."

[1383] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role.

[1384] 5. The generated data is sent to the device and the game interface is displayed.

[1385] 6. Users experience a story in which their avatar goes on an adventure to save a cat.

[1386] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[1387] This allows users to have a unique RPG experience that reflects their own characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[1388] The processing flow will be explained below.

[1389] Step 1:

[1390] The user downloads and installs the application.

[1391] Step 2:

[1392] The user launches the application, enters their information on the account registration screen, and creates an account.

[1393] Step 3:

[1394] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[1395] Step 4:

[1396] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[1397] Step 5:

[1398] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[1399] Step 6:

[1400] The server then supplies the saved post data to a generative AI model, which uses natural language processing (NLP) technology to analyze the sentiment of posts, extract frequently occurring keywords, and identify users' areas of interest.

[1401] Step 7:

[1402] A generative AI model identifies user characteristics and interests, such as "cat lover" or "travel lover" from posts.

[1403] Step 8:

[1404] The server generates characters and stories based on the analysis results. Using a generative AI model, it creates a character specific to the user (e.g., a cat-loving adventurer) and generates a story based on the user's interests (e.g., an adventure to save cats).

[1405] Step 9:

[1406] The server transmits the generated character and story data to the user's terminal.

[1407] Step 10:

[1408] The device displays the game interface based on the received character and story data, allowing the user to confirm their avatar and begin the game.

[1409] Step 11:

[1410] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[1411] Step 12:

[1412] The server synchronizes the data of multiple users in real time, so that the movements and progress of other players are also reflected on the user's device.

[1413] Step 13:

[1414] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[1415] Step 14:

[1416] When a user finishes a game session, the terminal transmits current game progress data to the server.

[1417] Step 15:

[1418] The server saves your game progress, allowing you to resume from the same point the next time you play.

[1419] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data.

[1420] Example 1

[1421] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1422] Today's users often demand personalized content based on their individual interests and characteristics. However, content generation using social networking service (SNS) data to date has faced challenges due to its inability to fully reflect user characteristics. Furthermore, there is no established method for providing an environment in which multiple users can play simultaneously in real time while synchronizing each user's operation data. Furthermore, there are still issues with SNS account authentication, secure data storage, and understanding daily patterns.

[1423] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1424] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for authenticating the linking of social networking accounts, means for analyzing the acquired posted data using natural language processing technology, means for creating characters and stories using a generative AI model, and means for transmitting operation data from the user's device to the server and receiving operation data from other users at the device in real time. This enables the generation of personalized characters and stories that reflect users' characteristics, providing a synchronized gaming experience in real time. Furthermore, the authentication means enables secure social networking and ensures data security.

[1425] A "social networking service account" is identification information that allows a user to access a social networking service.

[1426] "Posted data" refers to information such as text, images, and videos that users make public on social networking services.

[1427] "Analysis" is the process of processing collected data using natural language processing techniques to identify user characteristics and interests.

[1428] "Features" refer to individual attributes such as user preferences and tendencies obtained from analyzed data.

[1429] "Interests" indicate topics or activities that a user is particularly interested in.

[1430] A "character" is a user's avatar or a representative of a player within a game.

[1431] A "story" is the narrative or scenario that the player progresses through within the game.

[1432] A "terminal" is a user's device (e.g., smartphone, tablet) for running an application.

[1433] "Data synchronization" is the process of making all users' operation data consistent in real time when multiple users are playing a game simultaneously.

[1434] "Authentication means" refers to a method for linking a user's social networking service account with an application.

[1435] A "generative AI model" is an artificial intelligence model that automatically generates characters and stories based on the user's characteristics and interests.

[1436] "Natural language processing technology" is a technology that enables computers to understand and analyze human language.

[1437] "Operation data" is information relating to operations and actions performed by the user within the game.

[1438] "Data Storage Means" means a method or system for securely storing collected data.

[1439] "Daily patterns" are trends in daily activities and behaviors that are analyzed from data posted by users.

[1440] "Personalized" means customized based on the characteristics and interests of each individual user.

[1441] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that, and ultimately sends and displays them on the user's terminal.

[1442] This system provides an environment in which multiple users can play games simultaneously in real time, and is characterized by synchronizing the data of each user. The operation of the system is carried out as follows.

[1443] First, the user downloads and launches the application. This displays an account registration screen, where the user enters basic information such as their name and email address and links their social networking service account (e.g., Twitter, Facebook). Next, once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[1444] The collected posting data is stored on a server, which then supplies this data to a generative AI model, which uses natural language processing (NLP) techniques to analyze the posting data and identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading).

[1445] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. For example, if a user posts a lot about cats, the character will be given the attribute "cat lover" and the story will include an "adventure to rescue cats."

[1446] The generated character and story data is sent from the server to the user's device, which receives the data and displays the game interface, where the user can view their avatar and begin playing the game.

[1447] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other users are also reflected on the device. In this way, users can cooperate with each other to complete missions.

[1448] As a concrete example, let's assume that a user has linked their Twitter account. Suppose that the user's daily tweets often contain topics like "cats" and "travel." In this case, the following characters and stories will be generated.

[1449] 1. The user connects their Twitter account and launches the application.

[1450] 2. The server collects past tweet data through the Twitter API.

[1451] 3. The collected tweet data is analyzed using a generative AI model.

[1452] 4. The analysis results reveal that this user is a "cat lover" and a "travel lover."

[1453] 5. Based on the analysis, the server generates a story in which the cat plays an important role.

[1454] 6. The generated data is sent to the device and the game interface is displayed.

[1455] 7. Users experience a story in which their avatar goes on an adventure to save a cat.

[1456] 8. The server synchronizes all player data in real time, and everyone's actions are reflected in the game.

[1457] An example of a prompt you could give to a generative AI model would be:

[1458] "Identify the user's interests and characteristics from their posted data, and generate a unique character and story based on that data. For example, if the user loves cats and travels, generate a character with an adventure story about saving cats."

[1459] This allows users to have a unique RPG experience that reflects their characteristics and interests, and they can also enjoy the game cooperatively with other players in real time.

[1460] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1461] Program processing flow

[1462] Step 1:

[1463] A user downloads an application and launches it on their device. As input, the user installs and launches the application. As output, an account registration screen is displayed.

[1464] Specifically, when the application is launched for the first time, the terminal generates a user interface and displays an account registration screen.

[1465] Step 2:

[1466] The user enters their account registration information and links their social media account. As input, the user enters their name, email address, and social media link information. As output, an authentication token is obtained once linking is complete.

[1467] Specifically, the terminal receives user input, displays an authentication screen for the social networking service, and transmits an authentication token to the server once authentication is complete.

[1468] Step 3:

[1469] The server uses the SNS API to collect user posted data. As input, it receives an authentication token. As output, the user posted data is collected and saved in a database.

[1470] Specifically, the server accesses the SNS API using the authentication token, obtains the user's past posting data, and stores the collected data in a database.

[1471] Step 4:

[1472] The server supplies the collected posting data to the generative AI model for analysis. Posting data obtained from the database is supplied as input to the generative AI model. User characteristics and interests are identified as output.

[1473] Specifically, the generative AI model analyzes the data using natural language processing (NLP) techniques to identify the user's characteristics (e.g., cat lover, travel lover) and interests. The server then stores the analysis results in a database.

[1474] Step 5:

[1475] The server generates characters and stories based on the analysis results. As input, it generates prompt sentences based on the analysis results and feeds them to the generative AI model. As output, personalized characters and stories are generated.

[1476] Specifically, the server generates a prompt, for example, "If the user loves cats and travels, please generate a character with an adventure story about saving cats," and inputs this into the generative AI model. The generative AI model generates the character and story and returns the generated data to the server, which stores it in a database.

[1477] Step 6:

[1478] The server sends the generated character and story data to the user's device. As input, it receives the generated character and story data. As output, it sends the data to the user's device and displays the game interface.

[1479] Specifically, the server sends the generated data to the user's device, which receives the data and displays the game interface, allowing the user to view their avatar and story.

[1480] Step 7:

[1481] When a user starts a game, action data is sent from the device to the server as the user progresses. As input, the user's action data (e.g., movement, combat, item collection) is acquired. As output, the action data is sent to the server.

[1482] Specifically, the terminal captures the user's operations and transmits the data to the server.

[1483] Step 8:

[1484] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other users on their devices. As input, it receives the operation data of all users. As output, it sends the synchronized data in real time to each user's device.

[1485] Specifically, the server processes all users' operation data in real time and sends the synchronized data to each user's device. The device then reflects the movements of other users, enabling real-time cooperative play.

[1486] (Application example 1)

[1487] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] In recent autonomous vehicles, there is a growing demand for entertainment systems that allow passengers to make effective use of their time while on board. However, existing systems lack the means to provide a personalized entertainment experience. Furthermore, when there are multiple passengers, there is a problem in that they lack the functionality to allow passengers to interact with each other in real time. Therefore, the present invention aims to provide a system that uses users' social networking service data to provide a personalized and interactive entertainment experience.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1490] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify users' characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' terminals, means for synchronizing data for each user so that multiple users can play simultaneously in real time, and means for operating within the mobile communication terminal to provide users with an interactive entertainment experience. This allows users on board to enjoy a personalized entertainment experience based on their interests and characteristics, and if there are multiple users, they can interact with each other in real time.

[1491] Definitions of important words

[1492] "User's Social Networking Service Account" means the credentials used by a User to post and interact on a Social Media Platform.

[1493] "Posted data" refers to content such as text, images, and videos posted by users to social networking services, as well as metadata.

[1494] "Analysis" is a data processing method that uses natural language processing technology and machine learning algorithms to identify user characteristics and interests from collected posted data.

[1495] A "character" is a virtual entity that is generated exclusively for a user, reflects the characteristics of the user, and plays an active role within the story.

[1496] A "story" is entertainment content that includes a sequence of events or scenarios generated based on the user's interests.

[1497] "Terminal" refers to an electronic device that can be directly operated by a user, including a smartphone, tablet, computer, in-vehicle display, etc.

[1498] "Real-time" means that data and operations are processed instantly, without any delay.

[1499] "Synchronizing data" means matching data with each other in order to keep the operation data and progress status of multiple users in the same state.

[1500] "Mobile communication terminal" refers to an electronic device with communication functions intended for use in a vehicle, including displays and infotainment systems in autonomous vehicles.

[1501] An "interactive entertainment experience" is an experience in which users can interact with the content and influence the story or character behavior through actions and choices.

[1502] "Server" means a computer system running computational resources and software for collecting, analyzing, storing, and synchronizing data.

[1503] "Natural language processing technology" refers to the technology used by computers to understand, interpret, and generate human language, and uses machine learning algorithms and deep learning models.

[1504] A "generative AI model" is an artificial intelligence model that can generate new data from input data using machine learning algorithms.

[1505] "Multiple users on board" refers to multiple users inside an autonomous vehicle, each of whom can operate the system using their own terminal.

[1506] The OAuth 2.0 authentication system is a widely used protocol for authentication and authorization on the web, allowing third parties to access a user's resources.

[1507] A "real-time synchronization system" is a system that instantly processes changing data and shares the latest information with all related devices.

[1508] "Infotainment system" refers to a system that integrates information and entertainment within a vehicle, including displays, audio, and internet connectivity.

[1509] MODE FOR CARRYING OUT THE INVENTION

[1510] 1. System Program

[1511] The system that realizes this application example consists of the following program modules:

[1512] 1. User authentication module

[1513] 2. Data Collection Module

[1514] 3. Data Analysis Module

[1515] 4. Character Story Generation Module

[1516] 5. Data transmission and game interface module

[1517] 6. Real-time synchronization module

[1518] 2. Natural language explanation

[1519] User Authentication Module

[1520] When a user logs into the vehicle's entertainment system, an authentication module is started. The authentication module authenticates the user's social networking service account (e.g., Twitter, Facebook) using the OAuth 2.0 authentication system. This authentication allows the server to obtain an authentication token, which can then be used to collect the user's posted data.

[1521] Data Collection Module

[1522] The server uses the authentication token to retrieve user posted data through the SNS API. This includes content such as text, images, and videos. The collected data is securely stored in a database system. This module can use database systems such as Amazon RDS.

[1523] Data Analysis Module

[1524] The collected data is analyzed on the server using natural language processing (NLP) technology, particularly NLP models such as BERT and GPT-4. These models extract user characteristics and interests from text and analyze user trends and patterns.

[1525] Character Story Generation Module

[1526] Based on the analysis results, a generative AI model (e.g., GPT-4) generates a character and story specifically for the user. During this process, character attributes and storylines that match the user's characteristics and interests are generated. The server generates this data and prepares it for later transmission to the device.

[1527] Data transmission and game interface module

[1528] The generated characters and story are sent from the server to the vehicle's entertainment system. The user operates the interactive game interface using the in-vehicle display or touch panel. This module is implemented as an Android / iOS application.

[1529] Real-time Synchronization Module

[1530] The server synchronizes the data of multiple users riding in the vehicle in real time. Using Firebase and WebSocket technology, all users' operation data is updated simultaneously, and the progress of other users is also reflected in real time. In this way, users can interact with each other.

[1531] 3. Examples and prompts

[1532] Specific examples

[1533] For example, if a user links their Facebook account and it is analyzed that they frequently post about "cats" and "music" while on board, a story such as "Adventuring in a musical land where cats live" will be generated for that user. A unique feature of this service is that it allows passengers to enjoy the adventure together.

[1534] Prompt Sentence Examples

[1535] Example of prompt input:

[1536] Analyze the user's posted data and generate a story based on their interest in "cats" and "music." The user is already logged into the vehicle's entertainment system.

[1537] This allows users to spend their travel time enjoying personalized entertainment in an autonomous vehicle.

[1538] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1539] Program processing steps

[1540] Step 1:

[1541] When a user logs into the vehicle's entertainment system, the terminal displays an authentication screen. The user logs in with a social networking service account (e.g., Twitter, Facebook). As input, the user's authentication information is required, and as output, an authentication token is obtained. The terminal sends this token to the server. The authentication system uses OAuth 2.0.

[1542] Step 2:

[1543] The server uses the obtained authentication token to collect user posted data through the SNS API. The authentication token is required as input, and the user posted data is obtained as output. The server securely stores this data in a database. Amazon RDS is used as the database system.

[1544] Step 3:

[1545] The server analyzes the collected posting data using natural language processing technology. Posting data is required as input, and user characteristics and interests are obtained as output. Specifically, NLP models such as BERT and GPT-4 are used to identify user interest patterns (e.g., "cat lover" or "music lover"). This allows the user's tendencies and patterns to be identified.

[1546] Step 4:

[1547] The server uses a generative AI model (e.g., GPT-4) based on the analysis results to generate a character and story specifically for the user. The analysis results are required as input, and the characters and story are generated as output. For example, a story based on the user's interests is created, such as "Adventure in a musical land where cats live."

[1548] Step 5:

[1549] The server sends the generated characters and stories to the terminal. The generated characters and stories are required as input, and the data is displayed on the terminal as output. The terminal provides the user with an interactive game interface, which is achieved using an Android / iOS application.

[1550] Step 6:

[1551] The terminal collects user operation data and sends it to the server. The user operation data is required as input and sent to the server as output. The server also synchronizes other users' operation data in real time.

[1552] Step 7:

[1553] The server synchronizes all user operation data in real time using Firebase and WebSocket technology. Each user's operation data is required as input, and the game state updated in real time is obtained as output. This allows multiple users to share an interactive entertainment experience simultaneously.

[1554] Through these steps, users can enjoy a personalized and interactive entertainment experience inside their autonomous vehicle.

[1555] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1556] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[1557] Program processing

[1558] Data collection

[1559] When a user downloads and launches the application, an account registration screen appears. Here, the user must link their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token and uses this token to collect the user's posted data via the SNS API.

[1560] Data analysis

[1561] The collected post data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) techniques to identify user characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes user emotions (e.g., positive, negative, excited, calm, etc.) from the post.

[1562] Character and story generation

[1563] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[1564] Sending data and starting the game

[1565] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin playing the game. The music, background, and character dialogue are also dynamically adjusted according to the user's emotions.

[1566] Real-time play function

[1567] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[1568] Specific examples

[1569] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[1570] 1. The user connects their Twitter account and launches the application.

[1571] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[1572] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[1573] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[1574] 5. The generated data is sent to the device and the game interface is displayed.

[1575] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[1576] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[1577] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[1578] The processing flow will be explained below.

[1579] Step 1:

[1580] The user downloads and installs the application.

[1581] Step 2:

[1582] The user launches the application, enters their information on the account registration screen, and creates an account.

[1583] Step 3:

[1584] A user links their social networking service account (e.g., Twitter, Facebook). An authentication screen for linking is displayed, and the user enters their authentication information.

[1585] Step 4:

[1586] The server obtains the authentication token and uses the SNS API to collect the user's posted data. Specifically, it uses the Twitter API to retrieve past tweets.

[1587] Step 5:

[1588] The server securely stores the collected post data, and metadata such as post content, date and time, and sentiment analysis are stored in the database.

[1589] Step 6:

[1590] The server provides the saved post data to the generative AI model and emotion engine. The generative AI model analyzes the post data using natural language processing (NLP) technology to identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotion (e.g., positive, negative, excited, calm, etc.) from the post.

[1591] Step 7:

[1592] Based on the analysis results of the generative AI model and emotion engine, the server generates a character and story specifically for the user. For example, if a user posts many cat-related messages and the emotion engine recognizes positive emotions from those messages, the character will be given the attribute of "cat lover" and the story will include "a happy encounter with a cat."

[1593] Step 8:

[1594] The server transmits the generated character and story data to the user's terminal.

[1595] Step 9:

[1596] The device displays the game interface based on the character and story data received. The user can then confirm their avatar and begin the game. The in-game music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[1597] Step 10:

[1598] All actions performed by the user as they progress through the game (e.g., movement, combat, item collection, etc.) are transmitted from the device to the server.

[1599] Step 11:

[1600] The server synchronizes the data of multiple users in real time, so the movements and progress of other players are also reflected on the device.

[1601] Step 12:

[1602] The device receives synchronized data from the server and updates the game screen, allowing users to enjoy the game while cooperating or competing with other players.

[1603] Step 13:

[1604] When a user finishes a game session, the terminal transmits current game progress data to the server.

[1605] Step 14:

[1606] The server saves your game progress, allowing you to resume from the same point the next time you play.

[1607] The above are the specific processing steps for providing an RPG experience based on the user's SNS posting data and emotions.

[1608] Example 2

[1609] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1610] Current game systems and entertainment services struggle to provide personalized experiences that fully reflect each user's individual characteristics, interests, and emotions. Furthermore, when multiple users play simultaneously in real time, smoothly integrating all users' data and enabling cooperative play is also problematic. Furthermore, it is currently difficult to effectively utilize the data posted by users on social networking services and dynamically reflect it in the characters and story of the game. Systems that can solve these challenges and provide greater user satisfaction are needed.

[1611] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1612] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to users' devices, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for linking social networking service accounts using authentication means, means for analyzing the posted data acquired via an API using natural language processing technology, means for recognizing emotions from the analyzed posted data using an emotion engine, means for reflecting dynamic changes in the characters and stories based on the recognized emotions, means for dynamically adjusting music, backgrounds, character dialogue, etc. on the user's device based on the generated characters and stories, and means for transmitting actions performed by users as they progress through the game to the server and synchronizing operation data of other users in real time. This realizes a personalized gaming experience that reflects each user's individual characteristics and emotions, and enables smooth cooperative play in real time.

[1613] "User" means an individual or organization that has a social networking service account and uses this system.

[1614] A "server" is a computer system that collects, stores, and analyzes user data and transmits generated characters and stories to the user's device.

[1615] A "terminal" is a device (e.g., smartphone, tablet, computer) that a user uses to play a game.

[1616] A "Social Networking Service Account" is an individual account that a User has on an SNS platform.

[1617] "Posted data" refers to text, images, videos, and other content posted by users on social networking services.

[1618] "Collection methods" refer to the processes and tools used to obtain posted data through the API of a social networking service.

[1619] "Analysis means" refers to the process and tools for analyzing collected posted data using natural language processing technology and emotion recognition technology.

[1620] "Characteristics" refers to information such as individual personality, interests, and behavioral patterns identified from user posted data.

[1621] "Interests" are interests and preferences expressed by users through their posted data.

[1622] A "character" is an in-game avatar or virtual persona that is generated based on the user's characteristics and interests.

[1623] A "story" is a narrative scenario that is generated based on the user's characteristics and interests.

[1624] "Transmission means" refers to the process and tools that transfer the generated character and story data from the server to the user's terminal.

[1625] "Synchronization means" refers to the process and tools that transmit and receive operation data of multiple users between the server and the terminals in real time, ensuring consistency of in-game actions.

[1626] "Authentication Method" means the authentication technology and process for linking a social networking service account with the system.

[1627] "Natural language processing technology" refers to algorithms and methods for analyzing the text of posted data and understanding its meaning and sentiment.

[1628] The "emotion engine" is an algorithm and tool that recognizes user emotions from posted data and reflects them in the in-game experience.

[1629] "Dynamic adjustment" is the process of changing in-game music, backgrounds, character dialogue, and more in real time in response to the user's emotions and actions.

[1630] "Real-time cooperative play" is a play style in which multiple users play the game simultaneously, share data, and work together to complete missions.

[1631] The present invention is a system that collects posted data from users' social networking service accounts, analyzes that data to identify the user's characteristics and interests, generates characters and stories based on that data, and ultimately transmits and displays them on the user's device. It also provides an environment in which multiple users can play games simultaneously in real time, and synchronizes each user's data. Furthermore, by combining it with an emotion engine, it is possible to recognize emotions from user posts and reflect them in the game experience.

[1632] To implement this system, the following hardware and software are required:

[1633] Hardware

[1634] Server: A high-performance computer system that manages all processes of data collection, analysis, storage, and transmission.

[1635] Device: The device on which a user plays a game, such as a smartphone, tablet, or personal computer.

[1636] software

[1637] Generative AI model: An algorithm that uses natural language processing techniques to analyze user-submitted data and identify user characteristics and interests.

[1638] Emotion engine: Software that recognizes user emotions from posted data and reflects them in the game experience.

[1639] Data storage system: A database management system for securely storing collected data.

[1640] Real-time synchronization system: A network communication system for synchronizing data of multiple users in real time.

[1641] Specific operation of the system

[1642] Data collection

[1643] When a user downloads and launches the application, an account registration screen appears. Here, the user links their social networking service account (e.g., Twitter, Facebook). Once the linking is complete, the server obtains an authentication token, which is used to collect the user's posted data via the SNS API. At this time, high security is ensured by using authentication methods.

[1644] Data analysis

[1645] The collected posting data is stored on a server. The server then supplies this data to a generative AI model and an emotion engine. The generative AI model uses natural language processing technology to analyze the posting data and identify the user's characteristics (e.g., cat lover, travel lover) and interests (e.g., outdoors, reading). The emotion engine recognizes the user's emotions (e.g., positive, negative, excited, calm, etc.) from the posts.

[1646] Character and story generation

[1647] Based on the analysis results, the server generates a character and story specifically for the user. The generative AI model creates a personalized character that reflects the user's characteristics, and the story is similarly customized based on the user's interests. The character's behavior and story development also dynamically change based on the emotion engine's recognition results. For example, if a user posts many times about cats and the emotion engine recognizes positive emotions from those posts, the character will be given the attribute of "cat lover," and the story will include "a happy encounter with a cat."

[1648] Sending data and starting the game

[1649] The generated character and story data is sent from the server to the user's device. The device receives this data and displays the game interface, where the user can confirm their avatar and begin the game. In addition, the music, background, and character dialogue are dynamically adjusted according to the user's emotions.

[1650] Real-time play function

[1651] All actions taken by users as they progress through the game (e.g., movement, combat, item collection, etc.) are sent from the device to the server. The server synchronizes the data of multiple users in real time, and the movements and progress of other players are also reflected on the device. This allows users to cooperate with each other to complete missions.

[1652] Specific examples

[1653] Let's say a user connects their Twitter account. If their daily tweets are mostly about "cats" and "travel," and mostly positive content, the following characters and stories will be generated.

[1654] 1. The user connects their Twitter account and launches the application.

[1655] 2. The server collects past tweets through the Twitter API and analyzes the data using a generative AI model and sentiment engine.

[1656] 3. The analysis results reveal that this user is a "cat lover" and a "travel lover," and has many positive emotions.

[1657] 4. Based on the analysis results, the server generates an adventure story in which the cat plays an important role, and adds a happy essence to the story based on the results of the emotion engine.

[1658] 5. The generated data is sent to the device and the game interface is displayed.

[1659] 6. The user experiences a story in which their avatar goes on a "happy adventure to save a cat." The music and background are adjusted according to the user's emotions.

[1660] 7. If other players are also playing the game, the server will synchronize all player data in real time and everyone's actions will be reflected in the game.

[1661] Prompt Sentence Examples

[1662] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[1663] This allows users to have a unique RPG experience that reflects their own characteristics and emotions, and they can also enjoy the game cooperatively with other players in real time.

[1664] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1665] Program processing flow

[1666] Step 1: Download and launch the application

[1667] A user downloads an application from the Google Play Store or the App Store.

[1668] The device installs and launches the downloaded application.

[1669] Input: Application package

[1670] Output: Launched application interface

[1671] Step 2: Register an account and connect to social media

[1672] The device displays an account registration screen, and the user enters their SNS account information.

[1673] The server obtains an authentication token from the social media platform.

[1674] Input: User's social media account information

[1675] Output: Authentication token

[1676] Step 3: Data collection

[1677] The server uses the authentication token to collect user posting data from the SNS via API.

[1678] The server stores the collected data in a database.

[1679] Input: Authentication token

[1680] Output: User submitted data

[1681] Step 4: Data analysis

[1682] The server supplies the collected post data to the generative AI model and emotion engine.

[1683] The generative AI model uses natural language processing technology to analyze posted data and identify user characteristics and interests.

[1684] The emotion engine recognizes user emotions from posts.

[1685] Input: User submitted data

[1686] Output: User characteristics, interests, emotions

[1687] Step 5: Character and story generation

[1688] Based on the analysis results, the server generates a character and story specifically for the user.

[1689] It uses generative AI models to create characters and customized stories that reflect the user's characteristics.

[1690] Character behavior and story development are dynamically adjusted based on the recognition results of the emotion engine.

[1691] Input: User characteristics, interests, and emotions

[1692] Output: Character and story data

[1693] Step 6: Send data

[1694] The server transmits the generated character and story data to the user's terminal.

[1695] Input: Character and story data

[1696] Output: Data sent to the terminal

[1697] Step 7: Game Start and Dynamic Adjustments

[1698] The device uses the received data to display the game interface.

[1699] The user starts the game and experiences the characters and story.

[1700] The device dynamically adjusts the music, background, character dialogue, etc. according to the user's emotions.

[1701] Input: Received character and story data

[1702] Output: Dynamically adjusted game interface

[1703] Step 8: User actions and real-time synchronization

[1704] The actions that the user performs in the game (e.g., movement, combat, item collection) are sent from the device to the server.

[1705] The server synchronizes the data of multiple users in real time and reflects the movements and progress of other players on the device.

[1706] Input: User action data

[1707] Output: Real-time synchronized game data

[1708] Specific operation example

[1709] Let's assume that a user has linked their Twitter account. Their daily tweets are mostly about "cats" and "travel," and most of the content is positive.

[1710] 1. The user links their account and launches the application (Steps 1 and 2).

[1711] 2. The server collects past tweets via the Twitter API and stores them in a database (Step 3).

[1712] 3. Generative AI models and emotion engines analyze this data to identify user characteristics, interests, and emotions (Step 4).

[1713] 4. Based on the analysis results, the server generates an adventure story for users who are "cat lovers" or "travel lovers," adding elements of happiness (Step 5).

[1714] 5. The generated data is sent to the device and the game interface is displayed (Step 6).

[1715] 6. The user experiences an adventure story in which their avatar rescues a cat, with music and backgrounds dynamically adjusted to match their emotions. (Step 7)

[1716] 7. If other players are also playing the game at the same time, all player data is synchronized in real time (Step 8).

[1717] Prompt Sentence Examples

[1718] "I often post cat pictures and travel photos on Twitter. I tweet a lot of positive things, but what kind of characters and stories do they create?"

[1719] (Application example 2)

[1720] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1721] Many existing game systems that utilize social networking services collect users' interests and characteristics and generate characters and stories based on them. However, these systems are inadequate in terms of real-time play and emotional reflection, making it difficult to improve the quality of individual user experiences. Furthermore, there has been no system that can provide a shopping experience that reflects the user's emotions and interests. This has led to the problem of not being able to meet users' demands for a personalized shopping experience.

[1722] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1723] In this invention, the server includes means for collecting posted data from users' social networking service accounts, means for analyzing the collected posted data to identify user characteristics and interests, means for generating characters and stories based on the identified characteristics and interests, means for transmitting the generated characters and stories to the user's terminal, means for synchronizing each user's data so that multiple users can play simultaneously in real time, means for generating a personalized virtual environment by reflecting emotions recognized from each user's posted data, and means for providing an interactive shopping experience based on exploration and actions within the personalized virtual environment, thereby enabling a personalized shopping experience that reflects the user's interests and emotions.

[1724] A "social networking service account" is personal authentication information and profile information that enables a user to access and use a social networking service.

[1725] "Posted data" refers to information such as messages, images, and videos posted by users on social networking services.

[1726] "Analysis" is the process of classifying, understanding, and evaluating collected data using natural language processing techniques and machine learning models.

[1727] "Features" refer to attributes and information that indicate a user's interests, behavior, preferences, etc.

[1728] A "character" is a depiction of a virtual person, animal, etc. that is generated based on the user's characteristics and interests.

[1729] A "story" is a virtual narrative progression or scenario generated based on the user's interests and emotions.

[1730] A "terminal" is an electronic device such as a smartphone, tablet, or PC that a user accesses.

[1731] "Synchronization" refers to the process of matching the data and actions of multiple users in real time.

[1732] "Reflecting emotions" means influencing the behavior of the virtual environment and characters based on emotional information analyzed from user-submitted data.

[1733] "Virtual environment" refers to a computer-generated virtual space or virtual store.

[1734] An "interactive shopping experience" refers to an experience in which users explore, select, and purchase products in a virtual environment while interacting with other users in real time.

[1735] The system for realizing this invention comprises the following means: First, a user downloads and launches an application using their own terminal (smartphone, tablet, PC, etc.). The roles of the server, terminal, and user are explained in detail below.

[1736] Server Roles

[1737] 1. Data Collection:

[1738] When a user links their account with a social networking service (e.g., Twitter or Instagram), the server collects the user's posting data via the SNS API, using an authentication token.

[1739] 2. Data Analysis:

[1740] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine, utilizing a generative AI model (e.g., GPT-4) to identify user characteristics (e.g., specific hobbies and interests) and emotional states (e.g., positive, negative).

[1741] 3. Character and Story Generation:

[1742] The server generates characters and stories based on the identified user's interests, characteristics, and emotional state. A generative AI model automatically generates characters and stories personalized to the user.

[1743] 4. Real-time data synchronization:

[1744] The server has the ability to synchronize the data of multiple users in real time, providing an environment where users can play simultaneously.

[1745] 5. Creating personalized virtual environments:

[1746] Generate a virtual environment (e.g., a virtual store) that reflects the user's emotions and characteristics. The environment is customized based on the products and themes that the user is interested in.

[1747] Device Role

[1748] 1. Data reception and interface provision:

[1749] The user's device receives the generated characters and story sent from the server and displays them through a user interface, which also includes adjusting the music and background according to the emotions.

[1750] 2. Sending operational data:

[1751] When a user operates the device, the operation data (e.g., movement, selection, etc.) is sent to the server and synchronized in real time.

[1752] 3. Exploration and interaction within virtual environments:

[1753] Users can explore the virtual environment on their device and are provided with interactive experiences such as searching for products and completing missions.

[1754] User Roles

[1755] 1. Account linking and data provision:

[1756] A user links a social networking service account to the application and provides posting data.

[1757] 2. Gameplay and Shopping Experience:

[1758] Users not only enjoy individually customized characters and stories, but also have a shopping experience while connecting with other users in real time.

[1759] Specific examples

[1760] For example, if a user links their Instagram account and their posts are mostly about fashion and travel and contain positive content, they will be offered a personalized shopping adventure through the following steps:

[1761] 1. The user installs the application and connects their Instagram account.

[1762] 2. The server collects post data through the Instagram API and analyzes it using a generative AI model and emotion engine.

[1763] 3. Based on the analysis results, we recognize that the user is interested in "fashion" and "travel" and has positive emotions.

[1764] 4. The generative AI model generates a virtual fashion and travel-themed store and characters specifically for the user.

[1765] 5. The generated data is sent to the user's device, and their personalized shopping adventure begins.

[1766] Prompt Sentence Examples

[1767] "User 1 has provided us with their social media account and posting data. This user's posts are mostly about fashion and travel, and contain a lot of positive emotions. Please create a personalized virtual store that reflects the user's interests and emotions, and create a character and story just for the user."

[1768] In this way, users can have a unique shopping experience that reflects their own characteristics and interests, and can enjoy interacting with other users in real time.

[1769] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1770] Step 1:

[1771] The user installs the application and connects their social media account.

[1772] A user downloads and launches an application onto a device such as a smartphone or tablet. Next, they link their social networking service (SNS) account on the account registration screen. For example, they link their account by entering their Twitter or Instagram credentials. The input for this step is their SNS account credentials, and the output is an authentication token.

[1773] Step 2:

[1774] The server collects post data through the SNS API

[1775] The server uses the obtained authentication token to collect user post data through the SNS API. This data includes text, images, videos, etc., and is all stored on the server. The input of this step is the authentication token, and the output is the post data.

[1776] Step 3:

[1777] The server analyzes the collected submission data.

[1778] The server analyzes the collected posting data using natural language processing (NLP) technology and an emotion engine. NLP is used to analyze text data and identify user characteristics and interests. Image recognition technology is also used to extract information from image data. The emotion engine recognizes the user's emotions (positive, negative, etc.) from the posting data. The input of this step is the posting data, and the output is the user's characteristics, interests, and emotional information.

[1779] Step 4:

[1780] The server generates a character and story for the user.

[1781] Based on the analysis results, the server uses a generative AI model (e.g., GPT-4) to generate a character and story specifically for the user. The generated character reflects the user's characteristics, and the story incorporates the user's interests. The input for this step is the user's characteristics, interests, and emotional information, and the output is the generated character and story.

[1782] Step 5:

[1783] The server sends the generated character and story data to the user's device.

[1784] The server sends the generated character and story data to the user's device. The user's device displays a game interface based on the received data. The input for this step is the generated character data and story data, and the output is the game interface displayed on the user's device.

[1785] Step 6:

[1786] Users explore the virtual environment and engage in an interactive shopping experience

[1787] The user explores the virtual environment displayed on the device and controls the character. All exploration and actions within the virtual environment (e.g., selecting items, checking products) are sent to the server in real time. The server uses this operation data to provide an interactive shopping experience synchronized with other users. The input of this step is the user's operation data, and the output is a real-time synchronized shopping experience.

[1788] Step 7:

[1789] The server provides dynamic environment settings according to the user's emotions and actions.

[1790] The server dynamically adjusts the virtual environment settings (e.g., background music, visual effects) according to the emotions and real-time operation data analyzed from the user's posted data. The input of this step is the user's emotional information and operation data, and the output is the adjusted virtual environment settings.

[1791] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1792] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1793] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1794] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1795] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1796] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1797] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1798] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1799] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1800] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1801] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1802] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1803] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1804] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1805] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1806] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1807] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1808] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1809] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1810] Th...

Claims

1. means for collecting posting data from a user's social networking service account; A means for analyzing the collected posting data to identify user characteristics and interests; means for generating characters and stories based on the identified characteristics and interests; means for transmitting the generated characters and stories to a user's terminal; A means of synchronizing each user's data so that multiple users can play simultaneously in real time; A system including:

2. an authentication means for linking a social networking service account; A means for acquiring post data from a linked social networking service account via an API; A means for analyzing the acquired posted data using natural language processing technology; a means for generating characters and stories using a generative AI model based on the analysis results; A means for displaying the created characters and stories on a user's terminal; means for transmitting operation data from a user's terminal to a server and receiving operation data of other users synchronized in real time at the terminal; The system of claim 1 , comprising:

3. a data storage means for safely storing the collected submission data; A means of understanding the user's daily patterns based on the analyzed data; A means for adding characteristic attributes to a character generated based on the grasped daily patterns; A way to save your game progress and resume it the next time you play, The system of claim 1 , comprising:

Images